The core to the problem is that there is little to no Information Science employed in practice. The whole of IT is hit or miss, driven by personal ambition on one hand and hope on the other. Fear of being left behind is also a serious influence.
At the root of all this is that there is no sure way of assessing the value of expenditure on IT. There are some obvious examples where value for-money is unquestionable. Automated accounts, order processing and stock control are indispensable. Text processing, spreadsheets and e-mail are worth investing in for a percentage of the work force, but certainly not for everybody. Huge sums of money have been wasted on upgrading office systems, the main justification being that they are incompatible with older versions! This of course is the divide and conquer rule. The individuals in a company would love to have the sophisticated tools to play with and if someone else can have them, why can't they? It is staggering that no one seems to dare to demand a business case for this expenditure. The ability of the PC industry to persuade companies to waste significant sums of money stems from mass advertising. Few industries, if any, spend such huge sums on marketing, brain-washing users. Only recently did an article in Fortune (of all magazines) complain about how "hostile" the PC software was. This is a good trend but where is the mass voice of the press?
It is clear that mass advertising to the individual has badly influenced decision making in corporations, but bad investments aren't constrained to PCs. There has been a Leeming-like leap into ERP packages. There has unquestionably been some good marketing, especially by SAP, but we are talking about huge central investments in which the customer is represented by professional IT staff or highly paid consultants. And yet there is a growing awareness that the panacea of integration has not been delivered. These ERP systems appear to have many of the failings of the mixture of older systems. In truth I believe that the new ERP systems are potentially better than the mixture of applications they replaced, but the compliant is that only a small gain was made at huge cost.
Nowadays there is some serious literature being produced that is analysing some of the huge investment in IT against progress. Keith Devlin points out that prior to 1965 and the advent of large scale IT, US productivity grew 2% per annum, since when it has remained below 2%. Now this could well be due to other factors, but it is food for thought.
While it has been virtually impossible to justify the expenditure of office systems and difficult for commercial systems, we are now struggling with" information systems". In true keeping with the "press on in hope" attitude of the IT industry, vast sums have been spent on data warehousing. The success of some high profile ventures, WalMart in particular, frightened many companies into action. There have been many disasters. Data warehousing is difficult and requires full cooperation between business and technical analysts. And this exposes another core problem; in general the relationships between technicians and business people are strained and cooperation proves very difficult. In any case a data warehouse is only a tool for serving an information system and the lack of "information" skills has been a serious problem.
However it is probably the high profile of information systems that may well be the turning point for the IT industry. To evaluate the use of an information system, the impact on the whole organisation and the markets in which it trades, must be understood. And so there are serious signs that the academic world is becoming aware of "Information Science"!. From this should come some formal methods of evaluating sensitivity to investments. Until then it remains a "we think it is necessary" not "we know" world of IT. Fun but very inefficient.