It is becoming almost impossible to evaluate the true value of software. Despite spending a lot of effort planning upgrades to major ERP applications, the final result seldom satisfies many people, either simply because it is different (people are creatures of habit) or because the disruption with established practice interferes with business activities.
In most cases the new applications do offer advantages because they are replacing systems built some time ago which have not kept pace with business changes. The problem though is not whether a new system better, but is it a big enough improvement to justify the expense. In any case the decision to introduce a new system is based on assumptions which cannot be proven until the new system is live. When it is live and has been given a chance to settle down, then the benefits can be better evaluated. This however is a dubious practice, because it will have little impact; organisations are not going to change again!
The other major problem is the IT industry itself. There must of course be a input from the business managers, but a lot of the decisions about which package to adopt are made by the IT departments. It is the same people who are responsible for choosing a new system who are required to evaluate it. They are not going to willingly turn round and say that they had picked the wrong system. Thus since it is a very inexact science to evaluate return-on-investment (ROI) the results of any post-implementation study will be biased.
The situation is worse with office software than with business systems. Here the user’s opinion is far more influential. Unfortunately the users think that they know more than they do because they are heavily influenced by direct advertising. They use PCs at home and want the same thing at work, quite sensibly. Unfortunately they refuse to listen to professional advice. In particular they avoid the issue of the real cost of ownership, which is very different in a commercial environment to the home. In fact this should be a management issue, but management were also seduced by the PC and failed to give control to IT departments until it was too late. Today so many people are involved, from user to director, that no one is willing to address the problem of the very low ROI of office systems! Inertia is a very powerful thing.
It doesn’t help much either if we admit to some serious mistakes because business doesn’t stand still. Thus as soon as one problem is resolved others crop up. We have only touched the tip of the iceberg with e-commerce yet and we have no sooner upgraded Microsoft Office than another release is due. One can categorically state that the bottom-line benefits of upgrading to the newest version will be negligible and the ROI quite disastrous, but how can the power of mass marketing be resisted?
The biggest problem is how one can get unbiased, accurate advice, the answer to which, sadly, is "with difficulty". Consultancies need to be chosen with care. The larger outfits need to generate a lot of ongoing business to cover significant overheads. It is in their vested interests to recommend change for changes sake. They advise to use Unix and then when the change has been made and the consultancy needs diminish, they recommend a change to PC networks. Next they will be recommending a change to OSS and so on. Smaller, specialist consultancies should be a better bet, but one should check that the skills needed are actually available and not just a claim in a brochure.
Conferences and seminars were the best way of getting less biased information, not only from the speakers, but from conversations with fellow delegates. Conference fees are relatively high and there has been a significant cut in training budgets over the last five years so that there are fewer and fewer conferences to go to. The days of the huge exhibitions are over, at least for the time being. User groups are a good idea; although obviously biased towards the common product involved, they usually have alternative speakers and again the like minded delegates are a good source of opinion.
We need some unbiased magazines but they would have to be paid for; who would believe advice from a "free circulation" magazine? We seem to have cut off the sources of good advice which is not a healthy situation.
Martin Healey, pioneer development Intel-based computers en c/s-architecture. Director of a number of IT specialist companies and an Emeritus Professor of the University of Wales.
