The challenge of EMC

One of the great success stories of the recent IT history is EMC. They exploited the wonderful opportunity which was created when corporations expanded their choice of equipment to include mainframes, Unix machines and Intel-based servers.

The multiplicity of different architectures was considered unavoidable as the range of applications diversified, even though everyone knew that this would create a lot of management problems. I suspect that few organisations realised that there would also be such a growing demand for middleware to integrate these disparate systems. In any case there was a huge increase in the demand for online storage, both from the new generation of data processing applications (ERP) and the advent of CRM applications, with the attendant need for a data warehouse. At the same time office systems also created a huge increase in the need for storage as GUI text processing became the norm (a Word document is ten times bigger than the actual text, roll on XML editors and document systems!), as well as the growth of e-mail.
All of the systems vendors could supply the extra storage, but at a high price. Further few were able to cover all platforms. Thus the opportunity appeared for specialist storage sub-system vendors to move in. StorageTek, and HDS were well established, Data General successfully diversified into this area, but the big success story was EMC. They above all took advantage of the fantastic developments in magnetic recording devices, particularly disk drives. EMC don't make storage devices, they buy them and assemble them into subsystems. They took the RAID (Redundant Arrays of Inexpensive Discs) concept into their products, producing far more price effective fault tolerant systems than IBM, HP, et al, as well as being happy to cover multiple platforms.
However the same concepts have now been implemented by all the competition and with IBM buying a good customer like Sequent and HP taking over Compaq, the market is becoming more competitive. Add to this the down turn in spending on IT over the last year and EMC have been badly hit, laying off a lot of employees world wide.
The other major development has been the advent of Network Attached Storage (NAS) and Storage Area Networks (SAN), both attempting to provide a pool of storage for multiple systems. In particular developments are underway to enable multiple different systems to share common files, but not for some time yet and certainly not before long term industry wide standards can be agreed. However it is possible to partition a storage subsystem so that part of it is used by one computer and another part by a different one. If it were two totally independent systems this would be of little advantage compared to separate storage systems attached to the individual computers. However there is a big advantage when all the subsidiary requirements are considered. In particular back-up and archiving become much simpler, since it is applied to only a single sub-system. At the moment there is a big gap in back-up technologies. The hardware is remarkable, with low cost tape robots bringing high performance and high capacity to medium sized installations as well as the huge enterprise systems of the past. However the software is still host processor based, which raises the question in a mixed environment of which host. Data sets are moved from the storage sub-system to the host and then written back to the tape system. It is surprising that the storage systems themselves have not been developed yet to write directly across the storage Lan, bypassing the host. This is not trivial because it requires a lot of mapping of sectors to files, which means that file allocation tables (or equivalent) will have to be mirrored from each host to the storage system.
The bottom line of the above considerations then is obvious. The developments will focus on software rather than hardware. Storage management software products are going to be the next differentiator in SAN and NAS technologies. This will have to be a combination of host supported, as today, but also in the development of dedicated features of the next generation controllers.
This is the challenge that EMC must take up if they are to continue to keep out IBM, HDS, et al. It is clear that they know this and the majority of the development taking place at EMC labs outside Boston is focused on software today. Whether they can keep ahead of the competition in this arena yet remains to be seen.

 
Martin Healey, pioneer development Intel-based computers en c/s-architecture. Director of a number of IT specialist companies and an Emeritus Professor of the University of Wales.