Storage: All Roads Lead to the Cloud
The massive paradigm shift in the TMT landscape has profound implications for data storage and the companies involved in storage technologies. Web-based applications for both consumers and enterprises are driving a rapid expansion in data. At the same time, the rise of flash storage equipped portable devices is crowding out PCs with more capacious hard drive storage, while enterprises are coping with the storage requirements of their virtualized data centers. We believe that these trends portend a substantial long-term shift to public cloud-based storage solutions based on commodity components and proprietary implementations of open-source software. As such, we expect flash demand to flatten, commodity disk drive demand to reaccelerate, and storage system demand to peak and decline. These changes will also create significant opportunities for cloud data center operators and IT services consultants who can facilitate the transition for enterprises, and for services and applications that can exploit the cost and performance advantages of cloud-based storage.
PC hard disks and enterprise RAID arrays have dominated storage for 20 years. The first PCs relied on removable “floppy” disks for storage, but soon adopted the more reliable and capacious hard disk as standard. In the enterprise data center, hard disks had been the norm for mass storage since their introduction in the 1960’s, but really flourished with the adoption of RAID (Redundant Array of Independent Disks) techniques for combining multiple disk drives into a system with built in mechanisms for back up storage and failure recovery, introduced during the late ‘80’s. This paradigm of hard disk equipped devices supported by self contained data center storage systems is a lynch pin of the client/server architecture that rose in the early ‘90’s and that his been the primary principle behind enterprise computing ever since.
Data growth, driven by networked applications, is prodigious and expected to continue. In 2011, 437 billion GB of hard disk storage and 2.7 billion GB of flash memory were shipped. Over the next five years, we believe that industry projections of better than 40% annual growth in demand for storage capacity will prove conservative, driven by prodigious growth in the creation of data by users and in the capture of data by enterprises.
Consumers are shifting from PCs to portable devices, and with that, from local storage to cloud storage. The rapid shift from PCs to smartphones and tablets as the primary means for consumers to access information has catalyzed an acceleration of user created data, and has driven growth in portable and power-friendly flash storage at the expense of disk drives. However, flash capacity is over 11 times more expensive than hard disk – a strong impetus for users to take advantage of cheap, plentiful, and reliable disk storage in the cloud accessible via nearly ubiquitous high speed wireless access and for device flash configurations to level off. Given that cloud operators typically keep four redundant copies of each file in order to assure reliability, this shift should significantly increase demand for disk storage capacity.
Enterprise data centers are overwhelmed by “big data” and will eventually shift to lower cost, higher performance public cloud solutions for most future storage. McKinsey estimates that enterprises stored 7 billion Gigabytes of new information in 2010, with data growing at a better than 40% annual pace. The RAID systems favored by enterprises are expensive, inefficient and inflexible compared with public cloud “Storage as a Service” or storage bundled with cloud-based applications. Enterprise concerns about data security and transition costs vis a vis the public cloud are easing, and the shift will be cautious but pervasive.
Massive growth in cloud storage will commoditize storage technologies. The paradigm shift underway is funneling both consumer and enterprise storage demand toward the cloud. Most of this demand will be served by disk drives bought in bulk by cloud operators, assembled to their specs by contract manufacturers, and managed by proprietary software. Meanwhile, integrated storage system vendors, like EMC, IBM, HP and NetApp, will see enterprise data center demand crest and decline. Both cloud and enterprise data centers will make increasing use of flash for specific niche applications where fast storage access is critical, but prices will not threaten disk drives for volume applications for decades.
Winners: enterprise disk drives and SSD, cloud operators, IT consultants; Losers: storage system vendors, device disk drives and flash. Device disk drives are roughly 75% of industry sales today, making the transition toward portable platforms painful, yet the potential for growth from the cloud is underappreciated. Given slight valuations, even in the face of major industry consolidation, we believe that drive makers and their suppliers are eventual winners from the paradigm shift. Likewise, we see the biggest cloud operators – Google, Amazon, and Microsoft – as broadly advantaged by their scale and skill in delivering on-line storage to both enterprises and consumers. As enterprises navigate the shift to the cloud, we see IT consultants, such as IBM, Accenture, and Rackspace, as playing an important role. On the flip side, the shift to the cloud will be painful for storage system vendors. Flash memory vendors are enjoying an immediate boom from portable device growth, but we fear that price competition and limits to the amount of flash needed per device may yield future disappointments.
For our full research notes, please visit our published research site.