Inflection points are thrilling and slightly unnerving to live through. Like, for example, the inflection point the IT world is experiencing now.
We’re in a phase where the exponential growth of cloud computing feels unstoppable. Many industry observers are starting to predict not just that clouds will be pervasive, but that they will, before long, be the only platforms for IT infrastructure, applications, and commerce.
Big data analytics may be the trigger that accelerates us toward all things cloud. Big data analytics is the cloud’s killer application -- and also an inextricable component of many other hot, on-demand services that live natively in the cloud. After all, social, mobility, IoT, and security applications are huge sources of big data and also avidly consume both the data and the full range of advanced analytics that leverage it all.
One key metric indicating the rise of cloud services is the concomitant growth in adoption of on-demand data analytics that live in public clouds. Consider the recent Computerworld article with the headline screaming that "Cloud analytics is expected to keep growing and growing." The piece points to a recent study by analyst firm Research & Markets, which forecasts a CAGR (compound annual growth rate) in the global cloud analytics market of 26.29 percent from 2014 to 2019. The article also cites another analyst firm, Constellation Research, which puts the cloud analytics market’s CAGR even higher, at 46 percent through 2020. Various other analyst projections put cloud analytics market CAGR at more or less 25 percent indefinitely.
Consequently, I may need to revise my proclamation from two years ago, in which I stated that “it's far too soon to proclaim cloud the be-all, end-all platform for big data analytics.” My rationale for that statement was that “big data cloud approaches must prove their value in a competitive landscape where packaged software, appliances, expert integrated systems, and other deployment models have clear advantages in various circumstances.”
But now I’m not sure so sure that these older approaches can withstand the price-performance competition of the increasingly formidable cloud option. For big data and other applications, the cloud's value proposition derives from several key features: massive parallelism, petabyte scaling, on-demand resource pooling, elastic provisioning, self-service access and usage, and agile virtualization. More than that, its security and stability have continued to strengthen.
Perhaps I should have seen the writing on the wall. When, in that previous post, I spelled out the cloud’s sweet spot for big data use cases, I left loopholes a mile wide.
For starters, I pegged the public cloud’s chief use cases as the following: any enterprise application that is already hosted in the cloud, high-volume external data sources that require considerable pre-processing, elastic provisioning of very large but short-lived analytic sandboxes, and queryable off-premises archive. But when you consider that those use cases describe many social, mobile, IoT, streaming, and security applications, you have to admit that this explains much of the cloud space’s growth.
But to top it off, I noted that “if we scope ‘cloud’ to include private and hybrid deployment models, the range of suitable big-data applications is much broader.” And that also describes much of the uptake in enterprise cloud computing over the past few years. In fact, you’d be hard-pressed to identify any recent startup activity in the enterprise application market that doesn’t predicate its solution on either a public or hybrid public-private cloud deployment model, or on a “multi-cloud” model that involves two or more public cloud solutions.
So it’s clear that within the coming 10 years most customers will begin to provision most of their IT needs from cloud services. And this raises the question of whether, with all these trends pointing in cloud’s favor, it’s still possible that some day the trend may wane in favor of yet another new platform paradigm.
Will the spread of IoT-stoked “fog computing” -- in which data, processing, and other distributed resources are spread among trillions of edge nodes -- pave the way for more of a “peer-to-peer” or “mesh” networking model, away from the shared-service multitenant, software-as-a-service model currently in vogue?
It’s not too early to start looking beyond the cloud to the next evolutionary step in big-data analytics platforms. Thinking that the cloud is the omega of IT platforms is a bit like being Thomas J. Watson, Sr., who in the 1940s reputedly predicted that all the world would ever need would be five data-processing systems.
Here in the early 21st century, we technology professionals need to recognize that we too may have to adjust our visions as some unforeseen, disruptive, and all-consuming new data fabric takes shape.