A modest and geeky announcement by Google and a couple other big tech names on Thursday may be the summer’s biggest development in a continuing fight over the future of huge global computing.
Microsoft, IBM and Red Hat, along with others, are joining Google in creating open source software to manage computing workloads across thousands of computer servers. Amazon Web Services, by far the largest cloud computing company, was notably absent from the announcement.
Called Kubernetes, the software project draws on a former Google top secret: how the search giant efficiently built and deployed powerful applications like search, mail and maps over its cloud computing system, one of the world’s largest.
What Google kept proprietary it now hopes to sell to thousands of companies around the world as part of renting the public its computing power. So do Microsoft and IBM, with public clouds that chief executives at both companies say are crucial to their futures. Red Hat is a leading provider of services for the Linux computer operating system, which is used extensively in cloud servers.
“Some of the biggest enterprises in the world are throwing their hats into the ring,” said Miles Ward, an executive with Google Cloud Platform, Google’s cloud computing business. “We have built a stupefying amount of software using this.” Soon, he said, the rest of the world will, too.
Kubernetes may also prove useful in smaller cloud systems operated by private companies, in particular for deploying applications and bridging private and public clouds. “We are big believers, long term, in public clouds,” Mr. Ward said, “but there is still a couple trillion dollars in gear sitting around doing computing inside companies.”
That such computing infrastructure might be needed by a typical company says a lot about the scale of resources expected to be commonplace once things like big data analysis and managing billions of connected devices become commonplace in corporations.
That Google and others will donate such things free may say even more about the competitive environment. At almost the same time as the Kubernetes announcement, Amazon Web Services was having a large meeting in New York, marketing its version of cloud computing, which downplays the idea that companies should even have their own data centers. So far, that approach has worked well, but Google, Microsoft and IBM are all pushing variants on a hybrid model.
“Other companies say that everything is going to the public cloud — not in my lifetime,” said Paul Cormier, president of products and technology at Red Hat. “Red Hat and Google see things going in the same direction.”
In an email, an Amazon spokeswoman said that the web service had its own management system for workloads, which automates most functions efficiently. She declined to comment on the competitive environment or whether Amazon’s would follow its competitors into a hybrid business.
There is another way in which the announcement could rock the status quo. Kubernetes manages so-called “containers” inside the cloud, used for running different applications. The idea is to use the computing power in each of a cloud’s processing units, sometimes hundreds of thousands of computing nodes, as efficiently as possible. Google says it runs more than two billion containers a week for its services.
In this sense, containers are an improvement on virtualization, a means of increasing the efficiency of individual servers that remade the computer business a decade ago. Twitter and Airbnb already use an open source container software, called Mesos, to run applications.
That suggests that, in the long run, well-managed containers could also prove a threat to virtualization, a money maker for both Microsoft and VMware, a multibillion-dollar subsidiary of the data storage giant EMC.
“Virtualization no longer makes sense,” said Florian Leibert, co-founder and chief executive of Mesosphere, a container management company that is also working on the Kubernetes project. “This is the way to run computing at a big scale.”
Jason Zander, corporate vice president of Microsoft’s cloud business, said his company expected to keep using virtualization, as well as the bigger types of systems. “This won’t replace virtual machines,” he said. “There will be a spectrum of solutions.”
In an email, a VMware spokesman said the company thinks its customers are “thinking/wanting to figure out how to utilize containers within their existing VMware context.”