Much ado about supercomputers, and what it means for big data

NEW IBM DATA CENTRIC SYSTEMS FOR DOE
NEW IBM DATA CENTRIC SYSTEMS FOR DOE - - The U.S. Department of Energy has awarded IBM contracts valued at $325 million to develop and deliver the world’s most advanced “data centric” supercomputing systems at the Lawrence Livermore and Oak Ridge National Laboratories, helping them advance innovation and discovery in science, engineering and national defense. These OpenPOWER-based systems minimize data in motion and energy consumption, providing cost-effective modeling, simulation and Big Data analytics. (from left) Dr. Thom Mason, director, Oak Ridge National Laboratory; Dr. John E. Kelly III, Senior Vice President & director of IBM Research; Dr. Bill Goldstein, director of Lawrence Livermore National Laboratory. (Feature Photo Service)
Photograph by Rich Riggins/Feature Photo Service

One closely watched list in scientific circles is the Top500 ranking of the world’s faster supercomputers because of the sheer volume of data these room-sized systems are able to process.

The list is dominated by technologies in use throughout the United States and focused on data-intensive tasks such as counter-terrorism, basic science research, climate change simulations, or oil and gas exploration. Since last June, however, the Top500 has been led by a system at a Chinese university specializing in defense technology.

The U.S. government is determined to change that within the next three years. Last Friday, the Department of Energy (DOE) disclosed plans to spend $300 million on two new supercomputers that will be “five to seven times more powerful” than systems currently in use.

The technologies blessed by the contract—supplied by IBM (IBM), Nvidia (NVDA), and Mellanox (MLNX)—will be installed at the Oak Ridge National Laboratory and the Lawrence Livermore National Laboratory. The Argonne National Laboratory will also get a new supercomputer, but its supplier hasn’t been announced yet.

Aside from this short-term solution to analyzing ever-growing volumes of data, the DOE and the National Nuclear Security Administration are putting $100 million into a project called FastForward2. The aim of the initiative is to develop “exascale” computers 20 to 40 times faster than systems in use today. Advanced Micro Devices (AMD), Cray, IBM, Intel (INTC), and Nvidia are all participating in that initiative. By the way, Cray managed to score its own big contract last Friday: it will get $80 million to build its first system in the Middle East in almost two decades.

“High-performance computing is an essential component of the science and technology portfolio required to maintain U.S. competitiveness and ensure our economic and national security,” said DOE Secretary Ernest Moniz, in a statement.

Why are the two new supercomputers the DOE plans to install so much faster than what’s in use today? IBM touts the technology’s “data-centric” approach to processing information. Simply put, more analysis can take place where data is stored, reducing the processing power that has to be dedicated to retrieving it and moving it back and forth between storage hardware and processors.

When data does need to be moved around, it can be done at speeds of up to 17 petabytes per second—which is the equivalent of moving more than 100 billion digital photos.

In the traditional supercomputing model, advances in microprocessor speeds can only go so far, said Dave Turek, vice president of exascale computing at IBM. “It used to be all about the calculation, and now it is all about the data,” he said. “We have to put computing in places it never has been before.”

Aside from scientific applications such as the ones contemplated by the DOE, this unconventional approach could benefit industries highly dependent on high-performance computing technologies for analytics and business intelligence, such as financial services or oil and gas exploration. “Supercomputing design has to change, and it has to change in a way that very explicitly brings these analytics application to the fore. It’s all about accelerating the time to decision,” Turek said.

This item first appeared in the Nov. 17, 2014 edition of Data Sheet, Fortune’s daily newsletter on the business of technology. Sign up here.