Rochester team helps IBM create world's fastest computer

IBM made a triumphant return to the top of world’s fastest supercomputer list on Monday.

Summit, a water-cooled machine with deep roots in Rochester, took the No. 1 spot on the twice-a-year TOP500 list.

The supercomputer, still being installed by a mostly Rochester-based team at the Department of Energy’s Oak Ridge National Laboratory, was clocked running at 122.3 petaflops.

That adds up to a stunning 200 quadrillion calculations per second. It’s 120 times faster than IBM’s record-breaking Roadrunner supercomputer, which was the fastest in the world in 2008.

Sunway TaihuLight, a system developed by China’s National Research Center of Parallel Computer Engineering & Technology, fell to No. 2 on Monday after leading the list for the past two years. Its speed of 93 petaflops has remained unchanged since it came online in June 2016.

Much of Summit’s design, particularly the water-cooling system, came out of Rochester as well as orchestrating its manufacture in Guadalajara, Mexico, and its installation at the U.S. Department of Energy’s Oak Ridge National Laboratory in Oak Ridge, Tenn.

The supercomputer and its slightly slower sibling, Sierra (71.6 petaflops), were simultaneously designed and built by IBM, Nvidia, Mellanox Technology and Red Hat Linux software.

"What we did in Rochester was to integrate all of these pieces together into a solution," explained Rochester’s Andy Schram, IBM project executive for CORAL and high performance computing. "Rochester has been integrating technology (like the AS/400 server) for decades and I think that’s why they came back to here for this project."

While IBM has long maintained a presence on the Top500 supercomputer list, this marks its active return to the competition.

Big Blue was active in the sector in the early 2000s creating many record-breaking machines. However, its interest in supercomputers had dwindled and rolled to a stop for a period.

That changed in 2013, when IBM won the $325 million federal contract to build Summit and Sierra for CORAL, a collaboration between the U.S. Department of Energy, the National Nuclear Security Administration and the Office of Science.

"None of this stuff (technology) existed back then. You have to bid something, when you don’t know if you can make it," Schram said.

And the contract required that both supercomputers to be made and installed at the same time.

However, IBM with lots of help from its Rochester site has what could be described as "a unique set of skills."

"The capability to do this does not exist anywhere else in the world," said Ryan Paske, senior technical staff member for IBM’s Supply Chain New Products Engineering division. "IBM made two of the top computers in the world at the same time."

A key part of Sierra’s speed and power is the new water-cooling system that pumps water through tubes connecting the supercomputer’s 4,608 drawers. Flexible, metal "cold plates" support the water system.

"The cold plate is the technology allows us to package this much computing power in a very small container (each drawer). You couldn’t do this with air cooling," said Chris Marroquin, a system packaging design/integration engineer based in Rochester.

The flexible metal connector plates and the tubing allow the water system to be pulled out of the way for regular maintenance of the computer.

Marroquin says each drawer gets between half gallon of water to one gallon of water pumped through them every minute, depending on temperature. Once the water absorbs heat produced by IBM’s Power9 computer processors, it is circulated to bring the temp back down.

The major differences between Sierra and IBM’s earlier water-cooled machine called Blue Gene/Q is the architecture and the temperature. Blue Gene/Q used chilled water, while Sierra uses room-temperature water.

Of course, mixing water with electronics is tricky thing.

Sierra has 4,000 gallons pumped through its system every minute, according to Marroquin. That adds up to 100,000 potential leak points.

"And one leak is too many," he added.

While Summitt and Sierra are the stars of the moment, a 7-year-old IBM machine also made the top 10. Sequoia, an IBM BlueGene/Q system at Lawrence Livermore National Laboratory, hit the list at number eight running at 17.17 petaflops.

Big Blue had 19 machines on the top 500 list, which places it sixth among manufacturers on the list.

IBM’s return to number one and snagging three of the top 10 spots was positive, but the overall number of US-made machines dropped to a record low of 124, down from 145 in November. China has 206 on the list.

Looking to the future, Schram hopes this won’t be the last time IBM builds two supercomputers for the U.S. government.

He helped submit a bid earlier this month for the CORAL-2 project to build two computers to calculate at record exaflop speeds.

"We’ve put in a what we think in the very competitive bid. Hopefully, we’ll be doing this again in five years," said Schram with a grin.

AS/400

• In June of 1988, IBM Rochester rolled out the AS/400 midrange server, a new computer designed to be flexible and stable for businesses to use every day. It became wildly popular for its flexibility and longevity. 

The AS/400 quickly became one of the world's most popular business computing systems. The 400,000th AS/400 was presented on October 9, 1996, in Rochester to Greg LeMond, the three-time winner of the Tour de France bicycle race. By 1997, IBM had shipped nearly a half-million AS/400s. 

The AS/400 family was succeeded in 2000 by the IBM eServer iSeries.

Power processor

• Rochester IBM contributed to the development of the Power1 microprocessor in 1991.

The creation of Power1 launched a series of CPU units that came to power most of IBM's supercomputers.

Blue Gene/L

• The first IBM Blue Gene/L was built at the IBM lab in Rochester. On Sept. 29, 2004, Blue Gene/L surpassed NEC's Earth Simulator as the fastest computer in the world.

Blue Gene/L used 131,000 processors to routinely handle 280 trillion operations every second. A single scientist with a calculator would have to work nonstop for 177,000 years to perform the operations that Blue Gene could do in one second.

Each Blue Gene rack contained 1024 dual-processor nodes in a footprint that dramatically reduced floor space. The processors were engineered, so 32 fit on a single microchip.

Roadrunner

•  About 50 people at IBM Rochester developed and built the $133 million Roadrunner supercomputer for the Los Alamos National Laboratory in New Mexico in 2008.

On May 25, 2008, Roadrunner broke the elusive speed barrier of one petaflop — one thousand trillion calculations per second.

Roadrunner's time at the top of the supercomputer world did not last long. It was retired or unplugged as obsolete on March 31, 2013.

Blue Gene/Q Mira

• On Feb. 8, 2011, IBM announced the 10-petaflop Blue Gene/Q supercomputer Mira, in collaboration with the US Dept. of Energy's Argonne National Laboratory located near Chicago, Illinois.

Mira, which has a speed of 10 petaflops, was created to drive advances in designing ultra-efficient electric car batteries, understanding global climate change, exploring the evolution of our universe and more. 

Blue Gene/Q Sequoia

• With the help of Rochester, IBM introduced Blue Gene/Q. Blue Gene/Q used chilled water to cool the processors.

Sequoia, which is owned by the Department of Energy’s Lawrence Livermore National Laboratory, has 96 racks with 1.5 million of processors.

Sequoia was delivered to the Lawrence Livermore beginning in 2011 and was fully deployed in June 2012. It is part of the Advanced Simulation and Computing Program running nuclear simulations and advanced scientific research.

The third supercomputer design in the Blue Gene series, Blue Gene/Q has a peak performance of 20 petaflops.

Summit

Summit, which many Rochester IBM helped design, build and install, was developed for the Oak Ridge National Laboratory. It uses a water-cooled technology, though it does not require the water to be chilled as in the earlier Blue Gene/Q machine.

As of Monday, Summit was considered the fastest supercomputer in the world with its processing power is clocked at 200 petaflops.