Data-intensive supercomputer brings new users to high performance computing for science

Handling big data can sometimes feel like driving on an unpaved road for researchers with a need for speed and supercomputers.

“When you’re in the world of data, there are rocks and bumps in the way, and a lot of things that you have to take care of,” said Niall Gaffney, a former Hubble Space Telescope scientist who now heads the Data Intensive Computing group at the Texas Advanced Computing Center (TACC).

Gaffney led the effort to bring online a new kind of supercomputer, called Wrangler. Like the old Western cowboys who tamed wild horses, Wrangler tames beasts of big data, such as computing problems that involve analyzing thousands of files that need to be quickly opened, examined and cross-correlated.

Wrangler fills a gap in the supercomputing resources of XSEDE, the Extreme Science and Engineering Discovery Environment, supported by the National Science Foundation (NSF). XSEDE is a collection of advanced digital resources that scientists can easily use to share and analyze the massive datasets being produced in nearly every field of research today. In 2013, NSF awarded TACC and its academic partners Indiana University and the University of Chicago $11.2 million to build and operate Wrangler, a supercomputer to handle data-intensive high performance computing.

Wrangler was designed to work closely with the Stampede supercomputer, the 10th most powerful in the world according to the bi-annual Top500 list, and the flagship of TACC at The University of Texas at Austin (UT Austin). Stampede has computed over six million jobs for open science since it came online in 2013.

“We kept a lot of what was good with systems like Stampede,” said Gaffney, “but added new things to it like a very large flash storage system, a very large distributed spinning disc storage system, and high speed network access. This allows people who have data problems that weren’t being fulfilled by systems like Stampede and Lonestar to be able to do those in ways that they never could before.”

Gaffney made the analogy that supercomputers like Stampede are like racing sports cars, with fantastic compute engines optimized for going fast on smooth, well-defined race-tracks. Wrangler, on the other hand, is built like a rally car to go fast on unpaved, bumpy roads with muddy gravel.

“If you take a Ferrari off-road you may want to change the way that the suspension is done,” Gaffney said. “You want to change the way that the entire car is put together, even though it uses the same components, to build something suitable for people who have a different job.”

At the heart of Wrangler lie 600 terabytes of flash memory shared via PCI interconnect across Wrangler’s over 3,000 Haswell compute cores. “All parts of the system can access the same storage,” Gaffney said. “They can work in parallel together on the data that are stored inside this high-speed storage system to get larger results they couldn’t get otherwise.”

This massive amount of flash storage comes from DSSD, a startup co-founded by Andy Bechtolsheim of Sun Microsystems fame and acquired in May of 2015 by EMC. Bechtolsheim’s influence at TACC goes back to the ‘Magnum’ Infiniband network switch he led design on for the now-decommissioned Ranger supercomputer, the predecessor to Stampede.

What’s new is that DSSD took a shortcut between the CPU and the data. “The connection from the brain of the computer goes directly to the storage system. There’s no translation in between,” Gaffney said. “It actually allows people to compute directly with some of the fastest storage that you can get your hands on, with no bottlenecks in between.”

Read the rest of this in-depth report, here.