IBM, Nvidia, and Stone Ridge Technology today reported setting the performance record for a “billion cell” oil and gas reservoir simulation. Using IBM Minsky servers with Nvidia P100 GPUs and Stone Ridge’s ECHELON petroleum reservoir simulation software, the trio say their effort “shatters previous (Exxon) results using one-tenth the power and 1/100th of the space. The results were achieved in 92 minutes with 60 Power processors and 120 GPU accelerators and broke the previous published record (Aramco) of 20 hours using thousands of processors.”
The ‘billion cell” simulation represents a heady challenge typically tackled with ‘supercomputer’ class HPC infrastructure. The Minsky, of course, is the top of IBM’s Power server line and leverages Nvidia’s fastest GPU and NVLink interconnect. This simulation used 60 processors and 120 accelerators. IBM owed the systems – each Minsky had two Power8 CPUs with 256GB of memory, four Nvidia P100 GPUs, and InfiniBand EDR.
“This calculation is a very salient demonstration of the computational capability and density of solution that GPUs offer. That speed lets reservoir engineers run more models and ‘what-if’ scenarios than previously so they can produce oil more efficiently, open up fewer new fields and make responsible use of limited resources,” said Vincent Natoli, president of Stone Ridge Technology, in the official announcement. “By increasing compute performance and efficiency by more than an order of magnitude, we’re democratizing HPC for the reservoir simulation community.”
According to the collaborators, the data set was taken from public information and used to mimic large oil fields like those found in the middle east. Key code optimization included taking advantage of the CPU-GPU NVLink and GPU-GPU NVLink in the Power systems and also scaling the software to take advantage of 10s of Minsky systems in an HPC cluster.
The new solution, say the collaborators, is intended “to transform the price and performance for business critical High Performance Computing (HPC) applications for simulation and exploration.” The performance is impressive but not overly cheap. IBM estimates the cost of the 30 Minsky systems in the range of $1.5 million to $2 million. ECHELON is a standard Stone Ridge product and IBM and Stone Ridge plan to jointly sell the new solution into the oil and gas market.
Sumit Gupta, IBM vice president, High Performance Computing & Analytics, said, “The bottom line is that by running ECHELON on Minsky, users can achieve faster run-times using a fraction of the hardware. One recent effort used more than 700,000 processors in a server installation that occupies nearly half a football field. Stone Ridge did this calculation on two racks of IBM machines that could fit in the space of half a ping-pong table.”
IBM has been steadily ratcheting up efforts to showcase its Power systems – including Minsky – as it tries to wrestle market share in an x86 dominated landscape. Last month, the company spotlighted another Power8-based system – VOLTRON at Baylor College – which researchers used to assemble the 1.2 billion letter genome of the mosquito that carries the West Nile virus.
IBM and its collaborators argue “this latest advance” challenges misconceptions that GPUs can’t be efficient on complex application codes such as reservoir simulators and are better suited to simple, more naturally parallel applications such as seismic imaging.
They do note, “Billion cell models in the industry are rare in practice, but the calculation was accomplished to highlight the growing disparity in performance between new fully GPU based codes like ECHELON and equivalent legacy CPU codes. ECHELON scales from the cluster to the workstation and while it can turn over a billion cells on 30 servers, it can also run smaller models on a single server or even on a single Nvidia P100 board in a desktop workstation, the latter two use cases being more in the sweet spot for the industry.”