ORNL Summit Supercomputer Is Officially Here

By Tiffany Trader

June 8, 2018

Oak Ridge National Laboratory (ORNL) together with IBM and Nvidia celebrated the official unveiling of the Department of Energy (DOE) Summit supercomputer today at an event presided over by DOE Secretary Rick Perry. The partners, who collaborated to design and build the estimated $200-million dollar machine under the CORAL procurement program, heralded it as the world’s most powerful supercomputer with 200 peak petaflops for high-performance computing workloads and 3.3 peak exaops for emerging AI workloads.

The deployment encompasses 4,608 compute nodes, each containing two 22-core IBM Power9 processors and six Nvidia Tesla V100 GPUs, interconnected with dual-rail Mellanox EDR 100Gb/s InfiniBand. Summit is said to offer 8X more performance than its predecessor, Titan, which spans 18,688 AMD-Nvidia nodes. The new supercomputer has a power footprint of 13MW, not a significant increase over Titan’s 9MW considering the massive performance leap. Summit will include a 250PB IBM Spectrum Scale file system. This parallel file system, named Alpine

DOE Secretary Rick Perry at Summit unveiling

Perry upheld Summit’s installation as a sign of the United States’ global competitiveness and technological leadership:

“We know we’re in a competition and we know that this competition is real and it matters who gets there first,” said Perry. “Today [we] show the rest of the world that America is back in the game and we’re back in the game in a big way. Our national security, our economics, our scientific discovery, our energy research will be affected in a powerful way.”

Perry warned however that the U.S. also faces a challenge. “There are other nations that are racing to develop their technology; if we’re not dedicated and determined, the leadership we enjoy today could be the leadership of tomorrow and we don’t want that,” he said.

While this soft-launch (formal acceptance is scheduled for later this year) is an important milestone that is generating wide media attention, the HPC community proper is still awaiting and expects hard benchmarks; they won’t have to wait too much longer with the next Top500 list due out in two weeks. If Summit achieves the Linpack score that we’ve heard projected, roughly 120-petaflops, the United States could retake the Top500 crown from China, pending no surprises. China has held the top of the list since 2013 with the debut of the 33.9-petaflops (Linpack) Tianhe-2A. That machine fell to number two in 2016, when China stood up the 93-petaflops (Linpack) Sunway TaihuLight, which still holds the number one spot. The fastest U.S. machine is still the Oak Ridge Titan supercomputer, which entered the list at the number one position in November 2012 (with 17.6 Linpack petaflops) and now ranks fifth.

Perry emphasized the importance of supercomputing leadership to the United States’ administration, stating, “President Trump is determined to make America first in supercomputing.” He referenced the President’s March budget, noting it includes $677 million in funding for exascale activities, and indicated further funding increases are likely. (See our latest exascale budget coverage here.) The procurement process for Summit’s successor, named Frontier, is already underway. The plan is for the CORAL-2 machine to be the nation’s first capable exascale supercomputer with delivery timed for the second half of 2021.

The Linpack metric that the Top500 listing is based on, though imperfect, is a more meaningful way to rank machines than peak capability. Of course, the only benchmark that really matters is how a supercomputer performs on real applications. At the unveiling today, ORNL Director Thomas Zacharia noted that one of the earliest science applications carried out on Summit broke the mixed-precision exascale barrier.

Each Summit node uses six Nvidia Volta GPUs per two Power9 CPUs, tied together with Nvidia’s NVLink 2.0 technology (Image credit: Jason Richards/ORNL)

During early testing, researchers at Oak Ridge achieved 1.88 exaops using Summit’s V100 GPU Tensor cores to run a comparative genomics code that analyzes variation between human genome sequences. The run was carried out using a representative dataset on 4,000 nodes, achieving a computational efficiency of greater than 50 percent. Summit enabled a 25-fold speedup for the code compared to the lab’s previous leadership-class supercomputer Titan with the Tensor cores alone providing a 4.5-fold application speedup. (See ORNL’s writeup for more details.)

Summit, according to Oak Ridge and its partners, is poised to provide unprecedented computing power and deep learning capability to enable scientific discoveries that were previously impractical or impossible, and will advance research in energy, advanced materials and artificial intelligence (AI) and other domains. Its power will also be lent to improving the care of military veterans through a partnership with the US Department of Veterans Affairs that began in 2016.

Some of the science projects slated to run on Summit (as described by Oak Ridge):

Astrophysics

Exploding stars, known as supernovas, supply researchers with clues related to how heavy elements—including the gold in jewelry and iron in blood—seeded the universe.

The highly scalable FLASH code models this process at multiple scales—from the nuclear level to the large-scale hydrodynamics of a star’s final moments. On Summit, FLASH will go much further than previously possible, simulating supernova scenarios several thousand times longer and tracking about 12 times more elements than past projects.

“It’s at least a hundred times more computation than we’ve been able to do on earlier machines,” said ORNL computational astrophysicist Bronson Messer. “The sheer size of Summit will allow us to make very high-resolution models.”

Materials

Developing the next generation of materials, including compounds for energy storage, conversion and production, depends on subatomic understanding of material behavior. QMCPACK, a quantum Monte Carlo application, simulates these interactions using first-principles calculations.

Up to now, researchers have only been able to simulate tens of atoms because of QMCPACK’s high computational cost. Summit, however, can support materials composed of hundreds of atoms, a jump that aids the search for a more practical superconductor—a material that can transmit electricity with no energy loss.

“Summit’s large, on-node memory is very important for increasing the range of complexity in materials and physical phenomena,” said ORNL staff scientist Paul Kent. “Additionally, the much more powerful nodes are really going to help us extend the range of our simulations.”

Cancer Surveillance

One of the keys to combating cancer is developing tools that can automatically extract, analyze and sort existing health data to reveal previously hidden relationships between disease factors such as genes, biological markers and environment. Paired with unstructured data such as text-based reports and medical images, machine learning algorithms scaled on Summit will help supply medical researchers with a comprehensive view of the U.S. cancer population at a level of detail typically obtained only for clinical trial patients.

This cancer surveillance project is part of the CANcer Distributed Learning Environment, or CANDLE, a joint initiative between DOE and the National Cancer Institute.

“Essentially, we are training computers to read documents and abstract information using large volumes of data,” ORNL researcher Gina Tourassi said. “Summit enables us to explore much more complex models in a time efficient way so we can identify the ones that are most effective.”

Systems Biology

Applying machine learning and AI to genetic and biomedical datasets offers the potential toaccelerate understanding of human health and disease outcomes.

Using a mix of AI techniques on Summit, researchers will be able to identify patterns in the function, cooperation and evolution of human proteins and cellular systems. These patterns can collectively give rise to clinical phenotypes, observable traits of diseases such as Alzheimer’s, heart disease or addiction, and inform the drug discovery process.

Through a strategic partnership project between ORNL and the U.S. Department of Veterans Affairs, researchers are combining clinical and genomic data with machine learning and Summit’s advanced architecture to understand the genetic factors that contribute to conditions such as opioid addiction.

“The complexity of humans as a biological system is incredible,” said ORNL computational biologist Dan Jacobson. “Summit is enabling a whole new range of science that was simply not possible before it arrived.”

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

HPC in Life Sciences Part 1: CPU Choices, Rise of Data Lakes, Networking Challenges, and More

February 21, 2019

For the past few years HPCwire and leaders of BioTeam, a research computing consultancy specializing in life sciences, have convened to examine the state of HPC (and now AI) use in life sciences. Without HPC writ large, modern life sciences research would quickly grind to a halt. It’s true most life sciences research computing... Read more…

By John Russell

Arm Unveils Neoverse N1 Platform with up to 128-Cores

February 20, 2019

Following on its Neoverse roadmap announcement last October, Arm today revealed its next-gen Neoverse microarchitecture with compute and throughput-optimized silicon designs catered toward general-purpose cloud computing Read more…

By Tiffany Trader

The Internet of Criminal Things—Trust in the Gods but Verify!

February 20, 2019

“Are we under attack?” asked Professor Elmarie Biermann of the Cyber Security Institute during the recent South African Centre for High Performance Computing’s (CHPC) National Conference in Cape Town. A quick show Read more…

By Elizabeth Leake, STEM-Trek

HPE Extreme Performance Solutions

HPE and Intel® Omni-Path Architecture: How to Power a Cloud

Learn how HPE and Intel® Omni-Path Architecture provide critical infrastructure for leading Nordic HPC provider’s HPCFLOW cloud service.

powercloud_blog.jpgFor decades, HPE has been at the forefront of high-performance computing, and we’ve powered some of the fastest and most robust supercomputers in the world. Read more…

IBM Accelerated Insights

The Perils of Becoming Trapped in the Cloud

Terms like ‘open systems’ have been bandied about for decades. While modern computer systems are relatively open compared to their predecessors, there are still plenty of opportunities to become locked into proprietary interfaces. Read more…

Machine Learning Takes Heat for Science’s Reproducibility Crisis

February 19, 2019

Scientists are raising red flags about the accuracy and reproducibility of conclusions drawn by machine learning frameworks. Among the remedies are developing new ML systems that can question their own predictions, show Read more…

By George Leopold

HPC in Life Sciences Part 1: CPU Choices, Rise of Data Lakes, Networking Challenges, and More

February 21, 2019

For the past few years HPCwire and leaders of BioTeam, a research computing consultancy specializing in life sciences, have convened to examine the state of HPC (and now AI) use in life sciences. Without HPC writ large, modern life sciences research would quickly grind to a halt. It’s true most life sciences research computing... Read more…

By John Russell

Arm Unveils Neoverse N1 Platform with up to 128-Cores

February 20, 2019

Following on its Neoverse roadmap announcement last October, Arm today revealed its next-gen Neoverse microarchitecture with compute and throughput-optimized si Read more…

By Tiffany Trader

Insights from Optimized Codes on Cineca’s Marconi

February 15, 2019

What can you do with 381,392 CPU cores? For Cineca, it means enabling computational scientists to expand a large part of the world’s body of knowledge from the nanoscale to the astronomic, from calculating quantum effects in new materials to supporting bioinformatics for advanced healthcare research to screening millions of possible chemical combinations to attack a deadly virus. Read more…

By Ken Strandberg

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst o Read more…

By Tiffany Trader

UC Berkeley Paper Heralds Rise of Serverless Computing in the Cloud – Do You Agree?

February 13, 2019

Almost exactly ten years to the day from publishing of their widely-read, seminal paper on cloud computing, UC Berkeley researchers have issued another ambitious examination of cloud computing - Cloud Programming Simplified: A Berkeley View on Serverless Computing. The new work heralds the rise of ‘serverless computing’ as the next dominant phase of cloud computing. Read more…

By John Russell

Iowa ‘Grows Its Own’ to Fill the HPC Workforce Pipeline

February 13, 2019

The global workforce that supports advanced computing, scientific software and high-speed research networks is relatively small when you stop to consider the magnitude of the transformative discoveries it empowers. Technical conferences provide a forum where specialists convene to learn about the latest innovations and schedule face-time with colleagues from other institutions. Read more…

By Elizabeth Leake, STEM-Trek

Trump Signs Executive Order Launching U.S. AI Initiative

February 11, 2019

U.S. President Donald Trump issued an Executive Order (EO) today launching a U.S Artificial Intelligence Initiative. The new initiative - Maintaining American L Read more…

By John Russell

Celebrating Women in Science: Meet Four Women Leading the Way in HPC

February 11, 2019

One only needs to look around at virtually any CS/tech conference to realize that women are underrepresented, and that holds true of HPC. SC hosts over 13,000 H Read more…

By AJ Lauer

Quantum Computing Will Never Work

November 27, 2018

Amid the gush of money and enthusiastic predictions being thrown at quantum computing comes a proposed cold shower in the form of an essay by physicist Mikhail Read more…

By John Russell

Cray Unveils Shasta, Lands NERSC-9 Contract

October 30, 2018

Cray revealed today the details of its next-gen supercomputing architecture, Shasta, selected to be the next flagship system at NERSC. We've known of the code-name "Shasta" since the Argonne slice of the CORAL project was announced in 2015 and although the details of that plan have changed considerably, Cray didn't slow down its timeline for Shasta. Read more…

By Tiffany Trader

The Case Against ‘The Case Against Quantum Computing’

January 9, 2019

It’s not easy to be a physicist. Richard Feynman (basically the Jimi Hendrix of physicists) once said: “The first principle is that you must not fool yourse Read more…

By Ben Criger

AMD Sets Up for Epyc Epoch

November 16, 2018

It’s been a good two weeks, AMD’s Gary Silcott and Andy Parma told me on the last day of SC18 in Dallas at the restaurant where we met to discuss their show news and recent successes. Heck, it’s been a good year. Read more…

By Tiffany Trader

Intel Reportedly in $6B Bid for Mellanox

January 30, 2019

The latest rumors and reports around an acquisition of Mellanox focus on Intel, which has reportedly offered a $6 billion bid for the high performance interconn Read more…

By Doug Black

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst o Read more…

By Tiffany Trader

US Leads Supercomputing with #1, #2 Systems & Petascale Arm

November 12, 2018

The 31st Supercomputing Conference (SC) - commemorating 30 years since the first Supercomputing in 1988 - kicked off in Dallas yesterday, taking over the Kay Ba Read more…

By Tiffany Trader

Looking for Light Reading? NSF-backed ‘Comic Books’ Tackle Quantum Computing

January 28, 2019

Still baffled by quantum computing? How about turning to comic books (graphic novels for the well-read among you) for some clarity and a little humor on QC. The Read more…

By John Russell

Leading Solution Providers

SC 18 Virtual Booth Video Tour

Advania @ SC18 AMD @ SC18
ASRock Rack @ SC18
DDN Storage @ SC18
HPE @ SC18
IBM @ SC18
Lenovo @ SC18 Mellanox Technologies @ SC18
NVIDIA @ SC18
One Stop Systems @ SC18
Oracle @ SC18 Panasas @ SC18
Supermicro @ SC18 SUSE @ SC18 TYAN @ SC18
Verne Global @ SC18

Contract Signed for New Finnish Supercomputer

December 13, 2018

After the official contract signing yesterday, configuration details were made public for the new BullSequana system that the Finnish IT Center for Science (CSC Read more…

By Tiffany Trader

Deep500: ETH Researchers Introduce New Deep Learning Benchmark for HPC

February 5, 2019

ETH researchers have developed a new deep learning benchmarking environment – Deep500 – they say is “the first distributed and reproducible benchmarking s Read more…

By John Russell

IBM Quantum Update: Q System One Launch, New Collaborators, and QC Center Plans

January 10, 2019

IBM made three significant quantum computing announcements at CES this week. One was introduction of IBM Q System One; it’s really the integration of IBM’s Read more…

By John Russell

HPC Reflections and (Mostly Hopeful) Predictions

December 19, 2018

So much ‘spaghetti’ gets tossed on walls by the technology community (vendors and researchers) to see what sticks that it is often difficult to peer through Read more…

By John Russell

IBM Bets $2B Seeking 1000X AI Hardware Performance Boost

February 7, 2019

For now, AI systems are mostly machine learning-based and “narrow” – powerful as they are by today's standards, they're limited to performing a few, narro Read more…

By Doug Black

Nvidia’s Jensen Huang Delivers Vision for the New HPC

November 14, 2018

For nearly two hours on Monday at SC18, Jensen Huang, CEO of Nvidia, presented his expansive view of the future of HPC (and computing in general) as only he can do. Animated. Backstopped by a stream of data charts, product photos, and even a beautiful image of supernovae... Read more…

By John Russell

The Deep500 – Researchers Tackle an HPC Benchmark for Deep Learning

January 7, 2019

How do you know if an HPC system, particularly a larger-scale system, is well-suited for deep learning workloads? Today, that’s not an easy question to answer Read more…

By John Russell

Intel Confirms 48-Core Cascade Lake-AP for 2019

November 4, 2018

As part of the run-up to SC18, taking place in Dallas next week (Nov. 11-16), Intel is doling out info on its next-gen Cascade Lake family of Xeon processors, specifically the “Advanced Processor” version (Cascade Lake-AP), architected for high-performance computing, artificial intelligence and infrastructure-as-a-service workloads. Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This