Thomas Sterling’s ISC Keynote Celebrates Frontier and the Exascale Tour de Force

By John Russell

June 1, 2022

For a change, said Thomas Sterling, long-time ISC keynoter and HPC pioneer, picking a theme for his 2022 talk wasn’t a challenge.

Thomas Sterling, Indiana University

“What is the word you need that everyone will remember and agree to about a particular year? This was easy. It’s exaflops. I mean real exaflops, you know, the kind you can get your teeth into exaflops, not words like exascale or low-precision exaflops, or ‘we know what an exaflops is, right.’ No, Rmax [double precision performance],” said Sterling.

Hard to argue with that. Frontier’s groundbreaking 1.102 exaflops Linpack run to capture the latest Top500 crown has officially started the exascale era. Sterling was delighted to echo the news. This was his 19th ISC keynote, a feat in its own right. Sterling is professor of intelligent systems engineering and director of the AI Computing Systems Laboratory at Indiana University. He is perhaps best known as the “father of Beowulf” cluster architecture.

There was, as usual, a great deal more in Sterling’s talk (HPC Achievement and Impact 2022) – surging HPC in Europe, HPC’s identity crisis, worries-and-triumphs from the pandemic, and a brief spotlight on the Event Horizon Telescope’s computationally intense effort to image the black hole at the center of the Milky Way – but exascale computing took center stage. Presented here are a few of Sterling slides and comments (lightly edited). ISC has archived the video and will provide access to ISC attendees through the end of the year.

Frontier and the Exascale Tour de Force

Frontier’s performance, said Sterling, was a tour de force not only for itself but also for the wider array of efforts by DOE and the broad HPC computing community to reach exascale computing.

“Frontier is not just a machine. Frontier is the result of a process. That process has been taking place at Oak Ridge National Lab under the broader [guidance] of the Department of Energy, but also [with] other major agencies and goals of the U.S. And it is, in fact, [the result of] Titan, and Summit, and the dedication and commitment by many to making heterogeneous computing work for application programmers. That has to do with applications and algorithms. It also has to do with system software and tools. And yes, it has to do with architecture, and technology too. This is the culmination of more than a decade of focused attention to detail and maturation of an approach to HPC. They have produced this exciting result, which will do nothing but move the field forward,” said Sterling.

See HPCwire’s deep coverage of the Frontier system and the Top500.

Sterling cited the seminal role of the High Performance Computing & Communications (HPCC) program, established through the High-Performance Computing Act of 1991: “The HPCC program thirty years ago, which was [focused on] making really practical and effective HPC. So this was not just machines. This was also applications and systems software and tools.” He also praised the work of Exascale Computing Project (ECP) which focused on software and worked hand-in-glove with the hardware efforts to ensure there was a viable software ecosystem for new exascale systems, of which Frontier is first. 

What is HPC (and What is AI)?

While much of Sterling’s talk was celebratory, his darker comments on HPC’s so-called identity crisis were at least as interesting. HPC’s identity confusion has been a growing topic. The decline of Moore’s law, the rise of AI technologies and their infusion into HPC, and the emergence of heterogeneous systems have left many wondering what it means to be doing high performance computing. Sterling saved these comments for the end and didn’t offer an answer.

“What is HPC? I leave that as a question. I don’t attempt to answer it. But I will convey the sense that [if] HPC is a leading-edge technology with the focus on computing and we asked the question: what is performance? That may seem like a stupid question, right? High performance, what’s to talk about?

“Is it throughput? Or is it time-to-solution?”

Scanning the crowd, he said, “Those of you who look like we could have met 20 years ago here. Some of you would agree that it’s time-to-solution. Now I value the concept and the growth that has occurred with capacity computing. But this is a very serious problem because if it can’t include time-to-solution, then we’ve already put up a barrier that we will never pass,” said Sterling.

“The other question,” he said, “is what is AI? I was very afraid to do this slide and the fact is I didn’t want to tick anybody off (slide below). Here’s the thing. First of all, artificial intelligence is a terrible term. Yes, it might have implied intelligence from an artifact. The term was coined in 1958 or ’59. But it could also mean artificial as in not real and, honestly, the way we’re using it is more the latter. Let me simply say that not all of AI is machine learning, or deep learning, and not all of machine learning or deep learning is machine intelligence. I ask you to think about that.

“I think a right term would be machine intelligence and I’ll tell you what I think it needs to do — and this isn’t the Turing test. I think Turing was joking when he did it. It’s testing a human being against a computer and you wonder why would a computer: a) go along with that? and b) behave as stupid as humans are? So, meaning no disrespect to the guy (Alan Turing) on the [50£] bill in the UK, machine intelligence is machine understanding. Call that the Sterling test. If the machine understands – and that’s more than Winograd blocks rule; great piece of work in 1972 – then I think we’ve achieved machine intelligence,” said Sterling.

Not sure how much that clarifies things but it gives pause for thought.

Resurgent European HPC

Throughout ISC22, the robust resurgence of HPC in Europe was a central theme. Sterling noted progress under the European High Performance Computing Joint Undertaking (EuroHPC JU) program. He also noted the longer-term growing accessibility of HPC resources in Europe and showed a slide from an earlier Top500 presentation.

“It isn’t the splashy slide, but it’s the important one. Internationally, the field of HPC has converged in the available performance, whether it’s in peak or it’s spread across many deployed systems,” said Sterling

Sterling briefly talked about Europe’s growing efforts in quantum computing, particularly those focused integrating quantum computing with classical resources. He also said he thought quantum computers would not become general-purpose systems but would be used as accelerators for select problem types. Talking about the European Processor Initiative (EPI) to develop a microprocessor that not only emphasizes performance and utility, but also energy efficiency, he said, “I’ll simply say this is a major undertaking.”

Fugaku Today (and Yesterday); Zettascale tomorrow?

While Frontier is the new king of HPC, Sterling heaped praise on Fugaku, now second on the Top500. He lauded Fugaku’s race to deployment, impressive performance, and role as part of the global mobilization of HPC resources to fight the Covid pandemic.

“The science is amazing that we (the world) were able to build vaccinations that have a very strong positive effect. This brings me back to Fugaku because Fugaku, was more than [just] a delivered machine. When Covid-19 hit world, the people at Riken (and MEXT) decided they had to step up and accelerate the deployment of Fugaku by many months, in my opinion, close to a year. Even more amazing. They killed the red tape. Now, this probably saved time when administration was removed and collaborations were built between scientists who would design the new drugs and understand the mechanisms [of] the disease and at the same time [with] the computational scientist who knew how to use the machines, but didn’t know that microbiology. Fugaku became the centerpiece within Japan of driving this momentum forward. It was an extraordinary accomplishment.

“I also want to mention that the Japanese are not resting on their laurels, they’re already talking about their next machine to be delivered in 2029. Satoshi Matsuoka [speaking at ISC 2022] was not prepared to commit to a zettaflops. I’m not sure he was actually joking, but they are already working towards higher density, what they want to have the equal and significant impact,” said Sterling.

Reaching for the Sky with Event Horizon Telescope

A signature element of Sterling’s keynotes is tribute to a scientific achievement made possible by high performance computing. This year he talked about work by the Event Horizon Telescope [EHT] to image and analyze the black hole – Sagittarius A (estimated at four million solar masses) – at the heart of the Milky Way. It took 10 exaflops of compute over two days to resolve the image.

Showing a photo of Sagittarius A, he contrasted it with M87 – a photo of which he’d presented in a past keynote. M87 is one of the largest black holes so far identified. It is 1000 times larger than Sagittarius A and ~55 million light years away from Earth. “[The EHT is] a virtual computer but also a virtual antenna. This radio telescope comprises a half a dozen or so radio telescopes around the world. One of its virtues is one is able to point this virtual antenna at one object and watch it continuously as our Earth turns,” said Sterling.

“Believe it or not, it’s easier to look at a supermassive, distant black hole than it is to look at the center of the Milky Way, which is a mere 26,000 light years away. Why is this? The answer is because we get to look down on the top of those other spiral galaxies, while we’re looking directly into the dust clouds of our own. This is much harder. It’s also much smaller. The lower picture [in the slide] kind of gives you the impression of how tiny our own black hole is at 4 million solar masses. [Capturing the image and analyzing it] was only made possible by the use of high performance computing. And by the way, they couldn’t even use the internet to move the data.”

As always, Sterling packed a lot into his talk. He noted that AMD chips power many of the new systems. He mentioned the trend of HPC systems having longer lifespans. AI’s appetite for compute cycles would dominate HPC in the future, he suggested. Like many he cheered Cray’s strength after being purchased by HPE. He lauded the Advanced Graphic Intelligence Logical Environment program (AGILE) – hated the name, loved the mission: “The AGILE program aims to develop revolutionizing computer architectures and associated integrated circuit designs for a new class of high-performance, high efficiency, scalable computers that meet the needs of large-scale data analytic problems.”

We’ll see what next year brings.

Sterling Bio
Thomas Sterling is a Full Professor of Intelligent Systems Engineering at Indiana University (IU) serving as Director of the AI Computing Systems Laboratory at IU’s Luddy School of Informatics, Computing, and Engineering. Since receiving his Ph.D. from MIT as a Hertz Fellow in 1984, Dr. Sterling has engaged in applied research in parallel computing system structures, semantics, and operation in industry, government labs, and academia. Professional affiliations have included Harris Corp., IDA Supercomputing Research Center, NASA (GSFC, JPL), Un. of Maryland, Caltech, and LSU. Dr. Sterling is best known as the “father of Beowulf” for his pioneering research in commodity/Linux cluster computing for which he shared the Gordon Bell Prize in 1997. His current research is associated with innovative extreme scale computing through memory-centric non von Neumann architecture concepts to accelerate dynamic graph processing for AI including ML. In 2018, he co-founded the new tech company, Simultac LLC, and serves as its President and Chief Scientist. Dr. Sterling was the recipient of the 2013 Vanguard Award and is a Fellow of the AAAS. He has been selected this year to be inducted in the Space Technologies Hall of Fame. He is the co-author of seven books and holds six patents. Most recently, he co-authored the introductory textbook, “High Performance Computing”, published by Morgan-Kaufmann in 2018 which is going into 2nd edition.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Is Time Running Out for Compromise on America COMPETES/USICA Act?

June 22, 2022

You may recall that efforts proposed in 2020 to remake the National Science Foundation (Endless Frontier Act) have since expanded and morphed into two gigantic bills, the America COMPETES Act in the U.S. House of Representatives and the U.S. Innovation and Competition Act in the U.S. Senate. So far, efforts to reconcile the two pieces of legislation have snagged and recent reports... Read more…

Cerebras Systems Thinks Forward on AI Chips as it Claims Performance Win

June 22, 2022

Cerebras Systems makes the largest chip in the world, but is already thinking about its upcoming AI chips as learning models continue to grow at breakneck speed. The company’s latest Wafer Scale Engine chip is indeed the size of a wafer, and is made using TSMC’s 7nm process. The next chip will pack in more cores to handle the fast-growing compute needs of AI, said Andrew Feldman, CEO of Cerebras Systems. Read more…

AMD’s MI300 APUs to Power Exascale El Capitan Supercomputer

June 21, 2022

Additional details of the architecture of the exascale El Capitan supercomputer were disclosed today by Lawrence Livermore National Laboratory’s (LLNL) Terri Quinn in a presentation delivered to the 79th HPC User Forum Read more…

IDC Perspective on Integration of Quantum Computing and HPC

June 20, 2022

The insatiable need to compress time to insights from massive and complex datasets is fueling the demand for quantum computing integration into high performance computing (HPC) environments. Such an integration would allow enterprises to accelerate and optimize current HPC applications and processes by simulating and emulating them on today’s noisy... Read more…

Q&A with Intel’s Jeff McVeigh, an HPCwire Person to Watch in 2022

June 17, 2022

HPCwire presents our interview with Jeff McVeigh, vice president and general manager, Super Compute Group, Intel Corporation, and an HPCwire 2022 Person to Watch. McVeigh shares Intel's plans for the year ahead, his pers Read more…

AWS Solution Channel

Shutterstock 152995403

Bayesian ML Models at Scale with AWS Batch

This post was contributed by Ampersand’s Jeffrey Enos, Senior Machine Learning Engineer, Daniel Gerlanc, Senior Director for Data Science, and Brandon Willard, Data Science Lead. Read more…

Microsoft/NVIDIA Solution Channel

Shutterstock 261863138

Using Cloud-Based, GPU-Accelerated AI for Financial Risk Management

There are strict rules governing financial institutions with a number of global regulatory groups publishing financial compliance requirements. Financial institutions face many challenges and legal responsibilities for risk management, compliance violations, and failure to catch financial fraud. Read more…

Nvidia, Intel to Power Atos-Built MareNostrum 5 Supercomputer

June 16, 2022

The long-troubled, hotly anticipated MareNostrum 5 supercomputer finally has a vendor: Atos, which will be supplying a system that includes both Nvidia and Intel CPUs and GPUs across multiple partitions. The newly reimag Read more…

Is Time Running Out for Compromise on America COMPETES/USICA Act?

June 22, 2022

You may recall that efforts proposed in 2020 to remake the National Science Foundation (Endless Frontier Act) have since expanded and morphed into two gigantic bills, the America COMPETES Act in the U.S. House of Representatives and the U.S. Innovation and Competition Act in the U.S. Senate. So far, efforts to reconcile the two pieces of legislation have snagged and recent reports... Read more…

Cerebras Systems Thinks Forward on AI Chips as it Claims Performance Win

June 22, 2022

Cerebras Systems makes the largest chip in the world, but is already thinking about its upcoming AI chips as learning models continue to grow at breakneck speed. The company’s latest Wafer Scale Engine chip is indeed the size of a wafer, and is made using TSMC’s 7nm process. The next chip will pack in more cores to handle the fast-growing compute needs of AI, said Andrew Feldman, CEO of Cerebras Systems. Read more…

AMD’s MI300 APUs to Power Exascale El Capitan Supercomputer

June 21, 2022

Additional details of the architecture of the exascale El Capitan supercomputer were disclosed today by Lawrence Livermore National Laboratory’s (LLNL) Terri Read more…

IDC Perspective on Integration of Quantum Computing and HPC

June 20, 2022

The insatiable need to compress time to insights from massive and complex datasets is fueling the demand for quantum computing integration into high performance computing (HPC) environments. Such an integration would allow enterprises to accelerate and optimize current HPC applications and processes by simulating and emulating them on today’s noisy... Read more…

Q&A with Intel’s Jeff McVeigh, an HPCwire Person to Watch in 2022

June 17, 2022

HPCwire presents our interview with Jeff McVeigh, vice president and general manager, Super Compute Group, Intel Corporation, and an HPCwire 2022 Person to Watc Read more…

Nvidia, Intel to Power Atos-Built MareNostrum 5 Supercomputer

June 16, 2022

The long-troubled, hotly anticipated MareNostrum 5 supercomputer finally has a vendor: Atos, which will be supplying a system that includes both Nvidia and Inte Read more…

D-Wave Debuts Advantage2 Prototype; Seeks User Exploration and Feedback

June 16, 2022

Starting today, D-Wave Systems is providing access to a 500-plus-qubit prototype of its forthcoming 7000-qubit Advantage2 quantum annealing computer, which is d Read more…

AMD Opens Up Chip Design to the Outside for Custom Future

June 15, 2022

AMD is getting personal with chips as it sets sail to make products more to the liking of its customers. The chipmaker detailed a modular chip future in which customers can mix and match non-AMD processors in a custom chip package. "We are focused on making it easier to implement chips with more flexibility," said Mark Papermaster, chief technology officer at AMD during the analyst day meeting late last week. Read more…

Nvidia R&D Chief on How AI is Improving Chip Design

April 18, 2022

Getting a glimpse into Nvidia’s R&D has become a regular feature of the spring GTC conference with Bill Dally, chief scientist and senior vice president of research, providing an overview of Nvidia’s R&D organization and a few details on current priorities. This year, Dally focused mostly on AI tools that Nvidia is both developing and using in-house to improve... Read more…

Royalty-free stock illustration ID: 1919750255

Intel Says UCIe to Outpace PCIe in Speed Race

May 11, 2022

Intel has shared more details on a new interconnect that is the foundation of the company’s long-term plan for x86, Arm and RISC-V architectures to co-exist in a single chip package. The semiconductor company is taking a modular approach to chip design with the option for customers to cram computing blocks such as CPUs, GPUs and AI accelerators inside a single chip package. Read more…

The Final Frontier: US Has Its First Exascale Supercomputer

May 30, 2022

In April 2018, the U.S. Department of Energy announced plans to procure a trio of exascale supercomputers at a total cost of up to $1.8 billion dollars. Over the ensuing four years, many announcements were made, many deadlines were missed, and a pandemic threw the world into disarray. Now, at long last, HPE and Oak Ridge National Laboratory (ORNL) have announced that the first of those... Read more…

AMD/Xilinx Takes Aim at Nvidia with Improved VCK5000 Inferencing Card

March 8, 2022

AMD/Xilinx has released an improved version of its VCK5000 AI inferencing card along with a series of competitive benchmarks aimed directly at Nvidia’s GPU line. AMD says the new VCK5000 has 3x better performance than earlier versions and delivers 2x TCO over Nvidia T4. AMD also showed favorable benchmarks against several Nvidia GPUs, claiming its VCK5000 achieved... Read more…

Top500: Exascale Is Officially Here with Debut of Frontier

May 30, 2022

The 59th installment of the Top500 list, issued today from ISC 2022 in Hamburg, Germany, officially marks a new era in supercomputing with the debut of the first-ever exascale system on the list. Frontier, deployed at the Department of Energy’s Oak Ridge National Laboratory, achieved 1.102 exaflops in its fastest High Performance Linpack run, which was completed... Read more…

Newly-Observed Higgs Mode Holds Promise in Quantum Computing

June 8, 2022

The first-ever appearance of a previously undetectable quantum excitation known as the axial Higgs mode – exciting in its own right – also holds promise for developing and manipulating higher temperature quantum materials... Read more…

Nvidia Launches Hopper H100 GPU, New DGXs and Grace Superchips

March 22, 2022

The battle for datacenter dominance keeps getting hotter. Today, Nvidia kicked off its spring GTC event with new silicon, new software and a new supercomputer. Speaking from a virtual environment in the Nvidia Omniverse 3D collaboration and simulation platform, CEO Jensen Huang introduced the new Hopper GPU architecture and the H100 GPU... Read more…

PsiQuantum’s Path to 1 Million Qubits

April 21, 2022

PsiQuantum, founded in 2016 by four researchers with roots at Bristol University, Stanford University, and York University, is one of a few quantum computing startups that’s kept a moderately low PR profile. (That’s if you disregard the roughly $700 million in funding it has attracted.) The main reason is PsiQuantum has eschewed the clamorous public chase for... Read more…

Leading Solution Providers

Contributors

ISC 2022 Booth Video Tours

AMD
AWS
DDN
Dell
Intel
Lenovo
Microsoft
PENGUIN SOLUTIONS

AMD Opens Up Chip Design to the Outside for Custom Future

June 15, 2022

AMD is getting personal with chips as it sets sail to make products more to the liking of its customers. The chipmaker detailed a modular chip future in which customers can mix and match non-AMD processors in a custom chip package. "We are focused on making it easier to implement chips with more flexibility," said Mark Papermaster, chief technology officer at AMD during the analyst day meeting late last week. Read more…

Intel Reiterates Plans to Merge CPU, GPU High-performance Chip Roadmaps

May 31, 2022

Intel reiterated it is well on its way to merging its roadmap of high-performance CPUs and GPUs as it shifts over to newer manufacturing processes and packaging technologies in the coming years. The company is merging the CPU and GPU lineups into a chip (codenamed Falcon Shores) which Intel has dubbed an XPU. Falcon Shores... Read more…

India Launches Petascale ‘PARAM Ganga’ Supercomputer

March 8, 2022

Just a couple of weeks ago, the Indian government promised that it had five HPC systems in the final stages of installation and would launch nine new supercomputers this year. Now, it appears to be making good on that promise: the country’s National Supercomputing Mission (NSM) has announced the deployment of “PARAM Ganga” petascale supercomputer at Indian Institute of Technology (IIT)... Read more…

Nvidia Dominates MLPerf Inference, Qualcomm also Shines, Where’s Everybody Else?

April 6, 2022

MLCommons today released its latest MLPerf inferencing results, with another strong showing by Nvidia accelerators inside a diverse array of systems. Roughly fo Read more…

AMD’s MI300 APUs to Power Exascale El Capitan Supercomputer

June 21, 2022

Additional details of the architecture of the exascale El Capitan supercomputer were disclosed today by Lawrence Livermore National Laboratory’s (LLNL) Terri Read more…

Nvidia, Intel to Power Atos-Built MareNostrum 5 Supercomputer

June 16, 2022

The long-troubled, hotly anticipated MareNostrum 5 supercomputer finally has a vendor: Atos, which will be supplying a system that includes both Nvidia and Inte Read more…

Industry Consortium Forms to Drive UCIe Chiplet Interconnect Standard

March 2, 2022

A new industry consortium aims to establish a die-to-die interconnect standard – Universal Chiplet Interconnect Express (UCIe) – in support of an open chipl Read more…

Covid Policies at HPC Conferences Should Reflect HPC Research

June 6, 2022

Supercomputing has been indispensable throughout the Covid-19 pandemic, from modeling the virus and its spread to designing vaccines and therapeutics. But, desp Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire