HPC and the Spirit of St. Louis

By Thomas Sterling

June 20, 2012

Every year, as the International Supercomputing Conference in Germany approaches, our good friends here at HPCwire invite me to reflect on the trends of the past 12 months, not so much to provide a potentially tedious list of specific events, product deliveries, and TOP500 mantras but rather to convey a personal sense of what it all adds up to and possibly means for the future of HPC.

This year, to do so, is both easy and difficult. As a field, we are at an inflection point marked by significant progress and innovation in form and method, while at the same time we are confronted by uncertainty at a level that is at least uncomfortable for our system providers and possibly disruptive. There is certainly more contention about degree and direction of product and research investment, both within the US and internationally.

HPC has certainly entered a period of diversity far different than a decade ago, and that is not simply a function of the more than two orders of magnitude of Linpack performance in a time of Pax MPI. The easy part is to recite the buzz words of the year: “GPU”, “big data”, “clouds,” and “exascale.” If you are on one of these trains, then according to popular belief, you are on the fast track.

The mad dash to flops through the means of cramming as many ALUs as possible in the dense (and successful) form factor of a GPU, sometimes referred to as “accelerators,” has pushed passed the tipping point, with many, but not all major new installations incorporating these flops multipliers in their arsenal on the field of big iron.

Both NVIDIA and AMD are providing the punch in heterogeneity, although with vary different architectures. It’s actually interesting to watch NVIDIA on the wrong side of the PCI bus move ARM into their modules. AMD is moving their accelerator module into, or at least closer to, their multicore array. Choose your own benchmark (you actually should), but for some, the AMD strategy appears to be working, while the NVIDIA offering is clearly in the lead.

In the US, new big systems like Titan at Oak Ridge and Blue Waters at the University of Illinois are betting on this, and preparing to break through 10 petaflops based on Cray’s latest supercomputing offerings. Both China, with Tianhe-1A, and Japan, with TSUBAME 2, have taken a similar path, each with their own novel contributions.

But there are exceptions, even at the top end. Kei (K) in Kobe, the fastest machine at the time of this writing, has a more tightly integrated architecture provided by Fujitsu as it delivers about 10 petaflops and an array of IBM Blue Gene systems are banking on millions of lighter weight cores in a homogeneous system architecture to deliver an easier-to-program, and therefore more general class of computer, with lower power. (Note the GPUs are good on power as well.).

But programming remains a challenge, and if that is not hard enough, portability is even trickier, especially performance and scalability portability. There are on the order of 50,000-plus CUDA programmers but that does not mean the programming of large scalable systems incorporating GPUs is solved. OpenCL, a community-wide effort to provide an open programming methodology and one that addresses the problems somewhat more broadly, is in work and is attracting a growing body of users. OpenACC is an inchoate programming formalism with broader goals and an OpenMP-like touch and feel.

Many assert that we are looking at the system/programming family of the future. Others (and I’m among them) think it is a transitory phase, which will evolve into something as yet undefined. At least one heavy hitter, Intel, is betting on something all together different; their early MIC chip that defines a new manycore socket exhibiting homogeneity, reduced power, and generality. Clearly, the HPC community is not of a uniform opinion.

A very constructive movement that has gained momentum over the last year in the field of HPC is dubbed “big data.” In science and engineering more and more problems are challenged by the management, processing, and communication of potentially enormous amounts of associated data, whether observed by sensors or derived through simulation. The world’s largest telescopes, LIGO (Laser Interferometric Gravitational Observatory), and of course the LHC (Large Hadron Collider at CERN) are all examples of on-going experiments that generate constant streams of data that have to be dealt with. But biology and medical science also create an ever-growing body of data where cross correlations and data mining becomes an increasing challenge.

Storage capacity is only the beginning of the daunting problems confronting big data science. Communication bandwidth, latency, and reliability for data integrity, as well as power and cost are now and at an increasing pace continue to dominate big data science. Fortunately, unlike some other aspects of mostly flops-intense scientific computing, help will come from industry. This is because big data may generate big profits.

The needs of science in this realm are also manifest in the commercial space from large relational databases, through inventory and sales management, to social networks and search engines. These and other markets will drive technology advancement by the vendors that should have substantial impact on the science domain as well. But over-exuberance in our field is abundant and there are some well-intentioned practitioners in the big data arena who assert that this is THE problem in scientific computing. My message to them is: there are enough problems in HPC to go around.

Of course, according to some, the answer to the question of where to put all that data, or for that matter, where to process it (or any other kind of computing one might need to do) is obvious: it’s the cloud! Well maybe.

The value of clouds or “The Cloud” — I don’t know which — is real, permitting shared environments, data sources, services etc. among multiple people or communities and among the multiple platforms of a single individual. This is a rapidly moving capability and interface the full societal impact of which is probably unpredictable even to the most visionary among us but can be anticipating to be enormous and far reaching.

But for HPC, the utility of clouds in the future is, well, foggy. There are some sweet spots. Storage of data, larger than easily managed by a modest department, but smaller than some horrific size, is likely. The problem with ultra-large data sets is that they have to get moved. If they accrue slowly and are only lightly sampled, this can work. But if the entire data set has to be processed by local computing resources, then the intervening bandwidth provided by the internet simply may not be adequate.

On the computing side, there is an attraction to amortizing the cost and administration of a large array of computing resources across many users. Indeed, the accessibility of a system of very large scale that could not be acquired by any but a few institutions is a potential breakthrough in operational modality. But HPC reflects different forms of usage. The clouds can supply “throughput computing” and a significant percentage of the HPC workload is of this kind. Indeed, pools of resources including workstation farms across academic campuses and else where have been widely employed over decades.

But HPC has many computational challenges, single programs, that are tightly coupled and for which much of the programming challenge is performance tuning. Latencies have to be low, overheads even lower, and cost of information flow understood and stable. Clouds provide none of this in very large configurations. In some sense, this is their strength; successive requests are serviced by different configurations of available resources on demand. But for very large complex problems, they are not suitable, or at least less than optimal. Success of the cloud will require that we benefit from its advantages but not over-hype it and ultimately become disappointed.

People love milestones to mark progress and not just HPC people. In the last century two such captured the imagination of the world. One that I lived through was getting to the Moon with “one small step” provided Neil Armstrong in 1969. But another was a flight non-stop from New York to Paris by Charles Lindbergh in 1927 to claim the Raymond Orteig Prize. Today, the HPC community has self-defined our next milestone as exascale.

Over the last year, this objective has been codified by the US and internationally through meetings, plans, and programs. One international forum, the International Exascale Software Project (IESP), was completed after more than two years with its last of eight meetings in Kobe, Japan. The European Exascale Software Initiative (EESI) was also completed and is now succeeded by EESI-2. Plans are being considered in China, Japan, and Russia for their own path to exascale computing. In the US, the Department of Energy has launched at least three programs to develop a sufficient understanding and capability not just to get to exaflops, but to derive the right kind of exascale systems (hardware and software) and programming methodologies.

The Predictive Science Academic Alliance Program (PSAAP II) has just accepted proposals for exascale application development and system software. The co-design centers are also focused on the development of application algorithms and the systems upon which they are to run. The Modeling of Execution Model projects are exploring and quantifying the very principles upon which future exascale systems will be designed and operated. And the X-stack Program has just selected the teams that will develop next-generation system software and programming environments that will lead to exascale computing while providing nearer term utility as well.

But there is a difference between the milestones of 1927 and 1969 on the one hand, and that of the exascale, on the other. As extraordinary as Lindbergh’s historic accomplishment was, it was an end in itself. That cannot be the case for exascale computing. While our field has been guilty of stunt machines in the past, the cost and importance of achieving useful exascale capability, capacity, and application is too great to invest in merely claiming the first HPL exaflops Rmax run

And if some institution, agency, or nation does force such an artificial solution for a short-lived sense of glory, then surely the serious HPC community should mark this act with disdain. The future of HPC is the future of exascale but not merely such systems or benchmarks in and of themselves, but rather the scientific, medical, societal, and commercial breakthroughs that these systems will enable.

The Spirit of St Louis flew from New York to Paris. But it took a ship back to the US. It wouldn’t have made it if it had tried to do it in reverse. The head winds, which helped it fly east, would have impeded its progress west. The Spirit of Saint Louis now lives in the Smithsonian Air and Space Museum, the world’s most popular museum.

When viewing it from the 2nd floor, the discerning eye will notice a very peculiar thing; there is no front-looking window. Lindbergh could not see where he was going (although he did have a small periscope). From the side windows he could see where he was and guess what was coming next but he did not have the vision ahead.

HPC cannot afford to fly blind. We cannot just use our current position to assume we will make the right incremental progress towards our future destination. And we can’t just build an exascale computer to sit in a museum, even if it does run a benchmark. HPC is a tool for humanity to solve problems of importance when faced with so many critical challenges. No more stunt machines, please.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Arm Targets HPC with New Neoverse Platforms

September 22, 2020

UK-based semiconductor design company Arm today teased details of its Neoverse roadmap, introducing V1 (codenamed Zeus) and N2 (codenamed Perseus), Arm’s second generation N-series platform. The chip IP vendor said the Read more…

By Tiffany Trader

Microsoft’s Azure Quantum Platform Now Offers Toshiba’s ‘Simulated Bifurcation Machine’

September 22, 2020

While pure-play quantum computing (QC) gets most of the QC-related attention, there’s also been steady progress adapting quantum methods for select use on classical computers. Today, Microsoft announced that Toshiba’ Read more…

By John Russell

Oracle Cloud Deepens HPC Embrace with Launch of A100 Instances, Plans for Arm, More 

September 22, 2020

Oracle Cloud Infrastructure (OCI) continued its steady ramp-up of HPC capabilities today with a flurry of announcements. Topping the list is general availability of instances with Nvidia’s newest GPU, the A100. OCI als Read more…

By John Russell

IBM, CQC Enable Cloud-based Quantum Random Number Generation

September 21, 2020

IBM and Cambridge Quantum Computing (CQC) have partnered to achieve progress on one of the major business aspirations for quantum computing – the goal of generating verified, truly random numbers that can be used for a Read more…

By Todd R. Weiss

European Commission Declares €8 Billion Investment in Supercomputing

September 18, 2020

Just under two years ago, the European Commission formalized the EuroHPC Joint Undertaking (JU): a concerted HPC effort (comprising 32 participating states at current count) across the European Union and supplanting HPC Read more…

By Oliver Peckham

AWS Solution Channel

Next-generation aerospace modeling and simulation: benchmarking Amazon Web Services High Performance Computing services

The aerospace industry has been using Computational Fluid Dynamics (CFD) for decades to create and optimize designs digitally, from the largest passenger planes and fighter jets to gliders and drones. Read more…

Intel® HPC + AI Pavilion

Berlin Institute of Health: Putting HPC to Work for the World

Researchers from the Center for Digital Health at the Berlin Institute of Health (BIH) are using science to understand the pathophysiology of COVID-19, which can help to inform the development of targeted treatments. Read more…

Google Hires Longtime Intel Exec Bill Magro to Lead HPC Strategy

September 18, 2020

In a sign of the times, another prominent HPCer has made a move to a hyperscaler. Longtime Intel executive Bill Magro joined Google as chief technologist for high-performance computing, a newly created position that is a Read more…

By Tiffany Trader

Arm Targets HPC with New Neoverse Platforms

September 22, 2020

UK-based semiconductor design company Arm today teased details of its Neoverse roadmap, introducing V1 (codenamed Zeus) and N2 (codenamed Perseus), Arm’s seco Read more…

By Tiffany Trader

Oracle Cloud Deepens HPC Embrace with Launch of A100 Instances, Plans for Arm, More 

September 22, 2020

Oracle Cloud Infrastructure (OCI) continued its steady ramp-up of HPC capabilities today with a flurry of announcements. Topping the list is general availabilit Read more…

By John Russell

European Commission Declares €8 Billion Investment in Supercomputing

September 18, 2020

Just under two years ago, the European Commission formalized the EuroHPC Joint Undertaking (JU): a concerted HPC effort (comprising 32 participating states at c Read more…

By Oliver Peckham

Google Hires Longtime Intel Exec Bill Magro to Lead HPC Strategy

September 18, 2020

In a sign of the times, another prominent HPCer has made a move to a hyperscaler. Longtime Intel executive Bill Magro joined Google as chief technologist for hi Read more…

By Tiffany Trader

Future of Fintech on Display at HPC + AI Wall Street

September 17, 2020

Those who tuned in for Tuesday's HPC + AI Wall Street event got a peak at the future of fintech and lively discussion of topics like blockchain, AI for risk man Read more…

By Alex Woodie, Tiffany Trader and Todd R. Weiss

IBM’s Quantum Race to One Million Qubits

September 15, 2020

IBM today outlined its ambitious quantum computing technology roadmap at its virtual Quantum Summit. The eye-popping million qubit number is still far out, agrees IBM, but perhaps not that far out. Just as eye-popping is IBM’s nearer-term plan for a 1,000-plus qubit system named Condor... Read more…

By John Russell

Nvidia Commits to Buy Arm for $40B

September 14, 2020

Nvidia is acquiring semiconductor design company Arm Ltd. for $40 billion from SoftBank in a blockbuster deal that catapults the GPU chipmaker to a dominant position in the datacenter while helping troubled SoftBank reverse its financial woes. The deal, which has been rumored for... Read more…

By Todd R. Weiss and George Leopold

AMD’s Massive COVID-19 HPC Fund Adds 18 Institutions, 5 Petaflops of Power

September 14, 2020

Almost exactly five months ago, AMD announced its COVID-19 HPC Fund, an ongoing flow of resources and equipment to research institutions studying COVID-19 that began with an initial donation of $15 million. In June, AMD announced major equipment donations to several major institutions. Now, AMD is making its third major COVID-19 HPC Fund... Read more…

By Oliver Peckham

Supercomputer-Powered Research Uncovers Signs of ‘Bradykinin Storm’ That May Explain COVID-19 Symptoms

July 28, 2020

Doctors and medical researchers have struggled to pinpoint – let alone explain – the deluge of symptoms induced by COVID-19 infections in patients, and what Read more…

By Oliver Peckham

Nvidia Said to Be Close on Arm Deal

August 3, 2020

GPU leader Nvidia Corp. is in talks to buy U.K. chip designer Arm from parent company Softbank, according to several reports over the weekend. If consummated Read more…

By George Leopold

10nm, 7nm, 5nm…. Should the Chip Nanometer Metric Be Replaced?

June 1, 2020

The biggest cool factor in server chips is the nanometer. AMD beating Intel to a CPU built on a 7nm process node* – with 5nm and 3nm on the way – has been i Read more…

By Doug Black

Intel’s 7nm Slip Raises Questions About Ponte Vecchio GPU, Aurora Supercomputer

July 30, 2020

During its second-quarter earnings call, Intel announced a one-year delay of its 7nm process technology, which it says it will create an approximate six-month shift for its CPU product timing relative to prior expectations. The primary issue is a defect mode in the 7nm process that resulted in yield degradation... Read more…

By Tiffany Trader

HPE Keeps Cray Brand Promise, Reveals HPE Cray Supercomputing Line

August 4, 2020

The HPC community, ever-affectionate toward Cray and its eponymous founder, can breathe a (virtual) sigh of relief. The Cray brand will live on, encompassing th Read more…

By Tiffany Trader

Google Hires Longtime Intel Exec Bill Magro to Lead HPC Strategy

September 18, 2020

In a sign of the times, another prominent HPCer has made a move to a hyperscaler. Longtime Intel executive Bill Magro joined Google as chief technologist for hi Read more…

By Tiffany Trader

Neocortex Will Be First-of-Its-Kind 800,000-Core AI Supercomputer

June 9, 2020

Pittsburgh Supercomputing Center (PSC - a joint research organization of Carnegie Mellon University and the University of Pittsburgh) has won a $5 million award Read more…

By Tiffany Trader

Supercomputer Modeling Tests How COVID-19 Spreads in Grocery Stores

April 8, 2020

In the COVID-19 era, many people are treating simple activities like getting gas or groceries with caution as they try to heed social distancing mandates and protect their own health. Still, significant uncertainty surrounds the relative risk of different activities, and conflicting information is prevalent. A team of Finnish researchers set out to address some of these uncertainties by... Read more…

By Oliver Peckham

Leading Solution Providers

Contributors

Australian Researchers Break All-Time Internet Speed Record

May 26, 2020

If you’ve been stuck at home for the last few months, you’ve probably become more attuned to the quality (or lack thereof) of your internet connection. Even Read more…

By Oliver Peckham

Oracle Cloud Infrastructure Powers Fugaku’s Storage, Scores IO500 Win

August 28, 2020

In June, RIKEN shook the supercomputing world with its Arm-based, Fujitsu-built juggernaut: Fugaku. The system, which weighs in at 415.5 Linpack petaflops, topp Read more…

By Oliver Peckham

European Commission Declares €8 Billion Investment in Supercomputing

September 18, 2020

Just under two years ago, the European Commission formalized the EuroHPC Joint Undertaking (JU): a concerted HPC effort (comprising 32 participating states at c Read more…

By Oliver Peckham

Google Cloud Debuts 16-GPU Ampere A100 Instances

July 7, 2020

On the heels of the Nvidia’s Ampere A100 GPU launch in May, Google Cloud is announcing alpha availability of the A100 “Accelerator Optimized” VM A2 instance family on Google Compute Engine. The instances are powered by the HGX A100 16-GPU platform, which combines two HGX A100 8-GPU baseboards using... Read more…

By Tiffany Trader

DOD Orders Two AI-Focused Supercomputers from Liqid

August 24, 2020

The U.S. Department of Defense is making a big investment in data analytics and AI computing with the procurement of two HPC systems that will provide the High Read more…

By Tiffany Trader

Microsoft Azure Adds A100 GPU Instances for ‘Supercomputer-Class AI’ in the Cloud

August 19, 2020

Microsoft Azure continues to infuse its cloud platform with HPC- and AI-directed technologies. Today the cloud services purveyor announced a new virtual machine Read more…

By Tiffany Trader

Japan’s Fugaku Tops Global Supercomputing Rankings

June 22, 2020

A new Top500 champ was unveiled today. Supercomputer Fugaku, the pride of Japan and the namesake of Mount Fuji, vaulted to the top of the 55th edition of the To Read more…

By Tiffany Trader

Joliot-Curie Supercomputer Used to Build First Full, High-Fidelity Aircraft Engine Simulation

July 14, 2020

When industrial designers plan the design of a new element of a vehicle’s propulsion or exterior, they typically use fluid dynamics to optimize airflow and in Read more…

By Oliver Peckham

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This