HPC and the Spirit of St. Louis

By Thomas Sterling

June 20, 2012

Every year, as the International Supercomputing Conference in Germany approaches, our good friends here at HPCwire invite me to reflect on the trends of the past 12 months, not so much to provide a potentially tedious list of specific events, product deliveries, and TOP500 mantras but rather to convey a personal sense of what it all adds up to and possibly means for the future of HPC.

This year, to do so, is both easy and difficult. As a field, we are at an inflection point marked by significant progress and innovation in form and method, while at the same time we are confronted by uncertainty at a level that is at least uncomfortable for our system providers and possibly disruptive. There is certainly more contention about degree and direction of product and research investment, both within the US and internationally.

HPC has certainly entered a period of diversity far different than a decade ago, and that is not simply a function of the more than two orders of magnitude of Linpack performance in a time of Pax MPI. The easy part is to recite the buzz words of the year: “GPU”, “big data”, “clouds,” and “exascale.” If you are on one of these trains, then according to popular belief, you are on the fast track.

The mad dash to flops through the means of cramming as many ALUs as possible in the dense (and successful) form factor of a GPU, sometimes referred to as “accelerators,” has pushed passed the tipping point, with many, but not all major new installations incorporating these flops multipliers in their arsenal on the field of big iron.

Both NVIDIA and AMD are providing the punch in heterogeneity, although with vary different architectures. It’s actually interesting to watch NVIDIA on the wrong side of the PCI bus move ARM into their modules. AMD is moving their accelerator module into, or at least closer to, their multicore array. Choose your own benchmark (you actually should), but for some, the AMD strategy appears to be working, while the NVIDIA offering is clearly in the lead.

In the US, new big systems like Titan at Oak Ridge and Blue Waters at the University of Illinois are betting on this, and preparing to break through 10 petaflops based on Cray’s latest supercomputing offerings. Both China, with Tianhe-1A, and Japan, with TSUBAME 2, have taken a similar path, each with their own novel contributions.

But there are exceptions, even at the top end. Kei (K) in Kobe, the fastest machine at the time of this writing, has a more tightly integrated architecture provided by Fujitsu as it delivers about 10 petaflops and an array of IBM Blue Gene systems are banking on millions of lighter weight cores in a homogeneous system architecture to deliver an easier-to-program, and therefore more general class of computer, with lower power. (Note the GPUs are good on power as well.).

But programming remains a challenge, and if that is not hard enough, portability is even trickier, especially performance and scalability portability. There are on the order of 50,000-plus CUDA programmers but that does not mean the programming of large scalable systems incorporating GPUs is solved. OpenCL, a community-wide effort to provide an open programming methodology and one that addresses the problems somewhat more broadly, is in work and is attracting a growing body of users. OpenACC is an inchoate programming formalism with broader goals and an OpenMP-like touch and feel.

Many assert that we are looking at the system/programming family of the future. Others (and I’m among them) think it is a transitory phase, which will evolve into something as yet undefined. At least one heavy hitter, Intel, is betting on something all together different; their early MIC chip that defines a new manycore socket exhibiting homogeneity, reduced power, and generality. Clearly, the HPC community is not of a uniform opinion.

A very constructive movement that has gained momentum over the last year in the field of HPC is dubbed “big data.” In science and engineering more and more problems are challenged by the management, processing, and communication of potentially enormous amounts of associated data, whether observed by sensors or derived through simulation. The world’s largest telescopes, LIGO (Laser Interferometric Gravitational Observatory), and of course the LHC (Large Hadron Collider at CERN) are all examples of on-going experiments that generate constant streams of data that have to be dealt with. But biology and medical science also create an ever-growing body of data where cross correlations and data mining becomes an increasing challenge.

Storage capacity is only the beginning of the daunting problems confronting big data science. Communication bandwidth, latency, and reliability for data integrity, as well as power and cost are now and at an increasing pace continue to dominate big data science. Fortunately, unlike some other aspects of mostly flops-intense scientific computing, help will come from industry. This is because big data may generate big profits.

The needs of science in this realm are also manifest in the commercial space from large relational databases, through inventory and sales management, to social networks and search engines. These and other markets will drive technology advancement by the vendors that should have substantial impact on the science domain as well. But over-exuberance in our field is abundant and there are some well-intentioned practitioners in the big data arena who assert that this is THE problem in scientific computing. My message to them is: there are enough problems in HPC to go around.

Of course, according to some, the answer to the question of where to put all that data, or for that matter, where to process it (or any other kind of computing one might need to do) is obvious: it’s the cloud! Well maybe.

The value of clouds or “The Cloud” — I don’t know which — is real, permitting shared environments, data sources, services etc. among multiple people or communities and among the multiple platforms of a single individual. This is a rapidly moving capability and interface the full societal impact of which is probably unpredictable even to the most visionary among us but can be anticipating to be enormous and far reaching.

But for HPC, the utility of clouds in the future is, well, foggy. There are some sweet spots. Storage of data, larger than easily managed by a modest department, but smaller than some horrific size, is likely. The problem with ultra-large data sets is that they have to get moved. If they accrue slowly and are only lightly sampled, this can work. But if the entire data set has to be processed by local computing resources, then the intervening bandwidth provided by the internet simply may not be adequate.

On the computing side, there is an attraction to amortizing the cost and administration of a large array of computing resources across many users. Indeed, the accessibility of a system of very large scale that could not be acquired by any but a few institutions is a potential breakthrough in operational modality. But HPC reflects different forms of usage. The clouds can supply “throughput computing” and a significant percentage of the HPC workload is of this kind. Indeed, pools of resources including workstation farms across academic campuses and else where have been widely employed over decades.

But HPC has many computational challenges, single programs, that are tightly coupled and for which much of the programming challenge is performance tuning. Latencies have to be low, overheads even lower, and cost of information flow understood and stable. Clouds provide none of this in very large configurations. In some sense, this is their strength; successive requests are serviced by different configurations of available resources on demand. But for very large complex problems, they are not suitable, or at least less than optimal. Success of the cloud will require that we benefit from its advantages but not over-hype it and ultimately become disappointed.

People love milestones to mark progress and not just HPC people. In the last century two such captured the imagination of the world. One that I lived through was getting to the Moon with “one small step” provided Neil Armstrong in 1969. But another was a flight non-stop from New York to Paris by Charles Lindbergh in 1927 to claim the Raymond Orteig Prize. Today, the HPC community has self-defined our next milestone as exascale.

Over the last year, this objective has been codified by the US and internationally through meetings, plans, and programs. One international forum, the International Exascale Software Project (IESP), was completed after more than two years with its last of eight meetings in Kobe, Japan. The European Exascale Software Initiative (EESI) was also completed and is now succeeded by EESI-2. Plans are being considered in China, Japan, and Russia for their own path to exascale computing. In the US, the Department of Energy has launched at least three programs to develop a sufficient understanding and capability not just to get to exaflops, but to derive the right kind of exascale systems (hardware and software) and programming methodologies.

The Predictive Science Academic Alliance Program (PSAAP II) has just accepted proposals for exascale application development and system software. The co-design centers are also focused on the development of application algorithms and the systems upon which they are to run. The Modeling of Execution Model projects are exploring and quantifying the very principles upon which future exascale systems will be designed and operated. And the X-stack Program has just selected the teams that will develop next-generation system software and programming environments that will lead to exascale computing while providing nearer term utility as well.

But there is a difference between the milestones of 1927 and 1969 on the one hand, and that of the exascale, on the other. As extraordinary as Lindbergh’s historic accomplishment was, it was an end in itself. That cannot be the case for exascale computing. While our field has been guilty of stunt machines in the past, the cost and importance of achieving useful exascale capability, capacity, and application is too great to invest in merely claiming the first HPL exaflops Rmax run

And if some institution, agency, or nation does force such an artificial solution for a short-lived sense of glory, then surely the serious HPC community should mark this act with disdain. The future of HPC is the future of exascale but not merely such systems or benchmarks in and of themselves, but rather the scientific, medical, societal, and commercial breakthroughs that these systems will enable.

The Spirit of St Louis flew from New York to Paris. But it took a ship back to the US. It wouldn’t have made it if it had tried to do it in reverse. The head winds, which helped it fly east, would have impeded its progress west. The Spirit of Saint Louis now lives in the Smithsonian Air and Space Museum, the world’s most popular museum.

When viewing it from the 2nd floor, the discerning eye will notice a very peculiar thing; there is no front-looking window. Lindbergh could not see where he was going (although he did have a small periscope). From the side windows he could see where he was and guess what was coming next but he did not have the vision ahead.

HPC cannot afford to fly blind. We cannot just use our current position to assume we will make the right incremental progress towards our future destination. And we can’t just build an exascale computer to sit in a museum, even if it does run a benchmark. HPC is a tool for humanity to solve problems of importance when faced with so many critical challenges. No more stunt machines, please.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

UCSD, AIST Forge Tighter Alliance with AI-Focused MOU

January 18, 2018

The rich history of collaboration between UC San Diego and AIST in Japan is getting richer. The organizations entered into a five-year memorandum of understanding on January 10. The MOU represents the continuation of a 1 Read more…

By Tiffany Trader

New Blueprint for Converging HPC, Big Data

January 18, 2018

After five annual workshops on Big Data and Extreme-Scale Computing (BDEC), a group of international HPC heavyweights including Jack Dongarra (University of Tennessee), Satoshi Matsuoka (Tokyo Institute of Technology), Read more…

By John Russell

Researchers Measure Impact of ‘Meltdown’ and ‘Spectre’ Patches on HPC Workloads

January 17, 2018

Computer scientists from the Center for Computational Research, State University of New York (SUNY), University at Buffalo have examined the effect of Meltdown and Spectre security updates on the performance of popular H Read more…

By Tiffany Trader

HPE Extreme Performance Solutions

HPE and NREL Take Steps to Create a Sustainable, Energy-Efficient Data Center with an H2 Fuel Cell

As enterprises attempt to manage rising volumes of data, unplanned data center outages are becoming more common and more expensive. As the cost of downtime rises, enterprises lose out on productivity and valuable competitive advantage without access to their critical data. Read more…

Fostering Lustre Advancement Through Development and Contributions

January 17, 2018

Six months after organizational changes at Intel's High Performance Data (HPDD) division, most in the Lustre community have shed any initial apprehension around the potential changes that could affect or disrupt Lustre Read more…

By Carlos Aoki Thomaz

UCSD, AIST Forge Tighter Alliance with AI-Focused MOU

January 18, 2018

The rich history of collaboration between UC San Diego and AIST in Japan is getting richer. The organizations entered into a five-year memorandum of understandi Read more…

By Tiffany Trader

New Blueprint for Converging HPC, Big Data

January 18, 2018

After five annual workshops on Big Data and Extreme-Scale Computing (BDEC), a group of international HPC heavyweights including Jack Dongarra (University of Te Read more…

By John Russell

Researchers Measure Impact of ‘Meltdown’ and ‘Spectre’ Patches on HPC Workloads

January 17, 2018

Computer scientists from the Center for Computational Research, State University of New York (SUNY), University at Buffalo have examined the effect of Meltdown Read more…

By Tiffany Trader

Fostering Lustre Advancement Through Development and Contributions

January 17, 2018

Six months after organizational changes at Intel's High Performance Data (HPDD) division, most in the Lustre community have shed any initial apprehension aroun Read more…

By Carlos Aoki Thomaz

When the Chips Are Down

January 11, 2018

In the last article, "The High Stakes Semiconductor Game that Drives HPC Diversity," I alluded to the challenges facing the semiconductor industry and how that may impact the evolution of HPC systems over the next few years. I thought I’d lift the covers a little and look at some of the commercial challenges that impact the component technology we use in HPC. Read more…

By Dairsie Latimer

How Meltdown and Spectre Patches Will Affect HPC Workloads

January 10, 2018

There have been claims that the fixes for the Meltdown and Spectre security vulnerabilities, named the KPTI (aka KAISER) patches, are going to affect applicatio Read more…

By Rosemary Francis

Momentum Builds for US Exascale

January 9, 2018

2018 looks to be a great year for the U.S. exascale program. The last several months of 2017 revealed a number of important developments that help put the U.S. Read more…

By Alex R. Larzelere

ANL’s Rick Stevens on CANDLE, ARM, Quantum, and More

January 8, 2018

Late last year HPCwire caught up with Rick Stevens, associate laboratory director for computing, environment and life Sciences at Argonne National Laboratory, f Read more…

By John Russell

Inventor Claims to Have Solved Floating Point Error Problem

January 17, 2018

"The decades-old floating point error problem has been solved," proclaims a press release from inventor Alan Jorgensen. The computer scientist has filed for and Read more…

By Tiffany Trader

US Coalesces Plans for First Exascale Supercomputer: Aurora in 2021

September 27, 2017

At the Advanced Scientific Computing Advisory Committee (ASCAC) meeting, in Arlington, Va., yesterday (Sept. 26), it was revealed that the "Aurora" supercompute Read more…

By Tiffany Trader

Japan Unveils Quantum Neural Network

November 22, 2017

The U.S. and China are leading the race toward productive quantum computing, but it's early enough that ultimate leadership is still something of an open questi Read more…

By Tiffany Trader

AMD Showcases Growing Portfolio of EPYC and Radeon-based Systems at SC17

November 13, 2017

AMD’s charge back into HPC and the datacenter is on full display at SC17. Having launched the EPYC processor line in June along with its MI25 GPU the focus he Read more…

By John Russell

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

IBM Begins Power9 Rollout with Backing from DOE, Google

December 6, 2017

After over a year of buildup, IBM is unveiling its first Power9 system based on the same architecture as the Department of Energy CORAL supercomputers, Summit a Read more…

By Tiffany Trader

Fast Forward: Five HPC Predictions for 2018

December 21, 2017

What’s on your list of high (and low) lights for 2017? Volta 100’s arrival on the heels of the P100? Appearance, albeit late in the year, of IBM’s Power9? Read more…

By John Russell

GlobalFoundries Puts Wind in AMD’s Sails with 12nm FinFET

September 24, 2017

From its annual tech conference last week (Sept. 20), where GlobalFoundries welcomed more than 600 semiconductor professionals (reaching the Santa Clara venue Read more…

By Tiffany Trader

Leading Solution Providers

Chip Flaws ‘Meltdown’ and ‘Spectre’ Loom Large

January 4, 2018

The HPC and wider tech community have been abuzz this week over the discovery of critical design flaws that impact virtually all contemporary microprocessors. T Read more…

By Tiffany Trader

Perspective: What Really Happened at SC17?

November 22, 2017

SC is over. Now comes the myriad of follow-ups. Inboxes are filled with templated emails from vendors and other exhibitors hoping to win a place in the post-SC thinking of booth visitors. Attendees of tutorials, workshops and other technical sessions will be inundated with requests for feedback. Read more…

By Andrew Jones

Tensors Come of Age: Why the AI Revolution Will Help HPC

November 13, 2017

Thirty years ago, parallel computing was coming of age. A bitter battle began between stalwart vector computing supporters and advocates of various approaches to parallel computing. IBM skeptic Alan Karp, reacting to announcements of nCUBE’s 1024-microprocessor system and Thinking Machines’ 65,536-element array, made a public $100 wager that no one could get a parallel speedup of over 200 on real HPC workloads. Read more…

By John Gustafson & Lenore Mullin

Delays, Smoke, Records & Markets – A Candid Conversation with Cray CEO Peter Ungaro

October 5, 2017

Earlier this month, Tom Tabor, publisher of HPCwire and I had a very personal conversation with Cray CEO Peter Ungaro. Cray has been on something of a Cinderell Read more…

By Tiffany Trader & Tom Tabor

Flipping the Flops and Reading the Top500 Tea Leaves

November 13, 2017

The 50th edition of the Top500 list, the biannual publication of the world’s fastest supercomputers based on public Linpack benchmarking results, was released Read more…

By Tiffany Trader

GlobalFoundries, Ayar Labs Team Up to Commercialize Optical I/O

December 4, 2017

GlobalFoundries (GF) and Ayar Labs, a startup focused on using light, instead of electricity, to transfer data between chips, today announced they've entered in Read more…

By Tiffany Trader

How Meltdown and Spectre Patches Will Affect HPC Workloads

January 10, 2018

There have been claims that the fixes for the Meltdown and Spectre security vulnerabilities, named the KPTI (aka KAISER) patches, are going to affect applicatio Read more…

By Rosemary Francis

HPC Chips – A Veritable Smorgasbord?

October 10, 2017

For the first time since AMD's ill-fated launch of Bulldozer the answer to the question, 'Which CPU will be in my next HPC system?' doesn't have to be 'Whichever variety of Intel Xeon E5 they are selling when we procure'. Read more…

By Dairsie Latimer

  • arrow
  • Click Here for More Headlines
  • arrow
Share This