HPC and the Spirit of St. Louis

By Thomas Sterling

June 20, 2012

Every year, as the International Supercomputing Conference in Germany approaches, our good friends here at HPCwire invite me to reflect on the trends of the past 12 months, not so much to provide a potentially tedious list of specific events, product deliveries, and TOP500 mantras but rather to convey a personal sense of what it all adds up to and possibly means for the future of HPC.

This year, to do so, is both easy and difficult. As a field, we are at an inflection point marked by significant progress and innovation in form and method, while at the same time we are confronted by uncertainty at a level that is at least uncomfortable for our system providers and possibly disruptive. There is certainly more contention about degree and direction of product and research investment, both within the US and internationally.

HPC has certainly entered a period of diversity far different than a decade ago, and that is not simply a function of the more than two orders of magnitude of Linpack performance in a time of Pax MPI. The easy part is to recite the buzz words of the year: “GPU”, “big data”, “clouds,” and “exascale.” If you are on one of these trains, then according to popular belief, you are on the fast track.

The mad dash to flops through the means of cramming as many ALUs as possible in the dense (and successful) form factor of a GPU, sometimes referred to as “accelerators,” has pushed passed the tipping point, with many, but not all major new installations incorporating these flops multipliers in their arsenal on the field of big iron.

Both NVIDIA and AMD are providing the punch in heterogeneity, although with vary different architectures. It’s actually interesting to watch NVIDIA on the wrong side of the PCI bus move ARM into their modules. AMD is moving their accelerator module into, or at least closer to, their multicore array. Choose your own benchmark (you actually should), but for some, the AMD strategy appears to be working, while the NVIDIA offering is clearly in the lead.

In the US, new big systems like Titan at Oak Ridge and Blue Waters at the University of Illinois are betting on this, and preparing to break through 10 petaflops based on Cray’s latest supercomputing offerings. Both China, with Tianhe-1A, and Japan, with TSUBAME 2, have taken a similar path, each with their own novel contributions.

But there are exceptions, even at the top end. Kei (K) in Kobe, the fastest machine at the time of this writing, has a more tightly integrated architecture provided by Fujitsu as it delivers about 10 petaflops and an array of IBM Blue Gene systems are banking on millions of lighter weight cores in a homogeneous system architecture to deliver an easier-to-program, and therefore more general class of computer, with lower power. (Note the GPUs are good on power as well.).

But programming remains a challenge, and if that is not hard enough, portability is even trickier, especially performance and scalability portability. There are on the order of 50,000-plus CUDA programmers but that does not mean the programming of large scalable systems incorporating GPUs is solved. OpenCL, a community-wide effort to provide an open programming methodology and one that addresses the problems somewhat more broadly, is in work and is attracting a growing body of users. OpenACC is an inchoate programming formalism with broader goals and an OpenMP-like touch and feel.

Many assert that we are looking at the system/programming family of the future. Others (and I’m among them) think it is a transitory phase, which will evolve into something as yet undefined. At least one heavy hitter, Intel, is betting on something all together different; their early MIC chip that defines a new manycore socket exhibiting homogeneity, reduced power, and generality. Clearly, the HPC community is not of a uniform opinion.

A very constructive movement that has gained momentum over the last year in the field of HPC is dubbed “big data.” In science and engineering more and more problems are challenged by the management, processing, and communication of potentially enormous amounts of associated data, whether observed by sensors or derived through simulation. The world’s largest telescopes, LIGO (Laser Interferometric Gravitational Observatory), and of course the LHC (Large Hadron Collider at CERN) are all examples of on-going experiments that generate constant streams of data that have to be dealt with. But biology and medical science also create an ever-growing body of data where cross correlations and data mining becomes an increasing challenge.

Storage capacity is only the beginning of the daunting problems confronting big data science. Communication bandwidth, latency, and reliability for data integrity, as well as power and cost are now and at an increasing pace continue to dominate big data science. Fortunately, unlike some other aspects of mostly flops-intense scientific computing, help will come from industry. This is because big data may generate big profits.

The needs of science in this realm are also manifest in the commercial space from large relational databases, through inventory and sales management, to social networks and search engines. These and other markets will drive technology advancement by the vendors that should have substantial impact on the science domain as well. But over-exuberance in our field is abundant and there are some well-intentioned practitioners in the big data arena who assert that this is THE problem in scientific computing. My message to them is: there are enough problems in HPC to go around.

Of course, according to some, the answer to the question of where to put all that data, or for that matter, where to process it (or any other kind of computing one might need to do) is obvious: it’s the cloud! Well maybe.

The value of clouds or “The Cloud” — I don’t know which — is real, permitting shared environments, data sources, services etc. among multiple people or communities and among the multiple platforms of a single individual. This is a rapidly moving capability and interface the full societal impact of which is probably unpredictable even to the most visionary among us but can be anticipating to be enormous and far reaching.

But for HPC, the utility of clouds in the future is, well, foggy. There are some sweet spots. Storage of data, larger than easily managed by a modest department, but smaller than some horrific size, is likely. The problem with ultra-large data sets is that they have to get moved. If they accrue slowly and are only lightly sampled, this can work. But if the entire data set has to be processed by local computing resources, then the intervening bandwidth provided by the internet simply may not be adequate.

On the computing side, there is an attraction to amortizing the cost and administration of a large array of computing resources across many users. Indeed, the accessibility of a system of very large scale that could not be acquired by any but a few institutions is a potential breakthrough in operational modality. But HPC reflects different forms of usage. The clouds can supply “throughput computing” and a significant percentage of the HPC workload is of this kind. Indeed, pools of resources including workstation farms across academic campuses and else where have been widely employed over decades.

But HPC has many computational challenges, single programs, that are tightly coupled and for which much of the programming challenge is performance tuning. Latencies have to be low, overheads even lower, and cost of information flow understood and stable. Clouds provide none of this in very large configurations. In some sense, this is their strength; successive requests are serviced by different configurations of available resources on demand. But for very large complex problems, they are not suitable, or at least less than optimal. Success of the cloud will require that we benefit from its advantages but not over-hype it and ultimately become disappointed.

People love milestones to mark progress and not just HPC people. In the last century two such captured the imagination of the world. One that I lived through was getting to the Moon with “one small step” provided Neil Armstrong in 1969. But another was a flight non-stop from New York to Paris by Charles Lindbergh in 1927 to claim the Raymond Orteig Prize. Today, the HPC community has self-defined our next milestone as exascale.

Over the last year, this objective has been codified by the US and internationally through meetings, plans, and programs. One international forum, the International Exascale Software Project (IESP), was completed after more than two years with its last of eight meetings in Kobe, Japan. The European Exascale Software Initiative (EESI) was also completed and is now succeeded by EESI-2. Plans are being considered in China, Japan, and Russia for their own path to exascale computing. In the US, the Department of Energy has launched at least three programs to develop a sufficient understanding and capability not just to get to exaflops, but to derive the right kind of exascale systems (hardware and software) and programming methodologies.

The Predictive Science Academic Alliance Program (PSAAP II) has just accepted proposals for exascale application development and system software. The co-design centers are also focused on the development of application algorithms and the systems upon which they are to run. The Modeling of Execution Model projects are exploring and quantifying the very principles upon which future exascale systems will be designed and operated. And the X-stack Program has just selected the teams that will develop next-generation system software and programming environments that will lead to exascale computing while providing nearer term utility as well.

But there is a difference between the milestones of 1927 and 1969 on the one hand, and that of the exascale, on the other. As extraordinary as Lindbergh’s historic accomplishment was, it was an end in itself. That cannot be the case for exascale computing. While our field has been guilty of stunt machines in the past, the cost and importance of achieving useful exascale capability, capacity, and application is too great to invest in merely claiming the first HPL exaflops Rmax run

And if some institution, agency, or nation does force such an artificial solution for a short-lived sense of glory, then surely the serious HPC community should mark this act with disdain. The future of HPC is the future of exascale but not merely such systems or benchmarks in and of themselves, but rather the scientific, medical, societal, and commercial breakthroughs that these systems will enable.

The Spirit of St Louis flew from New York to Paris. But it took a ship back to the US. It wouldn’t have made it if it had tried to do it in reverse. The head winds, which helped it fly east, would have impeded its progress west. The Spirit of Saint Louis now lives in the Smithsonian Air and Space Museum, the world’s most popular museum.

When viewing it from the 2nd floor, the discerning eye will notice a very peculiar thing; there is no front-looking window. Lindbergh could not see where he was going (although he did have a small periscope). From the side windows he could see where he was and guess what was coming next but he did not have the vision ahead.

HPC cannot afford to fly blind. We cannot just use our current position to assume we will make the right incremental progress towards our future destination. And we can’t just build an exascale computer to sit in a museum, even if it does run a benchmark. HPC is a tool for humanity to solve problems of importance when faced with so many critical challenges. No more stunt machines, please.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Geospatial Data Research Leverages GPUs

August 17, 2017

MapD Technologies, the GPU-accelerated database specialist, said it is working with university researchers on leveraging graphics processors to advance geospatial analytics. The San Francisco-based company is collabor Read more…

By George Leopold

Intel, NERSC and University Partners Launch New Big Data Center

August 17, 2017

A collaboration between the Department of Energy’s National Energy Research Scientific Computing Center (NERSC), Intel and five Intel Parallel Computing Centers (IPCCs) has resulted in a new Big Data Center (BDC) that Read more…

By Linda Barney

Google Releases Deeplearn.js to Further Democratize Machine Learning

August 17, 2017

Spreading the use of machine learning tools is one of the goals of Google’s PAIR (People + AI Research) initiative, which was introduced in early July. Last week the cloud giant released deeplearn.js as part of that in Read more…

By John Russell

HPE Extreme Performance Solutions

Leveraging Deep Learning for Fraud Detection

Advancements in computing technologies and the expanding use of e-commerce platforms have dramatically increased the risk of fraud for financial services companies and their customers. Read more…

Spoiler Alert: Glimpse Next Week’s Solar Eclipse Via Simulation from TACC, SDSC, and NASA

August 17, 2017

Can’t wait to see next week’s solar eclipse? You can at least catch glimpses of what scientists expect it will look like. A team from Predictive Science Inc. (PSI), based in San Diego, working with Stampede2 at the Read more…

By John Russell

Microsoft Bolsters Azure With Cloud HPC Deal

August 15, 2017

Microsoft has acquired cloud computing software vendor Cycle Computing in a move designed to bring orchestration tools along with high-end computing access capabilities to the cloud. Terms of the acquisition were not disclosed. Read more…

By George Leopold

HPE Ships Supercomputer to Space Station, Final Destination Mars

August 14, 2017

With a manned mission to Mars on the horizon, the demand for space-based supercomputing is at hand. Today HPE and NASA sent the first off-the-shelf HPC system i Read more…

By Tiffany Trader

AMD EPYC Video Takes Aim at Intel’s Broadwell

August 14, 2017

Let the benchmarking begin. Last week, AMD posted a YouTube video in which one of its EPYC-based systems outperformed a ‘comparable’ Intel Broadwell-based s Read more…

By John Russell

Deep Learning Thrives in Cancer Moonshot

August 8, 2017

The U.S. War on Cancer, certainly a worthy cause, is a collection of programs stretching back more than 40 years and abiding under many banners. The latest is t Read more…

By John Russell

IBM Raises the Bar for Distributed Deep Learning

August 8, 2017

IBM is announcing today an enhancement to its PowerAI software platform aimed at facilitating the practical scaling of AI models on today’s fastest GPUs. Scal Read more…

By Tiffany Trader

IBM Storage Breakthrough Paves Way for 330TB Tape Cartridges

August 3, 2017

IBM announced yesterday a new record for magnetic tape storage that it says will keep tape storage density on a Moore's law-like path far into the next decade. Read more…

By Tiffany Trader

AMD Stuffs a Petaflops of Machine Intelligence into 20-Node Rack

August 1, 2017

With its Radeon “Vega” Instinct datacenter GPUs and EPYC “Naples” server chips entering the market this summer, AMD has positioned itself for a two-head Read more…

By Tiffany Trader

Cray Moves to Acquire the Seagate ClusterStor Line

July 28, 2017

This week Cray announced that it is picking up Seagate's ClusterStor HPC storage array business for an undisclosed sum. "In short we're effectively transitioning the bulk of the ClusterStor product line to Cray," said CEO Peter Ungaro. Read more…

By Tiffany Trader

How ‘Knights Mill’ Gets Its Deep Learning Flops

June 22, 2017

Intel, the subject of much speculation regarding the delayed, rewritten or potentially canceled “Aurora” contract (the Argonne Lab part of the CORAL “ Read more…

By Tiffany Trader

Nvidia’s Mammoth Volta GPU Aims High for AI, HPC

May 10, 2017

At Nvidia's GPU Technology Conference (GTC17) in San Jose, Calif., this morning, CEO Jensen Huang announced the company's much-anticipated Volta architecture a Read more…

By Tiffany Trader

Reinders: “AVX-512 May Be a Hidden Gem” in Intel Xeon Scalable Processors

June 29, 2017

Imagine if we could use vector processing on something other than just floating point problems.  Today, GPUs and CPUs work tirelessly to accelerate algorithms Read more…

By James Reinders

Quantum Bits: D-Wave and VW; Google Quantum Lab; IBM Expands Access

March 21, 2017

For a technology that’s usually characterized as far off and in a distant galaxy, quantum computing has been steadily picking up steam. Just how close real-wo Read more…

By John Russell

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Russian Researchers Claim First Quantum-Safe Blockchain

May 25, 2017

The Russian Quantum Center today announced it has overcome the threat of quantum cryptography by creating the first quantum-safe blockchain, securing cryptocurrencies like Bitcoin, along with classified government communications and other sensitive digital transfers. Read more…

By Doug Black

HPC Compiler Company PathScale Seeks Life Raft

March 23, 2017

HPCwire has learned that HPC compiler company PathScale has fallen on difficult times and is asking the community for help or actively seeking a buyer for its a Read more…

By Tiffany Trader

Trump Budget Targets NIH, DOE, and EPA; No Mention of NSF

March 16, 2017

President Trump’s proposed U.S. fiscal 2018 budget issued today sharply cuts science spending while bolstering military spending as he promised during the cam Read more…

By John Russell

Leading Solution Providers

Groq This: New AI Chips to Give GPUs a Run for Deep Learning Money

April 24, 2017

CPUs and GPUs, move over. Thanks to recent revelations surrounding Google’s new Tensor Processing Unit (TPU), the computing world appears to be on the cusp of Read more…

By Alex Woodie

CPU-based Visualization Positions for Exascale Supercomputing

March 16, 2017

In this contributed perspective piece, Intel’s Jim Jeffers makes the case that CPU-based visualization is now widely adopted and as such is no longer a contrarian view, but is rather an exascale requirement. Read more…

By Jim Jeffers, Principal Engineer and Engineering Leader, Intel

Google Debuts TPU v2 and will Add to Google Cloud

May 25, 2017

Not long after stirring attention in the deep learning/AI community by revealing the details of its Tensor Processing Unit (TPU), Google last week announced the Read more…

By John Russell

MIT Mathematician Spins Up 220,000-Core Google Compute Cluster

April 21, 2017

On Thursday, Google announced that MIT math professor and computational number theorist Andrew V. Sutherland had set a record for the largest Google Compute Engine (GCE) job. Sutherland ran the massive mathematics workload on 220,000 GCE cores using preemptible virtual machine instances. Read more…

By Tiffany Trader

Six Exascale PathForward Vendors Selected; DoE Providing $258M

June 15, 2017

The much-anticipated PathForward awards for hardware R&D in support of the Exascale Computing Project were announced today with six vendors selected – AMD Read more…

By John Russell

Top500 Results: Latest List Trends and What’s in Store

June 19, 2017

Greetings from Frankfurt and the 2017 International Supercomputing Conference where the latest Top500 list has just been revealed. Although there were no major Read more…

By Tiffany Trader

IBM Clears Path to 5nm with Silicon Nanosheets

June 5, 2017

Two years since announcing the industry’s first 7nm node test chip, IBM and its research alliance partners GlobalFoundries and Samsung have developed a proces Read more…

By Tiffany Trader

Messina Update: The US Path to Exascale in 16 Slides

April 26, 2017

Paul Messina, director of the U.S. Exascale Computing Project, provided a wide-ranging review of ECP’s evolving plans last week at the HPC User Forum. Read more…

By John Russell

  • arrow
  • Click Here for More Headlines
  • arrow
Share This