Jailbreaking HPC

By Gary Johnson

March 5, 2012

Is high performance computing in jail? Is number crunching a prisoner of policies and practices that are decades old?

The policies and practices that define the way we currently deploy and use our number-crunching HPC resources are based on assumptions that are, at best, several decades old. A lot has happened in the meantime, for example, the emergence of a sizeable and growing cadre of computational scientists and engineers, the internet and the web, continuous weakening of the coupling between work and location, the rise of data-intensive computing, commercial computing providers with staying power, and a broad consensus that the future health of our economy depends on intelligent and extensive exploitation of HPC. Perhaps it’s time to revisit those premises.

The major premises were that the engines of HPC are unique, its people are scarce, and the resources need to be held close in protected locations, its “temples.”

The engines of HPC are unique

In the early days of HPC, our “big iron” computers were crafted by hand and in small numbers. Those computers were in a class by themselves. We put them on display behind glass walls in machine rooms when they were new and, when they’d passed their prime. We retired them as museum pieces or lobby furniture. Have you ever sat on a Cray-1?

Since the late 90s, our HPC engines have been increasingly built from commodity parts and in scalable fashion. Indeed, those commodity parts are now reaching a degree of sophistication at the chip level that we previously only found at the machine level. HPC is now based on mass-produced hardware.

These computing systems have also become more modular – some to the extent of being assembled in Lego fashion from shipping-container-sized building blocks. Increasing numbers of computing floors are being operated in “lights out” mode, supported by very small numbers of on-site personnel.

The only real discriminators among HPC engines are: system balance (see: Number Crunching, Data Crunching and Energy Efficiency: the HPC Hat Trick); the size of the system one can afford; the floor space available to accommodate it; and the power bill one is prepared to pay.

The people of HPC are scarce

In those early days of HPC, there were no “computational scientists”. The term computational science was introduced by Ken Wilson around 1984. OK, maybe some of us were computational scientists but didn’t go by that name. The point is that computational science and engineering were just emerging as career fields and the number of practitioners was quite low.

Today, although there is still a supply gap, there are reasonably large numbers of computational scientists and engineers. They work in our universities, national labs and in industry. There are programs to educate them, journals to publish their work, focused meetings and conferences for them to attend, differentiated social networks for them to participate in, and human resources people who specialize in recruiting them.

HPC centers are temples

Back then, the scarcity of HPC users was complemented by a scarcity of people who were skilled at building and maintaining these specialized systems. That’s because it involved gluing various subsystems of exotic hardware and software together into functioning computers, tuning their physical environment, keeping them healthy and available, and making them accessible by remote users.

Additionally, teams of people were needed to offer training courses in the various aspects of this new thing, then called “supercomputing” and now called HPC. User support groups provided help in porting and tuning existing applications codes and expertise in writing new ones. Visualization specialists helped the end users create meaningful displays of their results, mostly by post-processing results on specialized visualization subsystems.

Given the exotic nature of supercomputing, the scarcity of the skilled people involved, and the complexity and interdependency of the tasks they performed, it made good sense to centralize HPC activities around a small number of centers. Today, there are vastly more HPC centers supporting various number-crunching applications in science and engineering.

Additionally, our definition of HPC has been enhanced to include the burgeoning field of data-intensive science. A whole new class of very large datacenters has come into existence to support the rapid growth and broad deployment of data-crunching applications in commerce, finance, marketing, social network analysis, and other fields. Furthermore, the locus of HPC activity seems to be shifting from the traditional protected locations (i.e., the temples) out into the enterprise. In concert with these developments, the operation of our computing centers has become more standardized and a larger pool of skilled people is available to support them.

Progress has overrun our premises

Depending on how you count it, we are three or four decades into this adventure we now call high performance computing. HPC has been very successful and its successes have altered the landscape of computing and its applications.

Our highest end computers are no longer unique but rather, by our own design decisions, scaled up versions of smaller and more common machines. Computational scientists and engineers are now routinely educated and employed, even if not in the numbers we might wish. HPC centers have lost a good bit of their original mystique as they have proliferated across the landscape.

Additional elements have also come into play. Ubiquitous access to the internet and world-wide web have made computing accessible to a large segment of the Earth’s population, not just those lucky few connected to some dedicated network. So, our HPC specialties can be practiced from just about anywhere. Commercial providers of computing services have reappeared, after a long hiatus , and have come back in much more resilient form (for example, Amazon, Cisco, Dell, Google, HP, IBM, Microsoft, and SGI).  It seems fair to say that commercially available computing services are here to stay.

On any given day, HPC in the Cloud is replete with news about these services. Indeed, we are told that some companies have gone so far as to compute exclusively in commercial clouds. Also, the Federal CIO has recently issued a memorandum encouraging federal agencies to develop trusted relationships with cloud service providers.

As for HPC applications, in the early days of supercomputing they were all about number crunching and we touted them as means for pushing back the frontiers of science and contributing seed corn for the future growth of science and engineering. There were no externally imposed milestones or deadlines associated with using computing in this mode for scientific discovery. Now, HPC is seen in much more immediate terms, as a problem-solving tool for economic success. Thus we desire a much swifter and more predictable transition from research into practice.

There has also been tremendous growth in data crunching applications, increasingly referred to as “big data.” Distinct from number crunching, these applications operate substantially outside the sphere of federal support. This other aspect of HPC is rapidly evolving and creating its own varieties of computing and data storage capabilities (for example, Facebook’s Open Compute Project and Open Storage Pod). Even the grande dame of supercomputing, Cray, has recently formed a Big Data Division. To follow the data-intensive computing and big data action, take a look at Datanami.

So, given the things that have changed and the new elements that have appeared, how might we adjust the premises on which we base our HPC policies and practices? Here are a few thoughts.

It’s about scalability

The elite machines in our computing ecosystem will always be, by definition, unique. We should take their care and feeding seriously. But the apex has now been pushed so high that computing seems really to be about “whateverscale.”

While are rightfully advocating for the development of exascale computers, most of our end-user applications compute at the terascale level, at best. They are thus “behind the computing curve” by a factor of 1,000,000, or soon will be. So, perhaps we should encourage innovation and growth of computing applications at whatever scale is appropriate to their immediate needs.

To the extent that we focus on moving “whateverscale” applications forward, their eventual transition to “apexscale” should be easier and come sooner. Furthermore, apexscale machines will be scaled-up versions of the smaller, whateverscale ones. Thus, the code development and architecture exploitation lessons learned at whateverscale will carry over to those top systems.

It’s about empowering people

In the early days of supercomputing, there were very few computational scientists , that is, people who were well versed in both their science field and in plying their science on our highest-end computing hardware. So, for allocating public resources, we created a dual review system. A traditional merit peer review was conducted to inform the process of allocating money and a separate “computational readiness” review was conducted to ensure that any compute cycles allocated would be used wisely.

Over the past several decades, we have been successful in producing lots of computational scientists and engineers – people who are very knowledgeable about both aspects of their profession. However, this dual review process is still in widespread use, especially by federal agencies. Thus, there is an opportunity to remove this form of “double jeopardy” by reverting to a single, comprehensive, review of computational science and engineering proposals. Doing this would also acknowledge our current generation of computational scientists and engineers for the skills they have worked hard to acquire. Emancipating and empowering computational scientists to better control their own fate can only serve to strengthen and grow the field.

It’s about the economy

When supercomputing was in its infancy, it was important to hold it close, to protect and nurture it. Consequently, our HPC resources were concentrated at federally funded research and development centers. Also, although there was a desire for these resources to have an economic impact, the kind of compulsion to do so that we see today was largely absent.

To be sure, there were a few companies that were early adopters of number-crunching HPC and continue to use it to great benefit. They generally employ their own internal computing resources. You can find some of them listed in the Top500 list, while others see competitive advantage in remaining invisible. What was missing then was an open, generally available, and well-supported commercial HPC infrastructure that could be used by much broader segments of the economy.

A few decades ago, there was an attempt to stand up commercial computer service bureaus and make supercomputing generally available. This attempt was less than successful, serving to reinforce the notion that custody of resources should be left to the government, its contractors, and perhaps a few other stalwarts.

Today, the computing and networking environment is vastly improved and our economic incentives to compute are intense. Data sciences and analytics have led the way in clearly establishing that commercial HPC is a viable enterprise. In many dimensions, this commercial form of HPC is now larger than our traditional government-sponsored form.

This presents all of us, including the federal HPC establishment, with the opportunity to embrace commercial HPC and to strengthen and grow it into a fabric to support the innovation in products and services that we’ve identified as necessary for the future health of our economy.

To be clear, embracing and using commercial HPC resources would not serve as a replacement for the vitally important work done by our government HPC complex. Rather, it would free this complex to pursue its essential role in HPC: pathfinding our computing future and making it real.

If HPC resources and services that are currently provided by the government complex can be provided commercially, then we should avail ourselves of this opportunity to build out our national computing infrastructure and actively support economic growth. If aspects of the current commercial offerings fall short of our number- and data-crunching needs, then we should aggressively transfer the knowledge and technologies necessary to remedy those shortcomings. Perhaps most importantly, we should use these commercial resources to the maximal extent feasible, to help grow and strengthen the marketplace and its offerings.

Rethinking things…

The premises on which we have built our HPC enterprise have clearly changed over the past several decades. While those premises served us well in getting HPC established, they may now be limiting its further development. Perhaps it’s time to jailbreak HPC.


About the author

Gary M. Johnson is the founder of Computational Science Solutions, LLC, whose mission is to develop, advocate, and implement solutions for the global computational science and engineering community.

Dr. Johnson specializes in management of high performance computing, applied mathematics, and computational science research activities; advocacy, development, and management of high performance computing centers; development of national science and technology policy; and creation of education and research programs in computational engineering and science.

He has worked in Academia, Industry and Government. He has held full professorships at Colorado State University and George Mason University, been a researcher at United Technologies Research Center, and worked for the Department of Defense, NASA, and the Department of Energy.

He is a graduate of the U.S. Air Force Academy; holds advanced degrees from Caltech and the von Karman Institute; and has a Ph.D. in applied sciences from the University of Brussels.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

HPE Launches Apollo 6500 Gen10 System as Part of AI Solution Push

March 21, 2018

HPE today announced the latest rev of its HPE Apollo 6500 platform, Gen10, along with a spate of new AI-oriented offerings designed to help customers optimize and scale up their AI and deep learning usage. Like is Gen Read more…

By Tiffany Trader

IBM Touts OpenPOWER Ecosystem, Announces New Customers, Products for AI and Hyperscale

March 20, 2018

At SC17 in Denver four months ago, Ken King, GM, OpenPOWER, IBM Systems Group, told a somewhat jaundiced trio of journalists that 2018 would, finally, after several years of expectations, be the year OpenPOWER and IBM’ Read more…

By Doug Black

Deep Learning at 15 PFlops Enables Training for Extreme Weather Identification at Scale

March 19, 2018

Petaflop per second deep learning training performance on the NERSC (National Energy Research Scientific Computing Center) Cori supercomputer has given climate scientists the ability to use machine learning to identify e Read more…

By Rob Farber

HPE Extreme Performance Solutions

Harness the Full Power of HPC Servers with an Effective Cooling Approach

High performance computing (HPC) innovation is rapidly transforming the way we operate – with an onslaught of cutting-edge technologies designed to optimize applications and workloads, increase productivity, and enable better business outcomes. Read more…

IBM Unveils New Cloud for Data Science and Engineering

March 19, 2018

Days ahead of its inaugural IBM Think mega-event, the multinational tech mainstay on Friday (March 16) unveiled a new cloud offering called Cloud Private Data that’s designed to help organizations utilize data science Read more…

By Alex Woodie

HPE Launches Apollo 6500 Gen10 System as Part of AI Solution Push

March 21, 2018

HPE today announced the latest rev of its HPE Apollo 6500 platform, Gen10, along with a spate of new AI-oriented offerings designed to help customers optimize a Read more…

By Tiffany Trader

IBM Touts OpenPOWER Ecosystem, Announces New Customers, Products for AI and Hyperscale

March 20, 2018

At SC17 in Denver four months ago, Ken King, GM, OpenPOWER, IBM Systems Group, told a somewhat jaundiced trio of journalists that 2018 would, finally, after sev Read more…

By Doug Black

Deep Learning at 15 PFlops Enables Training for Extreme Weather Identification at Scale

March 19, 2018

Petaflop per second deep learning training performance on the NERSC (National Energy Research Scientific Computing Center) Cori supercomputer has given climate Read more…

By Rob Farber

How the Cloud Is Falling Short for HPC

March 15, 2018

The last couple of years have seen cloud computing gradually build some legitimacy within the HPC world, but still the HPC industry lies far behind enterprise I Read more…

By Chris Downing

Stephen Hawking, Legendary Scientist, Dies at 76

March 14, 2018

Stephen Hawking passed away at his home in Cambridge, England, in the early morning of March 14; he was 76. Born on January 8, 1942, Hawking was an English theo Read more…

By Tiffany Trader

Hyperion Tackles Elusive Quantum Computing Landscape

March 13, 2018

Quantum computing - exciting and off-putting all at once - is a kaleidoscope of technology and market questions whose shapes and positions are far from settled. Read more…

By John Russell

Part Two: Navigating Life Sciences Choppy HPC Waters in 2018

March 8, 2018

2017 was not necessarily the best year to build a large HPC system for life sciences say Ari Berman, VP and GM of consulting services, and Aaron Gardner, direct Read more…

By John Russell

Google Chases Quantum Supremacy with 72-Qubit Processor

March 7, 2018

Google pulled ahead of the pack this week in the race toward "quantum supremacy," with the introduction of a new 72-qubit quantum processor called Bristlecone. Read more…

By Tiffany Trader

Inventor Claims to Have Solved Floating Point Error Problem

January 17, 2018

"The decades-old floating point error problem has been solved," proclaims a press release from inventor Alan Jorgensen. The computer scientist has filed for and Read more…

By Tiffany Trader

Japan Unveils Quantum Neural Network

November 22, 2017

The U.S. and China are leading the race toward productive quantum computing, but it's early enough that ultimate leadership is still something of an open questi Read more…

By Tiffany Trader

Researchers Measure Impact of ‘Meltdown’ and ‘Spectre’ Patches on HPC Workloads

January 17, 2018

Computer scientists from the Center for Computational Research, State University of New York (SUNY), University at Buffalo have examined the effect of Meltdown Read more…

By Tiffany Trader

IBM Begins Power9 Rollout with Backing from DOE, Google

December 6, 2017

After over a year of buildup, IBM is unveiling its first Power9 system based on the same architecture as the Department of Energy CORAL supercomputers, Summit a Read more…

By Tiffany Trader

Fast Forward: Five HPC Predictions for 2018

December 21, 2017

What’s on your list of high (and low) lights for 2017? Volta 100’s arrival on the heels of the P100? Appearance, albeit late in the year, of IBM’s Power9? Read more…

By John Russell

Russian Nuclear Engineers Caught Cryptomining on Lab Supercomputer

February 12, 2018

Nuclear scientists working at the All-Russian Research Institute of Experimental Physics (RFNC-VNIIEF) have been arrested for using lab supercomputing resources to mine crypto-currency, according to a report in Russia’s Interfax News Agency. Read more…

By Tiffany Trader

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Chip Flaws ‘Meltdown’ and ‘Spectre’ Loom Large

January 4, 2018

The HPC and wider tech community have been abuzz this week over the discovery of critical design flaws that impact virtually all contemporary microprocessors. T Read more…

By Tiffany Trader

Leading Solution Providers

GlobalFoundries, Ayar Labs Team Up to Commercialize Optical I/O

December 4, 2017

GlobalFoundries (GF) and Ayar Labs, a startup focused on using light, instead of electricity, to transfer data between chips, today announced they've entered in Read more…

By Tiffany Trader

How Meltdown and Spectre Patches Will Affect HPC Workloads

January 10, 2018

There have been claims that the fixes for the Meltdown and Spectre security vulnerabilities, named the KPTI (aka KAISER) patches, are going to affect applicatio Read more…

By Rosemary Francis

Perspective: What Really Happened at SC17?

November 22, 2017

SC is over. Now comes the myriad of follow-ups. Inboxes are filled with templated emails from vendors and other exhibitors hoping to win a place in the post-SC thinking of booth visitors. Attendees of tutorials, workshops and other technical sessions will be inundated with requests for feedback. Read more…

By Andrew Jones

V100 Good but not Great on Select Deep Learning Aps, Says Xcelerit

November 27, 2017

Wringing optimum performance from hardware to accelerate deep learning applications is a challenge that often depends on the specific application in use. A benc Read more…

By John Russell

Lenovo Unveils Warm Water Cooled ThinkSystem SD650 in Rampup to LRZ Install

February 22, 2018

This week Lenovo took the wraps off the ThinkSystem SD650 high-density server with third-generation direct water cooling technology developed in tandem with par Read more…

By Tiffany Trader

AMD Wins Another: Baidu to Deploy EPYC on Single Socket Servers

December 13, 2017

When AMD introduced its EPYC chip line in June, the company said a portion of the line was specifically designed to re-invigorate a single socket segment in wha Read more…

By John Russell

World Record: Quantum Computer with 46 Qubits Simulated

December 18, 2017

Scientists from the Jülich Supercomputing Centre have set a new world record. Together with researchers from Wuhan University and the University of Groningen, Read more…

New Blueprint for Converging HPC, Big Data

January 18, 2018

After five annual workshops on Big Data and Extreme-Scale Computing (BDEC), a group of international HPC heavyweights including Jack Dongarra (University of Te Read more…

By John Russell

  • arrow
  • Click Here for More Headlines
  • arrow
Share This