Jailbreaking HPC

By Gary Johnson

March 5, 2012

Is high performance computing in jail? Is number crunching a prisoner of policies and practices that are decades old?

The policies and practices that define the way we currently deploy and use our number-crunching HPC resources are based on assumptions that are, at best, several decades old. A lot has happened in the meantime, for example, the emergence of a sizeable and growing cadre of computational scientists and engineers, the internet and the web, continuous weakening of the coupling between work and location, the rise of data-intensive computing, commercial computing providers with staying power, and a broad consensus that the future health of our economy depends on intelligent and extensive exploitation of HPC. Perhaps it’s time to revisit those premises.

The major premises were that the engines of HPC are unique, its people are scarce, and the resources need to be held close in protected locations, its “temples.”

The engines of HPC are unique

In the early days of HPC, our “big iron” computers were crafted by hand and in small numbers. Those computers were in a class by themselves. We put them on display behind glass walls in machine rooms when they were new and, when they’d passed their prime. We retired them as museum pieces or lobby furniture. Have you ever sat on a Cray-1?

Since the late 90s, our HPC engines have been increasingly built from commodity parts and in scalable fashion. Indeed, those commodity parts are now reaching a degree of sophistication at the chip level that we previously only found at the machine level. HPC is now based on mass-produced hardware.

These computing systems have also become more modular – some to the extent of being assembled in Lego fashion from shipping-container-sized building blocks. Increasing numbers of computing floors are being operated in “lights out” mode, supported by very small numbers of on-site personnel.

The only real discriminators among HPC engines are: system balance (see: Number Crunching, Data Crunching and Energy Efficiency: the HPC Hat Trick); the size of the system one can afford; the floor space available to accommodate it; and the power bill one is prepared to pay.

The people of HPC are scarce

In those early days of HPC, there were no “computational scientists”. The term computational science was introduced by Ken Wilson around 1984. OK, maybe some of us were computational scientists but didn’t go by that name. The point is that computational science and engineering were just emerging as career fields and the number of practitioners was quite low.

Today, although there is still a supply gap, there are reasonably large numbers of computational scientists and engineers. They work in our universities, national labs and in industry. There are programs to educate them, journals to publish their work, focused meetings and conferences for them to attend, differentiated social networks for them to participate in, and human resources people who specialize in recruiting them.

HPC centers are temples

Back then, the scarcity of HPC users was complemented by a scarcity of people who were skilled at building and maintaining these specialized systems. That’s because it involved gluing various subsystems of exotic hardware and software together into functioning computers, tuning their physical environment, keeping them healthy and available, and making them accessible by remote users.

Additionally, teams of people were needed to offer training courses in the various aspects of this new thing, then called “supercomputing” and now called HPC. User support groups provided help in porting and tuning existing applications codes and expertise in writing new ones. Visualization specialists helped the end users create meaningful displays of their results, mostly by post-processing results on specialized visualization subsystems.

Given the exotic nature of supercomputing, the scarcity of the skilled people involved, and the complexity and interdependency of the tasks they performed, it made good sense to centralize HPC activities around a small number of centers. Today, there are vastly more HPC centers supporting various number-crunching applications in science and engineering.

Additionally, our definition of HPC has been enhanced to include the burgeoning field of data-intensive science. A whole new class of very large datacenters has come into existence to support the rapid growth and broad deployment of data-crunching applications in commerce, finance, marketing, social network analysis, and other fields. Furthermore, the locus of HPC activity seems to be shifting from the traditional protected locations (i.e., the temples) out into the enterprise. In concert with these developments, the operation of our computing centers has become more standardized and a larger pool of skilled people is available to support them.

Progress has overrun our premises

Depending on how you count it, we are three or four decades into this adventure we now call high performance computing. HPC has been very successful and its successes have altered the landscape of computing and its applications.

Our highest end computers are no longer unique but rather, by our own design decisions, scaled up versions of smaller and more common machines. Computational scientists and engineers are now routinely educated and employed, even if not in the numbers we might wish. HPC centers have lost a good bit of their original mystique as they have proliferated across the landscape.

Additional elements have also come into play. Ubiquitous access to the internet and world-wide web have made computing accessible to a large segment of the Earth’s population, not just those lucky few connected to some dedicated network. So, our HPC specialties can be practiced from just about anywhere. Commercial providers of computing services have reappeared, after a long hiatus , and have come back in much more resilient form (for example, Amazon, Cisco, Dell, Google, HP, IBM, Microsoft, and SGI).  It seems fair to say that commercially available computing services are here to stay.

On any given day, HPC in the Cloud is replete with news about these services. Indeed, we are told that some companies have gone so far as to compute exclusively in commercial clouds. Also, the Federal CIO has recently issued a memorandum encouraging federal agencies to develop trusted relationships with cloud service providers.

As for HPC applications, in the early days of supercomputing they were all about number crunching and we touted them as means for pushing back the frontiers of science and contributing seed corn for the future growth of science and engineering. There were no externally imposed milestones or deadlines associated with using computing in this mode for scientific discovery. Now, HPC is seen in much more immediate terms, as a problem-solving tool for economic success. Thus we desire a much swifter and more predictable transition from research into practice.

There has also been tremendous growth in data crunching applications, increasingly referred to as “big data.” Distinct from number crunching, these applications operate substantially outside the sphere of federal support. This other aspect of HPC is rapidly evolving and creating its own varieties of computing and data storage capabilities (for example, Facebook’s Open Compute Project and Open Storage Pod). Even the grande dame of supercomputing, Cray, has recently formed a Big Data Division. To follow the data-intensive computing and big data action, take a look at Datanami.

So, given the things that have changed and the new elements that have appeared, how might we adjust the premises on which we base our HPC policies and practices? Here are a few thoughts.

It’s about scalability

The elite machines in our computing ecosystem will always be, by definition, unique. We should take their care and feeding seriously. But the apex has now been pushed so high that computing seems really to be about “whateverscale.”

While are rightfully advocating for the development of exascale computers, most of our end-user applications compute at the terascale level, at best. They are thus “behind the computing curve” by a factor of 1,000,000, or soon will be. So, perhaps we should encourage innovation and growth of computing applications at whatever scale is appropriate to their immediate needs.

To the extent that we focus on moving “whateverscale” applications forward, their eventual transition to “apexscale” should be easier and come sooner. Furthermore, apexscale machines will be scaled-up versions of the smaller, whateverscale ones. Thus, the code development and architecture exploitation lessons learned at whateverscale will carry over to those top systems.

It’s about empowering people

In the early days of supercomputing, there were very few computational scientists , that is, people who were well versed in both their science field and in plying their science on our highest-end computing hardware. So, for allocating public resources, we created a dual review system. A traditional merit peer review was conducted to inform the process of allocating money and a separate “computational readiness” review was conducted to ensure that any compute cycles allocated would be used wisely.

Over the past several decades, we have been successful in producing lots of computational scientists and engineers – people who are very knowledgeable about both aspects of their profession. However, this dual review process is still in widespread use, especially by federal agencies. Thus, there is an opportunity to remove this form of “double jeopardy” by reverting to a single, comprehensive, review of computational science and engineering proposals. Doing this would also acknowledge our current generation of computational scientists and engineers for the skills they have worked hard to acquire. Emancipating and empowering computational scientists to better control their own fate can only serve to strengthen and grow the field.

It’s about the economy

When supercomputing was in its infancy, it was important to hold it close, to protect and nurture it. Consequently, our HPC resources were concentrated at federally funded research and development centers. Also, although there was a desire for these resources to have an economic impact, the kind of compulsion to do so that we see today was largely absent.

To be sure, there were a few companies that were early adopters of number-crunching HPC and continue to use it to great benefit. They generally employ their own internal computing resources. You can find some of them listed in the Top500 list, while others see competitive advantage in remaining invisible. What was missing then was an open, generally available, and well-supported commercial HPC infrastructure that could be used by much broader segments of the economy.

A few decades ago, there was an attempt to stand up commercial computer service bureaus and make supercomputing generally available. This attempt was less than successful, serving to reinforce the notion that custody of resources should be left to the government, its contractors, and perhaps a few other stalwarts.

Today, the computing and networking environment is vastly improved and our economic incentives to compute are intense. Data sciences and analytics have led the way in clearly establishing that commercial HPC is a viable enterprise. In many dimensions, this commercial form of HPC is now larger than our traditional government-sponsored form.

This presents all of us, including the federal HPC establishment, with the opportunity to embrace commercial HPC and to strengthen and grow it into a fabric to support the innovation in products and services that we’ve identified as necessary for the future health of our economy.

To be clear, embracing and using commercial HPC resources would not serve as a replacement for the vitally important work done by our government HPC complex. Rather, it would free this complex to pursue its essential role in HPC: pathfinding our computing future and making it real.

If HPC resources and services that are currently provided by the government complex can be provided commercially, then we should avail ourselves of this opportunity to build out our national computing infrastructure and actively support economic growth. If aspects of the current commercial offerings fall short of our number- and data-crunching needs, then we should aggressively transfer the knowledge and technologies necessary to remedy those shortcomings. Perhaps most importantly, we should use these commercial resources to the maximal extent feasible, to help grow and strengthen the marketplace and its offerings.

Rethinking things…

The premises on which we have built our HPC enterprise have clearly changed over the past several decades. While those premises served us well in getting HPC established, they may now be limiting its further development. Perhaps it’s time to jailbreak HPC.

—–

About the author

Gary M. Johnson is the founder of Computational Science Solutions, LLC, whose mission is to develop, advocate, and implement solutions for the global computational science and engineering community.

Dr. Johnson specializes in management of high performance computing, applied mathematics, and computational science research activities; advocacy, development, and management of high performance computing centers; development of national science and technology policy; and creation of education and research programs in computational engineering and science.

He has worked in Academia, Industry and Government. He has held full professorships at Colorado State University and George Mason University, been a researcher at United Technologies Research Center, and worked for the Department of Defense, NASA, and the Department of Energy.

He is a graduate of the U.S. Air Force Academy; holds advanced degrees from Caltech and the von Karman Institute; and has a Ph.D. in applied sciences from the University of Brussels.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Japan Meteorological Agency Takes Delivery of Pair of Crays

May 21, 2018

Cray has supplied two identical Cray XC50 supercomputers to the Japan Meteorological Agency (JMA) in northwestern Tokyo. Boasting more than 18 petaflops combined peak computing capacity, the new systems will extend the a Read more…

By Tiffany Trader

ASC18: Final Results Revealed & Wrapped Up

May 17, 2018

It was an exciting week at ASC18 in Nanyang, China. The student teams braved extreme heat, extremely difficult applications, and extreme competition in order to cross the cluster competition finish line. The gala awards ceremony took place on Wednesday. The auditorium was packed with student teams, various dignitaries, the media, and other interested parties. So what happened? Read more…

By Dan Olds

ASC18: Tough Applications & Tough Luck

May 17, 2018

The applications at the ASC18 Student Cluster Competition were tough. Tougher than the $3.99 steak special at your local greasy spoon restaurant. The apps are so tough that even Chuck Norris backs away from them slowly. Read more…

By Dan Olds

HPE Extreme Performance Solutions

HPC and AI Convergence is Accelerating New Levels of Intelligence

Data analytics is the most valuable tool in the digital marketplace – so much so that organizations are employing high performance computing (HPC) capabilities to rapidly collect, share, and analyze endless streams of data. Read more…

IBM Accelerated Insights

Mastering the Big Data Challenge in Cognitive Healthcare

Patrick Chain, genomics researcher at Los Alamos National Laboratory, posed a question in a recent blog: What if a nurse could swipe a patient’s saliva and run a quick genetic test to determine if the patient’s sore throat was caused by a cold virus or a bacterial infection? Read more…

Spring Meetings Underscore Quantum Computing’s Rise

May 17, 2018

The month of April 2018 saw four very important and interesting meetings to discuss the state of quantum computing technologies, their potential impacts, and the technology challenges ahead. These discussions happened in Read more…

By Alex R. Larzelere

Japan Meteorological Agency Takes Delivery of Pair of Crays

May 21, 2018

Cray has supplied two identical Cray XC50 supercomputers to the Japan Meteorological Agency (JMA) in northwestern Tokyo. Boasting more than 18 petaflops combine Read more…

By Tiffany Trader

ASC18: Final Results Revealed & Wrapped Up

May 17, 2018

It was an exciting week at ASC18 in Nanyang, China. The student teams braved extreme heat, extremely difficult applications, and extreme competition in order to cross the cluster competition finish line. The gala awards ceremony took place on Wednesday. The auditorium was packed with student teams, various dignitaries, the media, and other interested parties. So what happened? Read more…

By Dan Olds

Spring Meetings Underscore Quantum Computing’s Rise

May 17, 2018

The month of April 2018 saw four very important and interesting meetings to discuss the state of quantum computing technologies, their potential impacts, and th Read more…

By Alex R. Larzelere

Quantum Network Hub Opens in Japan

May 17, 2018

Following on the launch of its Q Commercial quantum network last December with 12 industrial and academic partners, the official Japanese hub at Keio University is now open to facilitate the exploration of quantum applications important to science and business. The news comes a week after IBM announced that North Carolina State University was the first U.S. university to join its Q Network. Read more…

By Tiffany Trader

Democratizing HPC: OSC Releases Version 1.3 of OnDemand

May 16, 2018

Making HPC resources readily available and easier to use for scientists who may have less HPC expertise is an ongoing challenge. Open OnDemand is a project by t Read more…

By John Russell

PRACE 2017 Annual Report: Exascale Aspirations; Industry Collaboration; HPC Training

May 15, 2018

The Partnership for Advanced Computing in Europe (PRACE) today released its annual report showcasing 2017 activities and providing a glimpse into thinking about Read more…

By John Russell

US Forms AI Brain Trust

May 11, 2018

Amid calls for a U.S. strategy for promoting AI development, the Trump administration is forming a senior-level panel to help coordinate government and industry research efforts. The Select Committee on Artificial Intelligence was announced Thursday (May 10) during a White House summit organized by the Office of Science and Technology Policy (OSTP). Read more…

By George Leopold

Emerging Advanced Scale Tech Trends Focus of Annual Tabor Conference

May 9, 2018

At Tabor Communications' annual Advanced Scale Forum (ASF) held this week in Austin, the focus was on enterprise adoption of HPC-class technologies and high performance data analytics (HPDA). It’s a confab that brings together end users (CIOs, IT planners, department heads) and vendors and encourages... Read more…

By the Editorial Team

MLPerf – Will New Machine Learning Benchmark Help Propel AI Forward?

May 2, 2018

Let the AI benchmarking wars begin. Today, a diverse group from academia and industry – Google, Baidu, Intel, AMD, Harvard, and Stanford among them – releas Read more…

By John Russell

How the Cloud Is Falling Short for HPC

March 15, 2018

The last couple of years have seen cloud computing gradually build some legitimacy within the HPC world, but still the HPC industry lies far behind enterprise I Read more…

By Chris Downing

Russian Nuclear Engineers Caught Cryptomining on Lab Supercomputer

February 12, 2018

Nuclear scientists working at the All-Russian Research Institute of Experimental Physics (RFNC-VNIIEF) have been arrested for using lab supercomputing resources to mine crypto-currency, according to a report in Russia’s Interfax News Agency. Read more…

By Tiffany Trader

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Deep Learning at 15 PFlops Enables Training for Extreme Weather Identification at Scale

March 19, 2018

Petaflop per second deep learning training performance on the NERSC (National Energy Research Scientific Computing Center) Cori supercomputer has given climate Read more…

By Rob Farber

Researchers Measure Impact of ‘Meltdown’ and ‘Spectre’ Patches on HPC Workloads

January 17, 2018

Computer scientists from the Center for Computational Research, State University of New York (SUNY), University at Buffalo have examined the effect of Meltdown Read more…

By Tiffany Trader

AI Cloud Competition Heats Up: Google’s TPUs, Amazon Building AI Chip

February 12, 2018

Competition in the white hot AI (and public cloud) market pits Google against Amazon this week, with Google offering AI hardware on its cloud platform intended Read more…

By Doug Black

US Plans $1.8 Billion Spend on DOE Exascale Supercomputing

April 11, 2018

On Monday, the United States Department of Energy announced its intention to procure up to three exascale supercomputers at a cost of up to $1.8 billion with th Read more…

By Tiffany Trader

Leading Solution Providers

Lenovo Unveils Warm Water Cooled ThinkSystem SD650 in Rampup to LRZ Install

February 22, 2018

This week Lenovo took the wraps off the ThinkSystem SD650 high-density server with third-generation direct water cooling technology developed in tandem with par Read more…

By Tiffany Trader

HPC and AI – Two Communities Same Future

January 25, 2018

According to Al Gara (Intel Fellow, Data Center Group), high performance computing and artificial intelligence will increasingly intertwine as we transition to Read more…

By Rob Farber

Inventor Claims to Have Solved Floating Point Error Problem

January 17, 2018

"The decades-old floating point error problem has been solved," proclaims a press release from inventor Alan Jorgensen. The computer scientist has filed for and Read more…

By Tiffany Trader

Google Chases Quantum Supremacy with 72-Qubit Processor

March 7, 2018

Google pulled ahead of the pack this week in the race toward "quantum supremacy," with the introduction of a new 72-qubit quantum processor called Bristlecone. Read more…

By Tiffany Trader

HPE Wins $57 Million DoD Supercomputing Contract

February 20, 2018

Hewlett Packard Enterprise (HPE) today revealed details of its massive $57 million HPC contract with the U.S. Department of Defense (DoD). The deal calls for HP Read more…

By Tiffany Trader

CFO Steps down in Executive Shuffle at Supermicro

January 31, 2018

Supermicro yesterday announced senior management shuffling including prominent departures, the completion of an audit linked to its delayed Nasdaq filings, and Read more…

By John Russell

Deep Learning Portends ‘Sea Change’ for Oil and Gas Sector

February 1, 2018

The billowing compute and data demands that spurred the oil and gas industry to be the largest commercial users of high-performance computing are now propelling Read more…

By Tiffany Trader

Nvidia Ups Hardware Game with 16-GPU DGX-2 Server and 18-Port NVSwitch

March 27, 2018

Nvidia unveiled a raft of new products from its annual technology conference in San Jose today, and despite not offering up a new chip architecture, there were still a few surprises in store for HPC hardware aficionados. Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Share This