Jailbreaking HPC

By Gary Johnson

March 5, 2012

Is high performance computing in jail? Is number crunching a prisoner of policies and practices that are decades old?

The policies and practices that define the way we currently deploy and use our number-crunching HPC resources are based on assumptions that are, at best, several decades old. A lot has happened in the meantime, for example, the emergence of a sizeable and growing cadre of computational scientists and engineers, the internet and the web, continuous weakening of the coupling between work and location, the rise of data-intensive computing, commercial computing providers with staying power, and a broad consensus that the future health of our economy depends on intelligent and extensive exploitation of HPC. Perhaps it’s time to revisit those premises.

The major premises were that the engines of HPC are unique, its people are scarce, and the resources need to be held close in protected locations, its “temples.”

The engines of HPC are unique

In the early days of HPC, our “big iron” computers were crafted by hand and in small numbers. Those computers were in a class by themselves. We put them on display behind glass walls in machine rooms when they were new and, when they’d passed their prime. We retired them as museum pieces or lobby furniture. Have you ever sat on a Cray-1?

Since the late 90s, our HPC engines have been increasingly built from commodity parts and in scalable fashion. Indeed, those commodity parts are now reaching a degree of sophistication at the chip level that we previously only found at the machine level. HPC is now based on mass-produced hardware.

These computing systems have also become more modular – some to the extent of being assembled in Lego fashion from shipping-container-sized building blocks. Increasing numbers of computing floors are being operated in “lights out” mode, supported by very small numbers of on-site personnel.

The only real discriminators among HPC engines are: system balance (see: Number Crunching, Data Crunching and Energy Efficiency: the HPC Hat Trick); the size of the system one can afford; the floor space available to accommodate it; and the power bill one is prepared to pay.

The people of HPC are scarce

In those early days of HPC, there were no “computational scientists”. The term computational science was introduced by Ken Wilson around 1984. OK, maybe some of us were computational scientists but didn’t go by that name. The point is that computational science and engineering were just emerging as career fields and the number of practitioners was quite low.

Today, although there is still a supply gap, there are reasonably large numbers of computational scientists and engineers. They work in our universities, national labs and in industry. There are programs to educate them, journals to publish their work, focused meetings and conferences for them to attend, differentiated social networks for them to participate in, and human resources people who specialize in recruiting them.

HPC centers are temples

Back then, the scarcity of HPC users was complemented by a scarcity of people who were skilled at building and maintaining these specialized systems. That’s because it involved gluing various subsystems of exotic hardware and software together into functioning computers, tuning their physical environment, keeping them healthy and available, and making them accessible by remote users.

Additionally, teams of people were needed to offer training courses in the various aspects of this new thing, then called “supercomputing” and now called HPC. User support groups provided help in porting and tuning existing applications codes and expertise in writing new ones. Visualization specialists helped the end users create meaningful displays of their results, mostly by post-processing results on specialized visualization subsystems.

Given the exotic nature of supercomputing, the scarcity of the skilled people involved, and the complexity and interdependency of the tasks they performed, it made good sense to centralize HPC activities around a small number of centers. Today, there are vastly more HPC centers supporting various number-crunching applications in science and engineering.

Additionally, our definition of HPC has been enhanced to include the burgeoning field of data-intensive science. A whole new class of very large datacenters has come into existence to support the rapid growth and broad deployment of data-crunching applications in commerce, finance, marketing, social network analysis, and other fields. Furthermore, the locus of HPC activity seems to be shifting from the traditional protected locations (i.e., the temples) out into the enterprise. In concert with these developments, the operation of our computing centers has become more standardized and a larger pool of skilled people is available to support them.

Progress has overrun our premises

Depending on how you count it, we are three or four decades into this adventure we now call high performance computing. HPC has been very successful and its successes have altered the landscape of computing and its applications.

Our highest end computers are no longer unique but rather, by our own design decisions, scaled up versions of smaller and more common machines. Computational scientists and engineers are now routinely educated and employed, even if not in the numbers we might wish. HPC centers have lost a good bit of their original mystique as they have proliferated across the landscape.

Additional elements have also come into play. Ubiquitous access to the internet and world-wide web have made computing accessible to a large segment of the Earth’s population, not just those lucky few connected to some dedicated network. So, our HPC specialties can be practiced from just about anywhere. Commercial providers of computing services have reappeared, after a long hiatus , and have come back in much more resilient form (for example, Amazon, Cisco, Dell, Google, HP, IBM, Microsoft, and SGI).  It seems fair to say that commercially available computing services are here to stay.

On any given day, HPC in the Cloud is replete with news about these services. Indeed, we are told that some companies have gone so far as to compute exclusively in commercial clouds. Also, the Federal CIO has recently issued a memorandum encouraging federal agencies to develop trusted relationships with cloud service providers.

As for HPC applications, in the early days of supercomputing they were all about number crunching and we touted them as means for pushing back the frontiers of science and contributing seed corn for the future growth of science and engineering. There were no externally imposed milestones or deadlines associated with using computing in this mode for scientific discovery. Now, HPC is seen in much more immediate terms, as a problem-solving tool for economic success. Thus we desire a much swifter and more predictable transition from research into practice.

There has also been tremendous growth in data crunching applications, increasingly referred to as “big data.” Distinct from number crunching, these applications operate substantially outside the sphere of federal support. This other aspect of HPC is rapidly evolving and creating its own varieties of computing and data storage capabilities (for example, Facebook’s Open Compute Project and Open Storage Pod). Even the grande dame of supercomputing, Cray, has recently formed a Big Data Division. To follow the data-intensive computing and big data action, take a look at Datanami.

So, given the things that have changed and the new elements that have appeared, how might we adjust the premises on which we base our HPC policies and practices? Here are a few thoughts.

It’s about scalability

The elite machines in our computing ecosystem will always be, by definition, unique. We should take their care and feeding seriously. But the apex has now been pushed so high that computing seems really to be about “whateverscale.”

While are rightfully advocating for the development of exascale computers, most of our end-user applications compute at the terascale level, at best. They are thus “behind the computing curve” by a factor of 1,000,000, or soon will be. So, perhaps we should encourage innovation and growth of computing applications at whatever scale is appropriate to their immediate needs.

To the extent that we focus on moving “whateverscale” applications forward, their eventual transition to “apexscale” should be easier and come sooner. Furthermore, apexscale machines will be scaled-up versions of the smaller, whateverscale ones. Thus, the code development and architecture exploitation lessons learned at whateverscale will carry over to those top systems.

It’s about empowering people

In the early days of supercomputing, there were very few computational scientists , that is, people who were well versed in both their science field and in plying their science on our highest-end computing hardware. So, for allocating public resources, we created a dual review system. A traditional merit peer review was conducted to inform the process of allocating money and a separate “computational readiness” review was conducted to ensure that any compute cycles allocated would be used wisely.

Over the past several decades, we have been successful in producing lots of computational scientists and engineers – people who are very knowledgeable about both aspects of their profession. However, this dual review process is still in widespread use, especially by federal agencies. Thus, there is an opportunity to remove this form of “double jeopardy” by reverting to a single, comprehensive, review of computational science and engineering proposals. Doing this would also acknowledge our current generation of computational scientists and engineers for the skills they have worked hard to acquire. Emancipating and empowering computational scientists to better control their own fate can only serve to strengthen and grow the field.

It’s about the economy

When supercomputing was in its infancy, it was important to hold it close, to protect and nurture it. Consequently, our HPC resources were concentrated at federally funded research and development centers. Also, although there was a desire for these resources to have an economic impact, the kind of compulsion to do so that we see today was largely absent.

To be sure, there were a few companies that were early adopters of number-crunching HPC and continue to use it to great benefit. They generally employ their own internal computing resources. You can find some of them listed in the Top500 list, while others see competitive advantage in remaining invisible. What was missing then was an open, generally available, and well-supported commercial HPC infrastructure that could be used by much broader segments of the economy.

A few decades ago, there was an attempt to stand up commercial computer service bureaus and make supercomputing generally available. This attempt was less than successful, serving to reinforce the notion that custody of resources should be left to the government, its contractors, and perhaps a few other stalwarts.

Today, the computing and networking environment is vastly improved and our economic incentives to compute are intense. Data sciences and analytics have led the way in clearly establishing that commercial HPC is a viable enterprise. In many dimensions, this commercial form of HPC is now larger than our traditional government-sponsored form.

This presents all of us, including the federal HPC establishment, with the opportunity to embrace commercial HPC and to strengthen and grow it into a fabric to support the innovation in products and services that we’ve identified as necessary for the future health of our economy.

To be clear, embracing and using commercial HPC resources would not serve as a replacement for the vitally important work done by our government HPC complex. Rather, it would free this complex to pursue its essential role in HPC: pathfinding our computing future and making it real.

If HPC resources and services that are currently provided by the government complex can be provided commercially, then we should avail ourselves of this opportunity to build out our national computing infrastructure and actively support economic growth. If aspects of the current commercial offerings fall short of our number- and data-crunching needs, then we should aggressively transfer the knowledge and technologies necessary to remedy those shortcomings. Perhaps most importantly, we should use these commercial resources to the maximal extent feasible, to help grow and strengthen the marketplace and its offerings.

Rethinking things…

The premises on which we have built our HPC enterprise have clearly changed over the past several decades. While those premises served us well in getting HPC established, they may now be limiting its further development. Perhaps it’s time to jailbreak HPC.

—–

About the author

Gary M. Johnson is the founder of Computational Science Solutions, LLC, whose mission is to develop, advocate, and implement solutions for the global computational science and engineering community.

Dr. Johnson specializes in management of high performance computing, applied mathematics, and computational science research activities; advocacy, development, and management of high performance computing centers; development of national science and technology policy; and creation of education and research programs in computational engineering and science.

He has worked in Academia, Industry and Government. He has held full professorships at Colorado State University and George Mason University, been a researcher at United Technologies Research Center, and worked for the Department of Defense, NASA, and the Department of Energy.

He is a graduate of the U.S. Air Force Academy; holds advanced degrees from Caltech and the von Karman Institute; and has a Ph.D. in applied sciences from the University of Brussels.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Geospatial Data Research Leverages GPUs

August 17, 2017

MapD Technologies, the GPU-accelerated database specialist, said it is working with university researchers on leveraging graphics processors to advance geospatial analytics. The San Francisco-based company is collabor Read more…

By George Leopold

Intel, NERSC and University Partners Launch New Big Data Center

August 17, 2017

A collaboration between the Department of Energy’s National Energy Research Scientific Computing Center (NERSC), Intel and five Intel Parallel Computing Centers (IPCCs) has resulted in a new Big Data Center (BDC) that Read more…

By Linda Barney

Google Releases Deeplearn.js to Further Democratize Machine Learning

August 17, 2017

Spreading the use of machine learning tools is one of the goals of Google’s PAIR (People + AI Research) initiative, which was introduced in early July. Last week the cloud giant released deeplearn.js as part of that in Read more…

By John Russell

HPE Extreme Performance Solutions

Leveraging Deep Learning for Fraud Detection

Advancements in computing technologies and the expanding use of e-commerce platforms have dramatically increased the risk of fraud for financial services companies and their customers. Read more…

Spoiler Alert: Glimpse Next Week’s Solar Eclipse Via Simulation from TACC, SDSC, and NASA

August 17, 2017

Can’t wait to see next week’s solar eclipse? You can at least catch glimpses of what scientists expect it will look like. A team from Predictive Science Inc. (PSI), based in San Diego, working with Stampede2 at the Read more…

By John Russell

Microsoft Bolsters Azure With Cloud HPC Deal

August 15, 2017

Microsoft has acquired cloud computing software vendor Cycle Computing in a move designed to bring orchestration tools along with high-end computing access capabilities to the cloud. Terms of the acquisition were not disclosed. Read more…

By George Leopold

HPE Ships Supercomputer to Space Station, Final Destination Mars

August 14, 2017

With a manned mission to Mars on the horizon, the demand for space-based supercomputing is at hand. Today HPE and NASA sent the first off-the-shelf HPC system i Read more…

By Tiffany Trader

AMD EPYC Video Takes Aim at Intel’s Broadwell

August 14, 2017

Let the benchmarking begin. Last week, AMD posted a YouTube video in which one of its EPYC-based systems outperformed a ‘comparable’ Intel Broadwell-based s Read more…

By John Russell

Deep Learning Thrives in Cancer Moonshot

August 8, 2017

The U.S. War on Cancer, certainly a worthy cause, is a collection of programs stretching back more than 40 years and abiding under many banners. The latest is t Read more…

By John Russell

IBM Raises the Bar for Distributed Deep Learning

August 8, 2017

IBM is announcing today an enhancement to its PowerAI software platform aimed at facilitating the practical scaling of AI models on today’s fastest GPUs. Scal Read more…

By Tiffany Trader

IBM Storage Breakthrough Paves Way for 330TB Tape Cartridges

August 3, 2017

IBM announced yesterday a new record for magnetic tape storage that it says will keep tape storage density on a Moore's law-like path far into the next decade. Read more…

By Tiffany Trader

AMD Stuffs a Petaflops of Machine Intelligence into 20-Node Rack

August 1, 2017

With its Radeon “Vega” Instinct datacenter GPUs and EPYC “Naples” server chips entering the market this summer, AMD has positioned itself for a two-head Read more…

By Tiffany Trader

Cray Moves to Acquire the Seagate ClusterStor Line

July 28, 2017

This week Cray announced that it is picking up Seagate's ClusterStor HPC storage array business for an undisclosed sum. "In short we're effectively transitioning the bulk of the ClusterStor product line to Cray," said CEO Peter Ungaro. Read more…

By Tiffany Trader

How ‘Knights Mill’ Gets Its Deep Learning Flops

June 22, 2017

Intel, the subject of much speculation regarding the delayed, rewritten or potentially canceled “Aurora” contract (the Argonne Lab part of the CORAL “ Read more…

By Tiffany Trader

Nvidia’s Mammoth Volta GPU Aims High for AI, HPC

May 10, 2017

At Nvidia's GPU Technology Conference (GTC17) in San Jose, Calif., this morning, CEO Jensen Huang announced the company's much-anticipated Volta architecture a Read more…

By Tiffany Trader

Reinders: “AVX-512 May Be a Hidden Gem” in Intel Xeon Scalable Processors

June 29, 2017

Imagine if we could use vector processing on something other than just floating point problems.  Today, GPUs and CPUs work tirelessly to accelerate algorithms Read more…

By James Reinders

Quantum Bits: D-Wave and VW; Google Quantum Lab; IBM Expands Access

March 21, 2017

For a technology that’s usually characterized as far off and in a distant galaxy, quantum computing has been steadily picking up steam. Just how close real-wo Read more…

By John Russell

Russian Researchers Claim First Quantum-Safe Blockchain

May 25, 2017

The Russian Quantum Center today announced it has overcome the threat of quantum cryptography by creating the first quantum-safe blockchain, securing cryptocurrencies like Bitcoin, along with classified government communications and other sensitive digital transfers. Read more…

By Doug Black

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

HPC Compiler Company PathScale Seeks Life Raft

March 23, 2017

HPCwire has learned that HPC compiler company PathScale has fallen on difficult times and is asking the community for help or actively seeking a buyer for its a Read more…

By Tiffany Trader

Trump Budget Targets NIH, DOE, and EPA; No Mention of NSF

March 16, 2017

President Trump’s proposed U.S. fiscal 2018 budget issued today sharply cuts science spending while bolstering military spending as he promised during the cam Read more…

By John Russell

Leading Solution Providers

Groq This: New AI Chips to Give GPUs a Run for Deep Learning Money

April 24, 2017

CPUs and GPUs, move over. Thanks to recent revelations surrounding Google’s new Tensor Processing Unit (TPU), the computing world appears to be on the cusp of Read more…

By Alex Woodie

CPU-based Visualization Positions for Exascale Supercomputing

March 16, 2017

In this contributed perspective piece, Intel’s Jim Jeffers makes the case that CPU-based visualization is now widely adopted and as such is no longer a contrarian view, but is rather an exascale requirement. Read more…

By Jim Jeffers, Principal Engineer and Engineering Leader, Intel

Google Debuts TPU v2 and will Add to Google Cloud

May 25, 2017

Not long after stirring attention in the deep learning/AI community by revealing the details of its Tensor Processing Unit (TPU), Google last week announced the Read more…

By John Russell

Six Exascale PathForward Vendors Selected; DoE Providing $258M

June 15, 2017

The much-anticipated PathForward awards for hardware R&D in support of the Exascale Computing Project were announced today with six vendors selected – AMD Read more…

By John Russell

Top500 Results: Latest List Trends and What’s in Store

June 19, 2017

Greetings from Frankfurt and the 2017 International Supercomputing Conference where the latest Top500 list has just been revealed. Although there were no major Read more…

By Tiffany Trader

IBM Clears Path to 5nm with Silicon Nanosheets

June 5, 2017

Two years since announcing the industry’s first 7nm node test chip, IBM and its research alliance partners GlobalFoundries and Samsung have developed a proces Read more…

By Tiffany Trader

MIT Mathematician Spins Up 220,000-Core Google Compute Cluster

April 21, 2017

On Thursday, Google announced that MIT math professor and computational number theorist Andrew V. Sutherland had set a record for the largest Google Compute Engine (GCE) job. Sutherland ran the massive mathematics workload on 220,000 GCE cores using preemptible virtual machine instances. Read more…

By Tiffany Trader

Messina Update: The US Path to Exascale in 16 Slides

April 26, 2017

Paul Messina, director of the U.S. Exascale Computing Project, provided a wide-ranging review of ECP’s evolving plans last week at the HPC User Forum. Read more…

By John Russell

  • arrow
  • Click Here for More Headlines
  • arrow
Share This