2009-2019: A Look Back on a Decade of Supercomputing

By Andrew Jones

December 15, 2009

As we turn the decade into the 2020s, we take a nostalgic look back at the last ten years of supercomputing. It’s amazing to think how much has changed in that time. Many of our older readers will recall how things were before the official Planetary Supercomputing Facilities at Shanghai, Oak Ridge and Saclay were established. Strange as it may seem now, each country — in fact, each university or company — had its own supercomputer!

Hindsight is easier, of course, but it is interesting to review how this major change in supercomputing came to happen over the last few years.

At the start of the decade, each major university, research centre or company using simulation & modelling had its own HPC resources — they owned it or leased it, operated it, housed it, etc. In addition, some countries (US, UK, Germany, etc.) operated their own national resources for open research. The national facilities were larger than individual institutions could afford, and access to these was usually by a mechanism known as “peer review” — the prospective user would write a short case describing how their science would benefit from using the facility and a group of fellow scientists would judge if the science was worthy. (Note: they rated the science, almost never the quality of the computing implementation!) Very often these national supercomputers were reserved for capability computations, similar to today’s Strategic Simulation category at Shanghai.

The highest profile facilities were those in major research centres (e.g., universities, US DOE labs, etc.) but many commercial organisations had very large facilities too, although these weren’t as well publicised since companies had begun to recognise their use of HPC as a strategic competitive asset. The world’s fastest supercomputers were ranked twice yearly on the TOP500 list. One of the key uses of the TOP500 was for tracking the increasing performance of supercomputing power, usually through a plot showing performance on a vertical logarithmic axis against years on a horizontal axis, and especially two trends on this plot: the reasonably linear growth (on the log scale) of the performance of the fastest machine at any one time; and the smooth linearly (log scale) increasing sum of performance of the 500 systems on the list. The first spark towards the Planetary Supercomputing Facilities came when someone asked “what if we could actually use the compute power of that sum line at once?”

Another factor was the increasing cost of the facilities provision — from computer acquisition (capital) to power (both capital for infrastructure and recurrent for operations) to site management (recurrent and capital, project management, etc.).

Based on this, a number of collaborations started to occur. In Europe, over 20 countries joined together for the two-year PRACE initiative to explore how a pan-European supercomputer service could work in practice. Much was learned from that project and the influences can be seen in the three Planetary Supercomputing Facilities. In the US, ORNL, originally a DOE open science national supercomputing centre, started to host other national facilities (initially for NSF, NOAA and DoD). In fact, ORNL was probably the first planetary supercomputing facility in practice, even though, as we know, Shanghai was the first official Planetary Supercomputing Facility.

People started to realise that operating these large supercomputers was not the interesting part of HPC, and was in fact a very specialist job. As more and more aggregation between national operating sites occurred, and as the scale limited the potential sites (due to power constraints, etc.), it became apparent that there would only be a few sites worldwide capable of fulfilling the growth predicted by the original TOP500 trends.

Then of course came what I call “the public realisation”. Politicians, the public, and Boards finally got it. Supercomputing made a difference. It wasn’t just big rooms of computers costing lots of tax dollars. It was a tool to underpin science, and often to propel it forward. It was a tool for accelerating any properly-formulated computational task, many even with impact on daily life. Better weather predictions. Better design and safety testing of household products. Consumer video/image processing (I remember trying to do early video processing on my own PC!). Speech processing — think how that has revolutionized mobile communications since the early days of typing email messages on BlackBerrys and the like.

And then the critical step — businesses and researchers finally understood that their competitive asset was the capabilities of their modelling software and user expertise — not the hardware itself. Successful businesses rushed to establish a lead over their competitors by investing in their modelling capability — especially robustness (getting trustable predictions/analysis), scalability (being able to process much larger datasets than before) and performance (driving down time to solutions).

As this “software arms race” was put into practice (led by the commercial users) — slowly at first but then with a surge of investment in robust scalable high performance software — money spent on hardware ceased to be the competitive difference. Coupled with the massive increase in demand for HPC resources following the public realisation, and the challenges of managing large facilities, this led to the announcement of the first Planetary Supercomputer Facility in Shanghai. Whilst there was initially preferential access for Chinese domestic users, anyone in the world could use the facility — from consumers to researchers to businesses. After years of trying to exploit commodity components, HPC itself became a commodity service. And this was true HPC, supporting tightly-coupled large simulations, not the earlier attempts at something daftly called “cloud computing,” which only really supported large numbers of very small jobs. The facility shocked the world with its scale — being larger not only than the then top machine on the TOP500, but also larger than the sum of the 500 systems.

The business case for individual ownership of HPC facilities worldwide suddenly became dramatically tougher to justify, with Shanghai providing all classes of computer resources at scale, including the various specialist processing types. Everyone got better HPC, whether capacity or capability, and cheaper HPC than they could ever provide locally. The consumer demand drove innovations in ease-of-use and accounting that previously were only ambitions of seemingly-perpetual academic research.

The international agreements from research funding agencies on behalf of their user communities and from consumer HPC brokers soon followed, confirming the official Planetary Supercomputing Facility status. Within a year, the US had followed suit, securing global agreement for Oak Ridge as the second official Planetary Supercomputing Facility, and of course deployed even more powerful resources than Shanghai.

Soon, the main security concerns had been solved. Network bandwidth that plagued earlier global collaborations went away, as data rarely needed to leave the facilities (or if so, only to transfer between Oak Ridge and Shanghai, which now had massive dedicated bandwidth). Anything that might be done with the data could be done at Oak Ridge or Shanghai — the data never needed to go anywhere else.

With the opening last year of the third and final Planetary Supercomputing Facility at Saclay, the world’s HPC is now ready to sprint into the next decade. We have now left the housing and daily care of the hardware to the specialists. The volume of public and private demand has set the scene for strong HPC provision into the future. We have the three official global providers to ensure consumer choice, with its competitive benefits, but few enough providers to underpin their business cases for the most capable possible HPC infrastructure.

With the pervasiveness of HPC in consumer, business and research arenas, and the long overdue acceptance of the truth that the software capabilities and performance at scale was the competitive asset, “can program HPC at scale” is now more than ever a valuable item for your CV.

For all this astounding progress, I wonder how quaint today’s world will seem when we look back from 2030. After all, just imagine someone reading this in 2009!

2009 Author’s Note: This is not intended to be a prediction nor vision for the next decade, merely some seasonal fun looking at some unlikely extremes of how our community might develop. After all, we’ve had reports saying “it’s the software” for years — so are the chances of us finally doing anything about it more or less likely than the Planetary Supercomputing Facilities?

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Glimpses of Today’s Total Solar Eclipse

August 21, 2017

Here are a few arresting images posted by NASA of today’s total solar eclipse. Such astronomical events have always captured our imagination and it’s not hard to understand why such occurrences were often greeted wit Read more…

By John Russell

Tech Giants Outline Battle Plans for Future HPC Market

August 21, 2017

Four companies engaged in a cage fight for leadership in the emerging HPC market of the 2020s are, despite deep differences in some areas, in violent agreement on at least one thing: the power consumption and latency pen Read more…

By Doug Black

Geospatial Data Research Leverages GPUs

August 17, 2017

MapD Technologies, the GPU-accelerated database specialist, said it is working with university researchers on leveraging graphics processors to advance geospatial analytics. The San Francisco-based company is collabor Read more…

By George Leopold

HPE Extreme Performance Solutions

Leveraging Deep Learning for Fraud Detection

Advancements in computing technologies and the expanding use of e-commerce platforms have dramatically increased the risk of fraud for financial services companies and their customers. Read more…

Intel, NERSC and University Partners Launch New Big Data Center

August 17, 2017

A collaboration between the Department of Energy’s National Energy Research Scientific Computing Center (NERSC), Intel and five Intel Parallel Computing Centers (IPCCs) has resulted in a new Big Data Center (BDC) that Read more…

By Linda Barney

Tech Giants Outline Battle Plans for Future HPC Market

August 21, 2017

Four companies engaged in a cage fight for leadership in the emerging HPC market of the 2020s are, despite deep differences in some areas, in violent agreement Read more…

By Doug Black

Microsoft Bolsters Azure With Cloud HPC Deal

August 15, 2017

Microsoft has acquired cloud computing software vendor Cycle Computing in a move designed to bring orchestration tools along with high-end computing access capabilities to the cloud. Terms of the acquisition were not disclosed. Read more…

By George Leopold

HPE Ships Supercomputer to Space Station, Final Destination Mars

August 14, 2017

With a manned mission to Mars on the horizon, the demand for space-based supercomputing is at hand. Today HPE and NASA sent the first off-the-shelf HPC system i Read more…

By Tiffany Trader

AMD EPYC Video Takes Aim at Intel’s Broadwell

August 14, 2017

Let the benchmarking begin. Last week, AMD posted a YouTube video in which one of its EPYC-based systems outperformed a ‘comparable’ Intel Broadwell-based s Read more…

By John Russell

Deep Learning Thrives in Cancer Moonshot

August 8, 2017

The U.S. War on Cancer, certainly a worthy cause, is a collection of programs stretching back more than 40 years and abiding under many banners. The latest is t Read more…

By John Russell

IBM Raises the Bar for Distributed Deep Learning

August 8, 2017

IBM is announcing today an enhancement to its PowerAI software platform aimed at facilitating the practical scaling of AI models on today’s fastest GPUs. Scal Read more…

By Tiffany Trader

IBM Storage Breakthrough Paves Way for 330TB Tape Cartridges

August 3, 2017

IBM announced yesterday a new record for magnetic tape storage that it says will keep tape storage density on a Moore's law-like path far into the next decade. Read more…

By Tiffany Trader

AMD Stuffs a Petaflops of Machine Intelligence into 20-Node Rack

August 1, 2017

With its Radeon “Vega” Instinct datacenter GPUs and EPYC “Naples” server chips entering the market this summer, AMD has positioned itself for a two-head Read more…

By Tiffany Trader

How ‘Knights Mill’ Gets Its Deep Learning Flops

June 22, 2017

Intel, the subject of much speculation regarding the delayed, rewritten or potentially canceled “Aurora” contract (the Argonne Lab part of the CORAL “ Read more…

By Tiffany Trader

Nvidia’s Mammoth Volta GPU Aims High for AI, HPC

May 10, 2017

At Nvidia's GPU Technology Conference (GTC17) in San Jose, Calif., this morning, CEO Jensen Huang announced the company's much-anticipated Volta architecture a Read more…

By Tiffany Trader

Reinders: “AVX-512 May Be a Hidden Gem” in Intel Xeon Scalable Processors

June 29, 2017

Imagine if we could use vector processing on something other than just floating point problems.  Today, GPUs and CPUs work tirelessly to accelerate algorithms Read more…

By James Reinders

Russian Researchers Claim First Quantum-Safe Blockchain

May 25, 2017

The Russian Quantum Center today announced it has overcome the threat of quantum cryptography by creating the first quantum-safe blockchain, securing cryptocurrencies like Bitcoin, along with classified government communications and other sensitive digital transfers. Read more…

By Doug Black

Quantum Bits: D-Wave and VW; Google Quantum Lab; IBM Expands Access

March 21, 2017

For a technology that’s usually characterized as far off and in a distant galaxy, quantum computing has been steadily picking up steam. Just how close real-wo Read more…

By John Russell

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Groq This: New AI Chips to Give GPUs a Run for Deep Learning Money

April 24, 2017

CPUs and GPUs, move over. Thanks to recent revelations surrounding Google’s new Tensor Processing Unit (TPU), the computing world appears to be on the cusp of Read more…

By Alex Woodie

HPC Compiler Company PathScale Seeks Life Raft

March 23, 2017

HPCwire has learned that HPC compiler company PathScale has fallen on difficult times and is asking the community for help or actively seeking a buyer for its a Read more…

By Tiffany Trader

Leading Solution Providers

Trump Budget Targets NIH, DOE, and EPA; No Mention of NSF

March 16, 2017

President Trump’s proposed U.S. fiscal 2018 budget issued today sharply cuts science spending while bolstering military spending as he promised during the cam Read more…

By John Russell

Google Debuts TPU v2 and will Add to Google Cloud

May 25, 2017

Not long after stirring attention in the deep learning/AI community by revealing the details of its Tensor Processing Unit (TPU), Google last week announced the Read more…

By John Russell

CPU-based Visualization Positions for Exascale Supercomputing

March 16, 2017

In this contributed perspective piece, Intel’s Jim Jeffers makes the case that CPU-based visualization is now widely adopted and as such is no longer a contrarian view, but is rather an exascale requirement. Read more…

By Jim Jeffers, Principal Engineer and Engineering Leader, Intel

Six Exascale PathForward Vendors Selected; DoE Providing $258M

June 15, 2017

The much-anticipated PathForward awards for hardware R&D in support of the Exascale Computing Project were announced today with six vendors selected – AMD Read more…

By John Russell

Top500 Results: Latest List Trends and What’s in Store

June 19, 2017

Greetings from Frankfurt and the 2017 International Supercomputing Conference where the latest Top500 list has just been revealed. Although there were no major Read more…

By Tiffany Trader

IBM Clears Path to 5nm with Silicon Nanosheets

June 5, 2017

Two years since announcing the industry’s first 7nm node test chip, IBM and its research alliance partners GlobalFoundries and Samsung have developed a proces Read more…

By Tiffany Trader

Messina Update: The US Path to Exascale in 16 Slides

April 26, 2017

Paul Messina, director of the U.S. Exascale Computing Project, provided a wide-ranging review of ECP’s evolving plans last week at the HPC User Forum. Read more…

By John Russell

Graphcore Readies Launch of 16nm Colossus-IPU Chip

July 20, 2017

A second $30 million funding round for U.K. AI chip developer Graphcore sets up the company to go to market with its “intelligent processing unit” (IPU) in Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Share This