HPC Clouds — Alto Cirrus or Cumulonimbus

By Thomas Sterling and Dylan Stark

November 21, 2008

The “cloud” model of exporting user workload and services to remote, distributed and virtual environments is emerging as a powerful paradigm for improving efficiency of client and server operations, enhancing quality of service, and enabling early access to unprecedented resources for many small enterprises. From single users to major commercial organizations, cloud computing is finding numerous niche opportunities, often by simplifying rapid availability of new capabilities, with minimum time to deployment and return on requirements. Yet, one domain that challenges this model in its characteristics and needs is high performance computing (HPC).

The unique demands and decades’ long experiences of HPC on the one hand hunger for the level of service that clouds promise while on the other hand impose stringent properties, at least in some cases, that may be beyond the potential of this otherwise remarkable trend. The question is, can cloud computing reach the ethereal heights of Alto Cirrus for HPC, or will it inflict the damaging thunderclap of cumulonimbus?

While HPC immediately invokes images of TOP500 machines, the petaflops performance regime, and applications that boldly compute where no machine has calculated before, in truth this domain is multivariate with many distinct class of demand. The potential role and impact of cloud computing to HPC must be viewed across the range of disparate uses embodied by the HPC community. One possible delineation of the field (in order of most stringent first) is:

  1. Highest possible delivered capability performance (strong scaling).
  2. Weak scaling single applications.
  3. Capacity, or throughput job-stream, computing.
  4. Management of massive data sets, possibly geographically distributed.
  5. Analysis and visualization of data sets.
  6. Management and administrative workloads supporting the HPC community.

Consideration of these distinct workflows exposes opportunities for the potential exploitation of the cloud model and the benefits this might convey. Starting from the bottom of the list, the HPC community involves many everyday data processing requirements that are similar to any business or academic institution. Already some of the general infrastructure needs are quietly being outsourced to cloud-like services including databases, email, web-management, information retrieval and distribution, and other routine but critical functions. However, many of these tasks can be provided by the local set of distributed workstation and small enterprise servers. Therefore the real benefit is in reducing cost of software maintenance and per head cost of software licenses, rather than reduction of cost of hardware facilities.

Offloading tasks directly associated with doing computational science, such as data analysis and visualization, are appropriate to the use of cloud services in certain cases. This is particularly true for smaller organizations that do not have the full set of software systems that are appropriate to the local requirements. Occasionally, availability of mid-scale hardware resources, such as enterprise servers, may be useful as well if queue times do not impede fast turnaround. This domain can be expanded to include the frequent introduction of new or upgraded software packages not readily available at the local site, even if open source. Where such software is provided by ISVs, the cost of ownership or licensing may exceed the budget or even the need of occasional use.

Offerings by cloud providers may find preferable incentives for use of such software. It also removes the need for local expertise in installing, tuning, and maintaining such arcane packages. This is particularly true for small groups or individual researchers. However, a recurring theme is that HPC users tend to be in environments that incorporate high levels of expertise including motivated students and young researchers, and therefore are more likely to have access to such capabilities. The use of clouds in this case will be determined by the peculiarities of the individual and his/her situation.

Although HPC is often equated to FLOPS, it is as dependent, even sometimes more so, on bytes. Much science is data oriented, comprising data acquisition, product generation, organization, correlation, archiving, mining, and presentation. Massive data sets, especially those that are intrinsically distributed among many sites are a particularly rich target for cloud services. Maintenance of large tertiary storage facilities is particularly difficult and expensive, even for the most facilities rich environments. Data management is one area of HPC in which commercial enterprises are significantly advanced, even with respect to scientific computing expertise, with significant commercial investment being applied compared to the rarified boutique scientific computing community.

One very important factor is that confidence in data integrity of large archives may ultimately be higher among cloud resource suppliers both because of their potentially distributed nature removes issues of single point failure (like hurricanes, lightening strikes, floods), and their ability to exploit substantial investments available due to economy of scale. But one, perhaps insurmountable, challenge may impose fundamental limits in the use of clouds for data storage for some mission-critical HPC user agencies and commercial research institutions: data security. Where the potential damage for leakage or corruption of data would be strategic in nature for national security or intellectual property protection, it may be implausible that such data, no matter what the quantity or putative guarantees, will be trusted to remote and sometimes unspecified service entities.

Throughput computing is an area of strong promise for HPC in the exploitation of the emerging cloud systems. Cloud services are particularly well suited for the provisioning of resources to handle application loads of many sequential or slightly parallel (everything will have to become multicore) application tasks limited to size-constrained SMP units, such as for moderate duration parametric studies. In this case, cloud services have the potential to greatly enhance an HPC institution’s available resources and operational flexibility while improving efficiency and reducing overall cost of equipment and maintenance personnel. By offloading throughput computing workloads to cloud resources, HPC investments may be better applied to those resources unique to the needs of STEM applications not adequately served by the widely-available cloud-class processing services. However, this is tempered by the important constraint discussed above related to workloads that are security or IP sensitive.

The final two regimes of the HPC scientific and technical computing arena prove more problematic for clouds. Although weak scaling applications, where the problem size grows with the system scale such that granularity of concurrency remains approximately constant, may be suitable for a subset of the class of machines available within a cloud, the virtualization demanded by the cloud environment will preclude the hardware-specific performance tuning essential to effective HPC application execution. Virtualization is an important means of achieving user productivity, but as yet it is not a path to optimal performance, especially for high scale supercomputer grade commodity clusters (e.g., Beowulf) and MPPs (e.g., Cray XT3/4/5 and IBM BG/L/P/Q). And, while auto-tuning (as part of an autonomic framework) may one day offer a path to scalable performance, current practices at this time by users of major applications demand hands-on access to the detailed specifics of the physical machine.

Where the HPC community is already plagued with sometimes single-digit efficiencies for highly-tuned codes that may run for weeks or months to completion, the loss of substantially more performance to virtualization is untenable in many cases. An additional challenge relates to I/O bandwidth, which is sometimes a serious bottleneck if not balanced with the application needs that cannot be ensured by the abstraction of the cloud. Also, the problem of checkpoint and restart is critical to major application runs but may not be a robust service incorporated as part of most cloud systems. Therefore, a suitable system would need to make appropriate guarantees with respect to the availability of hardware and software configurations that would not be representative of the broad class of clouds.

Finally, the most challenging aspect of HPC is the constantly advancing architecture and application of capability computing systems. In their most pure form they enable strong scaling where response time is reduced for fixed sized applications with increasing system scale. Such systems imply a premium cost not just because of their mammoth size comprising upwards of a million cores and tens of terabytes of main memory, but also because of their unique design and limited market, which results in the loss of economy of scale. Even when integrating many commodity devices such as microprocessors and DRAM components, the cost of such systems may be tens of millions to over a hundred million dollars.

With the very high bandwidth, low latency internal networks with specialized functionality (e.g., combining networks) and high bandwidth storage area networks for attached secondary storage, there are few commercial user domains that can help offset the NRE costs of such major and optimized computing systems. It is unlikely that a business model can be constructed that would justify such systems being made available through cloud economics. Added to this are the same issues with virtualization versus performance optimization through hands-on performance tuning as described above. Therefore, it is unlikely that clouds will satisfy capability computing challenges for computational science in the foreseeable future.

The evolution of the cloud paradigm is an important maturing of the power of microelectronics, distributed computing, the Internet, and the rapidly expanding role of computing in all aspects of human enterprise and social context. The HPC and scientific computing community will benefit in tangential ways from the cloud environments as they evolve and where appropriate. However, challenges of virtualization and performance optimization, security and intellectual property protection, and unique requirements of scale and functionality, will result in certain critical aspects of the requirements of HPC falling outside the domain of cloud computing, relying instead on the strong foundation upon which HPC is well grounded.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Top Ten Ways AI Affects HPC in 2019

March 26, 2019

AI workloads are becoming ubiquitous, including running on the world’s fastest computers — thereby changing what we call HPC forever. As every organization plans for the future, AI workloads are on our minds — how Read more…

By James Reinders

GTC 2019: Chief Scientist Bill Dally Provides Glimpse into Nvidia Research Engine

March 22, 2019

Amid the frenzy of GTC this week – Nvidia’s annual conference showcasing all things GPU (and now AI) – William Dally, chief scientist and SVP of research, provided a brief but insightful portrait of Nvidia’s rese Read more…

By John Russell

ORNL Helps Identify Challenges of Extremely Heterogeneous Architectures

March 21, 2019

Exponential growth in classical computing over the last two decades has produced hardware and software that support lightning-fast processing speeds, but advancements are topping out as computing architectures reach thei Read more…

By Laurie Varma

HPE Extreme Performance Solutions

HPE and Intel® Omni-Path Architecture: How to Power a Cloud

Learn how HPE and Intel® Omni-Path Architecture provide critical infrastructure for leading Nordic HPC provider’s HPCFLOW cloud service.

powercloud_blog.jpgFor decades, HPE has been at the forefront of high-performance computing, and we’ve powered some of the fastest and most robust supercomputers in the world. Read more…

IBM Accelerated Insights

Insurance: Where’s the Risk?

Insurers are facing extreme competitive challenges in their core businesses. Property and Casualty (P&C) and Life and Health (L&H) firms alike are highly impacted by the ongoing globalization, increasing regulation, and digital transformation of their client bases. Read more…

Interview with 2019 Person to Watch Jim Keller

March 21, 2019

On the heels of Intel's reaffirmation that it will deliver the first U.S. exascale computer in 2021, which will feature the company's new Intel Xe architecture, we bring you our interview with our 2019 Person to Watch Jim Keller, head of the Silicon Engineering Group at Intel. Read more…

By HPCwire Editorial Team

Top Ten Ways AI Affects HPC in 2019

March 26, 2019

AI workloads are becoming ubiquitous, including running on the world’s fastest computers — thereby changing what we call HPC forever. As every organization Read more…

By James Reinders

GTC 2019: Chief Scientist Bill Dally Provides Glimpse into Nvidia Research Engine

March 22, 2019

Amid the frenzy of GTC this week – Nvidia’s annual conference showcasing all things GPU (and now AI) – William Dally, chief scientist and SVP of research, Read more…

By John Russell

At GTC: Nvidia Expands Scope of Its AI and Datacenter Ecosystem

March 19, 2019

In the high-stakes race to provide the AI life-cycle solution of choice, three of the biggest horses in the field are IBM, Intel and Nvidia. While the latter is only a fraction of the size of its two bigger rivals, and has been in business for only a fraction of the time, Nvidia continues to impress with an expanding array of new GPU-based hardware, software, robotics, partnerships and... Read more…

By Doug Black

Nvidia Debuts Clara AI Toolkit with Pre-Trained Models for Radiology Use

March 19, 2019

AI’s push into healthcare got a boost yesterday with Nvidia’s release of the Clara Deploy AI toolkit which includes 13 pre-trained models for use in radiolo Read more…

By John Russell

It’s Official: Aurora on Track to Be First US Exascale Computer in 2021

March 18, 2019

The U.S. Department of Energy along with Intel and Cray confirmed today that an Intel/Cray supercomputer, "Aurora," capable of sustained performance of one exaf Read more…

By Tiffany Trader

Why Nvidia Bought Mellanox: ‘Future Datacenters Will Be…Like High Performance Computers’

March 14, 2019

“Future datacenters of all kinds will be built like high performance computers,” said Nvidia CEO Jensen Huang during a phone briefing on Monday after Nvidia revealed scooping up the high performance networking company Mellanox for $6.9 billion. Read more…

By Tiffany Trader

Oil and Gas Supercloud Clears Out Remaining Knights Landing Inventory: All 38,000 Wafers

March 13, 2019

The McCloud HPC service being built by Australia’s DownUnder GeoSolutions (DUG) outside Houston is set to become the largest oil and gas cloud in the world th Read more…

By Tiffany Trader

Quick Take: Trump’s 2020 Budget Spares DoE-funded HPC but Slams NSF and NIH

March 12, 2019

U.S. President Donald Trump’s 2020 budget request, released yesterday, proposes deep cuts in many science programs but seems to spare HPC funding by the Depar Read more…

By John Russell

Quantum Computing Will Never Work

November 27, 2018

Amid the gush of money and enthusiastic predictions being thrown at quantum computing comes a proposed cold shower in the form of an essay by physicist Mikhail Read more…

By John Russell

The Case Against ‘The Case Against Quantum Computing’

January 9, 2019

It’s not easy to be a physicist. Richard Feynman (basically the Jimi Hendrix of physicists) once said: “The first principle is that you must not fool yourse Read more…

By Ben Criger

Why Nvidia Bought Mellanox: ‘Future Datacenters Will Be…Like High Performance Computers’

March 14, 2019

“Future datacenters of all kinds will be built like high performance computers,” said Nvidia CEO Jensen Huang during a phone briefing on Monday after Nvidia revealed scooping up the high performance networking company Mellanox for $6.9 billion. Read more…

By Tiffany Trader

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst o Read more…

By Tiffany Trader

Intel Reportedly in $6B Bid for Mellanox

January 30, 2019

The latest rumors and reports around an acquisition of Mellanox focus on Intel, which has reportedly offered a $6 billion bid for the high performance interconn Read more…

By Doug Black

Looking for Light Reading? NSF-backed ‘Comic Books’ Tackle Quantum Computing

January 28, 2019

Still baffled by quantum computing? How about turning to comic books (graphic novels for the well-read among you) for some clarity and a little humor on QC. The Read more…

By John Russell

It’s Official: Aurora on Track to Be First US Exascale Computer in 2021

March 18, 2019

The U.S. Department of Energy along with Intel and Cray confirmed today that an Intel/Cray supercomputer, "Aurora," capable of sustained performance of one exaf Read more…

By Tiffany Trader

Contract Signed for New Finnish Supercomputer

December 13, 2018

After the official contract signing yesterday, configuration details were made public for the new BullSequana system that the Finnish IT Center for Science (CSC Read more…

By Tiffany Trader

Leading Solution Providers

SC 18 Virtual Booth Video Tour

Advania @ SC18 AMD @ SC18
ASRock Rack @ SC18
DDN Storage @ SC18
HPE @ SC18
IBM @ SC18
Lenovo @ SC18 Mellanox Technologies @ SC18
NVIDIA @ SC18
One Stop Systems @ SC18
Oracle @ SC18 Panasas @ SC18
Supermicro @ SC18 SUSE @ SC18 TYAN @ SC18
Verne Global @ SC18

Deep500: ETH Researchers Introduce New Deep Learning Benchmark for HPC

February 5, 2019

ETH researchers have developed a new deep learning benchmarking environment – Deep500 – they say is “the first distributed and reproducible benchmarking s Read more…

By John Russell

IBM Quantum Update: Q System One Launch, New Collaborators, and QC Center Plans

January 10, 2019

IBM made three significant quantum computing announcements at CES this week. One was introduction of IBM Q System One; it’s really the integration of IBM’s Read more…

By John Russell

IBM Bets $2B Seeking 1000X AI Hardware Performance Boost

February 7, 2019

For now, AI systems are mostly machine learning-based and “narrow” – powerful as they are by today's standards, they're limited to performing a few, narro Read more…

By Doug Black

The Deep500 – Researchers Tackle an HPC Benchmark for Deep Learning

January 7, 2019

How do you know if an HPC system, particularly a larger-scale system, is well-suited for deep learning workloads? Today, that’s not an easy question to answer Read more…

By John Russell

HPC Reflections and (Mostly Hopeful) Predictions

December 19, 2018

So much ‘spaghetti’ gets tossed on walls by the technology community (vendors and researchers) to see what sticks that it is often difficult to peer through Read more…

By John Russell

Arm Unveils Neoverse N1 Platform with up to 128-Cores

February 20, 2019

Following on its Neoverse roadmap announcement last October, Arm today revealed its next-gen Neoverse microarchitecture with compute and throughput-optimized si Read more…

By Tiffany Trader

Move Over Lustre & Spectrum Scale – Here Comes BeeGFS?

November 26, 2018

Is BeeGFS – the parallel file system with European roots – on a path to compete with Lustre and Spectrum Scale worldwide in HPC environments? Frank Herold Read more…

By John Russell

France to Deploy AI-Focused Supercomputer: Jean Zay

January 22, 2019

HPE announced today that it won the contract to build a supercomputer that will drive France’s AI and HPC efforts. The computer will be part of GENCI, the Fre Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This