DARPA Selects Cray and IBM for Final Phase of HPCS

By Michael Feldman

November 24, 2006

This week, the Defense Advanced Research Projects Agency (DARPA) selected Cray and IBM as the two Phase III developers for the High Productivity Computing Systems (HPCS) program. Initiated in 2002, the program is designed to produce a new generation of cost-effective, highly productive petascale systems for national security, scientific research and industrial users. The first two phases of HPCS were devoted to critical concept studies and assessments, preliminary research and development, and risk reduction engineering. Over the next four years, the third and final phase of the program will encompass development and demonstration of the HPCS technologies, culminating in a prototype system by each of the two vendors in 2010.

“This is a great day for Cray and the worldwide supercomputing community,” said Peter Ungaro, Cray's president and CEO. “The DARPA HPCS program is an important force that is shaping the future of HPC and the entire computer industry. With this Phase III award, DARPA has recognized Cray as a leading innovator with the technology, vision and expertise required to deliver world-class, revolutionary supercomputing systems.”

“IBM, DARPA and the mission partners will collaborate to develop a powerful and innovative design that will enhance the ability of supercomputers to help government, businesses and individuals,” said Bill Zeitler, senior vice president, IBM Systems and Technology Group. “We believe this new system will accelerate scientific breakthroughs, improve our nation's competitiveness and create new market opportunities.”

The DARPA-led program will use money contributed by the NSA and DOE to help fund the effort. Over the next four years, Cray will receive $250 million for their effort, while IBM will receive $244 million. The vendors and their contractors are also expected to make substantial investments in their own systems. According to DARPA, both IBM and Cray are obligated to provide at least 50 percent of the government funding amount in company cost-share.

“The vendors would not be producing these systems where it not for the investment by DARPA.” said HPCS program manager William Harrod, in a DARPA conference call on Wednesday. “They of course would have product lines, but they would not be nearly as aggressive in terms of performance and the ability to deliver productivity to their custoners. The key here is the ability to deliver productivity to the users. One can construct large systems, but then using them and getting performance out of them is the significant challenge. That's the problem we're trying to attack here.”

Harrod noted that high productivity computing will be a key technology for meeting our national security requirements and to enhance our economic competitiveness. “High productivity computing contributes substantially to the design and development of advanced vehicles and weapons, planning and execution of operational military scenarios, the intelligence problems of cryptanalysis and image processing, the maintenance of our nuclear stockpile, and is a key enabler for science and discovery in security-related fields,” he said.

In Phase III of the program, Cray and IBM will complete the hardware and software designs and technical development of their respective systems. The intention is to create machines capable of two petaflops of sustained performance, scalable to four petaflops. This represents a 10-fold performance increase compared to what was available in 2002. Even more significant is the requirement to increase the GUPS (Giga Updates Per Second) performance, which measures a system's ability to perform random memory accesses. This is especially important for applications which process irregular data structures, such as certain critical national security applications. The goal is to achieve GUPS performance of between 8,000 and 64,000. The current high mark goes to IBM Blue Gene/L, which achieves just 35 GUPS.

DARPA has specified some important HPCS Phase III milestones:

   1. Critical design review for software in 18 months.
   2. Critical design review for hardware in 30 months.
   3. Subsystem demonstration in December 2009.
   4. Final prototypes due in 2010.

DARPA will require that the prototype systems developed under HPCS to be at least one quarter of the size needed by the agency's mission partners — the NSA, DOE and NNSA. By the end of 2010, both Cray and IBM will have to demonstrate functional systems that will be evaluated by selected government HPC users.

To ensure economic viability, both vendors will be required to prepare a business plan for the development and commercialization of their products. The idea is to make the new technologies applicable to a range of systems, not just high-end government deployments. By ensuring that these systems are commercially viable, the government will not be the sole customer and thus, will not have to bear the entire burden of driving the evolution of these technologies.

Cray Cascade

The HPCS system being developed by Cray, called Cascade, is based on the company's vision of 'Adaptive Computing', a heterogeneous processing model in which the system software and the compiler/runtime code will assume the responsibility of mapping user applications on to the underlying processor hardware. Cascade will feature extremely high bandwidth global memory, advanced synchronization and multiple processor architectures (scalar, vector, multithreaded, and hardware acclerator). Cray says they will exploit the technology of a variety of partners in areas such as software tools and compilers (The Portand Group – PGI), file systems (Cluster File Systems), and storage (DataDirect Networks). In addition, Cray will rely heavily on AMD's multi-core Opteron processor and HyperTransport technologies.

Though Cray could not commit to the level of Opteron technology for the prototype in the 2010 time frame, 8-core AMD processors are expected to be available within the next two or three years. And while there are no specific plans to use AMD's ATI-derived GPU technology today, Cray CEO Peter Ungaro said that they are looking forward to working with AMD to bring their GPU technology into the Cascade system, as another accelerator option.

Over the next four years, Cray will incorporate elements of the Cascade program into commercially available products, including the peak-petaflops supercomputer, code-named “Baker,” that will be delivered to the Department of Energy's Oak Ridge National Laboratory (ORNL). In addition, ORNL will be one of Cray's Phase III partners, focused around scaling from both the systems perspective and the performance of key applications.


For the IBM HPCS effort, called PERCS, the company plans to make use of their POWER-based computer technologies. According to IBM, the DARPA award will substantially increase research and development activities into mainline IBM technologies planned to be delivered in 2010 and beyond, such as the next generation POWER7 processor, the AIX operating system, IBM's General Parallel File System, IBM's Parallel Environment and IBM's Interconnect and Storage Subsystems — technologies that are driving IBM's commercial product portfolio. IBM also plans to develop a robust HPC software stack and development tools to improve programmer productivity.

“These DARPA initiatives will propel IBM to far exceed the traditional 2X performance improvement over 18 months,” said Ravi Arimilli, IBM Fellow and Principal Investigator of POWER7. “We are embarking on a bold journey to deliver a 100X improvement in sustained performance over 48 months with a simpler and easy to use platform. Harnessing the development capabilities of IBM towards this disruptive design will drive the frontiers of science and business.”

Challenges Ahead

The difficulties of accomplishing all this in four years are considerable. At last week's “High Productivity Computing and Usable Petascale Systems” panel at SC06 in Tampa, Panelists Steve Scott (Cray), Rama Govindaraju (IBM), Jim Mitchell (Sun Microsystems) and Bob Lucas (University of Southern California) gave their perspectives on the challenges of DARPA's HPCS program. Jeremy Kepner (MIT Lincoln Laboratory) organized and chaired the panel and also participated in the discussion. Among the group, there was broad consensus about the the biggest challenges for HPCS systems.

It was generally agreed that petascale software (system and application) is trailing petascale hardware. The complexity of programming at this level will limit immediate exploitation of these systems. It was pointed out that increasing (peak) FLOPS is relatively easy to accomplish by just adding more floating-point hardware, but to use the FLOPS productively requires software that can be parallelized. Sun's Mitchell said that until the software scales, these systems will only be used as capacity machines – and rather expensive ones at that. Govindaraju agreed by noting that “peak performance is growing away from sustained performance.”

On the positive side, the use of a flatter memory hierarchy will increase performance and be easier to program compared to the distributed memory model in cluster architectures. This will help to raise the level of software abstraction, one of the key enablers for high productivity. However, the HPCS languages themselves, which are still under development, are not short-term solutions. In the interim, the more established Paritioned Global Address Space (PGAS) Languages, such as Universal Parallel C (UPC), Co-Array Fortran (CAF) and Titanium need to be made more available to give developers access to more productive software environments.

Another common theme from the panel involved system reliability of extremely large machines. When systems scale to tens of thousands or hundreds of thousands of processors and hundreds of terabytes of memory, the MTBF rates are such that a systematic approach must be developed in order to manage component failure — or as Mitchell observed, the ability to “compute through failure.” Ideally this means that hardware and system software mechanisms must be in place to insulate application code from system-level failures. Many of the panelists thought that software resiliency was probably the most important software technology for HPCS systems.

Cray CTO Steve Scott offered a message of optimism. He observed that petaflop hardware is only two years away and the importance of increasing global bandwidth, scaling software for multi-core processors and establishing system resiliency is well-understood, if not yet solved. The development of the hardware, operating system software and programming language environments are all underway. “I think we're going to get there,” concluded Scott.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Automated Optimization Boosts ResNet50 Performance by 1.77x on Intel CPUs

October 23, 2018

From supercomputers to cell phones, every system and software device in our digital panoply has a growing number of settings that, if not optimized, constrain performance, wasting precious cycles and watts. In the f Read more…

By Tiffany Trader

South Africa CHPC: Home Grown Dynasty

October 22, 2018

Before the build up to the final event in the 2018 Student Cluster Competition season (the SC18 competition in Dallas), I want to take a moment to write about one of the great inspirational stories of these competitions. Read more…

By Dan Olds

NSF Launches Quantum Computing Faculty Fellows Program

October 22, 2018

Efforts to expand quantum computing research capacity continue to accelerate. The National Science Foundation today announced a Quantum Computing & Information Science Faculty Fellows (QCIS-FF) program aimed at devel Read more…

By John Russell

HPE Extreme Performance Solutions

One Small Step Toward Mars: One Giant Leap for Supercomputing

Since the days of the Space Race between the U.S. and the former Soviet Union, we have continually sought ways to perform experiments in space. Read more…

IBM Accelerated Insights

Join IBM at SC18 and Learn to Harness the Next Generation of AI-focused Supercomputing

Blurring the lines between HPC and AI

Today’s high performance computers are helping clients gain insights at an unprecedented pace. The intersection of artificial intelligence (AI) and HPC can transform industries while solving some of the world’s toughest challenges. Read more…

Democratization of HPC Part 3: Ninth Graders Tap HPC in the Cloud to Design Flying Boats

October 18, 2018

This is the third in a series of articles demonstrating the growing acceptance of high-performance computing (HPC) in new user communities and application areas. In this article we present UberCloud use case #208 on how Read more…

By Wolfgang Gentzsch and Håkon Bull Hove

Automated Optimization Boosts ResNet50 Performance by 1.77x on Intel CPUs

October 23, 2018

From supercomputers to cell phones, every system and software device in our digital panoply has a growing number of settings that, if not optimized, constrain  Read more…

By Tiffany Trader

South Africa CHPC: Home Grown Dynasty

October 22, 2018

Before the build up to the final event in the 2018 Student Cluster Competition season (the SC18 competition in Dallas), I want to take a moment to write about o Read more…

By Dan Olds

Penguin Computing Launches Consultancy for Piecing AI Strategies Together

October 18, 2018

AI stands before the HPC industry as a beacon of great expectations, yet market research repeatedly shows that AI adoption is commonly stuck in the talking phas Read more…

By Tiffany Trader

When Water Quality—Not Quantity—Hinders HPC Cooling

October 18, 2018

Attention has been paid to the sheer quantity of water consumed by supercomputers’ cooling towers – and rightly so, as they can require thousands of gallons per minute to cool. But in the background, another factor can emerge, bottlenecking efficiency and raising costs: water quality. Read more…

By Oliver Peckham

Paper Offers ‘Proof’ of Quantum Advantage on Some Problems

October 18, 2018

Is quantum computing worth all the effort being poured into it or should we just wait for classical computing to catch up? An IBM blog today posed those questio Read more…

By John Russell

Dell EMC to Supply U Michigan’s Great Lakes Cluster

October 16, 2018

The University of Michigan (U-M) today announced Dell EMC is the lead vendor for U-M’s $4.8 million Great Lakes HPC cluster scheduled for deployment in first Read more…

By John Russell

Houston to Field Massive, ‘Geophysically Configured’ Cloud Supercomputer

October 11, 2018

Based on some news stories out today, one might get the impression that the next system to crack number one on the Top500 would be an industrial oil and gas mon Read more…

By Tiffany Trader

Nvidia Platform Pushes GPUs into Machine Learning, High Performance Data Analytics

October 10, 2018

GPU leader Nvidia, generally associated with deep learning, autonomous vehicles and other higher-end enterprise and scientific workloads (and gaming, of course) Read more…

By Doug Black

TACC Wins Next NSF-funded Major Supercomputer

July 30, 2018

The Texas Advanced Computing Center (TACC) has won the next NSF-funded big supercomputer beating out rivals including the National Center for Supercomputing Ap Read more…

By John Russell

IBM at Hot Chips: What’s Next for Power

August 23, 2018

With processor, memory and networking technologies all racing to fill in for an ailing Moore’s law, the era of the heterogeneous datacenter is well underway, Read more…

By Tiffany Trader

Requiem for a Phi: Knights Landing Discontinued

July 25, 2018

On Monday, Intel made public its end of life strategy for the Knights Landing "KNL" Phi product set. The announcement makes official what has already been wide Read more…

By Tiffany Trader

CERN Project Sees Orders-of-Magnitude Speedup with AI Approach

August 14, 2018

An award-winning effort at CERN has demonstrated potential to significantly change how the physics based modeling and simulation communities view machine learni Read more…

By Rob Farber

House Passes $1.275B National Quantum Initiative

September 17, 2018

Last Thursday the U.S. House of Representatives passed the National Quantum Initiative Act (NQIA) intended to accelerate quantum computing research and developm Read more…

By John Russell

Summit Supercomputer is Already Making its Mark on Science

September 20, 2018

Summit, now the fastest supercomputer in the world, is quickly making its mark in science – five of the six finalists just announced for the prestigious 2018 Read more…

By John Russell

New Deep Learning Algorithm Solves Rubik’s Cube

July 25, 2018

Solving (and attempting to solve) Rubik’s Cube has delighted millions of puzzle lovers since 1974 when the cube was invented by Hungarian sculptor and archite Read more…

By John Russell

D-Wave Breaks New Ground in Quantum Simulation

July 16, 2018

Last Friday D-Wave scientists and colleagues published work in Science which they say represents the first fulfillment of Richard Feynman’s 1982 notion that Read more…

By John Russell

Leading Solution Providers

HPC on Wall Street 2018 Booth Video Tours Playlist


Dell EMC





TACC’s ‘Frontera’ Supercomputer Expands Horizon for Extreme-Scale Science

August 29, 2018

The National Science Foundation and the Texas Advanced Computing Center announced today that a new system, called Frontera, will overtake Stampede 2 as the fast Read more…

By Tiffany Trader

HPE No. 1, IBM Surges, in ‘Bucking Bronco’ High Performance Server Market

September 27, 2018

Riding healthy U.S. and global economies, strong demand for AI-capable hardware and other tailwind trends, the high performance computing server market jumped 28 percent in the second quarter 2018 to $3.7 billion, up from $2.9 billion for the same period last year, according to industry analyst firm Hyperion Research. Read more…

By Doug Black

Intel Announces Cooper Lake, Advances AI Strategy

August 9, 2018

Intel's chief datacenter exec Navin Shenoy kicked off the company's Data-Centric Innovation Summit Wednesday, the day-long program devoted to Intel's datacenter Read more…

By Tiffany Trader

Germany Celebrates Launch of Two Fastest Supercomputers

September 26, 2018

The new high-performance computer SuperMUC-NG at the Leibniz Supercomputing Center (LRZ) in Garching is the fastest computer in Germany and one of the fastest i Read more…

By Tiffany Trader

MLPerf – Will New Machine Learning Benchmark Help Propel AI Forward?

May 2, 2018

Let the AI benchmarking wars begin. Today, a diverse group from academia and industry – Google, Baidu, Intel, AMD, Harvard, and Stanford among them – releas Read more…

By John Russell

Houston to Field Massive, ‘Geophysically Configured’ Cloud Supercomputer

October 11, 2018

Based on some news stories out today, one might get the impression that the next system to crack number one on the Top500 would be an industrial oil and gas mon Read more…

By Tiffany Trader

Aerodynamic Simulation Reveals Best Position in a Peloton of Cyclists

July 5, 2018

Eindhoven University of Technology (TU/e) and KU Leuven research group conducts the largest numerical simulation ever done in the sport industry and cycling discipline. The goal was to understand the aerodynamic interactions in the peloton, i.e., the main pack of cyclists in a race. Read more…

No Go for GloFo at 7nm; and the Fujitsu A64FX post-K CPU

September 5, 2018

It’s been a news worthy couple of weeks in the semiconductor and HPC industry. There were several HPC relevant disclosures at Hot Chips 2018 to whet appetites Read more…

By Dairsie Latimer

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This