HPCwire: IBM Enters Petascale Era with Blue Gene/P

The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing

HPCwire >> Features

June 29, 2007

IBM Enters Petascale Era with Blue Gene/P

by Michael Feldman, HPCwire Editor

Page: 1 of 2
1 | 2 All »

This week at the International Supercomputing Conference (ISC) in Dresden, Germany, IBM unveiled its next-generation Blue Gene architecture -- Blue Gene/P. The new model is intended for users looking for petaflop-level computing and beyond. Like its Blue Gene/L predecessor, Blue Gene/P is targeted for big science applications and the very highest end of commercial HPC. According to IBM, Blue Gene/P is two and a half times more powerful than the Blue Gene/L generation and requires only slightly more power. A relatively modest two-rack Blue Gene/P configuration that IBM deployed in-house ended up as number 31 on the new Top500 list announced this week.

The previous-generation Blue Gene/L machines represent some of the fastest systems in the world. The Lawrence Livermore system currently holds the top spot on the Top500 list and a number of other Blue Gene/L installations are scattered throughout the list. But this is the end of the line for Blue Gene/L. IBM manufacturing will now switch over to the P line. Blue Gene/L purchases currently in the pipeline represent the last machines of the first generation.

The Blue Gene/P architecture is based on a quad-core PowerPC 450 ASIC chip, where each chip is capable of 13.6 gigaflops. A compute node includes the ASIC chip along with 2 GB of SDRAM DDR2 memory. Thirty-two of these nodes are aggregated onto a board and 32 of the boards are placed in a 6-foot high rack. The result is a 4096-core rack, which provides 13.9 teraflops (peak) of processing power. This represents the smallest Blue Gene/P system you can buy.

Although the original Blue Gene/L was dual-core, it did not implement cache coherency in the hardware. By contrast, Blue Gene/P is designed around cache coherent quad-core chips, so they can be treated as SMP nodes in the same manner as any multicore-based commodity cluster. This makes the new Blue Gene more suitable for multithreaded workloads based on standard software technologies like OpenMP.

Compared to Blue Gene/L, the new generation uses slightly faster PowerPC processors (850 MHz versus 700 MHz) and twice as many cores per chip (4 versus 2). L3 cache has been doubled from 4 MB to 8 MB and main memory per compute node has been quadrupled from 512 MB to 2 GB. Main memory bandwidth has also increased -- from 5.6 to 13.6 GB/sec. In addition, the 3-D Torus and Tree networks have been upgraded, essentially more than doubling the bandwidth and cutting latencies in half. The increased capabilities provide a 2.4x increase in performance over Blue Gene/L, using roughly the same floor space and slightly more power.

A single 13.9-teraflop Blue Gene/P rack draws just 40 kilowatts, yielding 0.35 gigaflops/watt -- possibly the best performance/watt metric of any general-purpose computing system on the planet. SiCortex's MIPS-based cluster systems come close at around 0.32 gigaflops/watt. For comparison, Blue Gene/L offers a lower, but very respectable, 0.23 gigaflops/watt. Most x86-based high performance computing systems are an order of magnitude lower than that. As users build Blue Gene/P systems that scale to hundreds of teraflops and beyond, power efficiencies become even more critical.

And while not every customer will use Blue Gene/P to build petaflop systems, IBM anticipates at least one customer will put enough Blue Gene/P racks together to reach a sustained (Linpack) petaflop as early as next year. Apparently IBM has a few prospects that are considering purchasing the 80 or so Blue Gene/P racks required to build a such a machine. The architecture is actually designed to scale up 256 racks, which would come close to three Linpack petaflops. However, there are few customers who would know what to do with such power, and the cost would probably be prohibitive even for that select group. IBM realizes that, although there are many HPC customers with computational problems bigger than their machines, there are only so many organizations that have the right combination of money, workload, and software experience that's required to take advantage of machines like Blue Gene/P.

In any case, the Blue Gene/P sales pipeline is already filling up. The U.S. Dept. of Energy's Argonne National Laboratory, Argonne, Ill., will deploy the first Blue Gene/P system this fall. Argonne's initial Blue Gene/P system will be a 114-teraflop machine, and the lab is on track to eventually install about half a petaflop. Argonne currently has a Blue Gene/L system and will continue to operate that machine through at least 2008.

Explaining the lab's motivation to increase their Blue Gene investment, Ray Bair, Division Director for the Argonne Leadership Class Facility said: "Blue Gene has been a resounding success for scientific computing since its inception, both for DOE's INCITE program at Argonne National Laboratory and in diverse science programs at institutions around the world. The breadth and scale of science problems that can be addressed with Blue Gene was another important factor. IBM designed Blue Gene/P with petascale scientific computing in mind, making performance and functionality improvements from top to bottom while preserving Blue Gene's extraordinary balance."

Other installations are being planned as well. In Germany, the Max Planck Society and Forschungszentrum Jülich are scheduled to begin installing Blue Gene/P systems in late 2007. Other Blue Gene/P deployments are being planned by Stony Brook University and Brookhaven National Laboratory in Upton, N.Y., and the Science and Technology Facilities Council, Daresbury Laboratory in Cheshire, England.

Page: 1 of 2
1 | 2 All »

Article Tools

Share Options

(Digg, Technorati, more)

Subscribe

Subscribe to HPCwire

Discussion

There are 0 discussion items posted.

Join the Discussion

You must be a registered user in order to comment on HPCwire stories.

Members enjoy:

	Access to Special Content, including Podcasts, Webcasts, and Whitepapers and more!
	Ability to take an active role in the discussions that are shaping the HPC technology discipline!
	Special offers available only to HPCwire members!

Become a Registered User Today!

Registered Users Log in Here to Comment

Email Address:	Password (case sensitive)

Remember me	Forgot Password?

RSS Feeds

Feeds by Topic

Feeds by Industry

Feeds by Content Type

Top Headlines

Intel Partners See 'Easy' Upgrade Path With Xeon 5600 Chips

Mar 18 | ChannelWeb | Westmere parts already showing up in HPC machines. Read more...

AMD: OEMs primed for Opteron 6100s

Mar 17 | The Register | But what about the tier ones? Read more...

Arrival of the Desktop Supercomputer

Mar 17 | Cadalyst Magazine | A new generation of workstations is changing the nature of technical computing. Read more...

Scheduling HPC In The Cloud

Mar 17 | Linux Magazine | Latest iteration of Sun Grid Engine able to tap into Cloud. Read more...

Tailoring Medicine with Supercomputers

Mar 16 | Bio-IT World | Biotech firm builds genetic models from patient data. Read more...

Read more headlines...

Featured Whitepapers

Virtualization for Aggregation And The vSMP Architecture™

Jan 12 | | In-depth look at vSMP Foundation server virtualization technology, technical implementation, use cases and capabilities. The technical whitepaper provides an architectural overview and details on the three vSMP Foundation products: vSMP Foundation for SMP, vSMP Foundation for Cluster and vSMP Foundation for Cloud.

Copper Cable Technologies for High Performance Computing

Jan 18 | | This white paper discusses Gore’s copper cable assemblies, and how they continue to exceed the standards for providing reliable, cost-effective solutions for high-performance computer applications.

View the White Paper Library

Multimedia

Webcast: Virtualized Data Center Roundtable

Join this online panel discussion for live Q&A with leading industry experts, analysts, and end-users to discuss the latest innovations, best practices, barriers to implementation, and measurable benefits of server virtualization with a particular focus on today's real world solutions.

Webcast: Watch SC09 Birds of a Feather Video: Scalable Fault-Tolerant HPC Supercomputers

Learn about scalable fault-tolerant architectures and examples of energy efficient and scalable supercomputing clusters using dual QDR InfiniBand to combine capacity computing with network failover capabilities with the help of programming languages such as MPI and a robust Linux cluster management package.

Webcast: High Performance Computing for a Smarter Planet

LIVE@SCO9: The IBM team discusses new innovations in hardware, software and services that help clients better understand their workloads and get insight from their R&D efforts. Technology demonstrations include the soon-to-be-released Power7 HPC processor, the DCS990 system with 2.4 petabytes of storage, the xCAT management tool, secure HPC cloud computing and more. Winners of two HPCwire Readers' and Editors’ Choice Awards! Take the IBM virtual tour at SC09 or more information go online to: http://www-03.ibm.com/systems/deepcomputing/sc09.html