Visit additional Tabor Communication Publications
March 15, 2012
In high performance computing, Hewlett-Packard is best known for supplying bread-and-butter HPC systems, built with standard processors and interconnects. But the company's research arm has been devising a manycore chipset, which would outrun the average-sized HPC cluster of today. The design represents a radical leap in performance, and if implemented, would fulfill the promise of exascale computing.
The architecture, known as Corona, first conceived back in 2008, consists of a 256-core CPU, an optical memory module, integrated nanophotonics and 3D chip stacking employing through-silicon-vias (TSVs). At peak output, Corona should deliver 10 teraflops of performance. That's assuming 16 nm CMOS process technology, which is expected to be on tap by 2017.
The Corona design is aimed squarely at data-intensive types of application, whose speed is limited by the widening gap between CPU performance and available bandwidth to DRAM -- the so-called memory wall. Basically any workload whose data does not fit into processor cache is a candidate. This includes not just traditional big data applications, but also a whole bunch of interesting HPC simulations and analytics codes that have to manipulate large or irregular data sets, and are thus memory-constrained.
At the CPU level, Corona contains 256 cores, each supporting up to four threads simultaneously. The Corona cores themselves are nothing exotic. The HP researchers originally assumed low-power Intel x86 Penryn and Silverthorne CPU core architectures for their design simulations, but presumably ARM or other low-power designs could be substituted.
The processor is divided into 16 quad-core "clusters," with an integrated memory controller on every cluster. The rationale for the hierarchy is to ensure that memory bandwidth grows in concert with the core count and local memory access maintains low latency.
The processor is stacked with the memory controller/L2 cache, the analog electronics and the optical die (which includes on-chip lasers). Everything is hooked together by a 20 TB/sec dense wavelength division multiplexing (DWDM) crossbar, enabling cache coherency between cores, as well as superfast access to that cache.
The memory module, known as optically connected memory (OCM), is a separate chip stack made up of DRAM chips, plus the optical die and interface. It's connected to the CPU stack at a still rather impressive 10 TB/sec.
To put that into perspective, the current crop of commercial processors have to get by with just a fraction of that bandwidth. The latest 8-core Intel E5-2600 Xeons, for example, can manage about 80 GB/sec of memory bandwidth and the SPARC64 VIIIfx CPU, of K computer fame, supports 64 GB/sec. Even GPUs, which generally support bigger memory pipes (but have to feed hundreds of cores), are bandwidth constrained. NVIDIA fastest Tesla card, the M2090, maxes out at 177 GB/sec.
The main function of Corona's optical interconnect is to redress the worsening bytes-to-flop ratio that HPC'ers have been lamenting about for over a decade. For memory-constrained applications, it's preferable to have a byte-to-flop ratio of at least one. Back in the good old days of the late 20th century, computers delivered 8 bytes or more per flop. Now, for current CPUs and GPUs, it's down to between a half and a quarter of byte per flop.
The primary reasons for the poor ratio are the pin limitations on multicore processors, the inability to extend chip-level communication links across an entire node or computer, and the energy costs of electrical signaling. Photonics ameliorates these problems significantly since light is a much more efficient communication medium than electrons -- something long-haul network providers discovered awhile ago.
Energy efficiency, in particular, is a hallmark of photonic communication. The HP researchers calculate that a memory system using an electrical interconnect to drive 10 GB/sec of data to DRAM would take 80 watts. Using nanophotonics and DRAMs optimized to read or write just a cache line at a time, they think they achieve the same bandwidth with just 8 watts.
The trick is to get the optical hardware down onto the silicon. Thanks to recent advances in integrated photonics, the technology is getting close. For example, the Corona design specifies crystalline and silicon dioxide for the wave guides, which are two commonly used materials in CMOS manufacturing. Slightly more exotic is the use of Germanium for the receptors (to absorb the light so that it can be converted back into electrical signals), a less often used, but still CMOS-compatible material. Finally, for the light source, the Corona designers opted for mode-locked lasers, since they believe a single device can provide up to 64 wavelengths of light for the DWDM interconnect.
Using the SPLASH-2, the second version of the Stanford Parallel Applications for Shared Memory benchmark suite, the HP researchers demonstrated a performance improvement of 2 to 6 times on Corona compared to a similar system outfitted with an electrical interconnect, and those speed increases were achieved using much less power. They also showed significant performance improvements on five of the six HPC Challenge benchmarks: PTRANS (22X), STREAM (19X), GUPS (19X), MPI (19X), FFT (2X). DGEMM, which is not bandwidth limited, showed no improvement.
It's not all a slam dunk, however. 3D chipmaking and TSV technology is still a work in progress. And integrating photonic hardware using CMOS is in its infancy. But integrated photonics, 3D chip stacking, and the use of low-power cores for computation are all hot technologies now, especially for those in the supercomputing community looking down the road to exascale. The UHPC project (now apparently stuck in Phase 1) that was aimed at developing low-power extreme-scale computing, attracted proposals from Intel, MIT, NVIDIA, and Sandia that incorporated one or more of these technologies.
With Corona though, you get the whole package, so to speak. But all of the work to date appears to be with simulated hardware, and there was no mention in any of the research work of plans to create a working prototype. So whether this is destined to remain a research project at HP or something that gets transformed into a commercial offering remains to be seen.
May 23, 2013 |
The study of climate change is one of those scientific problems where it is almost essential to model the entire Earth to attain accurate results and make worthwhile predictions. In an attempt to make climate science more accessible to smaller research facilities, NASA introduced what they call ‘Climate in a Box,’ a system they note acts as a desktop supercomputer.
May 22, 2013 |
At some point in the not-too-distant future, building powerful, miniature computing systems will be considered a hobby for high schoolers, just as robotics or even Lego-building are today. That could be made possible through recent advancements made with the Raspberry Pi computers.
May 16, 2013 |
When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
May 15, 2013 |
Supercomputers at the Department of Energy’s National Energy Research Scientific Computing Center (NERSC) have worked on important computational problems such as collapse of the atomic state, the optimization of chemical catalysts, and now modeling popping bubbles.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/15/2013 | Bull | “50% of HPC users say their largest jobs scale to 120 cores or less.” How about yours? Are your codes ready to take advantage of today’s and tomorrow’s ultra-parallel HPC systems? Download this White Paper by Analysts Intersect360 Research to see what Bull and Intel’s Center for Excellence in Parallel Programming can do for your codes.
In this demonstration of SGI DMF ZeroWatt disk solution, Dr. Eng Lim Goh, SGI CTO, discusses a function of SGI DMF software to reduce costs and power consumption in an exascale (Big Data) storage datacenter.
The Cray CS300-AC cluster supercomputer offers energy efficient, air-cooled design based on modular, industry-standard platforms featuring the latest processor and network technologies and a wide range of datacenter cooling requirements.