Big Science, Tiny Microservers: IBM Research Pushes 64-Bit Possibilities

By Nicole Hemsoth

April 10, 2014

Four years ago, a friend dropped a Sheeva Plug into the hands of Ronald Luijten, a system designer at IBM Research in Zurich. At the time, neither could have realized the development cycle this simple gift would spark.

If you’re not familiar, Sheeva Plugs are compact devices that look a lot like your laptop power adapter, except instead of an electrical output plug, there’s a handy gigabit Ethernet port. Luitjen, whose primary interests lie in data movement and energy management, immediately saw the potential. He put his minimalist inclinations to work, and within a few months, had a VNC, an OS and a web server running from a USB attached hard drive. What struck him the most, however, was when he measured it from the mains and found the whole thing was running at a mere 4.3 watts. “I couldn’t believe this,” he said. “When I thought about it further, I saw it was the beginning of a revolution.”

This discovery coincided with a much larger project Luitjen was involved with at IBM Research. In conjunction with ASTRON, a team tapped some of Big Blue’s best minds to help the Square Kilometer Array (SKA) team discover new solutions to solve the unprecedented power, compute and data movement challenges inherent to measuring the Big Bang. Over the next decade, SKA researchers will be able to look back 13.8 billion years (and over a billion dollars) with 2 million antennae that will pull together a signal at the end of each day based on 10-14 exabytes of data, culminating in a daily condensed dose of info in the petabyte range. To do this will require well over what the exascale machines of the 2020 timeframe will offer but there’s another problem. The signals are being collected at the most radio wave-free locations one earth,  which happen to be places where there’s no power grid or internet.

This was the perfect set of conditions for IBM and SKA/ASTRON researchers to think outside of the power-hungry boxes that are required to feed this kind of science. And the perfect opportunity for an ultra low-power approach that recognizes that the compute is easy–it’s the data movement that’s the real power drain. Since altering the speed of light is out of the question, the only answer seems to be integrating as much as possible into a neat whole. While some of that technology still needs to mature (particularly in areas like stacked memory), Luitjen was able to demonstrate how big compute and little movement can be lashed together for maximum efficiency and multiple workloads.

But this isn’t all in the name of grand science. In addition to seeing a path to helping SKA with its noble mission, IBM too was able to see a path to meeting the “compute is free but data is not” paradigm. Luijten says their needs were specific; they wanted to see a microserver that could provide an ultra low-power “datacenter in a box” that could leverage commodity parts and condensed packaging. Further, it would have to be true 64-bit to be of commercial value (which meant no ARM since it wasn’t on the near horizon then), and would have to run a server-class operating system.

Building off the lesson learned during his Sheeva Plug jaunt, Luijten set to work with the one and only 64-bit chip on the market. In this case, it was the P5020 chip from Freescale—a product made specifically for the embedded market, thus without any of the software required for doing anything other than powering small devices operating on custom code. He says the Linux that came in the box was limited and he couldn’t even run the compiler. There was certainly no OS to meet IBM’s eventual needs, but with the help of a colleague and folks at Freescale, Luijten was able to get Fedora up and running on the 2.0 GHz Power-based architecture. And so the DOME Microserver was born.

fedora_bootGetting Fedora to sing on the DOME was one the first hurdle; the absence of an ecosystem was an incredible challenge and multiple iterations of attempting the use of different OS approaches that blended server and embedded realms. He imagined that finally being able to implement a functional server-class OS would be half of the trouble–that the real challenges were ahead in being able to build some functionality application-wise around that.

However, to Luijten’s surprise, just two days after the Fedora success, they were able to get IBM’s DB2 up and running on the tiny motherboard. Without compiling. This is indeed the same DB2 that requires ultra-pricey System X datacenters at a much greater up-front and of course, operational/power cost.

Luijten relayed a quick story about how he had a chat with upper management on the development side at IBM about what they were able to do and he flat-out denied it was possible. “He probably still doesn’t believe it to this day,” he laughed. But sure enough, he said, they had a program that was running for weeks on a single node end atop DB2 with a PHP app on a web browser that could kick through a basket of workloads on the Freescale-carried DB2 engine, all at around 55 watts.

DOMEThe very small team (just Luijten, another comrade and a group of researchers at Freescale) grabbed the chance to take hold of the new incarnation of the chip, which moved them from dual-core to 12 cores—a major leap that didn’t require a recompile to run DB2 again. The newest part, the T4240 runs at 60 watts but comes with some major enhancements to his aims in terms of threading (this is “true threading” he says, not hyperthreading), bumps to three memory channels, and moves them down to 28nm (versus 45 nm).

comparison_slide

The datacenter in a box approach with 128 of these boards using the newest chip yields 1536 cores and 3072 threads with between 3 or 6 TB of DRAM with a novel hot water cooling (ala SuperMUC) installation makes this a rather compelling idea for cloud datacenters and of course, for power-aware, poor folks who want to their commercial or research applications to run in a lightweight, cheap way. As for HPC, it’s all about potential and possibilities at this point versus anything practical. Again, this is a proof of concept project. Benchmark results and scaling capabilities will be forthcoming, but for anyone who wants a firsthand lesson in some of the lessons of a non-existent software ecosystem, the ARM guys aren’t the only ones to look to for war stories.

Just as a side note, while sitting with Luijten at the IDC User Forum this week, we set the little server node motherboard next to my iPhone—it was just a tad longer, do some mental comparisons for size scale or take a look below at his part versus a BlueGene board. Sitting this next to a Calexda or Moonshot offers about the same viewing experience.

comparison_shmarison

Microservers should package the entire server node motherboard into a single microchip, leaving off some elements that wouldn’t make sense (including DRAM, power conversion logic and NOR Flash since they don’t fit), says Luijten. There are many motherboards that have graphics and such, but this is pared down.

And yes, this was from a conversation at an HPC-centric event, which might strike some of you as a bit strange. Luijten says that he definitely does not do HPC but Earl Joseph believes strongly that the DOME microserver project is a perfect example of the type of technology that could be disruptive to the industry going forward. It’s power constrained, price-aware, and performance-oriented. While the specs on the flops front are in short order (you can do some quick math based on what Freescale has made available—not shabby for the size and power envelope), Joseph is spot-on. This was one of the more compelling presentations during the two days in Santa Fe and based on sideline conversations, one of the most widely-discussed.

It should be noted that these aren’t coming to a rack near you anytime soon. It’s still a research project, but it’s one that Freescale isn’t taking lightly, even if it’s not been as mainstream at IBM as Luijten might like to see one day. This would make a pretty compelling cloud server for Freescale and they’re working with him now to run some benchmarks to get a better baseline on the performance capabilities that will be shared in a press release eventually.

What IBM will do with the eventual success or interest in the concept on the development side remains anyone’s best guess—especially as the first drums of the ARM invasion can be heard beating in the not-so-far distance. “IBM sold off its SystemX business is because the moment a technology becomes commodity, they get out of the game,” Luijten  reflected. They can’t sustain a business on driving a commodity market, hence they’re looking now to things like cognitive computing, among other efforts.

He says that while IBM is not incredibly interested in what he’s working on now, at least in any serious product-driven way, he’s found that with research like this, it helps to be more than just a good technical engineer. “Someone said I’m like an entrepreneur,” he laughed. “It’s not enough to develop this technology, it has to be marketed and you have to find interest however you can.”

We’ll close with the most recent development/progress via one of his slides. And of course, we’ll continue to watch this, even if it’s remote from the HPC we’re looking at now.

status_slide

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Nvidia Leads Alpha MLPerf Benchmarking Round

December 12, 2018

Seven months after the launch of its AI benchmarking suite, the MLPerf consortium is releasing the first round of results based on submissions from Nvidia, Google and Intel. Of the seven benchmarks encompassed in version Read more…

By Tiffany Trader

Neural Network ‘Synapse’ Technology Showcased at IEEE Meeting

December 12, 2018

There’s nice snapshot of advancing work to develop improved neural network “synapse” technologies posted yesterday on IEEE Spectrum. Lower power, ease of use, manufacturability, and performance are all key paramete Read more…

By John Russell

IBM, Nvidia in AI Data Pipeline, Processing, Storage Union

December 11, 2018

IBM and Nvidia today announced a new turnkey AI solution that combines IBM Spectrum Scale scale-out file storage with Nvidia’s GPU-based DGX-1 AI server to provide what the companies call the “the highest performance Read more…

By Doug Black

HPE Extreme Performance Solutions

AI Can Be Scary. But Choosing the Wrong Partners Can Be Mortifying!

As you continue to dive deeper into AI, you will discover it is more than just deep learning. AI is an extremely complex set of machine learning, deep learning, reinforcement, and analytics algorithms with varying compute, storage, memory, and communications needs. Read more…

IBM Accelerated Insights

Blurring the Lines Between HPC and AI @ SC18

The dominant topic at SC18 was the convergence of HPC and Artificial Intelligence (AI) with some of the biggest research and enterprise HPC users providing perspectives on how HPC and AI are moving closer together. Read more…

Is Amazon’s Plunge into Server Chips a Watershed Moment?

December 11, 2018

For several years now the big cloud providers – Amazon, Microsoft Azure, Google, et al – have been transforming from technology consumers into technology creators in hardware and software. The most recent example bei Read more…

By John Russell

Nvidia Leads Alpha MLPerf Benchmarking Round

December 12, 2018

Seven months after the launch of its AI benchmarking suite, the MLPerf consortium is releasing the first round of results based on submissions from Nvidia, Goog Read more…

By Tiffany Trader

IBM, Nvidia in AI Data Pipeline, Processing, Storage Union

December 11, 2018

IBM and Nvidia today announced a new turnkey AI solution that combines IBM Spectrum Scale scale-out file storage with Nvidia’s GPU-based DGX-1 AI server to pr Read more…

By Doug Black

Is Amazon’s Plunge into Server Chips a Watershed Moment?

December 11, 2018

For several years now the big cloud providers – Amazon, Microsoft Azure, Google, et al – have been transforming from technology consumers into technology cr Read more…

By John Russell

Mellanox Uses Univa to Extend Silicon Design HPC Operation to Azure

December 11, 2018

Call it a corollary to Murphy’s Law: When a system is most in demand, when end users are most dependent on the system performing as required, when it’s crunch time – that’s when the system is most likely to blow up. Or make you wait in line to use it. Read more…

By Doug Black

Topology Can Help Us Find Patterns in Weather

December 6, 2018

Topology--the study of shapes--seems to be all the rage. You could even say that data has shape, and shape matters. Shapes are comfortable and familiar concepts, so it is intriguing to see that many applications are being recast to use topology. For instance, looking for weather and climate patterns. Read more…

By James Reinders

Zettascale by 2035? China Thinks So

December 6, 2018

Exascale machines (of at least a 1 exaflops peak) are anticipated to arrive by around 2020, a few years behind original predictions; and given extreme-scale performance challenges are not getting any easier, it makes sense that researchers are already looking ahead to the next big 1,000x performance goal post: zettascale computing. Read more…

By Tiffany Trader

Robust Quantum Computers Still a Decade Away, Says Nat’l Academies Report

December 5, 2018

The National Academies of Science, Engineering, and Medicine yesterday released a report – Quantum Computing: Progress and Prospects – whose optimism about Read more…

By John Russell

Revisiting the 2008 Exascale Computing Study at SC18

November 29, 2018

A report published a decade ago conveyed the results of a study aimed at determining if it were possible to achieve 1000X the computational power of the the Read more…

By Scott Gibson

Quantum Computing Will Never Work

November 27, 2018

Amid the gush of money and enthusiastic predictions being thrown at quantum computing comes a proposed cold shower in the form of an essay by physicist Mikhail Read more…

By John Russell

Cray Unveils Shasta, Lands NERSC-9 Contract

October 30, 2018

Cray revealed today the details of its next-gen supercomputing architecture, Shasta, selected to be the next flagship system at NERSC. We've known of the code-name "Shasta" since the Argonne slice of the CORAL project was announced in 2015 and although the details of that plan have changed considerably, Cray didn't slow down its timeline for Shasta. Read more…

By Tiffany Trader

IBM at Hot Chips: What’s Next for Power

August 23, 2018

With processor, memory and networking technologies all racing to fill in for an ailing Moore’s law, the era of the heterogeneous datacenter is well underway, Read more…

By Tiffany Trader

House Passes $1.275B National Quantum Initiative

September 17, 2018

Last Thursday the U.S. House of Representatives passed the National Quantum Initiative Act (NQIA) intended to accelerate quantum computing research and developm Read more…

By John Russell

Summit Supercomputer is Already Making its Mark on Science

September 20, 2018

Summit, now the fastest supercomputer in the world, is quickly making its mark in science – five of the six finalists just announced for the prestigious 2018 Read more…

By John Russell

AMD Sets Up for Epyc Epoch

November 16, 2018

It’s been a good two weeks, AMD’s Gary Silcott and Andy Parma told me on the last day of SC18 in Dallas at the restaurant where we met to discuss their show news and recent successes. Heck, it’s been a good year. Read more…

By Tiffany Trader

US Leads Supercomputing with #1, #2 Systems & Petascale Arm

November 12, 2018

The 31st Supercomputing Conference (SC) - commemorating 30 years since the first Supercomputing in 1988 - kicked off in Dallas yesterday, taking over the Kay Ba Read more…

By Tiffany Trader

CERN Project Sees Orders-of-Magnitude Speedup with AI Approach

August 14, 2018

An award-winning effort at CERN has demonstrated potential to significantly change how the physics based modeling and simulation communities view machine learni Read more…

By Rob Farber

Leading Solution Providers

SC 18 Virtual Booth Video Tour

Advania @ SC18 AMD @ SC18
ASRock Rack @ SC18
DDN Storage @ SC18
HPE @ SC18
IBM @ SC18
Lenovo @ SC18 Mellanox Technologies @ SC18
NVIDIA @ SC18
One Stop Systems @ SC18
Oracle @ SC18 Panasas @ SC18
Supermicro @ SC18 SUSE @ SC18 TYAN @ SC18
Verne Global @ SC18

TACC’s ‘Frontera’ Supercomputer Expands Horizon for Extreme-Scale Science

August 29, 2018

The National Science Foundation and the Texas Advanced Computing Center announced today that a new system, called Frontera, will overtake Stampede 2 as the fast Read more…

By Tiffany Trader

HPE No. 1, IBM Surges, in ‘Bucking Bronco’ High Performance Server Market

September 27, 2018

Riding healthy U.S. and global economies, strong demand for AI-capable hardware and other tailwind trends, the high performance computing server market jumped 28 percent in the second quarter 2018 to $3.7 billion, up from $2.9 billion for the same period last year, according to industry analyst firm Hyperion Research. Read more…

By Doug Black

Nvidia’s Jensen Huang Delivers Vision for the New HPC

November 14, 2018

For nearly two hours on Monday at SC18, Jensen Huang, CEO of Nvidia, presented his expansive view of the future of HPC (and computing in general) as only he can do. Animated. Backstopped by a stream of data charts, product photos, and even a beautiful image of supernovae... Read more…

By John Russell

Germany Celebrates Launch of Two Fastest Supercomputers

September 26, 2018

The new high-performance computer SuperMUC-NG at the Leibniz Supercomputing Center (LRZ) in Garching is the fastest computer in Germany and one of the fastest i Read more…

By Tiffany Trader

Houston to Field Massive, ‘Geophysically Configured’ Cloud Supercomputer

October 11, 2018

Based on some news stories out today, one might get the impression that the next system to crack number one on the Top500 would be an industrial oil and gas mon Read more…

By Tiffany Trader

Intel Confirms 48-Core Cascade Lake-AP for 2019

November 4, 2018

As part of the run-up to SC18, taking place in Dallas next week (Nov. 11-16), Intel is doling out info on its next-gen Cascade Lake family of Xeon processors, specifically the “Advanced Processor” version (Cascade Lake-AP), architected for high-performance computing, artificial intelligence and infrastructure-as-a-service workloads. Read more…

By Tiffany Trader

Google Releases Machine Learning “What-If” Analysis Tool

September 12, 2018

Training machine learning models has long been time-consuming process. Yesterday, Google released a “What-If Tool” for probing how data point changes affect a model’s prediction. The new tool is being launched as a new feature of the open source TensorBoard web application... Read more…

By John Russell

The Convergence of Big Data and Extreme-Scale HPC

August 31, 2018

As we are heading towards extreme-scale HPC coupled with data intensive analytics like machine learning, the necessary integration of big data and HPC is a curr Read more…

By Rob Farber

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This