Big Science, Tiny Microservers: IBM Research Pushes 64-Bit Possibilities

By Nicole Hemsoth

April 10, 2014

Four years ago, a friend dropped a Sheeva Plug into the hands of Ronald Luijten, a system designer at IBM Research in Zurich. At the time, neither could have realized the development cycle this simple gift would spark.

If you’re not familiar, Sheeva Plugs are compact devices that look a lot like your laptop power adapter, except instead of an electrical output plug, there’s a handy gigabit Ethernet port. Luitjen, whose primary interests lie in data movement and energy management, immediately saw the potential. He put his minimalist inclinations to work, and within a few months, had a VNC, an OS and a web server running from a USB attached hard drive. What struck him the most, however, was when he measured it from the mains and found the whole thing was running at a mere 4.3 watts. “I couldn’t believe this,” he said. “When I thought about it further, I saw it was the beginning of a revolution.”

This discovery coincided with a much larger project Luitjen was involved with at IBM Research. In conjunction with ASTRON, a team tapped some of Big Blue’s best minds to help the Square Kilometer Array (SKA) team discover new solutions to solve the unprecedented power, compute and data movement challenges inherent to measuring the Big Bang. Over the next decade, SKA researchers will be able to look back 13.8 billion years (and over a billion dollars) with 2 million antennae that will pull together a signal at the end of each day based on 10-14 exabytes of data, culminating in a daily condensed dose of info in the petabyte range. To do this will require well over what the exascale machines of the 2020 timeframe will offer but there’s another problem. The signals are being collected at the most radio wave-free locations one earth,  which happen to be places where there’s no power grid or internet.

This was the perfect set of conditions for IBM and SKA/ASTRON researchers to think outside of the power-hungry boxes that are required to feed this kind of science. And the perfect opportunity for an ultra low-power approach that recognizes that the compute is easy–it’s the data movement that’s the real power drain. Since altering the speed of light is out of the question, the only answer seems to be integrating as much as possible into a neat whole. While some of that technology still needs to mature (particularly in areas like stacked memory), Luitjen was able to demonstrate how big compute and little movement can be lashed together for maximum efficiency and multiple workloads.

But this isn’t all in the name of grand science. In addition to seeing a path to helping SKA with its noble mission, IBM too was able to see a path to meeting the “compute is free but data is not” paradigm. Luijten says their needs were specific; they wanted to see a microserver that could provide an ultra low-power “datacenter in a box” that could leverage commodity parts and condensed packaging. Further, it would have to be true 64-bit to be of commercial value (which meant no ARM since it wasn’t on the near horizon then), and would have to run a server-class operating system.

Building off the lesson learned during his Sheeva Plug jaunt, Luijten set to work with the one and only 64-bit chip on the market. In this case, it was the P5020 chip from Freescale—a product made specifically for the embedded market, thus without any of the software required for doing anything other than powering small devices operating on custom code. He says the Linux that came in the box was limited and he couldn’t even run the compiler. There was certainly no OS to meet IBM’s eventual needs, but with the help of a colleague and folks at Freescale, Luijten was able to get Fedora up and running on the 2.0 GHz Power-based architecture. And so the DOME Microserver was born.

fedora_bootGetting Fedora to sing on the DOME was one the first hurdle; the absence of an ecosystem was an incredible challenge and multiple iterations of attempting the use of different OS approaches that blended server and embedded realms. He imagined that finally being able to implement a functional server-class OS would be half of the trouble–that the real challenges were ahead in being able to build some functionality application-wise around that.

However, to Luijten’s surprise, just two days after the Fedora success, they were able to get IBM’s DB2 up and running on the tiny motherboard. Without compiling. This is indeed the same DB2 that requires ultra-pricey System X datacenters at a much greater up-front and of course, operational/power cost.

Luijten relayed a quick story about how he had a chat with upper management on the development side at IBM about what they were able to do and he flat-out denied it was possible. “He probably still doesn’t believe it to this day,” he laughed. But sure enough, he said, they had a program that was running for weeks on a single node end atop DB2 with a PHP app on a web browser that could kick through a basket of workloads on the Freescale-carried DB2 engine, all at around 55 watts.

DOMEThe very small team (just Luijten, another comrade and a group of researchers at Freescale) grabbed the chance to take hold of the new incarnation of the chip, which moved them from dual-core to 12 cores—a major leap that didn’t require a recompile to run DB2 again. The newest part, the T4240 runs at 60 watts but comes with some major enhancements to his aims in terms of threading (this is “true threading” he says, not hyperthreading), bumps to three memory channels, and moves them down to 28nm (versus 45 nm).

comparison_slide

The datacenter in a box approach with 128 of these boards using the newest chip yields 1536 cores and 3072 threads with between 3 or 6 TB of DRAM with a novel hot water cooling (ala SuperMUC) installation makes this a rather compelling idea for cloud datacenters and of course, for power-aware, poor folks who want to their commercial or research applications to run in a lightweight, cheap way. As for HPC, it’s all about potential and possibilities at this point versus anything practical. Again, this is a proof of concept project. Benchmark results and scaling capabilities will be forthcoming, but for anyone who wants a firsthand lesson in some of the lessons of a non-existent software ecosystem, the ARM guys aren’t the only ones to look to for war stories.

Just as a side note, while sitting with Luijten at the IDC User Forum this week, we set the little server node motherboard next to my iPhone—it was just a tad longer, do some mental comparisons for size scale or take a look below at his part versus a BlueGene board. Sitting this next to a Calexda or Moonshot offers about the same viewing experience.

comparison_shmarison

Microservers should package the entire server node motherboard into a single microchip, leaving off some elements that wouldn’t make sense (including DRAM, power conversion logic and NOR Flash since they don’t fit), says Luijten. There are many motherboards that have graphics and such, but this is pared down.

And yes, this was from a conversation at an HPC-centric event, which might strike some of you as a bit strange. Luijten says that he definitely does not do HPC but Earl Joseph believes strongly that the DOME microserver project is a perfect example of the type of technology that could be disruptive to the industry going forward. It’s power constrained, price-aware, and performance-oriented. While the specs on the flops front are in short order (you can do some quick math based on what Freescale has made available—not shabby for the size and power envelope), Joseph is spot-on. This was one of the more compelling presentations during the two days in Santa Fe and based on sideline conversations, one of the most widely-discussed.

It should be noted that these aren’t coming to a rack near you anytime soon. It’s still a research project, but it’s one that Freescale isn’t taking lightly, even if it’s not been as mainstream at IBM as Luijten might like to see one day. This would make a pretty compelling cloud server for Freescale and they’re working with him now to run some benchmarks to get a better baseline on the performance capabilities that will be shared in a press release eventually.

What IBM will do with the eventual success or interest in the concept on the development side remains anyone’s best guess—especially as the first drums of the ARM invasion can be heard beating in the not-so-far distance. “IBM sold off its SystemX business is because the moment a technology becomes commodity, they get out of the game,” Luijten  reflected. They can’t sustain a business on driving a commodity market, hence they’re looking now to things like cognitive computing, among other efforts.

He says that while IBM is not incredibly interested in what he’s working on now, at least in any serious product-driven way, he’s found that with research like this, it helps to be more than just a good technical engineer. “Someone said I’m like an entrepreneur,” he laughed. “It’s not enough to develop this technology, it has to be marketed and you have to find interest however you can.”

We’ll close with the most recent development/progress via one of his slides. And of course, we’ll continue to watch this, even if it’s remote from the HPC we’re looking at now.

status_slide

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Exascale Computing Project Names Doug Kothe as Director

September 20, 2017

The Department of Energy’s Exascale Computing Project (ECP) has named Doug Kothe as its new director effective October 1. He replaces Paul Messina, who is stepping down after two years to return to Argonne National L Read more…

Takeaways from the Milwaukee HPC User Forum

September 19, 2017

Milwaukee’s elegant Pfister Hotel hosted approximately 100 attendees for the 66th HPC User Forum (September 5-7, 2017). In the original home city of Pabst Blue Ribbon and Harley Davidson motorcycles the agenda addresse Read more…

By Merle Giles

NSF Awards $10M to Extend Chameleon Cloud Testbed Project

September 19, 2017

The National Science Foundation has awarded a second phase, $10 million grant to the Chameleon cloud computing testbed project led by University of Chicago with partners at the Texas Advanced Computing Center (TACC), Ren Read more…

By John Russell

HPE Extreme Performance Solutions

HPE Prepares Customers for Success with the HPC Software Portfolio

High performance computing (HPC) software is key to harnessing the full power of HPC environments. Development and management tools enable IT departments to streamline installation and maintenance of their systems as well as create, optimize, and run their HPC applications. Read more…

NERSC Simulations Shed Light on Fusion Reaction Turbulence

September 19, 2017

Understanding fusion reactions in detail – particularly plasma turbulence – is critical to the effort to bring fusion power to reality. Recent work including roughly 70 million hours of compute time at the National E Read more…

Exascale Computing Project Names Doug Kothe as Director

September 20, 2017

The Department of Energy’s Exascale Computing Project (ECP) has named Doug Kothe as its new director effective October 1. He replaces Paul Messina, who is s Read more…

Takeaways from the Milwaukee HPC User Forum

September 19, 2017

Milwaukee’s elegant Pfister Hotel hosted approximately 100 attendees for the 66th HPC User Forum (September 5-7, 2017). In the original home city of Pabst Blu Read more…

By Merle Giles

Kathy Yelick Charts the Promise and Progress of Exascale Science

September 15, 2017

On Friday, Sept. 8, Kathy Yelick of Lawrence Berkeley National Laboratory and the University of California, Berkeley, delivered the keynote address on “Breakt Read more…

By Tiffany Trader

DARPA Pledges Another $300 Million for Post-Moore’s Readiness

September 14, 2017

The Defense Advanced Research Projects Agency (DARPA) launched a giant funding effort to ensure the United States can sustain the pace of electronic innovation vital to both a flourishing economy and a secure military. Under the banner of the Electronics Resurgence Initiative (ERI), some $500-$800 million will be invested in post-Moore’s Law technologies. Read more…

By Tiffany Trader

IBM Breaks Ground for Complex Quantum Chemistry

September 14, 2017

IBM has reported the use of a novel algorithm to simulate BeH2 (beryllium-hydride) on a quantum computer. This is the largest molecule so far simulated on a quantum computer. The technique, which used six qubits of a seven-qubit system, is an important step forward and may suggest an approach to simulating ever larger molecules. Read more…

By John Russell

Cubes, Culture, and a New Challenge: Trish Damkroger Talks about Life at Intel—and Why HPC Matters More Than Ever

September 13, 2017

Trish Damkroger wasn’t looking to change jobs when she attended SC15 in Austin, Texas. Capping a 15-year career within Department of Energy (DOE) laboratories, she was acting Associate Director for Computation at Lawrence Livermore National Laboratory (LLNL). Her mission was to equip the lab’s scientists and research partners with resources that would advance their cutting-edge work... Read more…

By Jan Rowell

EU Funds 20 Million Euro ARM+FPGA Exascale Project

September 7, 2017

At the Barcelona Supercomputer Centre on Wednesday (Sept. 6), 16 partners gathered to launch the EuroEXA project, which invests €20 million over three-and-a-half years into exascale-focused research and development. Led by the Horizon 2020 program, EuroEXA picks up the banner of a triad of partner projects — ExaNeSt, EcoScale and ExaNoDe — building on their work... Read more…

By Tiffany Trader

MIT-IBM Watson AI Lab Targets Algorithms, AI Physics

September 7, 2017

Investment continues to flow into artificial intelligence research, especially in key areas such as AI algorithms that promise to move the technology from speci Read more…

By George Leopold

How ‘Knights Mill’ Gets Its Deep Learning Flops

June 22, 2017

Intel, the subject of much speculation regarding the delayed, rewritten or potentially canceled “Aurora” contract (the Argonne Lab part of the CORAL “ Read more…

By Tiffany Trader

Reinders: “AVX-512 May Be a Hidden Gem” in Intel Xeon Scalable Processors

June 29, 2017

Imagine if we could use vector processing on something other than just floating point problems.  Today, GPUs and CPUs work tirelessly to accelerate algorithms Read more…

By James Reinders

NERSC Scales Scientific Deep Learning to 15 Petaflops

August 28, 2017

A collaborative effort between Intel, NERSC and Stanford has delivered the first 15-petaflops deep learning software running on HPC platforms and is, according Read more…

By Rob Farber

Russian Researchers Claim First Quantum-Safe Blockchain

May 25, 2017

The Russian Quantum Center today announced it has overcome the threat of quantum cryptography by creating the first quantum-safe blockchain, securing cryptocurrencies like Bitcoin, along with classified government communications and other sensitive digital transfers. Read more…

By Doug Black

Oracle Layoffs Reportedly Hit SPARC and Solaris Hard

September 7, 2017

Oracle’s latest layoffs have many wondering if this is the end of the line for the SPARC processor and Solaris OS development. As reported by multiple sources Read more…

By John Russell

Google Debuts TPU v2 and will Add to Google Cloud

May 25, 2017

Not long after stirring attention in the deep learning/AI community by revealing the details of its Tensor Processing Unit (TPU), Google last week announced the Read more…

By John Russell

Six Exascale PathForward Vendors Selected; DoE Providing $258M

June 15, 2017

The much-anticipated PathForward awards for hardware R&D in support of the Exascale Computing Project were announced today with six vendors selected – AMD Read more…

By John Russell

Top500 Results: Latest List Trends and What’s in Store

June 19, 2017

Greetings from Frankfurt and the 2017 International Supercomputing Conference where the latest Top500 list has just been revealed. Although there were no major Read more…

By Tiffany Trader

Leading Solution Providers

IBM Clears Path to 5nm with Silicon Nanosheets

June 5, 2017

Two years since announcing the industry’s first 7nm node test chip, IBM and its research alliance partners GlobalFoundries and Samsung have developed a proces Read more…

By Tiffany Trader

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Graphcore Readies Launch of 16nm Colossus-IPU Chip

July 20, 2017

A second $30 million funding round for U.K. AI chip developer Graphcore sets up the company to go to market with its “intelligent processing unit” (IPU) in Read more…

By Tiffany Trader

Google Releases Deeplearn.js to Further Democratize Machine Learning

August 17, 2017

Spreading the use of machine learning tools is one of the goals of Google’s PAIR (People + AI Research) initiative, which was introduced in early July. Last w Read more…

By John Russell

EU Funds 20 Million Euro ARM+FPGA Exascale Project

September 7, 2017

At the Barcelona Supercomputer Centre on Wednesday (Sept. 6), 16 partners gathered to launch the EuroEXA project, which invests €20 million over three-and-a-half years into exascale-focused research and development. Led by the Horizon 2020 program, EuroEXA picks up the banner of a triad of partner projects — ExaNeSt, EcoScale and ExaNoDe — building on their work... Read more…

By Tiffany Trader

Amazon Debuts New AMD-based GPU Instances for Graphics Acceleration

September 12, 2017

Last week Amazon Web Services (AWS) streaming service, AppStream 2.0, introduced a new GPU instance called Graphics Design intended to accelerate graphics. The Read more…

By John Russell

Cray Moves to Acquire the Seagate ClusterStor Line

July 28, 2017

This week Cray announced that it is picking up Seagate's ClusterStor HPC storage array business for an undisclosed sum. "In short we're effectively transitioning the bulk of the ClusterStor product line to Cray," said CEO Peter Ungaro. Read more…

By Tiffany Trader

IBM Advances Web-based Quantum Programming

September 5, 2017

IBM Research is pairing its Jupyter-based Data Science Experience notebook environment with its cloud-based quantum computer, IBM Q, in hopes of encouraging a new class of entrepreneurial user to solve intractable problems that even exceed the capabilities of the best AI systems. Read more…

By Alex Woodie

  • arrow
  • Click Here for More Headlines
  • arrow
Share This