Big Science, Tiny Microservers: IBM Research Pushes 64-Bit Possibilities

By Nicole Hemsoth

April 10, 2014

Four years ago, a friend dropped a Sheeva Plug into the hands of Ronald Luijten, a system designer at IBM Research in Zurich. At the time, neither could have realized the development cycle this simple gift would spark.

If you’re not familiar, Sheeva Plugs are compact devices that look a lot like your laptop power adapter, except instead of an electrical output plug, there’s a handy gigabit Ethernet port. Luitjen, whose primary interests lie in data movement and energy management, immediately saw the potential. He put his minimalist inclinations to work, and within a few months, had a VNC, an OS and a web server running from a USB attached hard drive. What struck him the most, however, was when he measured it from the mains and found the whole thing was running at a mere 4.3 watts. “I couldn’t believe this,” he said. “When I thought about it further, I saw it was the beginning of a revolution.”

This discovery coincided with a much larger project Luitjen was involved with at IBM Research. In conjunction with ASTRON, a team tapped some of Big Blue’s best minds to help the Square Kilometer Array (SKA) team discover new solutions to solve the unprecedented power, compute and data movement challenges inherent to measuring the Big Bang. Over the next decade, SKA researchers will be able to look back 13.8 billion years (and over a billion dollars) with 2 million antennae that will pull together a signal at the end of each day based on 10-14 exabytes of data, culminating in a daily condensed dose of info in the petabyte range. To do this will require well over what the exascale machines of the 2020 timeframe will offer but there’s another problem. The signals are being collected at the most radio wave-free locations one earth,  which happen to be places where there’s no power grid or internet.

This was the perfect set of conditions for IBM and SKA/ASTRON researchers to think outside of the power-hungry boxes that are required to feed this kind of science. And the perfect opportunity for an ultra low-power approach that recognizes that the compute is easy–it’s the data movement that’s the real power drain. Since altering the speed of light is out of the question, the only answer seems to be integrating as much as possible into a neat whole. While some of that technology still needs to mature (particularly in areas like stacked memory), Luitjen was able to demonstrate how big compute and little movement can be lashed together for maximum efficiency and multiple workloads.

But this isn’t all in the name of grand science. In addition to seeing a path to helping SKA with its noble mission, IBM too was able to see a path to meeting the “compute is free but data is not” paradigm. Luijten says their needs were specific; they wanted to see a microserver that could provide an ultra low-power “datacenter in a box” that could leverage commodity parts and condensed packaging. Further, it would have to be true 64-bit to be of commercial value (which meant no ARM since it wasn’t on the near horizon then), and would have to run a server-class operating system.

Building off the lesson learned during his Sheeva Plug jaunt, Luijten set to work with the one and only 64-bit chip on the market. In this case, it was the P5020 chip from Freescale—a product made specifically for the embedded market, thus without any of the software required for doing anything other than powering small devices operating on custom code. He says the Linux that came in the box was limited and he couldn’t even run the compiler. There was certainly no OS to meet IBM’s eventual needs, but with the help of a colleague and folks at Freescale, Luijten was able to get Fedora up and running on the 2.0 GHz Power-based architecture. And so the DOME Microserver was born.

fedora_bootGetting Fedora to sing on the DOME was one the first hurdle; the absence of an ecosystem was an incredible challenge and multiple iterations of attempting the use of different OS approaches that blended server and embedded realms. He imagined that finally being able to implement a functional server-class OS would be half of the trouble–that the real challenges were ahead in being able to build some functionality application-wise around that.

However, to Luijten’s surprise, just two days after the Fedora success, they were able to get IBM’s DB2 up and running on the tiny motherboard. Without compiling. This is indeed the same DB2 that requires ultra-pricey System X datacenters at a much greater up-front and of course, operational/power cost.

Luijten relayed a quick story about how he had a chat with upper management on the development side at IBM about what they were able to do and he flat-out denied it was possible. “He probably still doesn’t believe it to this day,” he laughed. But sure enough, he said, they had a program that was running for weeks on a single node end atop DB2 with a PHP app on a web browser that could kick through a basket of workloads on the Freescale-carried DB2 engine, all at around 55 watts.

DOMEThe very small team (just Luijten, another comrade and a group of researchers at Freescale) grabbed the chance to take hold of the new incarnation of the chip, which moved them from dual-core to 12 cores—a major leap that didn’t require a recompile to run DB2 again. The newest part, the T4240 runs at 60 watts but comes with some major enhancements to his aims in terms of threading (this is “true threading” he says, not hyperthreading), bumps to three memory channels, and moves them down to 28nm (versus 45 nm).

comparison_slide

The datacenter in a box approach with 128 of these boards using the newest chip yields 1536 cores and 3072 threads with between 3 or 6 TB of DRAM with a novel hot water cooling (ala SuperMUC) installation makes this a rather compelling idea for cloud datacenters and of course, for power-aware, poor folks who want to their commercial or research applications to run in a lightweight, cheap way. As for HPC, it’s all about potential and possibilities at this point versus anything practical. Again, this is a proof of concept project. Benchmark results and scaling capabilities will be forthcoming, but for anyone who wants a firsthand lesson in some of the lessons of a non-existent software ecosystem, the ARM guys aren’t the only ones to look to for war stories.

Just as a side note, while sitting with Luijten at the IDC User Forum this week, we set the little server node motherboard next to my iPhone—it was just a tad longer, do some mental comparisons for size scale or take a look below at his part versus a BlueGene board. Sitting this next to a Calexda or Moonshot offers about the same viewing experience.

comparison_shmarison

Microservers should package the entire server node motherboard into a single microchip, leaving off some elements that wouldn’t make sense (including DRAM, power conversion logic and NOR Flash since they don’t fit), says Luijten. There are many motherboards that have graphics and such, but this is pared down.

And yes, this was from a conversation at an HPC-centric event, which might strike some of you as a bit strange. Luijten says that he definitely does not do HPC but Earl Joseph believes strongly that the DOME microserver project is a perfect example of the type of technology that could be disruptive to the industry going forward. It’s power constrained, price-aware, and performance-oriented. While the specs on the flops front are in short order (you can do some quick math based on what Freescale has made available—not shabby for the size and power envelope), Joseph is spot-on. This was one of the more compelling presentations during the two days in Santa Fe and based on sideline conversations, one of the most widely-discussed.

It should be noted that these aren’t coming to a rack near you anytime soon. It’s still a research project, but it’s one that Freescale isn’t taking lightly, even if it’s not been as mainstream at IBM as Luijten might like to see one day. This would make a pretty compelling cloud server for Freescale and they’re working with him now to run some benchmarks to get a better baseline on the performance capabilities that will be shared in a press release eventually.

What IBM will do with the eventual success or interest in the concept on the development side remains anyone’s best guess—especially as the first drums of the ARM invasion can be heard beating in the not-so-far distance. “IBM sold off its SystemX business is because the moment a technology becomes commodity, they get out of the game,” Luijten  reflected. They can’t sustain a business on driving a commodity market, hence they’re looking now to things like cognitive computing, among other efforts.

He says that while IBM is not incredibly interested in what he’s working on now, at least in any serious product-driven way, he’s found that with research like this, it helps to be more than just a good technical engineer. “Someone said I’m like an entrepreneur,” he laughed. “It’s not enough to develop this technology, it has to be marketed and you have to find interest however you can.”

We’ll close with the most recent development/progress via one of his slides. And of course, we’ll continue to watch this, even if it’s remote from the HPC we’re looking at now.

status_slide

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

HPC in Life Sciences Part 1: CPU Choices, Rise of Data Lakes, Networking Challenges, and More

February 21, 2019

For the past few years HPCwire and leaders of BioTeam, a research computing consultancy specializing in life sciences, have convened to examine the state of HPC (and now AI) use in life sciences. Without HPC writ large, modern life sciences research would quickly grind to a halt. It’s true most life sciences research computing... Read more…

By John Russell

Arm Unveils Neoverse N1 Platform with up to 128-Cores

February 20, 2019

Following on its Neoverse roadmap announcement last October, Arm today revealed its next-gen Neoverse microarchitecture with compute and throughput-optimized silicon designs catered toward general-purpose cloud computing Read more…

By Tiffany Trader

The Internet of Criminal Things—Trust in the Gods but Verify!

February 20, 2019

“Are we under attack?” asked Professor Elmarie Biermann of the Cyber Security Institute during the recent South African Centre for High Performance Computing’s (CHPC) National Conference in Cape Town. A quick show Read more…

By Elizabeth Leake, STEM-Trek

HPE Extreme Performance Solutions

HPE and Intel® Omni-Path Architecture: How to Power a Cloud

Learn how HPE and Intel® Omni-Path Architecture provide critical infrastructure for leading Nordic HPC provider’s HPCFLOW cloud service.

powercloud_blog.jpgFor decades, HPE has been at the forefront of high-performance computing, and we’ve powered some of the fastest and most robust supercomputers in the world. Read more…

IBM Accelerated Insights

The Perils of Becoming Trapped in the Cloud

Terms like ‘open systems’ have been bandied about for decades. While modern computer systems are relatively open compared to their predecessors, there are still plenty of opportunities to become locked into proprietary interfaces. Read more…

Machine Learning Takes Heat for Science’s Reproducibility Crisis

February 19, 2019

Scientists are raising red flags about the accuracy and reproducibility of conclusions drawn by machine learning frameworks. Among the remedies are developing new ML systems that can question their own predictions, show Read more…

By George Leopold

HPC in Life Sciences Part 1: CPU Choices, Rise of Data Lakes, Networking Challenges, and More

February 21, 2019

For the past few years HPCwire and leaders of BioTeam, a research computing consultancy specializing in life sciences, have convened to examine the state of HPC (and now AI) use in life sciences. Without HPC writ large, modern life sciences research would quickly grind to a halt. It’s true most life sciences research computing... Read more…

By John Russell

Arm Unveils Neoverse N1 Platform with up to 128-Cores

February 20, 2019

Following on its Neoverse roadmap announcement last October, Arm today revealed its next-gen Neoverse microarchitecture with compute and throughput-optimized si Read more…

By Tiffany Trader

Insights from Optimized Codes on Cineca’s Marconi

February 15, 2019

What can you do with 381,392 CPU cores? For Cineca, it means enabling computational scientists to expand a large part of the world’s body of knowledge from the nanoscale to the astronomic, from calculating quantum effects in new materials to supporting bioinformatics for advanced healthcare research to screening millions of possible chemical combinations to attack a deadly virus. Read more…

By Ken Strandberg

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst o Read more…

By Tiffany Trader

UC Berkeley Paper Heralds Rise of Serverless Computing in the Cloud – Do You Agree?

February 13, 2019

Almost exactly ten years to the day from publishing of their widely-read, seminal paper on cloud computing, UC Berkeley researchers have issued another ambitious examination of cloud computing - Cloud Programming Simplified: A Berkeley View on Serverless Computing. The new work heralds the rise of ‘serverless computing’ as the next dominant phase of cloud computing. Read more…

By John Russell

Iowa ‘Grows Its Own’ to Fill the HPC Workforce Pipeline

February 13, 2019

The global workforce that supports advanced computing, scientific software and high-speed research networks is relatively small when you stop to consider the magnitude of the transformative discoveries it empowers. Technical conferences provide a forum where specialists convene to learn about the latest innovations and schedule face-time with colleagues from other institutions. Read more…

By Elizabeth Leake, STEM-Trek

Trump Signs Executive Order Launching U.S. AI Initiative

February 11, 2019

U.S. President Donald Trump issued an Executive Order (EO) today launching a U.S Artificial Intelligence Initiative. The new initiative - Maintaining American L Read more…

By John Russell

Celebrating Women in Science: Meet Four Women Leading the Way in HPC

February 11, 2019

One only needs to look around at virtually any CS/tech conference to realize that women are underrepresented, and that holds true of HPC. SC hosts over 13,000 H Read more…

By AJ Lauer

Quantum Computing Will Never Work

November 27, 2018

Amid the gush of money and enthusiastic predictions being thrown at quantum computing comes a proposed cold shower in the form of an essay by physicist Mikhail Read more…

By John Russell

Cray Unveils Shasta, Lands NERSC-9 Contract

October 30, 2018

Cray revealed today the details of its next-gen supercomputing architecture, Shasta, selected to be the next flagship system at NERSC. We've known of the code-name "Shasta" since the Argonne slice of the CORAL project was announced in 2015 and although the details of that plan have changed considerably, Cray didn't slow down its timeline for Shasta. Read more…

By Tiffany Trader

The Case Against ‘The Case Against Quantum Computing’

January 9, 2019

It’s not easy to be a physicist. Richard Feynman (basically the Jimi Hendrix of physicists) once said: “The first principle is that you must not fool yourse Read more…

By Ben Criger

AMD Sets Up for Epyc Epoch

November 16, 2018

It’s been a good two weeks, AMD’s Gary Silcott and Andy Parma told me on the last day of SC18 in Dallas at the restaurant where we met to discuss their show news and recent successes. Heck, it’s been a good year. Read more…

By Tiffany Trader

Intel Reportedly in $6B Bid for Mellanox

January 30, 2019

The latest rumors and reports around an acquisition of Mellanox focus on Intel, which has reportedly offered a $6 billion bid for the high performance interconn Read more…

By Doug Black

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst o Read more…

By Tiffany Trader

US Leads Supercomputing with #1, #2 Systems & Petascale Arm

November 12, 2018

The 31st Supercomputing Conference (SC) - commemorating 30 years since the first Supercomputing in 1988 - kicked off in Dallas yesterday, taking over the Kay Ba Read more…

By Tiffany Trader

Looking for Light Reading? NSF-backed ‘Comic Books’ Tackle Quantum Computing

January 28, 2019

Still baffled by quantum computing? How about turning to comic books (graphic novels for the well-read among you) for some clarity and a little humor on QC. The Read more…

By John Russell

Leading Solution Providers

SC 18 Virtual Booth Video Tour

Advania @ SC18 AMD @ SC18
ASRock Rack @ SC18
DDN Storage @ SC18
HPE @ SC18
IBM @ SC18
Lenovo @ SC18 Mellanox Technologies @ SC18
NVIDIA @ SC18
One Stop Systems @ SC18
Oracle @ SC18 Panasas @ SC18
Supermicro @ SC18 SUSE @ SC18 TYAN @ SC18
Verne Global @ SC18

Contract Signed for New Finnish Supercomputer

December 13, 2018

After the official contract signing yesterday, configuration details were made public for the new BullSequana system that the Finnish IT Center for Science (CSC Read more…

By Tiffany Trader

Deep500: ETH Researchers Introduce New Deep Learning Benchmark for HPC

February 5, 2019

ETH researchers have developed a new deep learning benchmarking environment – Deep500 – they say is “the first distributed and reproducible benchmarking s Read more…

By John Russell

IBM Quantum Update: Q System One Launch, New Collaborators, and QC Center Plans

January 10, 2019

IBM made three significant quantum computing announcements at CES this week. One was introduction of IBM Q System One; it’s really the integration of IBM’s Read more…

By John Russell

HPC Reflections and (Mostly Hopeful) Predictions

December 19, 2018

So much ‘spaghetti’ gets tossed on walls by the technology community (vendors and researchers) to see what sticks that it is often difficult to peer through Read more…

By John Russell

IBM Bets $2B Seeking 1000X AI Hardware Performance Boost

February 7, 2019

For now, AI systems are mostly machine learning-based and “narrow” – powerful as they are by today's standards, they're limited to performing a few, narro Read more…

By Doug Black

Nvidia’s Jensen Huang Delivers Vision for the New HPC

November 14, 2018

For nearly two hours on Monday at SC18, Jensen Huang, CEO of Nvidia, presented his expansive view of the future of HPC (and computing in general) as only he can do. Animated. Backstopped by a stream of data charts, product photos, and even a beautiful image of supernovae... Read more…

By John Russell

The Deep500 – Researchers Tackle an HPC Benchmark for Deep Learning

January 7, 2019

How do you know if an HPC system, particularly a larger-scale system, is well-suited for deep learning workloads? Today, that’s not an easy question to answer Read more…

By John Russell

Intel Confirms 48-Core Cascade Lake-AP for 2019

November 4, 2018

As part of the run-up to SC18, taking place in Dallas next week (Nov. 11-16), Intel is doling out info on its next-gen Cascade Lake family of Xeon processors, specifically the “Advanced Processor” version (Cascade Lake-AP), architected for high-performance computing, artificial intelligence and infrastructure-as-a-service workloads. Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This