EU Projects Unite on Heterogeneous ARM-based Exascale Prototype

By Tiffany Trader

February 24, 2016

A trio of partner projects based in Europe – Exanest, Exanode and Ecoscale – are working in close collaboration to develop the building blocks for an exascale architecture prototype that will, as they describe, put the power of ten million computers into a single supercomputer. The effort is unique in seeking to advance the ARM64 + FPGA architecture as a foundational “general-purpose” exascale platform.

Funded for three years as part of Europe’s Horizon2020 program, the partners are coordinating their efforts with the goal of building an early “straw man” prototype late this year that will consist of more than one-thousand energy-efficient ARM cores, reconfigurable logic, plus advanced storage, memory, cooling and packaging technologies.

Exanest is the project partner that is focused on the system level, including interconnection, storage, packaging and cooling. And as the name implies, Exanode is responsible for the compute node and the memory of that compute node. Ecoscale focuses on employing and managing reconfigurable logic as accelerators within the system.

Exanest

Manolis Katevenis, the project coordinator for Exanest and head of computer architecture at FORTH-ICS in Greece, explains that Exanest has set an early target of 2016 to build this “relatively-large” first prototype, comprised of at least one-thousand ARM cores.

He says, “We are starting early with a prototype based on existing technology because we want system software to be developed and applications to start being ported and tuned. For the remainder of the two years, there will be ongoing software development, plus research on interconnects, storage and cooling technologies. We also believe that there will be new interesting compute nodes coming out from our partner projects and we will use such nodes.”

In discussing target workloads, Katevenis emphasizes flexibility and breadth, echoing the sentiments we are hearing from across the HPC community. The goal for this platform is to be able to support a range of applications, both on the traditional compute and physics side and the data-intensive side. A look at the Exanest partner list hints at the kind of high-performance applications that will be supported: astrophysics, nuclear physics, simulation-based engineering, and even in-memory databases with partner MonetDB Solutions. Allinea will be providing the ARMv8 profiling and debugging tools.

Although the projects are still in the specification phase, they will be making selections with the aim of overcoming the specific challenges related to exascale. Areas of focus include compact packaging, permanent storage, interconnection, resilience and application behavior. Some of the design decisions were revealed in this poster from Exanest that shows a diagram of the daughterboard and blade design. Note that Xilinx is a key partner.

Exanest daughterboard and blade design

To achieve a complete prototype capable of running real-world benchmarks and applications by 2018, the primary partners are collaborating with a number of other academic groups and industry partners using co-design principles to develop the hardware and software elements. This is a classic public-private arrangement where academic and industrial partners join forces and industrial partners benefit by being able to reuse the technology that is developed.

On the technology side, packaging and cooling is a key focus for Exanest, which will rely on Iceotope, the immersive cooling vendor, to design an innovative cooling environment. The first prototype will employ Iceotope technology and there is the expectation that technology with even higher power density will be developed as the project progresses.

One of the primary criteria for the project partners is low-energy consumption for the main processor. They have chosen 64-bit ARM processors as their main compute engine. Katevenis affirms that having a processor that consumes dramatically less power allows many more cores to be packaged in the same physical volume and within the same total power consumption budget. “One way we will achieve scale is this low-power consumption,” says the project lead, “but another is by having accelerators to provide floating point performance boost to appropriate applications.”

As for topology, the Exanest team is discussing the family of networks that includes fat trees and Dragonfly topology. They will be linking blades through optical fibers that they can plug and unplug allowing them to experiment with more than one topology. Exanest will also be using FPGAs for building the interconnection network so they can experiment with novel protocols.

Exanode

Denis Dutoit, the project coordinator for Exanode, tells HPCwire the goal of that project is to build a node-level prototype with technologies that exhibit exascale potential. The three building blocks are heterogeneous compute elements (ARM-v8 low-power processors plus various accelerators, namely FPGAs although ASICs and GPGPUs may also be explored); 3D interposer integration for compute density; and, continuing the efforts of the EUROSERVER project, an advanced memory scheme for low-latency, high-bandwidth memory access, scalable to exabyte levels.

ExaNoDe_Figures_04-1024x768

Dutoit, who is the strategic marketing manager, architecture, IC design and embedded software division at CEA-Leti, notes that this is a technology driven project at the start, but on top of this prototype, there will be a complete software stack for HPC capability. Evaluation will be done first will be done on the node level, explains Dutoit. They will utilize emulated hardware first and representative HPC applications to evaluate at the level nodes, but after that, Exanest will reuse these compute nodes and integrate them into their complete machine to do the full testing and evaluation with real applications.

There will be a formal effort to productize the resulting technology through a partnership with Kaleao, a UK company that focuses on energy-efficient, compact hyperconverged platforms.

Ecoscale

Iakovos Mavroidis, project coordinator for Ecoscale, says that while there are three main projects, he sees it as one big project with Ecoscale dedicated to reconfigurable computing.

A member of Computer Architecture and VLSI Systems (CARV) Laboratory of FORTH-ICS and a member of Telecommunication Systems Institute, Mavroidis notes that the main problem being addressed is how to improve today’s HPC servers. Simple scaling without improving technologies is unfeasible due to utility costs and power consumption limitations. Ecoscale is tackling these challenges by proposing a scale-out hybrid MPI+OpenCL programming environment and a runtime system, along with a hardware architecture which is tailored to the needs of HPC applications. The programming model and runtime system follows a hierarchical approach where the system is partitioned into multiple autonomous workers (i.e. compute nodes).

ecoscale_framework

“The main focus of Ecoscale is to support shared partitioned reconfigurable resources, accessed by these compute nodes,” says Mavroidis. “The intention is to have a global notion of the reconfigurable resources so that each compute node can access remote reconfigurable resources not only its own local resources. The logic can also be shared by several compute nodes working in parallel.” To accomplish this, workers are interconnected in a tree-like structure in order to form larger Partitioned Global Address Space (PGAS) partitions, which are further hierarchically interconnected via an MPI protocol.

“The virtualization will happen automatically in hardware and it has to be done because reconfigurable resources are very limited unless remote access is enabled,” states Mavroidis. “The aim is to provide a user-friendly way for the programmer to use all the reconfigurable logic in the system. This requires a very high-speed low-latency interconnection topology and this is what Exanest will provide.”

Mavroidis explains there must be means for the programmer to access the system and at a higher-level the run-time system has to be redefined to understand the needs of the application so it can reconfigure the machine. He believes that in order to fully implement this, there will need to be innovation in all the layers of the stack, and also the programming model itself will also need to be redefined. The partners are aiming to support most of the existing and common HPC libraries in order to make this architecture available to most of the existing applications.

The main focus of Ecoscale is to automate out the complexity of FPGA programming. Anyone who has watched FPGAs struggle to get a foothold in HPC knows this is not an easy task, but the need for low-power performance is driving interest and innovation. “The programmer should not have to be aware that the machine uses reconfigurable computing, but rather be able to write the program using high-level programming model such as MPI or Standard C,” states Mavroidis.

On a related note, Exanest project partner BeeGFS has just announced that the BeeGFS parallel file system is now available as open source from www.beegfs.com. “Although BeeGFS can already run out of the box on ARM systems today, this project [Exanest] will give us the opportunity to make sure that we can deliver the maximum performance on this architecture as well,” shares Bernd Lietzow, BeeGFS head for Exanest.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Ayar Labs to Demo Photonics Chiplet in FPGA Package at Hot Chips

August 19, 2019

Silicon startup Ayar Labs continues to gain momentum with its DARPA-backed optical chiplet technology that puts advanced electronics and optics on the same chip using standard CMOS fabrication. At Hot Chips 31 in Stanfor Read more…

By Tiffany Trader

Talk to Me: Nvidia Claims NLP Inference, Training Records

August 15, 2019

Nvidia says it’s achieved significant advances in conversation natural language processing (NLP) training and inference, enabling more complex, immediate-response interchanges between customers and chatbots. And the co Read more…

By Doug Black

Trump Administration and NIST Issue AI Standards Development Plan

August 14, 2019

Efforts to develop AI are gathering steam fast. On Monday, the White House issued a federal plan to help develop technical standards for AI following up on a mandate contained in the Administration’s AI Executive Order Read more…

By John Russell

AWS Solution Channel

Efficiency and Cost-Optimization for HPC Workloads – AWS Batch and Amazon EC2 Spot Instances

High Performance Computing on AWS leverages the power of cloud computing and the extreme scale it offers to achieve optimal HPC price/performance. With AWS you can right size your services to meet exactly the capacity requirements you need without having to overprovision or compromise capacity. Read more…

HPE Extreme Performance Solutions

Bring the combined power of HPC and AI to your business transformation

FPGA (Field Programmable Gate Array) acceleration cards are not new, as they’ve been commercially available since 1984. Typically, the emphasis around FPGAs has centered on the fact that they’re programmable accelerators, and that they can truly offer workload specific hardware acceleration solutions without requiring custom silicon. Read more…

IBM Accelerated Insights

Cloudy with a Chance of Mainframes

[Connect with HPC users and learn new skills in the IBM Spectrum LSF User Community.]

Rapid rates of change sometimes result in unexpected bedfellows. Read more…

Scientists to Tap Exascale Computing to Unlock the Mystery of our Accelerating Universe

August 14, 2019

The universe and everything in it roared to life with the Big Bang approximately 13.8 billion years ago. It has continued expanding ever since. While we have a good understanding of the early universe, its fate billions Read more…

By Rob Johnson

Ayar Labs to Demo Photonics Chiplet in FPGA Package at Hot Chips

August 19, 2019

Silicon startup Ayar Labs continues to gain momentum with its DARPA-backed optical chiplet technology that puts advanced electronics and optics on the same chip Read more…

By Tiffany Trader

Scientists to Tap Exascale Computing to Unlock the Mystery of our Accelerating Universe

August 14, 2019

The universe and everything in it roared to life with the Big Bang approximately 13.8 billion years ago. It has continued expanding ever since. While we have a Read more…

By Rob Johnson

AI is the Next Exascale – Rick Stevens on What that Means and Why It’s Important

August 13, 2019

Twelve years ago the Department of Energy (DOE) was just beginning to explore what an exascale computing program might look like and what it might accomplish. Today, DOE is repeating that process for AI, once again starting with science community town halls to gather input and stimulate conversation. The town hall program... Read more…

By Tiffany Trader and John Russell

Cray Wins NNSA-Livermore ‘El Capitan’ Exascale Contract

August 13, 2019

Cray has won the bid to build the first exascale supercomputer for the National Nuclear Security Administration (NNSA) and Lawrence Livermore National Laborator Read more…

By Tiffany Trader

AMD Launches Epyc Rome, First 7nm CPU

August 8, 2019

From a gala event at the Palace of Fine Arts in San Francisco yesterday (Aug. 7), AMD launched its second-generation Epyc Rome x86 chips, based on its 7nm proce Read more…

By Tiffany Trader

Lenovo Drives Single-Socket Servers with AMD Epyc Rome CPUs

August 7, 2019

No summer doldrums here. As part of the AMD Epyc Rome launch event in San Francisco today, Lenovo announced two new single-socket servers, the ThinkSystem SR635 Read more…

By Doug Black

Building Diversity and Broader Engagement in the HPC Community

August 7, 2019

Increasing diversity and inclusion in HPC is a community-building effort. Representation of both issues and individuals matters - the more people see HPC in a w Read more…

By AJ Lauer

Xilinx vs. Intel: FPGA Market Leaders Launch Server Accelerator Cards

August 6, 2019

The two FPGA market leaders, Intel and Xilinx, both announced new accelerator cards this week designed to handle specialized, compute-intensive workloads and un Read more…

By Doug Black

High Performance (Potato) Chips

May 5, 2006

In this article, we focus on how Procter & Gamble is using high performance computing to create some common, everyday supermarket products. Tom Lange, a 27-year veteran of the company, tells us how P&G models products, processes and production systems for the betterment of consumer package goods. Read more…

By Michael Feldman

Supercomputer-Powered AI Tackles a Key Fusion Energy Challenge

August 7, 2019

Fusion energy is the Holy Grail of the energy world: low-radioactivity, low-waste, zero-carbon, high-output nuclear power that can run on hydrogen or lithium. T Read more…

By Oliver Peckham

Cray, AMD to Extend DOE’s Exascale Frontier

May 7, 2019

Cray and AMD are coming back to Oak Ridge National Laboratory to partner on the world’s largest and most expensive supercomputer. The Department of Energy’s Read more…

By Tiffany Trader

Graphene Surprises Again, This Time for Quantum Computing

May 8, 2019

Graphene is fascinating stuff with promise for use in a seeming endless number of applications. This month researchers from the University of Vienna and Institu Read more…

By John Russell

AMD Verifies Its Largest 7nm Chip Design in Ten Hours

June 5, 2019

AMD announced last week that its engineers had successfully executed the first physical verification of its largest 7nm chip design – in just ten hours. The AMD Radeon Instinct Vega20 – which boasts 13.2 billion transistors – was tested using a TSMC-certified Calibre nmDRC software platform from Mentor. Read more…

By Oliver Peckham

TSMC and Samsung Moving to 5nm; Whither Moore’s Law?

June 12, 2019

With reports that Taiwan Semiconductor Manufacturing Co. (TMSC) and Samsung are moving quickly to 5nm manufacturing, it’s a good time to again ponder whither goes the venerable Moore’s law. Shrinking feature size has of course been the primary hallmark of achieving Moore’s law... Read more…

By John Russell

Deep Learning Competitors Stalk Nvidia

May 14, 2019

There is no shortage of processing architectures emerging to accelerate deep learning workloads, with two more options emerging this week to challenge GPU leader Nvidia. First, Intel researchers claimed a new deep learning record for image classification on the ResNet-50 convolutional neural network. Separately, Israeli AI chip startup Hailo.ai... Read more…

By George Leopold

Cray Wins NNSA-Livermore ‘El Capitan’ Exascale Contract

August 13, 2019

Cray has won the bid to build the first exascale supercomputer for the National Nuclear Security Administration (NNSA) and Lawrence Livermore National Laborator Read more…

By Tiffany Trader

Leading Solution Providers

ISC 2019 Virtual Booth Video Tour

CRAY
CRAY
DDN
DDN
DELL EMC
DELL EMC
GOOGLE
GOOGLE
ONE STOP SYSTEMS
ONE STOP SYSTEMS
PANASAS
PANASAS
VERNE GLOBAL
VERNE GLOBAL

Nvidia Embraces Arm, Declares Intent to Accelerate All CPU Architectures

June 17, 2019

As the Top500 list was being announced at ISC in Frankfurt today with an upgraded petascale Arm supercomputer in the top third of the list, Nvidia announced its Read more…

By Tiffany Trader

Top500 Purely Petaflops; US Maintains Performance Lead

June 17, 2019

With the kick-off of the International Supercomputing Conference (ISC) in Frankfurt this morning, the 53rd Top500 list made its debut, and this one's for petafl Read more…

By Tiffany Trader

AMD Launches Epyc Rome, First 7nm CPU

August 8, 2019

From a gala event at the Palace of Fine Arts in San Francisco yesterday (Aug. 7), AMD launched its second-generation Epyc Rome x86 chips, based on its 7nm proce Read more…

By Tiffany Trader

A Behind-the-Scenes Look at the Hardware That Powered the Black Hole Image

June 24, 2019

Two months ago, the first-ever image of a black hole took the internet by storm. A team of scientists took years to produce and verify the striking image – an Read more…

By Oliver Peckham

Cray – and the Cray Brand – to Be Positioned at Tip of HPE’s HPC Spear

May 22, 2019

More so than with most acquisitions of this kind, HPE’s purchase of Cray for $1.3 billion, announced last week, seems to have elements of that overused, often Read more…

By Doug Black and Tiffany Trader

Chinese Company Sugon Placed on US ‘Entity List’ After Strong Showing at International Supercomputing Conference

June 26, 2019

After more than a decade of advancing its supercomputing prowess, operating the world’s most powerful supercomputer from June 2013 to June 2018, China is keep Read more…

By Tiffany Trader

In Wake of Nvidia-Mellanox: Xilinx to Acquire Solarflare

April 25, 2019

With echoes of Nvidia’s recent acquisition of Mellanox, FPGA maker Xilinx has announced a definitive agreement to acquire Solarflare Communications, provider Read more…

By Doug Black

Qualcomm Invests in RISC-V Startup SiFive

June 7, 2019

Investors are zeroing in on the open standard RISC-V instruction set architecture and the processor intellectual property being developed by a batch of high-flying chip startups. Last fall, Esperanto Technologies announced a $58 million funding round. Read more…

By George Leopold

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This