Architecting for AI Workloads

March 4, 2019

Artificial intelligence has come of age. To capitalize fully on the opportunities, organizations need to design high-performance computing architectures for AI workloads.

After years of talking about the promise of artificial intelligence, enterprises around the world are now diving headfirst into AI-driven processes and business models. From financial services to manufacturing, from healthcare to retail, enterprises are now “all in” with AI and its supporting computing models — notably machine and deep learning. The same holds true for universities and government agencies. They are using AI for countless pursuits, from driving groundbreaking scientific discoveries to protecting our national security.

This widespread embrace of all things AI is fueled by the rise of more powerful processors and accelerators, advanced tools and techniques for data analytics, more precise algorithms and — most of all — an explosion of data, driven to a large degree by the Internet of Things. When you put it all together, you’ve got what it takes to put AI to work in countless applications.

Architecting for AI workloads

To capitalize fully on the opportunities in today’s data-driven world, IT organizations need to design high-performance computing architectures to accommodate demanding AI workloads. The HPC and AI community has started optimizing AI frameworks and developer tools to address performance needs, allowing for much larger batch sizes to be processed on industry standard CPUs. Within the last year, Intel® has seen up to 241x training performance gains through optimized frameworks with Intel® Math Kernel Library (MKL) on Intel Xeon® Scalable Processors over Haswell processors. This can take your time to train from hours to minutes, while these optimizations provide the eco-system greater access to AI capabilities.

This shift to AI-focused infrastructure is happening today as organizations roll out systems that bring together the capabilities of HPC, data analytics and AI. This is the case with the University of Cambridge’s latest supercomputer, called Cumulus. This groundbreaking system was designed to serve as a single HPC cluster that supports researchers’ needs for data analytics, machine learning and large-scale data processing. The goal is to solve extremely difficult big data, simulation and AI challenges.

To meet this goal, the Cumulus architecture was designed to address the broad range of system challenges, including those at the compute, network, storage and software layers. A key objective was to make the infrastructure perform well for diverse, data-intensive research workloads.

The Cumulus system provides more than 2 petaflops of performance, powered by Dell EMC PowerEdge™ servers and Intel Xeon Scalable processors, all connected via the Intel Omni-Path Architecture (OPA). The system incorporates OpenStack® software to control pools of compute, storage and networking resources and make them readily accessible to users via a cloud interface.

Solving for I/O bottlenecks

This architectural foundation alone doesn’t necessarily solve today’s persistent I/O challenges in HPC clusters. Here’s the problem: While data-processing power has raced forward in recent years, storage I/O limitations have created bottlenecks that slow time to insight, particularly for researchers running data-centric workloads that interact continuously with data storage systems.

The Cumulus system removes these bottlenecks with a unique solution called the Data Accelerator (aka DAC), which is designed into the network topology. DAC incorporates technologies from Dell EMC, Intel  and Cambridge University. In this architecture, the DAC nodes work in conjunction with the Distributed Name Space (DNE) feature in the Lustre file system and Intel® Omni-Path switches to accelerate system I/O.

The results of this accelerated architecture have been rather amazing. With DAC under the hood, Cumulus provides more than 500 GB/s of I/O read performance, which makes it the UK’s fastest HPC I/O platform, according to the university’s Research Computing Service, which operates the Cumulus cluster.[1]

In benchmark testing, the Cumulus system achieved an IO-500 score of 158.7, which ranked the system third on the November 2018 IO-500 list. For system users, these numbers equate to big improvements in I/O performance for data-intensive HPC and AI workloads — and faster time to insight.

Building the right foundation for new and emerging workloads

For organizations searching for the right IT foundation for AI workloads, Intel offers expert insights in its high-level Guide to Developing an AI Infrastructure Strategy. The options outlined in this guide range from starting from scratch with your current systems to outsourcing your entire solution. One of these options is to build a broad platform that is designed to support a wide range of AI workloads — which is the approach the University of Cambridge took with its Cumulus system.

The guide explains: “This approach is similar to the emerging ‘platform’ architecture we now see prevalent across IT — that is, an approach that provides a highly scalable infrastructure layer that can be managed as a single pool, using virtualization and software-defined orchestration across server processing, storage and networking.”[2]

The guide presents this broad-platform infrastructure strategy in terms of a three-tier stack, with hardware, software and process layers that work together to enable AI workloads. A few highlights from this architecture:

  • At the hardware layer, communication between devices and systems is based around an ultra-high speed backbone, such as the Intel® Omni-Path.
  • The software layer includes operating system and virtualization layers, which support a library of AI-specific modules. These modules enable algorithmic processing and analytics, data management and I/O, as well as the delivery of data sources and the visualization of analysis results.
  • The process layer runs the business logic of the AI application, using library modules to deliver capabilities like image recognition.

Intel notes that this architecture results in a platform-based approach that offers a single point of configuration and a unique deployment target.

Key takeaways

The rise of artificial intelligence creates unprecedented opportunities for today’s enterprises. To fully capitalize on these opportunities, your organization needs a scalable HPC infrastructure that is specifically designed to incorporate the latest processor and fabric technologies, accommodate massive amounts of data, and leverage technologies to accelerate the data storage I/O and AI workloads.

To learn more

For a closer and more technical look at the University of Cambridge’s use of the Data Accelerator, visit the Research Computing Services’ Data Accelerator site. And for a broader look at the university’s Cumulus cluster, read the Dell EMC case study “UK Science Cloud.”

 

The Convergence of HPC, Analytics and AI

High-performance computing, data analytics and artificial intelligence no longer live in separate domains. These complementary technologies are rapidly converging as organizations work to gain greater value from the data they capture and store.


[1] Dell EMC case study, “UK Science Cloud,” November 2018.

 

[2] Intel, “Select the Best Infrastructure Strategy to Support Your AI Solution,” March 2018.

 

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Stampede2 ‘Shocks’ with New Shock Turbulence Insights

August 19, 2019

Shockwaves play roles in everything from high-speed aircraft to supernovae – and now, supercomputer-powered research from the Texas A&M University and the Texas Advanced Computing Center (TACC) is helping to shed l Read more…

By Oliver Peckham

Nanosheet Transistors: The Last Step in Moore’s Law?

August 19, 2019

Forget for a moment the clamor around the decline of Moore’s Law. It's had a brilliant run, something to be marveled at given it’s not a law at all. Squeezing out the last bit of performance that roughly corresponds Read more…

By John Russell

Ayar Labs to Demo Photonics Chiplet in FPGA Package at Hot Chips

August 19, 2019

Silicon startup Ayar Labs continues to gain momentum with its DARPA-backed optical chiplet technology that puts advanced electronics and optics on the same chip using standard CMOS fabrication. At Hot Chips 31 in Stanfor Read more…

By Tiffany Trader

AWS Solution Channel

Efficiency and Cost-Optimization for HPC Workloads – AWS Batch and Amazon EC2 Spot Instances

High Performance Computing on AWS leverages the power of cloud computing and the extreme scale it offers to achieve optimal HPC price/performance. With AWS you can right size your services to meet exactly the capacity requirements you need without having to overprovision or compromise capacity. Read more…

HPE Extreme Performance Solutions

Bring the combined power of HPC and AI to your business transformation

FPGA (Field Programmable Gate Array) acceleration cards are not new, as they’ve been commercially available since 1984. Typically, the emphasis around FPGAs has centered on the fact that they’re programmable accelerators, and that they can truly offer workload specific hardware acceleration solutions without requiring custom silicon. Read more…

IBM Accelerated Insights

Keys to Attracting the Newest HPC Talent – Post-Millennials

[Connect with HPC users and learn new skills in the IBM Spectrum LSF User Community.]

For engineers and scientists growing up in the 80s, the current state of HPC makes perfect sense. Read more…

Talk to Me: Nvidia Claims NLP Inference, Training Records

August 15, 2019

Nvidia says it’s achieved significant advances in conversation natural language processing (NLP) training and inference, enabling more complex, immediate-response interchanges between customers and chatbots. And the co Read more…

By Doug Black

Ayar Labs to Demo Photonics Chiplet in FPGA Package at Hot Chips

August 19, 2019

Silicon startup Ayar Labs continues to gain momentum with its DARPA-backed optical chiplet technology that puts advanced electronics and optics on the same chip Read more…

By Tiffany Trader

Scientists to Tap Exascale Computing to Unlock the Mystery of our Accelerating Universe

August 14, 2019

The universe and everything in it roared to life with the Big Bang approximately 13.8 billion years ago. It has continued expanding ever since. While we have a Read more…

By Rob Johnson

AI is the Next Exascale – Rick Stevens on What that Means and Why It’s Important

August 13, 2019

Twelve years ago the Department of Energy (DOE) was just beginning to explore what an exascale computing program might look like and what it might accomplish. Today, DOE is repeating that process for AI, once again starting with science community town halls to gather input and stimulate conversation. The town hall program... Read more…

By Tiffany Trader and John Russell

Cray Wins NNSA-Livermore ‘El Capitan’ Exascale Contract

August 13, 2019

Cray has won the bid to build the first exascale supercomputer for the National Nuclear Security Administration (NNSA) and Lawrence Livermore National Laborator Read more…

By Tiffany Trader

AMD Launches Epyc Rome, First 7nm CPU

August 8, 2019

From a gala event at the Palace of Fine Arts in San Francisco yesterday (Aug. 7), AMD launched its second-generation Epyc Rome x86 chips, based on its 7nm proce Read more…

By Tiffany Trader

Lenovo Drives Single-Socket Servers with AMD Epyc Rome CPUs

August 7, 2019

No summer doldrums here. As part of the AMD Epyc Rome launch event in San Francisco today, Lenovo announced two new single-socket servers, the ThinkSystem SR635 Read more…

By Doug Black

Building Diversity and Broader Engagement in the HPC Community

August 7, 2019

Increasing diversity and inclusion in HPC is a community-building effort. Representation of both issues and individuals matters - the more people see HPC in a w Read more…

By AJ Lauer

Xilinx vs. Intel: FPGA Market Leaders Launch Server Accelerator Cards

August 6, 2019

The two FPGA market leaders, Intel and Xilinx, both announced new accelerator cards this week designed to handle specialized, compute-intensive workloads and un Read more…

By Doug Black

High Performance (Potato) Chips

May 5, 2006

In this article, we focus on how Procter & Gamble is using high performance computing to create some common, everyday supermarket products. Tom Lange, a 27-year veteran of the company, tells us how P&G models products, processes and production systems for the betterment of consumer package goods. Read more…

By Michael Feldman

Supercomputer-Powered AI Tackles a Key Fusion Energy Challenge

August 7, 2019

Fusion energy is the Holy Grail of the energy world: low-radioactivity, low-waste, zero-carbon, high-output nuclear power that can run on hydrogen or lithium. T Read more…

By Oliver Peckham

Cray, AMD to Extend DOE’s Exascale Frontier

May 7, 2019

Cray and AMD are coming back to Oak Ridge National Laboratory to partner on the world’s largest and most expensive supercomputer. The Department of Energy’s Read more…

By Tiffany Trader

Graphene Surprises Again, This Time for Quantum Computing

May 8, 2019

Graphene is fascinating stuff with promise for use in a seeming endless number of applications. This month researchers from the University of Vienna and Institu Read more…

By John Russell

AMD Verifies Its Largest 7nm Chip Design in Ten Hours

June 5, 2019

AMD announced last week that its engineers had successfully executed the first physical verification of its largest 7nm chip design – in just ten hours. The AMD Radeon Instinct Vega20 – which boasts 13.2 billion transistors – was tested using a TSMC-certified Calibre nmDRC software platform from Mentor. Read more…

By Oliver Peckham

TSMC and Samsung Moving to 5nm; Whither Moore’s Law?

June 12, 2019

With reports that Taiwan Semiconductor Manufacturing Co. (TMSC) and Samsung are moving quickly to 5nm manufacturing, it’s a good time to again ponder whither goes the venerable Moore’s law. Shrinking feature size has of course been the primary hallmark of achieving Moore’s law... Read more…

By John Russell

Cray Wins NNSA-Livermore ‘El Capitan’ Exascale Contract

August 13, 2019

Cray has won the bid to build the first exascale supercomputer for the National Nuclear Security Administration (NNSA) and Lawrence Livermore National Laborator Read more…

By Tiffany Trader

Deep Learning Competitors Stalk Nvidia

May 14, 2019

There is no shortage of processing architectures emerging to accelerate deep learning workloads, with two more options emerging this week to challenge GPU leader Nvidia. First, Intel researchers claimed a new deep learning record for image classification on the ResNet-50 convolutional neural network. Separately, Israeli AI chip startup Hailo.ai... Read more…

By George Leopold

Leading Solution Providers

ISC 2019 Virtual Booth Video Tour

CRAY
CRAY
DDN
DDN
DELL EMC
DELL EMC
GOOGLE
GOOGLE
ONE STOP SYSTEMS
ONE STOP SYSTEMS
PANASAS
PANASAS
VERNE GLOBAL
VERNE GLOBAL

Nvidia Embraces Arm, Declares Intent to Accelerate All CPU Architectures

June 17, 2019

As the Top500 list was being announced at ISC in Frankfurt today with an upgraded petascale Arm supercomputer in the top third of the list, Nvidia announced its Read more…

By Tiffany Trader

Top500 Purely Petaflops; US Maintains Performance Lead

June 17, 2019

With the kick-off of the International Supercomputing Conference (ISC) in Frankfurt this morning, the 53rd Top500 list made its debut, and this one's for petafl Read more…

By Tiffany Trader

AMD Launches Epyc Rome, First 7nm CPU

August 8, 2019

From a gala event at the Palace of Fine Arts in San Francisco yesterday (Aug. 7), AMD launched its second-generation Epyc Rome x86 chips, based on its 7nm proce Read more…

By Tiffany Trader

A Behind-the-Scenes Look at the Hardware That Powered the Black Hole Image

June 24, 2019

Two months ago, the first-ever image of a black hole took the internet by storm. A team of scientists took years to produce and verify the striking image – an Read more…

By Oliver Peckham

Cray – and the Cray Brand – to Be Positioned at Tip of HPE’s HPC Spear

May 22, 2019

More so than with most acquisitions of this kind, HPE’s purchase of Cray for $1.3 billion, announced last week, seems to have elements of that overused, often Read more…

By Doug Black and Tiffany Trader

Chinese Company Sugon Placed on US ‘Entity List’ After Strong Showing at International Supercomputing Conference

June 26, 2019

After more than a decade of advancing its supercomputing prowess, operating the world’s most powerful supercomputer from June 2013 to June 2018, China is keep Read more…

By Tiffany Trader

In Wake of Nvidia-Mellanox: Xilinx to Acquire Solarflare

April 25, 2019

With echoes of Nvidia’s recent acquisition of Mellanox, FPGA maker Xilinx has announced a definitive agreement to acquire Solarflare Communications, provider Read more…

By Doug Black

Qualcomm Invests in RISC-V Startup SiFive

June 7, 2019

Investors are zeroing in on the open standard RISC-V instruction set architecture and the processor intellectual property being developed by a batch of high-flying chip startups. Last fall, Esperanto Technologies announced a $58 million funding round. Read more…

By George Leopold

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This