HiPEAC Keynote: In-Memory Computing Steps Closer to Practical Reality

By John Russell

January 21, 2021

Pursuit of in-memory computing has long been an active area with recent progress showing promise. Just how in-memory computing works, how close it is to practical application, and what are some of the key opportunities and challenges facing in-memory computing comprised the core of IBM researcher Evangelos Eleftheriou’s keynote at the annual HiPEAC conference held virtually this week.

The HiPEAC – High Performance Embedded Architecture and Compilation – project began as one of Europe’s Horizon 2020 projects intended to advance HPC. Coincident with this year’s virtual conference, the organization released HiPEAC Vision 2021, which perhaps paradoxically but hopefully, suggests Europe “seize the opportunity presented by the influence of the COVID-19 pandemic” to develop user-centered IT systems that prioritize security and convenience.

The HiPEAC Vision 2021 document is freely available for download. This year’s conference also included an Industry Day, keynoted by AMD’s Brad McCredie discussing the path to exascale (HPCwire will have coverage later). Not surprisingly much of the agenda looked at AI writ large.

Eleftheriou’s talk dug into in-memory computing and the big problem it tackles, which is data transfer overhead in terms of latency and energy consumption. His snapshot of the landscape looked at how phase change memory and more familiar SRAM/DRAM technology can be used as alternatives to traditional von Neumann compute architectures. He emphasized how deep learning techniques can be implemented using in-memory computing.

“In the last five-plus years, AI has become synonymous with deep learning. And the progress in this area is fast and dramatic. We are at a point where, for example, image and speech recognition can produce accuracies close to or better than the human brain. Most of the fundamental algorithmic developments around deep learning go back decades,” said Eleftheriou, an IBM Fellow working at the IBM’s Zurich Research Laboratory.

The key challenge in computing today, he said, is the mushrooming of compute requirements caused by AI adoption. “For really large AI training jobs, [compute requirements] are doubling every 3.5 months. Clearly, this is unsustainable without fundamental hardware and software innovation,” he said.

As it turns out matrix vector multiplication is the dominant function. “[F]or the three most common AI applications, such as speech recognition, natural language processing, and computer vision, which involve RNNs, LSTM’s and CNNs, the matrix vector multiplications constitute 70 to 90% of the total deep learning operations. The rest of the operations are simple element wise operations with single or multiple word operands,” Eleftheriou said.

Broadly, the core issue is moving data between the processor and the memory in conventional von Neumann computer architecture. There’s a cost for every data transfer that in-memory computing can reduce.

“If I want to transfer data from a DRAM, the energy consumption is 640 picojoules per word, then if I go down to the SRAM, it is 50 picojoules and when I have a small local SRAM, it’s about 5 picojoules. So compare these energy consumptions for data transfer to the cost of an a bit multiplication. So it is only 0.2 picojoules per eight bits, which is a factor of about 3,000 less than transferring data from DRAM. Although this study was back in 2014, for the 45 nanometer CMOS technology node, there are recent studies that show that the problem gets exacerbated as you move down the lithography node,” said Eleftheriou.

Eleftheriou, like others, argue the need to solve the data transfer and management problem is driving opportunities 1) to develop future systems that minimize data movement for performing computation directly-or-near where the data resides, and 2) to introduce novel computational primitives, like matrix vector multiplication that facilitate the deep learning workloads. Such innovations are under way. He noted 3D XPoint memory technology is opening new horizons for running faster and with lower power requirements on many applications.

In-memory computing will also drive innovation, he said.

“Memory is not just a place where you park your data, but an active participant in computing. And here (slide below) we see the various incarnations of memory on the on the left-hand side, we have the classical you know DRAM, SRAM, Flash, where information is stored as charge. In DRAM, we have an explicit capacitor. In flash, we have the floating gate, if I want to do computation, we use the Coulomb’s Law. On the right-hand side, we have this emerging memory technology under the name memristors, where information is stored as resistance

The new approaches, he said, “possess sufficient richness of dynamics for non von Neumann computing. There is a range of in memory logic and arithmetic operations that can be performed using both charge-based and resistance-based memory devices. This slide shows examples of logical operations. On the left-hand side, we can see schematics of bitwise AND and OR operations using two memory devices. Data are stored as resistance logic state value. Then with the appropriate choice of the current thresholds in the sense amplifier, it’s possible to implement AND and OR logical operations. On the right-hand side, we can see schematics of again bitwise AND and OR operations using DRAM and SRAM cells. In the case of DRAM, the operands are stored in the cells A and B and then there is a third DRAM cell that plays the role of a selector which used to decide whether AND or OR is realized.”

 

Using these techniques, Eleftheriou said it is possible to implement matrix multiply operations that are critical and abundant in DL. The example shown below is using resistive memory.

“Let’s assume now that I want to perform a matrix vector multiplication (slide below). On the left-hand side, you see a matrix, three rows two columns, multiplied by a two-dimensional vector that resolves as a three-dimensional vector. What we do is, we map the values – the elements of the matrix – into conductance values on the crossbar array; in other words we exploit the analog storage capability of the resistive memory. Then we map the two-dimensional vector into the read voltage at the input of the crossbar array. And the result automatically appears as current at the columns of the crossbar array. Therefore, I can have in-place matrix vector multiply with time complexity of 01 time complexity. And you can do the same thing for the transposed matrix and vector multiplication; in this case you just apply the voltages at the columns, and you get the current at the rows,” explained Eleftheriou.

Many challenges remain. Eleftheriou noted a new software stack would be needed. “You have different computational primitives. you have matrix vector multiplications. you have a different control, you know, that you need when you implement a data flow engine, different compilers are needed, and so on,” he said. Drift remains a problem for resistive memory, he added, but efforts are ongoing to cope with that.

Practical devices are still a ways off. “There are many startups out there working on it. I believe it will be a couple of years until we see products on the market. That’s my guess,” he said.

There was a good deal more detail in Eleftheriou’s presentation. If a video become available HPCwire will update this article with a link.

Slides sourced from Evangelos Eleftheriou’s (IBM) HiPEAC keynote 

Feature image: IBM PCM chip

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

IBM Quantum Challenge 2021 – Let the Programming Begin!

May 17, 2021

Looking to sharpen or perhaps simply explore your quantum programming skills? On Thursday, IBM fires up its IBM Quantum Challenge 2021 marking the fifth anniversary of IBM Quantum Experience cloud services and the 40th  Read more…

Q&A with Altair CEO James Scapa, an HPCwire Person to Watch in 2021

May 14, 2021

Chairman, CEO and co-founder of Altair James R. Scapa closed several acquisitions for the company in 2020, including the purchase and integration of Univa and Ellexus. Scapa founded Altair more than 35 years ago with two Read more…

HLRS HPC Helps to Model Muscle Movements

May 13, 2021

The growing scale of HPC is allowing simulation of more and more complex systems at greater detail than ever before, particularly in the biological research spheres. Now, researchers at the University of Stuttgart are le Read more…

Behind the Met Office’s Procurement of a Billion-Dollar Microsoft System

May 13, 2021

The UK’s national weather service, the Met Office, caused shockwaves of curiosity a few weeks ago when it formally announced that its forthcoming billion-dollar supercomputer – expected to be the most powerful weather and climate-focused supercomputer in the world when it launches in 2022... Read more…

AMD, GlobalFoundries Commit to $1.6 Billion Wafer Supply Deal

May 13, 2021

AMD plans to purchase $1.6 billion worth of wafers from GlobalFoundries in the 2022 to 2024 timeframe, the chipmaker revealed today (May 13) in an SEC filing. In the face of global semiconductor shortages and record-high demand, AMD is renegotiating its Wafer Supply Agreement and bumping up capacity. Read more…

AWS Solution Channel

Numerical weather prediction on AWS Graviton2

The Weather Research and Forecasting (WRF) model is a numerical weather prediction (NWP) system designed to serve both atmospheric research and operational forecasting needs. Read more…

Hyperion Offers Snapshot of Quantum Computing Market

May 13, 2021

The nascent quantum computer (QC) market will grow 27 percent annually (CAGR) reaching $830 million in 2024 according to an update provided today by analyst firm Hyperion Research at the HPC User Forum being held this we Read more…

Behind the Met Office’s Procurement of a Billion-Dollar Microsoft System

May 13, 2021

The UK’s national weather service, the Met Office, caused shockwaves of curiosity a few weeks ago when it formally announced that its forthcoming billion-dollar supercomputer – expected to be the most powerful weather and climate-focused supercomputer in the world when it launches in 2022... Read more…

AMD, GlobalFoundries Commit to $1.6 Billion Wafer Supply Deal

May 13, 2021

AMD plans to purchase $1.6 billion worth of wafers from GlobalFoundries in the 2022 to 2024 timeframe, the chipmaker revealed today (May 13) in an SEC filing. In the face of global semiconductor shortages and record-high demand, AMD is renegotiating its Wafer Supply Agreement and bumping up capacity. Read more…

Hyperion Offers Snapshot of Quantum Computing Market

May 13, 2021

The nascent quantum computer (QC) market will grow 27 percent annually (CAGR) reaching $830 million in 2024 according to an update provided today by analyst fir Read more…

Hyperion: HPC Server Market Ekes 1 Percent Gain in 2020, Storage Poised for ‘Tipping Point’

May 12, 2021

The HPC User Forum meeting taking place virtually this week (May 11-13) kicked off with Hyperion Research’s market update, covering the 2020 period. Although Read more…

IBM Debuts Qiskit Runtime for Quantum Computing; Reports Dramatic Speed-up

May 11, 2021

In conjunction with its virtual Think event, IBM today introduced an enhanced Qiskit Runtime Software for quantum computing, which it says demonstrated 120x spe Read more…

AMD Chipmaker TSMC to Use AMD Chips for Chipmaking

May 8, 2021

TSMC has tapped AMD to support its major manufacturing and R&D workloads. AMD will provide its Epyc Rome 7702P CPUs – with 64 cores operating at a base cl Read more…

Fast Pass Through (Some of) the Quantum Landscape with ORNL’s Raphael Pooser

May 7, 2021

In a rather remarkable way, and despite the frequent hype, the behind-the-scenes work of developing quantum computing has dramatically accelerated in the past f Read more…

IBM Research Debuts 2nm Test Chip with 50 Billion Transistors

May 6, 2021

IBM Research today announced the successful prototyping of the world's first 2 nanometer chip, fabricated with silicon nanosheet technology on a standard 300mm Read more…

AMD Chipmaker TSMC to Use AMD Chips for Chipmaking

May 8, 2021

TSMC has tapped AMD to support its major manufacturing and R&D workloads. AMD will provide its Epyc Rome 7702P CPUs – with 64 cores operating at a base cl Read more…

Intel Launches 10nm ‘Ice Lake’ Datacenter CPU with Up to 40 Cores

April 6, 2021

The wait is over. Today Intel officially launched its 10nm datacenter CPU, the third-generation Intel Xeon Scalable processor, codenamed Ice Lake. With up to 40 Read more…

Julia Update: Adoption Keeps Climbing; Is It a Python Challenger?

January 13, 2021

The rapid adoption of Julia, the open source, high level programing language with roots at MIT, shows no sign of slowing according to data from Julialang.org. I Read more…

CERN Is Betting Big on Exascale

April 1, 2021

The European Organization for Nuclear Research (CERN) involves 23 countries, 15,000 researchers, billions of dollars a year, and the biggest machine in the worl Read more…

HPE Launches Storage Line Loaded with IBM’s Spectrum Scale File System

April 6, 2021

HPE today launched a new family of storage solutions bundled with IBM’s Spectrum Scale Erasure Code Edition parallel file system (description below) and featu Read more…

10nm, 7nm, 5nm…. Should the Chip Nanometer Metric Be Replaced?

June 1, 2020

The biggest cool factor in server chips is the nanometer. AMD beating Intel to a CPU built on a 7nm process node* – with 5nm and 3nm on the way – has been i Read more…

Saudi Aramco Unveils Dammam 7, Its New Top Ten Supercomputer

January 21, 2021

By revenue, oil and gas giant Saudi Aramco is one of the largest companies in the world, and it has historically employed commensurate amounts of supercomputing Read more…

Quantum Computer Start-up IonQ Plans IPO via SPAC

March 8, 2021

IonQ, a Maryland-based quantum computing start-up working with ion trap technology, plans to go public via a Special Purpose Acquisition Company (SPAC) merger a Read more…

Leading Solution Providers

Contributors

AMD Launches Epyc ‘Milan’ with 19 SKUs for HPC, Enterprise and Hyperscale

March 15, 2021

At a virtual launch event held today (Monday), AMD revealed its third-generation Epyc “Milan” CPU lineup: a set of 19 SKUs -- including the flagship 64-core, 280-watt 7763 part --  aimed at HPC, enterprise and cloud workloads. Notably, the third-gen Epyc Milan chips achieve 19 percent... Read more…

Can Deep Learning Replace Numerical Weather Prediction?

March 3, 2021

Numerical weather prediction (NWP) is a mainstay of supercomputing. Some of the first applications of the first supercomputers dealt with climate modeling, and Read more…

Livermore’s El Capitan Supercomputer to Debut HPE ‘Rabbit’ Near Node Local Storage

February 18, 2021

A near node local storage innovation called Rabbit factored heavily into Lawrence Livermore National Laboratory’s decision to select Cray’s proposal for its CORAL-2 machine, the lab’s first exascale-class supercomputer, El Capitan. Details of this new storage technology were revealed... Read more…

African Supercomputing Center Inaugurates ‘Toubkal,’ Most Powerful Supercomputer on the Continent

February 25, 2021

Historically, Africa hasn’t exactly been synonymous with supercomputing. There are only a handful of supercomputers on the continent, with few ranking on the Read more…

GTC21: Nvidia Launches cuQuantum; Dips a Toe in Quantum Computing

April 13, 2021

Yesterday Nvidia officially dipped a toe into quantum computing with the launch of cuQuantum SDK, a development platform for simulating quantum circuits on GPU-accelerated systems. As Nvidia CEO Jensen Huang emphasized in his keynote, Nvidia doesn’t plan to build... Read more…

New Deep Learning Algorithm Solves Rubik’s Cube

July 25, 2018

Solving (and attempting to solve) Rubik’s Cube has delighted millions of puzzle lovers since 1974 when the cube was invented by Hungarian sculptor and archite Read more…

The History of Supercomputing vs. COVID-19

March 9, 2021

The COVID-19 pandemic poses a greater challenge to the high-performance computing community than any before. HPCwire's coverage of the supercomputing response t Read more…

Microsoft to Provide World’s Most Powerful Weather & Climate Supercomputer for UK’s Met Office

April 22, 2021

More than 14 months ago, the UK government announced plans to invest £1.2 billion ($1.56 billion) into weather and climate supercomputing, including procuremen Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire