IBM Phase Change Device Shows Promise for Emerging AI Apps

By John Russell

August 3, 2016

IBM Research today announced a significant advance in phase-change memristive technology – based on chalcogenide-based phase-change materials – that IBM says has the potential to achieve much higher neuronal circuit densities and lower power in neuromorphic computing. Also, for the first time, researchers demonstrated randomly spiking neurons based on the technology and were able to detect “temporal correlations in parallel data streams and in sub-Nyquist representation of high-bandwidth signals.”

The announcement coincided with the publishing of a paper, Stochastic phase-changing neurons, in Nature Nanotechnology. The work, largely performed at IBM Research Zurich, is part of a sizable body of exciting research currently occurring in the neuromorphic space.

Turning on two large-scale neuromorphic computers – SpiNNaker and BrainScaleS – this spring and making them readily available to researchers marked another major step forward in pursuit of brain-inspired computing. These machines, based on different neuromorphic architectures, mimic much more closely the way brains process information. Why this is important is perhaps best illustrated by an earlier effort to map neuron-like processing onto a traditional supercomputer, in this case Japan’s K Computer.

“They used about 65000 processors [to create] a billion very simple neurons. Call it one percent of the brain – but it’s not really a brain, [just] a very simple network model. This machine consumes the power of 30MW and the system runs 1500X slower than real-time biology,” said Karlheinz Meier, a leader in the Human Brain Project and whose group developed the BrainScaleS machine. In terms of energy efficiency, this K Computer effort was “ten billion times less energy efficient.”

Evangelos Eleftheriou, IBM Fellow
Evangelos Eleftheriou, IBM Fellow

The IBM research takes substantial steps forward on several fronts. “We have been researching phase-change materials for memory applications for over a decade, and our progress in the past 24 months has been remarkable,” said IBM Fellow Evangelos Eleftheriou. “In this period, we have discovered and published new memory techniques, including projected memory, stored 3 bits per cell in phase-change memory for the first time, and now are demonstrating the powerful capabilities of phase-change-based artificial neurons, which can perform various computational primitives such as data-correlation detection and unsupervised learning at high speeds using very little energy.”

IBM scientists organized hundreds of artificial neurons into populations and used them to represent fast and complex signals. Moreover, the artificial neurons have been shown to sustain billions of switching cycles, which would correspond to multiple years of operation at an update frequency of 100 Hz. The energy required for each neuron update was less than five picojoules and the average power less than 120 microwatts — for comparison, 60 million microwatts power a 60 watt lightbulb.

As Eleftheriou notes memristor technology is hardly new, and as promising as it is, there have also been obstacles, notably variability and scaling, but the IBM approach overcomes some of those issues.

Co-author Dr. Abu Sebastian, IBM Research, told HPCwire, “Variability is indeed a big issue for resistive memory technologies such as the ones based on metal-oxides (TiOx, HfOx etc.). But not so much in phase change memory (PCM) devices. We still have a certain amount of inter-device and intra-device variability even in PCM devices. Clearly this type of randomness would be undesirable for memory-type of applications. However, for neuromorphic applications, this could even be advantageous as we show in our paper. The stochastic firing response of the phase change neurons and their ability to represent high frequency signals arise from this randomness or variability.”

IBM Phase Change Device
IBM Phase Change Device

Just the mechanism of IBM’s phase change device technology is fascinating. The artificial neurons designed by IBM scientists in Zurich consist of phase-change materials, including germanium antimony telluride, which exhibit two stable states, an amorphous one (without a clearly defined structure) and a crystalline one (with structure). These materials are the basis of re-writable Blue-ray discs. However, the artificial neurons do not store digital information; they are analog, just like the synapses and neurons in our biological brain.

In the published demonstration, the team applied a series of electrical pulses to the artificial neurons, which resulted in the progressive crystallization of the phase-change material, ultimately causing the neuron to fire. In neuroscience, this function is known as the integrate-and-fire property of biological neurons. This is the foundation for event-based computation and, in principle, is similar to how our brain triggers a response when we touch something hot.

The authors write in the conclusion:

“The ability to represent the membrane potential in artificial spiking neurons renders phase-change devices a promising technology for extremely dense memristive neuro-synaptic arrays. A particularly interesting property is their scalability down to the nanometer scale and the fast and well-understood dynamics of the amorphous-to-crystalline transition. The high speed and low energy at which phase-change neurons operate will be particularly useful in emerging applications such as processing of event-based sensory information, low-power perceptual decision making and probabilistic inference in uncertain conditions. We also envisage that distributed analysis of rapidly emerging, pervasive data sources such as social media data and the ‘Internet of Things’ could benefit from low-power, memristive computational primitives.”

A figure from the paper, shown below, illustrates the principal.

IBM.Fig1

Artificial neuron based on a phase-change device, with an array of plastic synapses at its input. Schematic of an artificial neuron that consists of the input (dendrites), the soma (which comprises the neuronal membrane and the spike event generation mechanism) and the output (axon). The dendrites may be connected to plastic synapses interfacing the neuron with other neurons in a network. The key computational element is the neuronal membrane, which stores the membrane potential in the phase configuration of a nanoscale phase-change device. Owing to their inherent nanosecond-timescale dynamics, nanometre-length-scale dimensions and native stochasticity, these devices enable the emulation of large and dense populations of neurons for bioinspired signal representation and computation.

 

“By exploiting the physics of reversible amorphous-to-crystal phase transitions, we show that the temporal integration of postsynaptic potentials can be achieved on a nanosecond timescale. Moreover, we show that this is inherently stochastic because of the melt-quench-induced reconfiguration of the atomic structure occurring when the neuron is reset,” write the researchers in the paper.

Mimicking threshold-based neuronal models with traditional technology can be problematic, note the researchers. “Emulating these by means of conventional CMOS circuits, such as current-mode, voltage-mode and subthreshold transistor circuits, is relatively complex and hinders seamless integration with highly dense synaptic arrays. Moreover, conventional CMOS solutions rely on storing the membrane potential in a capacitor. Even with a drastic scaling of the technology node, realizing the capacitance densities measured in biological neuronal membranes is challenging.”

(A brief note on the phase-change device attributes is included at the end of this article.)

One interesting aspect of the work is demonstration of stochastic behavior of artificial neurons based on the phase-change technology. The authors believe they can turn this into an advantage. The stochastic behavior of neurons in nature results from many sources such as the ionic conductance noise, the chaotic motion of charge carriers due to thermal noise, inter-neuron morphologic variabilities and background noise. This complexity has “restricted the implementation of artificial noisy integrate-and-fire neurons in software simulations”, despite their importance in bio-inspired computation and applications in signal and information processing.

The phase change devices, it turns out, also exhibit stochastic behavior for similar reasons. Think of a population of 1000 artificial neurons based on IBM’s phase-change technology. Broadly, when fed the same signal trains, they will fire at slightly differing rates that track well with standard probability models. This can be extremely useful when using a population of artificial neurons for some tasks.

“The relatively complex computational tasks, such as Bayesian inference, that stochastic neuronal populations can perform with collocated processing and storage render them attractive as a possible alternative to von-Neumann-based algorithms in future cognitive computers.”

Here’s a link to a Youtube video by IBM on the technology posted today.

Note on Phase-Change Device Attributes Excerpted From the Paper

The mushroom-type phase-change devices used in the experiments were fabricated in the 90 nm technology node with the bottom electrode created using a sublithographic key-hole process27. The phase-change material was doped Ge2Sb2Te5. All experiments were conducted on phase-change devices that had been heavily cycled. The bottom electrode had a radius of 20 nm and a length of 65 nm. The phase-change material was 100 nm thick and extended to the top electrode, the radius of which was 100 nm. For the single- neuron experiments, the phase-change device was operated in series with a resistor of 5 kΩ. The experiments using multiple neurons and experiments with neuronal populations were based on a crossbar topology in which 100 phase-change devices were interconnected in a 10 × 10 array unit, with a lateral field-effect transistor used as the access device. We used multiple array units to reach population sizes of up to 500 neurons.

Link to paper: http://www.nature.com/nnano/journal/v11/n8/full/nnano.2016.70.html

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

NSF Budget Approved for $8.3B in 2020, a 2.5% Increase

January 16, 2020

The National Science Foundation (NSF) has been spared a President Trump-proposed budget cut that would have rolled back its funding to 2012 levels. Congress passed legislation last month that sets the budget at $8.3 bill Read more…

By Staff report

NOAA Updates Its Massive, Supercomputer-Generated Climate Dataset

January 15, 2020

As Australia burns, understanding and mitigating the climate crisis is more urgent than ever. Now, by leveraging the computing resources at the National Energy Research Scientific Computing Center (NERSC), the U.S. National Oceanic and Atmospheric Administration (NOAA) has updated its 20th Century Reanalysis Project (20CR) dataset... Read more…

By Oliver Peckham

Atos-AMD System to Quintuple Supercomputing Power at European Centre for Medium-Range Weather Forecasts

January 15, 2020

The United Kingdom-based European Centre for Medium-Range Weather Forecasts (ECMWF), a supercomputer-powered weather forecasting organization backed by most of the countries in Europe, has signed a four-year, $89-million Read more…

By Oliver Peckham

Julia Programming’s Dramatic Rise in HPC and Elsewhere

January 14, 2020

Back in 2012 a paper by four computer scientists including Alan Edelman of MIT introduced Julia, A Fast Dynamic Language for Technical Computing. At the time, the gold standard programming languages for fast performance Read more…

By John Russell

Quantum Computing, ML Drive 2019 Patent Awards

January 14, 2020

The dizzying pace of technology innovation often fueled by the growing availability of computing horsepower is underscored by the race to develop unique designs and application that can be patented. Among the goals of ma Read more…

By George Leopold

AWS Solution Channel

Challenging the barriers to High Performance Computing in the Cloud

Cloud computing helps democratize High Performance Computing by placing powerful computational capabilities in the hands of more researchers, engineers, and organizations who may lack access to sufficient on-premises infrastructure. Read more…

IBM Accelerated Insights

Intelligent HPC – Keeping Hard Work at Bay(es)

Since the dawn of time, humans have looked for ways to make their lives easier. Over the centuries human ingenuity has given us inventions such as the wheel and simple machines – which help greatly with tasks that would otherwise be extremely laborious. Read more…

Andrew Jones Joins Microsoft Azure HPC Team

January 13, 2020

Andrew Jones announced today he is joining Microsoft as part of the Azure HPC engineering & product team in early February. Jones makes the move after nearly 12 years at the UK HPC consultancy Numerical Algorithms Gr Read more…

By Staff report

Atos-AMD System to Quintuple Supercomputing Power at European Centre for Medium-Range Weather Forecasts

January 15, 2020

The United Kingdom-based European Centre for Medium-Range Weather Forecasts (ECMWF), a supercomputer-powered weather forecasting organization backed by most of Read more…

By Oliver Peckham

Julia Programming’s Dramatic Rise in HPC and Elsewhere

January 14, 2020

Back in 2012 a paper by four computer scientists including Alan Edelman of MIT introduced Julia, A Fast Dynamic Language for Technical Computing. At the time, t Read more…

By John Russell

White House AI Regulatory Guidelines: ‘Remove Impediments to Private-sector AI Innovation’

January 9, 2020

When it comes to new technology, it’s been said government initially stays uninvolved – then gets too involved. The White House’s guidelines for federal a Read more…

By Doug Black

IBM Touts Quantum Network Growth, Improving QC Quality, and Battery Research

January 8, 2020

IBM today announced its Q (quantum) Network community had grown to 100-plus – Delta Airlines and Los Alamos National Laboratory are among most recent addition Read more…

By John Russell

HPCwire Awards Highlight Supercomputing Achievements in the Sciences

January 7, 2020

In November at SC19 in Denver, the HPCwire Readers’ and Editors’ Choice awards program celebrated its 16th year of honoring remarkable achievements in high-performance computing. With categories ranging from Best Use of HPC in Energy to Top HPC-Enabled Scientific Achievement, many of the winners contributed to groundbreaking developments in the sciences. This editorial highlights those awards. Read more…

By Oliver Peckham

Blasts from the (Recent) Past and Hopes for the Future

December 23, 2019

What does 2020 look like to you? What did 2019 look like? Lots happened but the main trends were carryovers from 2018 – AI messaging again blanketed everything; the roll-out of new big machines and exascale announcements continued; processor diversity and system disaggregation kicked up a notch; hyperscalers continued flexing their muscles (think AWS and its Graviton2 processor); and the U.S. and China continued their awkward trade war. Read more…

By John Russell

ARPA-E Applies ML to Power Generation Designs

December 19, 2019

The U.S. Energy Department’s research arm is leveraging machine learning technologies to simplify the design process for energy systems ranging from photovolt Read more…

By George Leopold

Focused on ‘Silicon TAM,’ Intel Puts Gary Patton, Former GlobalFoundries CTO, in Charge of Design Enablement

December 12, 2019

Change within Intel’s upper management – and to its company mission – has continued as a published report has disclosed that chip technology heavyweight G Read more…

By Doug Black

Using AI to Solve One of the Most Prevailing Problems in CFD

October 17, 2019

How can artificial intelligence (AI) and high-performance computing (HPC) solve mesh generation, one of the most commonly referenced problems in computational engineering? A new study has set out to answer this question and create an industry-first AI-mesh application... Read more…

By James Sharpe

D-Wave’s Path to 5000 Qubits; Google’s Quantum Supremacy Claim

September 24, 2019

On the heels of IBM’s quantum news last week come two more quantum items. D-Wave Systems today announced the name of its forthcoming 5000-qubit system, Advantage (yes the name choice isn’t serendipity), at its user conference being held this week in Newport, RI. Read more…

By John Russell

SC19: IBM Changes Its HPC-AI Game Plan

November 25, 2019

It’s probably fair to say IBM is known for big bets. Summit supercomputer – a big win. Red Hat acquisition – looking like a big win. OpenPOWER and Power processors – jury’s out? At SC19, long-time IBMer Dave Turek sketched out a different kind of bet for Big Blue – a small ball strategy, if you’ll forgive the baseball analogy... Read more…

By John Russell

Cray, Fujitsu Both Bringing Fujitsu A64FX-based Supercomputers to Market in 2020

November 12, 2019

The number of top-tier HPC systems makers has shrunk due to a steady march of M&A activity, but there is increased diversity and choice of processing compon Read more…

By Tiffany Trader

Crystal Ball Gazing: IBM’s Vision for the Future of Computing

October 14, 2019

Dario Gil, IBM’s relatively new director of research, painted a intriguing portrait of the future of computing along with a rough idea of how IBM thinks we’ Read more…

By John Russell

Kubernetes, Containers and HPC

September 19, 2019

Software containers and Kubernetes are important tools for building, deploying, running and managing modern enterprise applications at scale and delivering enterprise software faster and more reliably to the end user — while using resources more efficiently and reducing costs. Read more…

By Daniel Gruber, Burak Yenier and Wolfgang Gentzsch, UberCloud

Intel Debuts New GPU – Ponte Vecchio – and Outlines Aspirations for oneAPI

November 17, 2019

Intel today revealed a few more details about its forthcoming Xe line of GPUs – the top SKU is named Ponte Vecchio and will be used in Aurora, the first plann Read more…

By John Russell

Dell Ramps Up HPC Testing of AMD Rome Processors

October 21, 2019

Dell Technologies is wading deeper into the AMD-based systems market with a growing evaluation program for the latest Epyc (Rome) microprocessors from AMD. In a Read more…

By John Russell

Leading Solution Providers

SC 2019 Virtual Booth Video Tour

AMD
AMD
ASROCK RACK
ASROCK RACK
AWS
AWS
CEJN
CJEN
CRAY
CRAY
DDN
DDN
DELL EMC
DELL EMC
IBM
IBM
MELLANOX
MELLANOX
ONE STOP SYSTEMS
ONE STOP SYSTEMS
PANASAS
PANASAS
SIX NINES IT
SIX NINES IT
VERNE GLOBAL
VERNE GLOBAL
WEKAIO
WEKAIO

Julia Programming’s Dramatic Rise in HPC and Elsewhere

January 14, 2020

Back in 2012 a paper by four computer scientists including Alan Edelman of MIT introduced Julia, A Fast Dynamic Language for Technical Computing. At the time, t Read more…

By John Russell

IBM Unveils Latest Achievements in AI Hardware

December 13, 2019

“The increased capabilities of contemporary AI models provide unprecedented recognition accuracy, but often at the expense of larger computational and energet Read more…

By Oliver Peckham

SC19: Welcome to Denver

November 17, 2019

A significant swath of the HPC community has come to Denver for SC19, which began today (Sunday) with a rich technical program. As is customary, the ribbon cutt Read more…

By Tiffany Trader

With the Help of HPC, Astronomers Prepare to Deflect a Real Asteroid

September 26, 2019

For years, NASA has been running simulations of asteroid impacts to understand the risks (and likelihoods) of asteroids colliding with Earth. Now, NASA and the European Space Agency (ESA) are preparing for the next, crucial step in planetary defense against asteroid impacts: physically deflecting a real asteroid. Read more…

By Oliver Peckham

Jensen Huang’s SC19 – Fast Cars, a Strong Arm, and Aiming for the Cloud(s)

November 20, 2019

We’ve come to expect Nvidia CEO Jensen Huang’s annual SC keynote to contain stunning graphics and lively bravado (with plenty of examples) in support of GPU Read more…

By John Russell

Top500: US Maintains Performance Lead; Arm Tops Green500

November 18, 2019

The 54th Top500, revealed today at SC19, is a familiar list: the U.S. Summit (ORNL) and Sierra (LLNL) machines, offering 148.6 and 94.6 petaflops respectively, Read more…

By Tiffany Trader

51,000 Cloud GPUs Converge to Power Neutrino Discovery at the South Pole

November 22, 2019

At the dead center of the South Pole, thousands of sensors spanning a cubic kilometer are buried thousands of meters beneath the ice. The sensors are part of Ic Read more…

By Oliver Peckham

IBM Opens Quantum Computing Center; Announces 53-Qubit Machine

September 19, 2019

Gauging progress in quantum computing is a tricky thing. IBM yesterday announced the opening of the IBM Quantum Computing Center in New York, with five 20-qubit Read more…

By John Russell

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This