IBM Phase Change Device Shows Promise for Emerging AI Apps

By John Russell

August 3, 2016

IBM Research today announced a significant advance in phase-change memristive technology – based on chalcogenide-based phase-change materials – that IBM says has the potential to achieve much higher neuronal circuit densities and lower power in neuromorphic computing. Also, for the first time, researchers demonstrated randomly spiking neurons based on the technology and were able to detect “temporal correlations in parallel data streams and in sub-Nyquist representation of high-bandwidth signals.”

The announcement coincided with the publishing of a paper, Stochastic phase-changing neurons, in Nature Nanotechnology. The work, largely performed at IBM Research Zurich, is part of a sizable body of exciting research currently occurring in the neuromorphic space.

Turning on two large-scale neuromorphic computers – SpiNNaker and BrainScaleS – this spring and making them readily available to researchers marked another major step forward in pursuit of brain-inspired computing. These machines, based on different neuromorphic architectures, mimic much more closely the way brains process information. Why this is important is perhaps best illustrated by an earlier effort to map neuron-like processing onto a traditional supercomputer, in this case Japan’s K Computer.

“They used about 65000 processors [to create] a billion very simple neurons. Call it one percent of the brain – but it’s not really a brain, [just] a very simple network model. This machine consumes the power of 30MW and the system runs 1500X slower than real-time biology,” said Karlheinz Meier, a leader in the Human Brain Project and whose group developed the BrainScaleS machine. In terms of energy efficiency, this K Computer effort was “ten billion times less energy efficient.”

Evangelos Eleftheriou, IBM Fellow
Evangelos Eleftheriou, IBM Fellow

The IBM research takes substantial steps forward on several fronts. “We have been researching phase-change materials for memory applications for over a decade, and our progress in the past 24 months has been remarkable,” said IBM Fellow Evangelos Eleftheriou. “In this period, we have discovered and published new memory techniques, including projected memory, stored 3 bits per cell in phase-change memory for the first time, and now are demonstrating the powerful capabilities of phase-change-based artificial neurons, which can perform various computational primitives such as data-correlation detection and unsupervised learning at high speeds using very little energy.”

IBM scientists organized hundreds of artificial neurons into populations and used them to represent fast and complex signals. Moreover, the artificial neurons have been shown to sustain billions of switching cycles, which would correspond to multiple years of operation at an update frequency of 100 Hz. The energy required for each neuron update was less than five picojoules and the average power less than 120 microwatts — for comparison, 60 million microwatts power a 60 watt lightbulb.

As Eleftheriou notes memristor technology is hardly new, and as promising as it is, there have also been obstacles, notably variability and scaling, but the IBM approach overcomes some of those issues.

Co-author Dr. Abu Sebastian, IBM Research, told HPCwire, “Variability is indeed a big issue for resistive memory technologies such as the ones based on metal-oxides (TiOx, HfOx etc.). But not so much in phase change memory (PCM) devices. We still have a certain amount of inter-device and intra-device variability even in PCM devices. Clearly this type of randomness would be undesirable for memory-type of applications. However, for neuromorphic applications, this could even be advantageous as we show in our paper. The stochastic firing response of the phase change neurons and their ability to represent high frequency signals arise from this randomness or variability.”

IBM Phase Change Device
IBM Phase Change Device

Just the mechanism of IBM’s phase change device technology is fascinating. The artificial neurons designed by IBM scientists in Zurich consist of phase-change materials, including germanium antimony telluride, which exhibit two stable states, an amorphous one (without a clearly defined structure) and a crystalline one (with structure). These materials are the basis of re-writable Blue-ray discs. However, the artificial neurons do not store digital information; they are analog, just like the synapses and neurons in our biological brain.

In the published demonstration, the team applied a series of electrical pulses to the artificial neurons, which resulted in the progressive crystallization of the phase-change material, ultimately causing the neuron to fire. In neuroscience, this function is known as the integrate-and-fire property of biological neurons. This is the foundation for event-based computation and, in principle, is similar to how our brain triggers a response when we touch something hot.

The authors write in the conclusion:

“The ability to represent the membrane potential in artificial spiking neurons renders phase-change devices a promising technology for extremely dense memristive neuro-synaptic arrays. A particularly interesting property is their scalability down to the nanometer scale and the fast and well-understood dynamics of the amorphous-to-crystalline transition. The high speed and low energy at which phase-change neurons operate will be particularly useful in emerging applications such as processing of event-based sensory information, low-power perceptual decision making and probabilistic inference in uncertain conditions. We also envisage that distributed analysis of rapidly emerging, pervasive data sources such as social media data and the ‘Internet of Things’ could benefit from low-power, memristive computational primitives.”

A figure from the paper, shown below, illustrates the principal.

IBM.Fig1

Artificial neuron based on a phase-change device, with an array of plastic synapses at its input. Schematic of an artificial neuron that consists of the input (dendrites), the soma (which comprises the neuronal membrane and the spike event generation mechanism) and the output (axon). The dendrites may be connected to plastic synapses interfacing the neuron with other neurons in a network. The key computational element is the neuronal membrane, which stores the membrane potential in the phase configuration of a nanoscale phase-change device. Owing to their inherent nanosecond-timescale dynamics, nanometre-length-scale dimensions and native stochasticity, these devices enable the emulation of large and dense populations of neurons for bioinspired signal representation and computation.

 

“By exploiting the physics of reversible amorphous-to-crystal phase transitions, we show that the temporal integration of postsynaptic potentials can be achieved on a nanosecond timescale. Moreover, we show that this is inherently stochastic because of the melt-quench-induced reconfiguration of the atomic structure occurring when the neuron is reset,” write the researchers in the paper.

Mimicking threshold-based neuronal models with traditional technology can be problematic, note the researchers. “Emulating these by means of conventional CMOS circuits, such as current-mode, voltage-mode and subthreshold transistor circuits, is relatively complex and hinders seamless integration with highly dense synaptic arrays. Moreover, conventional CMOS solutions rely on storing the membrane potential in a capacitor. Even with a drastic scaling of the technology node, realizing the capacitance densities measured in biological neuronal membranes is challenging.”

(A brief note on the phase-change device attributes is included at the end of this article.)

One interesting aspect of the work is demonstration of stochastic behavior of artificial neurons based on the phase-change technology. The authors believe they can turn this into an advantage. The stochastic behavior of neurons in nature results from many sources such as the ionic conductance noise, the chaotic motion of charge carriers due to thermal noise, inter-neuron morphologic variabilities and background noise. This complexity has “restricted the implementation of artificial noisy integrate-and-fire neurons in software simulations”, despite their importance in bio-inspired computation and applications in signal and information processing.

The phase change devices, it turns out, also exhibit stochastic behavior for similar reasons. Think of a population of 1000 artificial neurons based on IBM’s phase-change technology. Broadly, when fed the same signal trains, they will fire at slightly differing rates that track well with standard probability models. This can be extremely useful when using a population of artificial neurons for some tasks.

“The relatively complex computational tasks, such as Bayesian inference, that stochastic neuronal populations can perform with collocated processing and storage render them attractive as a possible alternative to von-Neumann-based algorithms in future cognitive computers.”

Here’s a link to a Youtube video by IBM on the technology posted today.

Note on Phase-Change Device Attributes Excerpted From the Paper

The mushroom-type phase-change devices used in the experiments were fabricated in the 90 nm technology node with the bottom electrode created using a sublithographic key-hole process27. The phase-change material was doped Ge2Sb2Te5. All experiments were conducted on phase-change devices that had been heavily cycled. The bottom electrode had a radius of 20 nm and a length of 65 nm. The phase-change material was 100 nm thick and extended to the top electrode, the radius of which was 100 nm. For the single- neuron experiments, the phase-change device was operated in series with a resistor of 5 kΩ. The experiments using multiple neurons and experiments with neuronal populations were based on a crossbar topology in which 100 phase-change devices were interconnected in a 10 × 10 array unit, with a lateral field-effect transistor used as the access device. We used multiple array units to reach population sizes of up to 500 neurons.

Link to paper: http://www.nature.com/nnano/journal/v11/n8/full/nnano.2016.70.html

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Supercomputer Modeling Shows How COVID-19 Spreads Through Populations

May 30, 2020

As many states begin to loosen the lockdowns and stay-at-home orders that have forced most Americans inside for the past two months, researchers are poring over the data, looking for signs of the dreaded second peak of t Read more…

By Oliver Peckham

SODALITE: Towards Automated Optimization of HPC Application Deployment

May 29, 2020

Developing and deploying applications across heterogeneous infrastructures like HPC or Cloud with diverse hardware is a complex problem. Enabling developers to describe the application deployment and optimising runtime p Read more…

By the SODALITE Team

What’s New in HPC Research: Astronomy, Weather, Security & More

May 29, 2020

In this bimonthly feature, HPCwire highlights newly published research in the high-performance computing community and related domains. From parallel programming to exascale to quantum computing, the details are here. Read more…

By Oliver Peckham

DARPA Looks to Automate Secure Silicon Designs

May 28, 2020

The U.S. military is ramping up efforts to secure semiconductors and its electronics supply chain by embedding defenses during the chip design phase. The automation effort also addresses the high cost and complexity of s Read more…

By George Leopold

COVID-19 HPC Consortium Expands to Europe, Reports on Research Projects

May 28, 2020

The COVID-19 HPC Consortium, a public-private effort delivering free access to HPC processing for scientists pursuing coronavirus research – some utilizing AI-based techniques – has expanded to more than 56 research Read more…

By Doug Black

AWS Solution Channel

Computational Fluid Dynamics on AWS

Over the past 30 years Computational Fluid Dynamics (CFD) has grown to become a key part of many engineering design processes. From aircraft design to modelling the blood flow in our bodies, the ability to understand the behaviour of fluids has enabled countless innovations and improved the time to market for many products. Read more…

What’s New in Computing vs. COVID-19: IceCube, TACC, Watson & More

May 28, 2020

Supercomputing, big data and artificial intelligence are crucial tools in the fight against the coronavirus pandemic. Around the world, researchers, corporations and governments are urgently devoting their computing reso Read more…

By Oliver Peckham

COVID-19 HPC Consortium Expands to Europe, Reports on Research Projects

May 28, 2020

The COVID-19 HPC Consortium, a public-private effort delivering free access to HPC processing for scientists pursuing coronavirus research – some utilizing AI Read more…

By Doug Black

$100B Plan Submitted for Massive Remake and Expansion of NSF

May 27, 2020

Legislation to reshape, expand - and rename - the National Science Foundation has been submitted in both the U.S. House and Senate. The proposal, which seems to Read more…

By John Russell

IBM Boosts Deep Learning Accuracy on Memristive Chips

May 27, 2020

IBM researchers have taken another step towards making in-memory computing based on phase change (PCM) memory devices a reality. Papers in Nature and Frontiers Read more…

By John Russell

Hats Over Hearts: Remembering Rich Brueckner

May 26, 2020

HPCwire and all of the Tabor Communications family are saddened by last week’s passing of Rich Brueckner. He was the ever-optimistic man in the Red Hat presiding over the InsideHPC media portfolio for the past decade and a constant presence at HPC’s most important events. Read more…

Nvidia Q1 Earnings Top Expectations, Datacenter Revenue Breaks $1B

May 22, 2020

Nvidia’s seemingly endless roll continued in the first quarter with the company announcing blockbuster earnings that exceeded Wall Street expectations. Nvidia Read more…

By Doug Black

Microsoft’s Massive AI Supercomputer on Azure: 285k CPU Cores, 10k GPUs

May 20, 2020

Microsoft has unveiled a supercomputing monster – among the world’s five most powerful, according to the company – aimed at what is known in scientific an Read more…

By Doug Black

HPC in Life Sciences 2020 Part 1: Rise of AMD, Data Management’s Wild West, More 

May 20, 2020

Given the disruption caused by the COVID-19 pandemic and the massive enlistment of major HPC resources to fight the pandemic, it is especially appropriate to re Read more…

By John Russell

AMD Epyc Rome Picked for New Nvidia DGX, but HGX Preserves Intel Option

May 19, 2020

AMD continues to make inroads into the datacenter with its second-generation Epyc "Rome" processor, which last week scored a win with Nvidia's announcement that Read more…

By Tiffany Trader

Supercomputer Modeling Tests How COVID-19 Spreads in Grocery Stores

April 8, 2020

In the COVID-19 era, many people are treating simple activities like getting gas or groceries with caution as they try to heed social distancing mandates and protect their own health. Still, significant uncertainty surrounds the relative risk of different activities, and conflicting information is prevalent. A team of Finnish researchers set out to address some of these uncertainties by... Read more…

By Oliver Peckham

[email protected] Turns Its Massive Crowdsourced Computer Network Against COVID-19

March 16, 2020

For gamers, fighting against a global crisis is usually pure fantasy – but now, it’s looking more like a reality. As supercomputers around the world spin up Read more…

By Oliver Peckham

[email protected] Rallies a Legion of Computers Against the Coronavirus

March 24, 2020

Last week, we highlighted [email protected], a massive, crowdsourced computer network that has turned its resources against the coronavirus pandemic sweeping the globe – but [email protected] isn’t the only game in town. The internet is buzzing with crowdsourced computing... Read more…

By Oliver Peckham

Global Supercomputing Is Mobilizing Against COVID-19

March 12, 2020

Tech has been taking some heavy losses from the coronavirus pandemic. Global supply chains have been disrupted, virtually every major tech conference taking place over the next few months has been canceled... Read more…

By Oliver Peckham

Supercomputer Simulations Reveal the Fate of the Neanderthals

May 25, 2020

For hundreds of thousands of years, neanderthals roamed the planet, eventually (almost 50,000 years ago) giving way to homo sapiens, which quickly became the do Read more…

By Oliver Peckham

DoE Expands on Role of COVID-19 Supercomputing Consortium

March 25, 2020

After announcing the launch of the COVID-19 High Performance Computing Consortium on Sunday, the Department of Energy yesterday provided more details on its sco Read more…

By John Russell

Steve Scott Lays Out HPE-Cray Blended Product Roadmap

March 11, 2020

Last week, the day before the El Capitan processor disclosures were made at HPE's new headquarters in San Jose, Steve Scott (CTO for HPC & AI at HPE, and former Cray CTO) was on-hand at the Rice Oil & Gas HPC conference in Houston. He was there to discuss the HPE-Cray transition and blended roadmap, as well as his favorite topic, Cray's eighth-gen networking technology, Slingshot. Read more…

By Tiffany Trader

Honeywell’s Big Bet on Trapped Ion Quantum Computing

April 7, 2020

Honeywell doesn’t spring to mind when thinking of quantum computing pioneers, but a decade ago the high-tech conglomerate better known for its control systems waded deliberately into the then calmer quantum computing (QC) waters. Fast forward to March when Honeywell announced plans to introduce an ion trap-based quantum computer whose ‘performance’ would... Read more…

By John Russell

Leading Solution Providers

SC 2019 Virtual Booth Video Tour

AMD
AMD
ASROCK RACK
ASROCK RACK
AWS
AWS
CEJN
CJEN
CRAY
CRAY
DDN
DDN
DELL EMC
DELL EMC
IBM
IBM
MELLANOX
MELLANOX
ONE STOP SYSTEMS
ONE STOP SYSTEMS
PANASAS
PANASAS
SIX NINES IT
SIX NINES IT
VERNE GLOBAL
VERNE GLOBAL
WEKAIO
WEKAIO

Contributors

Fujitsu A64FX Supercomputer to Be Deployed at Nagoya University This Summer

February 3, 2020

Japanese tech giant Fujitsu announced today that it will supply Nagoya University Information Technology Center with the first commercial supercomputer powered Read more…

By Tiffany Trader

Tech Conferences Are Being Canceled Due to Coronavirus

March 3, 2020

Several conferences scheduled to take place in the coming weeks, including Nvidia’s GPU Technology Conference (GTC) and the Strata Data + AI conference, have Read more…

By Alex Woodie

Exascale Watch: El Capitan Will Use AMD CPUs & GPUs to Reach 2 Exaflops

March 4, 2020

HPE and its collaborators reported today that El Capitan, the forthcoming exascale supercomputer to be sited at Lawrence Livermore National Laboratory and serve Read more…

By John Russell

Cray to Provide NOAA with Two AMD-Powered Supercomputers

February 24, 2020

The United States’ National Oceanic and Atmospheric Administration (NOAA) last week announced plans for a major refresh of its operational weather forecasting supercomputers, part of a 10-year, $505.2 million program, which will secure two HPE-Cray systems for NOAA’s National Weather Service to be fielded later this year and put into production in early 2022. Read more…

By Tiffany Trader

‘Billion Molecules Against COVID-19’ Challenge to Launch with Massive Supercomputing Support

April 22, 2020

Around the world, supercomputing centers have spun up and opened their doors for COVID-19 research in what may be the most unified supercomputing effort in hist Read more…

By Oliver Peckham

Summit Supercomputer is Already Making its Mark on Science

September 20, 2018

Summit, now the fastest supercomputer in the world, is quickly making its mark in science – five of the six finalists just announced for the prestigious 2018 Read more…

By John Russell

15 Slides on Programming Aurora and Exascale Systems

May 7, 2020

Sometime in 2021, Aurora, the first planned U.S. exascale system, is scheduled to be fired up at Argonne National Laboratory. Cray (now HPE) and Intel are the k Read more…

By John Russell

TACC Supercomputers Run Simulations Illuminating COVID-19, DNA Replication

March 19, 2020

As supercomputers around the world spin up to combat the coronavirus, the Texas Advanced Computing Center (TACC) is announcing results that may help to illumina Read more…

By Staff report

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This