IBM Phase Change Device Shows Promise for Emerging AI Apps

By John Russell

August 3, 2016

IBM Research today announced a significant advance in phase-change memristive technology – based on chalcogenide-based phase-change materials – that IBM says has the potential to achieve much higher neuronal circuit densities and lower power in neuromorphic computing. Also, for the first time, researchers demonstrated randomly spiking neurons based on the technology and were able to detect “temporal correlations in parallel data streams and in sub-Nyquist representation of high-bandwidth signals.”

The announcement coincided with the publishing of a paper, Stochastic phase-changing neurons, in Nature Nanotechnology. The work, largely performed at IBM Research Zurich, is part of a sizable body of exciting research currently occurring in the neuromorphic space.

Turning on two large-scale neuromorphic computers – SpiNNaker and BrainScaleS – this spring and making them readily available to researchers marked another major step forward in pursuit of brain-inspired computing. These machines, based on different neuromorphic architectures, mimic much more closely the way brains process information. Why this is important is perhaps best illustrated by an earlier effort to map neuron-like processing onto a traditional supercomputer, in this case Japan’s K Computer.

“They used about 65000 processors [to create] a billion very simple neurons. Call it one percent of the brain – but it’s not really a brain, [just] a very simple network model. This machine consumes the power of 30MW and the system runs 1500X slower than real-time biology,” said Karlheinz Meier, a leader in the Human Brain Project and whose group developed the BrainScaleS machine. In terms of energy efficiency, this K Computer effort was “ten billion times less energy efficient.”

Evangelos Eleftheriou, IBM Fellow
Evangelos Eleftheriou, IBM Fellow

The IBM research takes substantial steps forward on several fronts. “We have been researching phase-change materials for memory applications for over a decade, and our progress in the past 24 months has been remarkable,” said IBM Fellow Evangelos Eleftheriou. “In this period, we have discovered and published new memory techniques, including projected memory, stored 3 bits per cell in phase-change memory for the first time, and now are demonstrating the powerful capabilities of phase-change-based artificial neurons, which can perform various computational primitives such as data-correlation detection and unsupervised learning at high speeds using very little energy.”

IBM scientists organized hundreds of artificial neurons into populations and used them to represent fast and complex signals. Moreover, the artificial neurons have been shown to sustain billions of switching cycles, which would correspond to multiple years of operation at an update frequency of 100 Hz. The energy required for each neuron update was less than five picojoules and the average power less than 120 microwatts — for comparison, 60 million microwatts power a 60 watt lightbulb.

As Eleftheriou notes memristor technology is hardly new, and as promising as it is, there have also been obstacles, notably variability and scaling, but the IBM approach overcomes some of those issues.

Co-author Dr. Abu Sebastian, IBM Research, told HPCwire, “Variability is indeed a big issue for resistive memory technologies such as the ones based on metal-oxides (TiOx, HfOx etc.). But not so much in phase change memory (PCM) devices. We still have a certain amount of inter-device and intra-device variability even in PCM devices. Clearly this type of randomness would be undesirable for memory-type of applications. However, for neuromorphic applications, this could even be advantageous as we show in our paper. The stochastic firing response of the phase change neurons and their ability to represent high frequency signals arise from this randomness or variability.”

IBM Phase Change Device
IBM Phase Change Device

Just the mechanism of IBM’s phase change device technology is fascinating. The artificial neurons designed by IBM scientists in Zurich consist of phase-change materials, including germanium antimony telluride, which exhibit two stable states, an amorphous one (without a clearly defined structure) and a crystalline one (with structure). These materials are the basis of re-writable Blue-ray discs. However, the artificial neurons do not store digital information; they are analog, just like the synapses and neurons in our biological brain.

In the published demonstration, the team applied a series of electrical pulses to the artificial neurons, which resulted in the progressive crystallization of the phase-change material, ultimately causing the neuron to fire. In neuroscience, this function is known as the integrate-and-fire property of biological neurons. This is the foundation for event-based computation and, in principle, is similar to how our brain triggers a response when we touch something hot.

The authors write in the conclusion:

“The ability to represent the membrane potential in artificial spiking neurons renders phase-change devices a promising technology for extremely dense memristive neuro-synaptic arrays. A particularly interesting property is their scalability down to the nanometer scale and the fast and well-understood dynamics of the amorphous-to-crystalline transition. The high speed and low energy at which phase-change neurons operate will be particularly useful in emerging applications such as processing of event-based sensory information, low-power perceptual decision making and probabilistic inference in uncertain conditions. We also envisage that distributed analysis of rapidly emerging, pervasive data sources such as social media data and the ‘Internet of Things’ could benefit from low-power, memristive computational primitives.”

A figure from the paper, shown below, illustrates the principal.

IBM.Fig1

Artificial neuron based on a phase-change device, with an array of plastic synapses at its input. Schematic of an artificial neuron that consists of the input (dendrites), the soma (which comprises the neuronal membrane and the spike event generation mechanism) and the output (axon). The dendrites may be connected to plastic synapses interfacing the neuron with other neurons in a network. The key computational element is the neuronal membrane, which stores the membrane potential in the phase configuration of a nanoscale phase-change device. Owing to their inherent nanosecond-timescale dynamics, nanometre-length-scale dimensions and native stochasticity, these devices enable the emulation of large and dense populations of neurons for bioinspired signal representation and computation.

 

“By exploiting the physics of reversible amorphous-to-crystal phase transitions, we show that the temporal integration of postsynaptic potentials can be achieved on a nanosecond timescale. Moreover, we show that this is inherently stochastic because of the melt-quench-induced reconfiguration of the atomic structure occurring when the neuron is reset,” write the researchers in the paper.

Mimicking threshold-based neuronal models with traditional technology can be problematic, note the researchers. “Emulating these by means of conventional CMOS circuits, such as current-mode, voltage-mode and subthreshold transistor circuits, is relatively complex and hinders seamless integration with highly dense synaptic arrays. Moreover, conventional CMOS solutions rely on storing the membrane potential in a capacitor. Even with a drastic scaling of the technology node, realizing the capacitance densities measured in biological neuronal membranes is challenging.”

(A brief note on the phase-change device attributes is included at the end of this article.)

One interesting aspect of the work is demonstration of stochastic behavior of artificial neurons based on the phase-change technology. The authors believe they can turn this into an advantage. The stochastic behavior of neurons in nature results from many sources such as the ionic conductance noise, the chaotic motion of charge carriers due to thermal noise, inter-neuron morphologic variabilities and background noise. This complexity has “restricted the implementation of artificial noisy integrate-and-fire neurons in software simulations”, despite their importance in bio-inspired computation and applications in signal and information processing.

The phase change devices, it turns out, also exhibit stochastic behavior for similar reasons. Think of a population of 1000 artificial neurons based on IBM’s phase-change technology. Broadly, when fed the same signal trains, they will fire at slightly differing rates that track well with standard probability models. This can be extremely useful when using a population of artificial neurons for some tasks.

“The relatively complex computational tasks, such as Bayesian inference, that stochastic neuronal populations can perform with collocated processing and storage render them attractive as a possible alternative to von-Neumann-based algorithms in future cognitive computers.”

Here’s a link to a Youtube video by IBM on the technology posted today.

Note on Phase-Change Device Attributes Excerpted From the Paper

The mushroom-type phase-change devices used in the experiments were fabricated in the 90 nm technology node with the bottom electrode created using a sublithographic key-hole process27. The phase-change material was doped Ge2Sb2Te5. All experiments were conducted on phase-change devices that had been heavily cycled. The bottom electrode had a radius of 20 nm and a length of 65 nm. The phase-change material was 100 nm thick and extended to the top electrode, the radius of which was 100 nm. For the single- neuron experiments, the phase-change device was operated in series with a resistor of 5 kΩ. The experiments using multiple neurons and experiments with neuronal populations were based on a crossbar topology in which 100 phase-change devices were interconnected in a 10 × 10 array unit, with a lateral field-effect transistor used as the access device. We used multiple array units to reach population sizes of up to 500 neurons.

Link to paper: http://www.nature.com/nnano/journal/v11/n8/full/nnano.2016.70.html

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Talk to Me: Nvidia Claims NLP Inference, Training Records

August 15, 2019

Nvidia says it’s achieved significant advances in conversation natural language processing (NLP) training and inference, enabling more complex, immediate-response interchanges between customers and chatbots. And the co Read more…

By Doug Black

Trump Administration and NIST Issue AI Standards Development Plan

August 14, 2019

Efforts to develop AI are gathering steam fast. On Monday, the White House issued a federal plan to help develop technical standards for AI following up on a mandate contained in the Administration’s AI Executive Order Read more…

By John Russell

Scientists to Tap Exascale Computing to Unlock the Mystery of our Accelerating Universe

August 14, 2019

The universe and everything in it roared to life with the Big Bang approximately 13.8 billion years ago. It has continued expanding ever since. While we have a good understanding of the early universe, its fate billions Read more…

By Rob Johnson

AWS Solution Channel

Efficiency and Cost-Optimization for HPC Workloads – AWS Batch and Amazon EC2 Spot Instances

High Performance Computing on AWS leverages the power of cloud computing and the extreme scale it offers to achieve optimal HPC price/performance. With AWS you can right size your services to meet exactly the capacity requirements you need without having to overprovision or compromise capacity. Read more…

HPE Extreme Performance Solutions

Bring the combined power of HPC and AI to your business transformation

FPGA (Field Programmable Gate Array) acceleration cards are not new, as they’ve been commercially available since 1984. Typically, the emphasis around FPGAs has centered on the fact that they’re programmable accelerators, and that they can truly offer workload specific hardware acceleration solutions without requiring custom silicon. Read more…

IBM Accelerated Insights

Cloudy with a Chance of Mainframes

[Connect with HPC users and learn new skills in the IBM Spectrum LSF User Community.]

Rapid rates of change sometimes result in unexpected bedfellows. Read more…

Argonne Supercomputer Accelerates Cancer Prediction Research

August 13, 2019

In the fight against cancer, early prediction, which drastically improves prognoses, is critical. Now, new research by a team from Northwestern University – and accelerated by supercomputing resources at Argonne Nation Read more…

By Oliver Peckham

Scientists to Tap Exascale Computing to Unlock the Mystery of our Accelerating Universe

August 14, 2019

The universe and everything in it roared to life with the Big Bang approximately 13.8 billion years ago. It has continued expanding ever since. While we have a Read more…

By Rob Johnson

AI is the Next Exascale – Rick Stevens on What that Means and Why It’s Important

August 13, 2019

Twelve years ago the Department of Energy (DOE) was just beginning to explore what an exascale computing program might look like and what it might accomplish. Today, DOE is repeating that process for AI, once again starting with science community town halls to gather input and stimulate conversation. The town hall program... Read more…

By Tiffany Trader and John Russell

Cray Wins NNSA-Livermore ‘El Capitan’ Exascale Contract

August 13, 2019

Cray has won the bid to build the first exascale supercomputer for the National Nuclear Security Administration (NNSA) and Lawrence Livermore National Laborator Read more…

By Tiffany Trader

AMD Launches Epyc Rome, First 7nm CPU

August 8, 2019

From a gala event at the Palace of Fine Arts in San Francisco yesterday (Aug. 7), AMD launched its second-generation Epyc Rome x86 chips, based on its 7nm proce Read more…

By Tiffany Trader

Lenovo Drives Single-Socket Servers with AMD Epyc Rome CPUs

August 7, 2019

No summer doldrums here. As part of the AMD Epyc Rome launch event in San Francisco today, Lenovo announced two new single-socket servers, the ThinkSystem SR635 Read more…

By Doug Black

Building Diversity and Broader Engagement in the HPC Community

August 7, 2019

Increasing diversity and inclusion in HPC is a community-building effort. Representation of both issues and individuals matters - the more people see HPC in a w Read more…

By AJ Lauer

Xilinx vs. Intel: FPGA Market Leaders Launch Server Accelerator Cards

August 6, 2019

The two FPGA market leaders, Intel and Xilinx, both announced new accelerator cards this week designed to handle specialized, compute-intensive workloads and un Read more…

By Doug Black

Upcoming NSF Cyberinfrastructure Projects to Support ‘Long-Tail’ Users, AI and Big Data

August 5, 2019

The National Science Foundation is well positioned to support national priorities, as new NSF-funded HPC systems to come online in the upcoming year promise to Read more…

By Ken Chiacchia, Pittsburgh Supercomputing Center/XSEDE

High Performance (Potato) Chips

May 5, 2006

In this article, we focus on how Procter & Gamble is using high performance computing to create some common, everyday supermarket products. Tom Lange, a 27-year veteran of the company, tells us how P&G models products, processes and production systems for the betterment of consumer package goods. Read more…

By Michael Feldman

Supercomputer-Powered AI Tackles a Key Fusion Energy Challenge

August 7, 2019

Fusion energy is the Holy Grail of the energy world: low-radioactivity, low-waste, zero-carbon, high-output nuclear power that can run on hydrogen or lithium. T Read more…

By Oliver Peckham

Cray, AMD to Extend DOE’s Exascale Frontier

May 7, 2019

Cray and AMD are coming back to Oak Ridge National Laboratory to partner on the world’s largest and most expensive supercomputer. The Department of Energy’s Read more…

By Tiffany Trader

Graphene Surprises Again, This Time for Quantum Computing

May 8, 2019

Graphene is fascinating stuff with promise for use in a seeming endless number of applications. This month researchers from the University of Vienna and Institu Read more…

By John Russell

AMD Verifies Its Largest 7nm Chip Design in Ten Hours

June 5, 2019

AMD announced last week that its engineers had successfully executed the first physical verification of its largest 7nm chip design – in just ten hours. The AMD Radeon Instinct Vega20 – which boasts 13.2 billion transistors – was tested using a TSMC-certified Calibre nmDRC software platform from Mentor. Read more…

By Oliver Peckham

TSMC and Samsung Moving to 5nm; Whither Moore’s Law?

June 12, 2019

With reports that Taiwan Semiconductor Manufacturing Co. (TMSC) and Samsung are moving quickly to 5nm manufacturing, it’s a good time to again ponder whither goes the venerable Moore’s law. Shrinking feature size has of course been the primary hallmark of achieving Moore’s law... Read more…

By John Russell

Deep Learning Competitors Stalk Nvidia

May 14, 2019

There is no shortage of processing architectures emerging to accelerate deep learning workloads, with two more options emerging this week to challenge GPU leader Nvidia. First, Intel researchers claimed a new deep learning record for image classification on the ResNet-50 convolutional neural network. Separately, Israeli AI chip startup Hailo.ai... Read more…

By George Leopold

Cray Wins NNSA-Livermore ‘El Capitan’ Exascale Contract

August 13, 2019

Cray has won the bid to build the first exascale supercomputer for the National Nuclear Security Administration (NNSA) and Lawrence Livermore National Laborator Read more…

By Tiffany Trader

Leading Solution Providers

ISC 2019 Virtual Booth Video Tour

CRAY
CRAY
DDN
DDN
DELL EMC
DELL EMC
GOOGLE
GOOGLE
ONE STOP SYSTEMS
ONE STOP SYSTEMS
PANASAS
PANASAS
VERNE GLOBAL
VERNE GLOBAL

Nvidia Embraces Arm, Declares Intent to Accelerate All CPU Architectures

June 17, 2019

As the Top500 list was being announced at ISC in Frankfurt today with an upgraded petascale Arm supercomputer in the top third of the list, Nvidia announced its Read more…

By Tiffany Trader

Top500 Purely Petaflops; US Maintains Performance Lead

June 17, 2019

With the kick-off of the International Supercomputing Conference (ISC) in Frankfurt this morning, the 53rd Top500 list made its debut, and this one's for petafl Read more…

By Tiffany Trader

A Behind-the-Scenes Look at the Hardware That Powered the Black Hole Image

June 24, 2019

Two months ago, the first-ever image of a black hole took the internet by storm. A team of scientists took years to produce and verify the striking image – an Read more…

By Oliver Peckham

AMD Launches Epyc Rome, First 7nm CPU

August 8, 2019

From a gala event at the Palace of Fine Arts in San Francisco yesterday (Aug. 7), AMD launched its second-generation Epyc Rome x86 chips, based on its 7nm proce Read more…

By Tiffany Trader

Cray – and the Cray Brand – to Be Positioned at Tip of HPE’s HPC Spear

May 22, 2019

More so than with most acquisitions of this kind, HPE’s purchase of Cray for $1.3 billion, announced last week, seems to have elements of that overused, often Read more…

By Doug Black and Tiffany Trader

Chinese Company Sugon Placed on US ‘Entity List’ After Strong Showing at International Supercomputing Conference

June 26, 2019

After more than a decade of advancing its supercomputing prowess, operating the world’s most powerful supercomputer from June 2013 to June 2018, China is keep Read more…

By Tiffany Trader

In Wake of Nvidia-Mellanox: Xilinx to Acquire Solarflare

April 25, 2019

With echoes of Nvidia’s recent acquisition of Mellanox, FPGA maker Xilinx has announced a definitive agreement to acquire Solarflare Communications, provider Read more…

By Doug Black

Qualcomm Invests in RISC-V Startup SiFive

June 7, 2019

Investors are zeroing in on the open standard RISC-V instruction set architecture and the processor intellectual property being developed by a batch of high-flying chip startups. Last fall, Esperanto Technologies announced a $58 million funding round. Read more…

By George Leopold

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This