Interdisciplinary Team at UT Austin Envisions New Neuromorphic Computing Architecture

November 26, 2019

Nov. 26, 2019 — If you wanted to deliver a package across the street, you could program a powerful computer to do it, equipped with sensors and hardware capable of running multiple differential equations to track the movement and speed of each car. But a young child would be capable of doing the same task with little effort, says Alex Demkov, professor of Physics at The University of Texas at Austin.

Piezoelectric modulation of nonlinear optical response in BaTiO3 thin film. [Credit: Alexander A. Demkov et al, Appl. Phys. Lett. 113, 132902 (2018); doi: 10.1063/1.5045460]
Image courtesy of TACC.
“I guarantee you the brain is completely unaware that differential equations exist. And yet they solve the same problem of how to avoid collisions with fast approaching movement,” Demkov said. “A child trains from a young age, tossing rocks to learn spatial dimensions, and can do things we can scarcely train powerful computers to do.”

Not only that, the brain uses only a fraction of the energy computer chips require to accomplish the task, and can operate at room temperature.

Why is a physicist, with a background in developing new materials for advanced technologies, wondering about a child’s brain? Because his research has recently led him into the world of neuromorphic computing, an emerging field that emulates the workings of the brain to perform the tasks we now use silicon-based digital processing to accomplish.

Researchers in Demkov’s group study new nanomaterials in the UT Material Physics Lab. Image courtesy of TACC.

In the last century, computer processing power has grown rapidly, roughly doubling every two years (according to Moore’s Law), and in the process has transformed our world. But in recent years, progress has slowed as chips reach the physical limits of miniaturization and require unsustainable amounts of electricity to operate.

Alternative paradigms — some new, some old — are getting a fresh look from researchers. Neuromorphic computing is one such paradigm, as is optical computing where light takes the place of electrons as the transmitter of signals. Demkov happens to be an expert in the latter, having worked for two decades developing novel electronic materials for technologies with support from the Air Force, Navy, and National Science Foundation, often in collaboration with IBM and other industry partners.

In recent years, Demkov has been developing hybrid silicon-photonic technologies based on novel nano-scale materials like silicon-integrated barium titanate, a ceramic material that exhibits exotic and useful properties for information processing.

The challenge with optical computing in the past has been the difficulty in making devices that are both controllable and small enough to use in devices. However, barium titanate, Demkov has found, can transmit light and be switched on and off, using a variety of clever mechanisms, using very little power and on a very small scale. Moreover, it can be fused to silicon to integrate into chips that can, for instance, provide the computing and control mechanisms for aircraft at a fraction of the weight of today’s technologies.

Working with IBM and researchers at ETH-Zurich, Demkov recently demonstrated a system that is much more efficient than the current state of the art. The results were published in Nature Materials in 2018.

“Using new materials and old methods, we might be able to create new neuromorphic computers that are better than silicon-based computers at doing certain kinds of transformations,” Demkov said.

Reconsidering the Neuromorphic Canon

The material and optical characteristics of processors are just a few aspects of the problem that need to be solved to develop a rival for silicon semiconductors, which has a century-long head start in research.

To go from a promising new material to a device that can rival and surpass silicon, quantum computers, and a host of other contenders, Demkov has pulled together a team of researchers from across UT Austin and beyond with expertise in neuroscience, algorithmic development, circuit design, parallel computing, and device architectures to create something that exceeds what leading industry groups like Google, Intel, and Hewlett Packard Enterprise are imagining.

“This is our competitive advantage,” he said. “We have 30 brilliant, diverse researchers working on this problem. We have a good idea. And we are not beholden to existing technologies.”

If the use of silicon digital processors is a problem at the technological end of things, troubles at a conceptual level start with the use of a mathematical formulation of thought first enunciated in 1954.

In recent years, Demkov has been developing hybrid silicon-photonic technologies based on novel nano-scale materials. To transmit signals in this optical computing, electrons are replaced with light, which uses less power and space than conventional chips. Image courtesy of TACC.
In recent years, Demkov has been developing hybrid silicon-photonic technologies based on novel nano-scale materials. To transmit signals in this optical computing, electrons are replaced with light, which uses less power and space than conventional chips.

Traditional neuromorphic computing is rooted in the idea that neurons spike, or react, in order to communicate. This spiking takes the place of the on/off, 0/1 of digital gates – the root of computing.

However, neuroscientists have learned a great deal about the reality of neurons in the past six decades, which are much more complicated and interconnected than the old textbook description would lead one to believe.

“Hodgkin Huxley wrote this set of four equations based on the understanding of cell membrane and transfer circa 1954 and that’s what brain science has been about ever since,” Demkov said. “Now we have functional MRI, we have advanced microscopy, so we surely can develop better models.”

Work by Kristen Harris, a leading neuroscientist at UT Austin who uses neuroimaging and computing to create detailed 3D models of the brain, led Demkov to the realization that the mathematical models of neuro-processing are due for a rewrite.

“Brains are not one bit machines,” explained Harris. “Just based on the structure of synapses in the brain in different brain regions and under different activity levels, we’ve seen 26 different distinct synaptic types. It’s not an on off machine. It has graded levels of capability. And is not digital; it’s analog.”

“The simple explanation needs to be rethought and rearticulated as an algorithm,” Demkov concurred.

Harris was drawn to the project because of the diverse perspectives of the researchers involved. “They’re all thinking outside the box. But they didn’t realize how complicated the box was,” she said. “We may not even speak the same language when we start out, but the goal eventually will be to at least understand each other’s language and then begin to build bridges.”

Separate from the behavior and logic of neurons themselves, a key part of the project is the development of improved neural networks — a method of training computers to “learn” how to do human-like tasks, from identifying images to discovering new scientific theories.

Neural networks, when combined with large amounts of data, have shown themselves to be incredibly effective at solving problems that traditional simulation and modeling are incapable of handing. However, today’s state-of-the-art methods are still slow, require massive amounts of computer power, and have been limited in their applications and robustness.

Demkov and the UT team are eyeing new formulations of neural networks that may be able to work faster using non-linear, random connectivity.

“This particular kind of neuromorphic architecture is called the reservoir computer or an echo state machine,” he explained. “It turns out that there’s a way to realize this type of system in optics which is very, very neat. With neuromorphic computing, you didn’t compute anything. You just train the neural network to use spiking to say yes or no.”

Leading the device design effort on the project is Ray Chen, a chair in the Electrical and Computer Engineering department and director of the Nanophotonics and Optical Interconnects Research Lab at the Microelectronics Research Center.

As part of an Air Force-funded research project, Chen has been experimenting with optical neural networks that could use orders-of-magnitude lower power consumption compared to current CPUs and GPUs. At the Asia and South Pacific Design Automation Conference 2019, Chen presented a software-hardware co-designed slim optical neural networks which demonstrates 15 to 38 percent less phase shift variation than state-of-the-art systems with no accuracy loss and better noise robustness. Recently he proposed an optical neural network architecture to perform Fast Fourier transforms — a type of computations frequently used in engineering and science — that could potentially be three times smaller than previous designs with negligible accuracy degradation.

“A UT neuromorphic computing center would provide the vertical integration of different technology readiness level from basic science to system applications that can significantly upgrade the human and machine interface,” Chen said.

The researchers on the team have a secret weapon at their disposal: the supercomputers at UT’s Texas Advanced Computing Center, including Frontera and Stampede2, the #1 and #2 most powerful supercomputers at any U.S. university.

“These systems allow us to predict the characteristics of materials, circuits, and devices before we construct them, and help us come up with the optimal designs for systems that can do things we’ve never done before,” he said.

Down in the Material Physics Lab, which Demkov co-leads, he shows off a machine made of steel cylinders and thick electrical wires capable of creating ultra-pure silicon photonic materials — the only one of its kind in the world. There he builds and tests nano-scale components invisible to the naked eye that could one day be ubiquitous, allowing computation to further embed itself in our day to day lives.

Despite competition from the world’s most powerful technology companies, Demkov believes UT has the resources and expertise to become the leader in neuromorphic computing.

“Most of the people in industry are working on this problem in a vacuum and are trying to gather ideas from their limited understanding of the literature,” he said. “We have the engine which drives the literature in our midst, and together we can create something truly extraordinary.”


Source:  Aaron Dubrow, Texas Advanced Computing Center, The University of Texas at Austin

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Focused on ‘Silicon TAM,’ Intel Puts Gary Patton, Former GlobalFoundries CTO, in Charge of Design Enablement

December 12, 2019

Change within Intel’s upper management – and to its company mission – has continued as a published report has disclosed that chip technology heavyweight Gary Patton, GlobalFoundries’ CTO and R&D SVP as well a Read more…

By Doug Black

Quantum Bits: Rigetti Debuts New Gates, D-Wave Cuts NEC Deal, AWS Jumps into the Quantum Pool

December 12, 2019

There’s been flurry of significant news in the quantum computing world. Yesterday, Rigetti introduced a new family of gates that reduces circuit depth required on some problems and D-Wave struck a deal with NEC to coll Read more…

By John Russell

How Formula 1 Used Cloud HPC to Build the Next Generation of Racing

December 12, 2019

Formula 1, Rob Smedley explained, is maybe the biggest racing spectacle in the world, with five hundred million fans tuning in for every race. Smedley, a chief engineer with Formula 1’s performance engineering and anal Read more…

By Oliver Peckham

RPI Powers Up ‘AiMOS’ AI Supercomputer

December 11, 2019

Designed to push the frontiers of computing chip and systems performance optimized for AI workloads, an 8 petaflops (Linpack) IBM Power9-based supercomputer has been unveiled in upstate New York that will be used by IBM Read more…

By Doug Black

At SC19: Developing a Digital Twin

December 11, 2019

In the not too distant future, we can expect to see our skies filled with unmanned aerial vehicles (UAVs) delivering packages, maybe even people, from location to location. In such a world, there will also be a digital twin for each UAV in the fleet: a virtual model that will follow the UAV through its existence, evolving with time. Read more…

By Aaron Dubrow

AWS Solution Channel

Making High Performance Computing Affordable and Accessible for Small and Medium Businesses with HPC on AWS

High performance computing (HPC) brings a powerful set of tools to a broad range of industries, helping to drive innovation and boost revenue in finance, genomics, oil and gas extraction, and other fields. Read more…

IBM Accelerated Insights

GPU Scheduling and Resource Accounting: The Key to an Efficient AI Data Center

[Connect with LSF users and learn new skills in the IBM Spectrum LSF User Community!]

GPUs are the new CPUs

GPUs have become a staple technology in modern HPC and AI data centers. Read more…

Supercomputers Help Predict Carbon Dioxide Levels

December 10, 2019

The Earth’s terrestrial ecosystems – its lands, forests, jungles and so on – are crucial “sinks” for atmospheric carbon, holding nearly 30 percent of our annual CO2 emissions as they breathe in the carbon-rich Read more…

By Oliver Peckham

Focused on ‘Silicon TAM,’ Intel Puts Gary Patton, Former GlobalFoundries CTO, in Charge of Design Enablement

December 12, 2019

Change within Intel’s upper management – and to its company mission – has continued as a published report has disclosed that chip technology heavyweight G Read more…

By Doug Black

RPI Powers Up ‘AiMOS’ AI Supercomputer

December 11, 2019

Designed to push the frontiers of computing chip and systems performance optimized for AI workloads, an 8 petaflops (Linpack) IBM Power9-based supercomputer has Read more…

By Doug Black

At SC19: Developing a Digital Twin

December 11, 2019

In the not too distant future, we can expect to see our skies filled with unmanned aerial vehicles (UAVs) delivering packages, maybe even people, from location to location. In such a world, there will also be a digital twin for each UAV in the fleet: a virtual model that will follow the UAV through its existence, evolving with time. Read more…

By Aaron Dubrow

Intel’s Jim Clarke on its New Cryo-controller and why Intel isn’t Late to the Quantum Party

December 9, 2019

Intel today introduced the ‘first-of-its-kind’ cryo-controller chip for quantum computing and previewed a cryo-prober tool for characterizing quantum proces Read more…

By John Russell

On the Spack Track @SC19

December 5, 2019

At the annual supercomputing conference, SC19 in Denver, Colorado, there were Spack events each day of the conference. As a reflection of its grassroots heritage, nine sessions were planned by more than a dozen thought leaders from seven organizations, including three U.S. national Department of Energy (DOE) laboratories and Sylabs... Read more…

By Elizabeth Leake

Intel’s New Hyderabad Design Center Targets Exascale Era Technologies

December 3, 2019

Intel's Raja Koduri was in India this week to help launch a new 300,000 square foot design and engineering center in Hyderabad, which will focus on advanced com Read more…

By Tiffany Trader

AWS Debuts 7nm 2nd-Gen Graviton Arm Processor

December 3, 2019

The “x86 Big Bang,” in which market dominance of the venerable Intel CPU has exploded into fragments of processor options suited to varying workloads, has n Read more…

By Doug Black

Ride on the Wild Side – Squyres SC19 Mars Rovers Keynote

December 2, 2019

Reminding us of the deep and enabling connection between HPC and modern science is an important part of the SC Conference mission. And yes, HPC is a science its Read more…

By John Russell

Using AI to Solve One of the Most Prevailing Problems in CFD

October 17, 2019

How can artificial intelligence (AI) and high-performance computing (HPC) solve mesh generation, one of the most commonly referenced problems in computational engineering? A new study has set out to answer this question and create an industry-first AI-mesh application... Read more…

By James Sharpe

D-Wave’s Path to 5000 Qubits; Google’s Quantum Supremacy Claim

September 24, 2019

On the heels of IBM’s quantum news last week come two more quantum items. D-Wave Systems today announced the name of its forthcoming 5000-qubit system, Advantage (yes the name choice isn’t serendipity), at its user conference being held this week in Newport, RI. Read more…

By John Russell

DARPA Looks to Propel Parallelism

September 4, 2019

As Moore’s law runs out of steam, new programming approaches are being pursued with the goal of greater hardware performance with less coding. The Defense Advanced Projects Research Agency is launching a new programming effort aimed at leveraging the benefits of massive distributed parallelism with less sweat. Read more…

By George Leopold

Ayar Labs to Demo Photonics Chiplet in FPGA Package at Hot Chips

August 19, 2019

Silicon startup Ayar Labs continues to gain momentum with its DARPA-backed optical chiplet technology that puts advanced electronics and optics on the same chip Read more…

By Tiffany Trader

SC19: IBM Changes Its HPC-AI Game Plan

November 25, 2019

It’s probably fair to say IBM is known for big bets. Summit supercomputer – a big win. Red Hat acquisition – looking like a big win. OpenPOWER and Power processors – jury’s out? At SC19, long-time IBMer Dave Turek sketched out a different kind of bet for Big Blue – a small ball strategy, if you’ll forgive the baseball analogy... Read more…

By John Russell

Cray, Fujitsu Both Bringing Fujitsu A64FX-based Supercomputers to Market in 2020

November 12, 2019

The number of top-tier HPC systems makers has shrunk due to a steady march of M&A activity, but there is increased diversity and choice of processing compon Read more…

By Tiffany Trader

Crystal Ball Gazing: IBM’s Vision for the Future of Computing

October 14, 2019

Dario Gil, IBM’s relatively new director of research, painted a intriguing portrait of the future of computing along with a rough idea of how IBM thinks we’ Read more…

By John Russell

Intel Debuts New GPU – Ponte Vecchio – and Outlines Aspirations for oneAPI

November 17, 2019

Intel today revealed a few more details about its forthcoming Xe line of GPUs – the top SKU is named Ponte Vecchio and will be used in Aurora, the first plann Read more…

By John Russell

Leading Solution Providers

SC 2019 Virtual Booth Video Tour

AMD
AMD
CEJN
CJEN
ONE STOP SYSTEMS
ONE STOP SYSTEMS
PANASAS
PANASAS
SIX NINES IT
SIX NINES IT
VERNE GLOBAL
VERNE GLOBAL
WEKAIO
WEKAIO

Kubernetes, Containers and HPC

September 19, 2019

Software containers and Kubernetes are important tools for building, deploying, running and managing modern enterprise applications at scale and delivering enterprise software faster and more reliably to the end user — while using resources more efficiently and reducing costs. Read more…

By Daniel Gruber, Burak Yenier and Wolfgang Gentzsch, UberCloud

Dell Ramps Up HPC Testing of AMD Rome Processors

October 21, 2019

Dell Technologies is wading deeper into the AMD-based systems market with a growing evaluation program for the latest Epyc (Rome) microprocessors from AMD. In a Read more…

By John Russell

Cray Wins NNSA-Livermore ‘El Capitan’ Exascale Contract

August 13, 2019

Cray has won the bid to build the first exascale supercomputer for the National Nuclear Security Administration (NNSA) and Lawrence Livermore National Laborator Read more…

By Tiffany Trader

SC19: Welcome to Denver

November 17, 2019

A significant swath of the HPC community has come to Denver for SC19, which began today (Sunday) with a rich technical program. As is customary, the ribbon cutt Read more…

By Tiffany Trader

When Dense Matrix Representations Beat Sparse

September 9, 2019

In our world filled with unintended consequences, it turns out that saving memory space to help deal with GPU limitations, knowing it introduces performance pen Read more…

By James Reinders

Cerebras to Supply DOE with Wafer-Scale AI Supercomputing Technology

September 17, 2019

Cerebras Systems, which debuted its wafer-scale AI silicon at Hot Chips last month, has entered into a multi-year partnership with Argonne National Laboratory and Lawrence Livermore National Laboratory as part of a larger collaboration with the U.S. Department of Energy... Read more…

By Tiffany Trader

With the Help of HPC, Astronomers Prepare to Deflect a Real Asteroid

September 26, 2019

For years, NASA has been running simulations of asteroid impacts to understand the risks (and likelihoods) of asteroids colliding with Earth. Now, NASA and the European Space Agency (ESA) are preparing for the next, crucial step in planetary defense against asteroid impacts: physically deflecting a real asteroid. Read more…

By Oliver Peckham

Jensen Huang’s SC19 – Fast Cars, a Strong Arm, and Aiming for the Cloud(s)

November 20, 2019

We’ve come to expect Nvidia CEO Jensen Huang’s annual SC keynote to contain stunning graphics and lively bravado (with plenty of examples) in support of GPU Read more…

By John Russell

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This