Riding the Third Wave of AI without Quantum Computing

Oct. 23, 2019 — Rapid changes are occurring in the field of artificial intelligence (AI) as many computer scientists explore new ways to make systems faster and more efficient. One anticipated capability is quantum computing—technology that follows the laws of quantum physics, enabling processing power to exist in multiple states and perform multiple tasks at the same time. If realized in hardware, it would speed-up some computational problem-solving exponentially. UC San Diego Theoretical Physicist Max Di Ventra is catching this next wave of cutting-edge AI with an alternative and fundamentally different platform he calls “memcomputing,” which doesn’t require quantum capabilities.

Sketch of a memcomputing architecture. Apart from the input/output and a control unit, which directs the machine on what problem to solve, all computation is done by a memory unit, a “computational memory.” From F.L. Traversa and M. Di Ventra, IEEE Trans. Neural Networks Learn. Sys. 26, 2702 (2015). © 2015 IEEE. Image courtesy of UC San Diego.

“Using a physics-based approach, this novel computing paradigm employs memory to both process and store information on the same physical location, a property that somewhat mimics the computational principles of the human brain,” said the UC San Diego physics professor and author of “The Scientific Method: Reflections from a Practitioner (Oxford University Press, 2018).”

After years of trial and error, Di Ventra and his group developed all of the mathematics required for this new simple architecture, combining “memory” and “compute” and driven by a specialized “computational memory” unit, with performance that resembles quantum computing—without the overwhelming computational overhead. Now, with half-a-million dollars over 18 months from the Defense Advanced Research Projects Agency (DARPA), Di Ventra and his students are working to apply this new physics-based approach to AI.

“Our project, if successful, would have a large impact in the field of machine learning and artificial intelligence by showing that physics approaches can be of great help in fields of research that are traditionally dominated by computer scientists,” said Di Ventra.

With the DARPA funds, the team will apply memcomputing to the unsupervised learning, or pre-training, of Deep Belief Networks. These are systems of multi-layer neural networks (NNs) used to recognize, generate and group data. DiVentra will also propose a hardware architecture, using current technologies, to perform this task. Pre-training of NNs is a notoriously difficult problem, and researchers have all but abandoned it in favor of supervised learning. However, in order to have machines that adapt to external stimuli in real time and make decisions according to the context in which they operate—the goal of the third wave of AI—powerful new methods to train NNs in an unsupervised manner are required.

Demonstration that a memcomputing solver (named Falcon in the figure) outperforms, by orders of magnitude, state-of-the-art algorithms in solving difficult computational problems. From F. Sheldon, P. Cicotti, F.L. Traversa and M. Di Ventra, IEEE Trans. Neural Networks Learn. Sys. (2019). © 2019 IEEE. Image courtesy of UC San Diego.

Di Ventra explained that memcomputing accelerates the time to find feasible solutions to the most complex optimization problems in all industries.

“We have applied these emulations to a wide variety of difficult computational problems that are of interest to both academia and industry, and solved them orders of magnitude faster than traditional algorithms,” noted Di Ventra.

Unlike quantum computing, memcomputing employs non-quantum units so it can be realized in hardware with available technology and emulated in software on traditional computers. Current computing capabilities began with the work of Alan Turing, who helped decrypt German codes during WWII with his Bombe Machine. He also developed the Turing Machine, which became the basis for modern computers. John von Neumann devised the architecture for the Turing Machine, whereby the central processing unit (CPU) was separate from the memory unit. The so-called von Neumann Bottleneck in today’s computing is created precisely from the physical separation of the CPU and the memory unit: the CPU has to constantly insert and extract information from the memory, significantly slowing processing time.

“Memcomputing represents a radical departure from both our traditional computers, and algorithms that run on them, and quantum computers,” said Di Ventra. “It provides the necessary tools for the realization of an adaptable computational platform deployable in the field of artificial intelligence and offers strategic advantages to the Department of Defense in numerous applications,” said Di Ventra.

In view of the preliminary successes of memcomputing, Di Ventra has co-founded the company MemComputing, Inc., which is developing a software as a service, based on this technology, to solve the most challenging problems in academia and industry.

Source: Cynthia Dillon, UC San Diego