Adaptive computing company Knowm Inc. says it has cleared a major technological hurdle in the pursuit of practical artificial neural network (ANN) chips that has stymied deeper-pocketed competitors, including IBM and HP. As the first to develop memristors capable of bi-directional incremental learning capability, the Santa Fe, NM-based startup can move forward with its ANN chip architecture, the intended substrate for a new generation of advanced machine learning and artificial intelligence applications.
According to company CEO and co-founder Alex Nugent, its “thermodynamic RAM” (kT-RAM) general purpose synaptic processor architecture confers two major advantages over simulated approaches on conventional digital circuits:
- A merging of memory and processing avoids the need to move data around. Data can be collocated in the same place where it’s computed.
- Each kT-RAM memristor holds 6-9 bits of information. Rather than simply being an on-off switch, the memristor can vary its resistance over a wide range and in doing hold a lot of information.
The kT-RAM processor specification is based on the theory of ‘Anti-Hebbian and Hebbian’ (AHaH) computing – a nature-inspired approach developed by Nugent and software development lead Tim Molter. Nugent explained kT-RAM provides a general-purpose on-chip synaptic integration and learning resource for machine learning operations. This technology reduces synaptic integration and adaption to analog memristor circuit operations and is many orders of magnitude more efficient than digital computing methods.
Before it could fulfill the promise of the AHaH theory, however, Knowm needed a memristor with a certain combination of properties, including:
- Back End of Line (BEOL) CMOS compatibility
- Low thresholds of adaptation
- Resistance ranges from ~100kΩ to ~100MΩ
- Many intermediate resistance states, around 100 is ideal
- Non-volatile
- Voltage dependence
- Bi-directional incremental operation
Memristor expert Dr. Kris Campbell of Boise State University had demonstrated all of these requirements in her previous work, except for the higher resistance states and the bi-directional incremental property. Through a STTR program with the Air Force Research Laboratory, Knowm’s CEO approached Dr. Campbell a little over a year ago to see if she could fill in the missing pieces.
“For a while now, memristors have shown uni-directional incremental operation, which is to say, you can apply voltage across the device and increment it gradually in one direction, but when you reverse the voltage you get an all-or-nothing change,” Nugent explained to HPCwire. “This doesn’t work, we need something that can gradually step up or down. We need both. But this sudden-erase property is very difficult to deal with and many other groups have been struggling with it.”
Working with chalcogenide compounds, Dr. Campbell instantiated a breakthrough that enabled memristors with incrementally adjustable resistance in both directions rather than just one.
“This voltage-dependent bi-directional incremental property, this is what we are after in pairing memristors with machine learning because learning is an incremental operation,” said Nugent. “It’s what we call ‘nudging a weight.’ You nudge it positive, you nudge it negative. Increase the voltage and you can nudge it more. Reverse the voltage and you can reverse the direction. Learning algorithms amount to finding ways to nudge the weights around.”
Dr. Campbell was able to fabricate it in a unique way, said Nugent, describing her approach as the “yin to everyone else’s yang.”
“We demonstrated phenomenal bidirectional incremental operation at the exact resistance ranges that we need,” Nugent continued. “So this is, as far as we know, the final piece. We now have a device which is absolutely ideal for AHaH computing. Learning can now be reduced to an analog hardware operation.”
The advance follows on the heels of Knowm’s making its discrete memristors, emulator kits and CMOS-plus-memristor services available earlier this year. The new evaluation chip contains eight discrete memristors in a 16-pin ceramic DIP package. Pricing starts at $220 per unit with discounts applied automatically to bulk orders. Raw die are available as well.
Knowm’s plan is to build the world’s first truly adaptive neuromemristive processor with potential power efficiency gains of up to ten orders of magnitude over traditional computing architectures.
“The gap in efficiency that we are trying to close, that is the efficiency of brains and the efficiency of trying to bring computing on par with biology,” said Nugent.