Intel’s Silicon Brain System a Blueprint for Future AI Computing Architectures

By Agam Shah

April 24, 2024

Intel is releasing a whole arsenal of AI chips and systems hoping something will stick in the market. Its latest entry is a neuromorphic system called Hala Point. The system includes Intel’s research chip called Loihi 2, an exotic AI chip inspired by the design and functioning of the human brain.

“This rivals and exceeds levels achieved by architectures built on graphics processing units (GPU) and central processing units (CPU),” Intel claimed on its website.

It bears repeating: Hala Point is a research system, and the Loihi 2 chip is still in Intel’s lab and not commercially available yet. It won’t unseat the dominance of Nvidia GPUs anytime soon.

Instead, it may be a blueprint for a new computing architecture in which computing travels to the data. Today’s computing designs rely on data traveling to computing, which creates bottlenecks.

The Hala Point system and Loihi 2 system pursue a new concept of “sparse computing,” which the U.S. Department of Defense is studying as a computing architecture of the future.

The chip’s architecture is more effective at processing audio and video, which Intel highlighted in a research paper published last month.

The Promise of Intel’s Neurochip

Intel announced its first neuromorphic chip in 2017 as it encountered a predicament around scaling manufacturing capabilities and reducing chip sizes.

Current AI computing techniques involve moving data to compute, which is considered inefficient for AI due to bottlenecks in storage, memory, and processing. Loihi 2’s neuromorphic design focuses on efficient execution at the points of data, which frees up bandwidth and storage.

The proliferation of data makes current computing models unsustainable, especially for post-exascale systems, said William Harrod, a program manager at the U.S. Department of Defense’s Intelligence Advanced Research Projects Activity (IARPA), during a keynote at the Supercomputing 22 conference.

IARPA has a data-centric computing program to “define the future of computing based on the data movement problem, not on floating point units of ALUs,” Harrod said.

The program, called AGILE (Advanced Graphical Intelligence Logical Computing Environment), focuses on a new architecture that moves compute to data instead of vice versa to generate faster real-time results.

Intel, Qualcomm, and AMD are among the companies pushing for new chip designs and computer architectures for the program. The companies are submitting proposals to IARPA.

A research paper published last month by Intel’s researchers compared Loihi 2 to Nvidia’s Jetson Orin Nano. The researchers concluded that Loihi 2 had power and performance advantages over Orin Nano but that Nvidia’s chip was better when computing at scale.

False Promises of Silicon Brains

Neuromorphic chips have been under development at chip makers and universities for more than a decade.

The exotic chips were hyped as bringing a higher level of intelligence to computers, a role assumed by AI chips such as Nvidia’s GPUs. At the time, IBM and others were also developing neuromorphic chips.

Neuromorphic chips are probabilistic and factor in uncertainty and randomness in computing. These chips are designed to replicate the structure and functioning of the brain, which itself is still a mystery.

Most conventional AI chips operate on deterministic models, which are more precise and rely on machine learning and associated algorithms. However, AI chips can also incorporate probabilistic models.

Computing cores in neuromorphic chips play the role of neurons, which are interconnected (like synapses). Computing in neurons across synapses happens in parallel. The low-precision computing points reach conclusions by understanding trends and associations in the information.

A brain can recognize cats through many neurons working in parallel, and Intel’s chip is intended to function similarly.

Intel’s Hala Point scales up the number of computing neurons in its system to 1.15 billion, which is nowhere close to the scale of the human brain (approximately 86 billion). But the computational capacity is significantly larger than its 2020 neuromorphic system called Pohoiki Springs, which had 100 million neurons.

The chip can “support up to 20 quadrillion operations per second, or 20 petaops, with an efficiency exceeding 15 trillion 8-bit operations per second per watt (TOPS/W) when executing conventional deep neural networks,” Intel claimed. The claims have not been independently verified.

Origins of Neuromorphic Chips

Intel’s first silicon brain effort came out in 2017, but other chip developments were already underway. IBM was developing its own neuromorphic chips, and Qualcomm was developing its Zeroth chip.

In 2013, Qualcomm released Zeroth, and CEO Paul Jacobs teased the idea of smartphones being perceptive and anticipating the needs of smartphone users. His prediction took a while to come true, but smartphones like Google Pixel can now adapt to user needs.

Efforts in the U.S. and EU to fund brain-inspired chips in the 2010s vanished without a trace.

Starting in 2008, DARPA funded a program called Multiphase Synapse (Systems of Neuromorphic Adaptive Plastic Scalable Electronics), which involved IBM, Hewlett-Packard, Cornell, Stanford University, and other universities.

The EU funded a $1.6 billion Human Brain Project to understand how the brain works, and one component included developing a chip based on the design. The Human Brain Project has now ended.

IBM remains one of the few organizations developing a silicon brain, and last year, it released a chip called NorthPole, which focuses more on power efficiency as opposed to scaling.

Intel’s Neurochip

The Hala Point system includes the Loihi 2 chip, which is made using the Intel 4 process and has 128 cores per chip. Each chip includes up to 1 million digital neurons and 120 million synapses. The Loihi 2 throughput is achieved by electrical impulses that facilitate communication between neurons.

Loihi 2 is Intel’s second-generation neuromorphic research chip. (Credit: Intel Corporation)

The Hala Point chip has 1,152 Loihi 2 chips and 1.15 billion neurons with 128 billion synapses. The six-rack system consumes 2,600 watts of power, and the chips are organized in a mesh design, with six asynchronous parallel lanes to interconnect multiple Loihi 2 chips.

A 10Gbps Ethernet connector facilitates communication. Mesh configurations can range from one to “thousands of chips,” Intel said.

“Further development will enable applications of neuromorphic computing that overcome power and latency constraints that currently limit the real-world, real-time deployment of AI capabilities,” Intel said in a research paper.

The chips are artificial neural networks that can perform many concurrent audio and video functions. Computation is sparse and performed more at the point where data is located.

“While the GPUs, Tensor processors, and deep learning accelerators of today focus on dense matrix-based computation at a very high throughput, neuromorphic processors focus on sparse event-driven computation that minimizes activity and data movement,” Intel’s researchers wrote in the paper.

Intel’s research compared its chip on an artificial neural network versus a Jetson Orin Nano chip, which was released about a year ago.

Intel Hala Point system composed of Intel Loihi 2 neuromorphic processors. (Source: Intel Corporation)

The research paper concluded that neurons on the Loihi 2 chip activated only when needed, compared to the Jetson chip on the sparse computing model. That was more energy efficient and made better use of bandwidth and other computing resources.

However, the Jetson Orin held an advantage on larger operations when computing cores were fully fired up and operating at scale within comparable bandwidth.

Where the Chip Fits

The Loihi 2 survived Pat Gelsinger’s axe during restructuring, so the company may see value in the research. The Hala Point is deployed at Sandia National Laboratories, which tests a variety of chips and configurations for energy-efficient AI.

The chip is also part of Intel’s stable of AI chips, which includes Gaudi 3 and Ponte Vecchio GPU. It does not present any current threat to any CPU or GPU, including Intel’s offerings.

Neuromorphic is another chip concept as Intel mulls its future. Making the chip commercially viable is a challenge. Like quantum processors, these chips will require software frameworks and algorithms.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

ISC 2024 Takeaways: Love for Top500, Extending HPC Systems, and Media Bashing

May 23, 2024

The ISC High Performance show is typically about time-to-science, but breakout sessions also focused on Europe's tech sovereignty, server infrastructure, storage, throughput, and new computing technologies. This round Read more…

HPC Pioneer Gordon Bell Passed Away

May 22, 2024

Legendary computer scientist Gordon Bell passed away last Friday at his home in Coronado, CA. He was 89. The New York Times has a nice tribute piece. A long-time pioneer with Digital Equipment Corp, he pushed hard for de Read more…

ISC 2024 — A Few Quantum Gems and Slides from a Packed QC Agenda

May 22, 2024

If you were looking for quantum computing content, ISC 2024 was a good place to be last week — there were around 20 quantum computing related sessions. QC even earned a slide in Kathy Yelick’s opening keynote — Bey Read more…

Atos Outlines Plans to Get Acquired, and a Path Forward

May 21, 2024

Atos – via its subsidiary Eviden – is the second major supercomputer maker outside of HPE, while others have largely dropped out. The lack of integrators and Atos' financial turmoil have the HPC market worried. If Atos goes under, HPE will be the only major option for building large-scale systems. Read more…

Core42 Is Building Its 172 Million-core AI Supercomputer in Texas

May 20, 2024

UAE-based Core42 is building an AI supercomputer with 172 million cores which will become operational later this year. The system, Condor Galaxy 3, was announced earlier this year and will have 192 nodes with Cerebras Read more…

Google Announces Sixth-generation AI Chip, a TPU Called Trillium

May 17, 2024

On Tuesday May 14th, Google announced its sixth-generation TPU (tensor processing unit) called Trillium.  The chip, essentially a TPU v6, is the company's latest weapon in the AI battle with GPU maker Nvidia and clou Read more…

ISC 2024 Takeaways: Love for Top500, Extending HPC Systems, and Media Bashing

May 23, 2024

The ISC High Performance show is typically about time-to-science, but breakout sessions also focused on Europe's tech sovereignty, server infrastructure, storag Read more…

ISC 2024 — A Few Quantum Gems and Slides from a Packed QC Agenda

May 22, 2024

If you were looking for quantum computing content, ISC 2024 was a good place to be last week — there were around 20 quantum computing related sessions. QC eve Read more…

Atos Outlines Plans to Get Acquired, and a Path Forward

May 21, 2024

Atos – via its subsidiary Eviden – is the second major supercomputer maker outside of HPE, while others have largely dropped out. The lack of integrators and Atos' financial turmoil have the HPC market worried. If Atos goes under, HPE will be the only major option for building large-scale systems. Read more…

Google Announces Sixth-generation AI Chip, a TPU Called Trillium

May 17, 2024

On Tuesday May 14th, Google announced its sixth-generation TPU (tensor processing unit) called Trillium.  The chip, essentially a TPU v6, is the company's l Read more…

Europe’s Race towards Quantum-HPC Integration and Quantum Advantage

May 16, 2024

What an interesting panel, Quantum Advantage — Where are We and What is Needed? While the panelists looked slightly weary — their’s was, after all, one of Read more…

The Future of AI in Science

May 15, 2024

AI is one of the most transformative and valuable scientific tools ever developed. By harnessing vast amounts of data and computational power, AI systems can un Read more…

Some Reasons Why Aurora Didn’t Take First Place in the Top500 List

May 15, 2024

The makers of the Aurora supercomputer, which is housed at the Argonne National Laboratory, gave some reasons why the system didn't make the top spot on the Top Read more…

ISC 2024 Keynote: High-precision Computing Will Be a Foundation for AI Models

May 15, 2024

Some scientific computing applications cannot sacrifice accuracy and will always require high-precision computing. Therefore, conventional high-performance c Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

Atos Outlines Plans to Get Acquired, and a Path Forward

May 21, 2024

Atos – via its subsidiary Eviden – is the second major supercomputer maker outside of HPE, while others have largely dropped out. The lack of integrators and Atos' financial turmoil have the HPC market worried. If Atos goes under, HPE will be the only major option for building large-scale systems. Read more…

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, codenamed Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Some Reasons Why Aurora Didn’t Take First Place in the Top500 List

May 15, 2024

The makers of the Aurora supercomputer, which is housed at the Argonne National Laboratory, gave some reasons why the system didn't make the top spot on the Top Read more…

Leading Solution Providers

Contributors

Eyes on the Quantum Prize – D-Wave Says its Time is Now

January 30, 2024

Early quantum computing pioneer D-Wave again asserted – that at least for D-Wave – the commercial quantum era has begun. Speaking at its first in-person Ana Read more…

The GenAI Datacenter Squeeze Is Here

February 1, 2024

The immediate effect of the GenAI GPU Squeeze was to reduce availability, either direct purchase or cloud access, increase cost, and push demand through the roof. A secondary issue has been developing over the last several years. Even though your organization secured several racks... Read more…

The NASA Black Hole Plunge

May 7, 2024

We have all thought about it. No one has done it, but now, thanks to HPC, we see what it looks like. Hold on to your feet because NASA has released videos of wh Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

Intel Plans Falcon Shores 2 GPU Supercomputing Chip for 2026  

August 8, 2023

Intel is planning to onboard a new version of the Falcon Shores chip in 2026, which is code-named Falcon Shores 2. The new product was announced by CEO Pat Gel Read more…

GenAI Having Major Impact on Data Culture, Survey Says

February 21, 2024

While 2023 was the year of GenAI, the adoption rates for GenAI did not match expectations. Most organizations are continuing to invest in GenAI but are yet to Read more…

How the Chip Industry is Helping a Battery Company

May 8, 2024

Chip companies, once seen as engineering pure plays, are now at the center of geopolitical intrigue. Chip manufacturing firms, especially TSMC and Intel, have b Read more…

Q&A with Nvidia’s Chief of DGX Systems on the DGX-GB200 Rack-scale System

March 27, 2024

Pictures of Nvidia's new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing po Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire