Q&A with Jim Keller, CTO of Tenstorrent, and an HPCwire Person to Watch in 2021

By HPCwire Editorial Team

April 22, 2021

HPCwire recently announced our 2021 People to Watch and will be running featured interviews with these 14 thought leaders and HPC influencers in the coming weeks. First up we are happy to bring you our interview with Jim Keller, president and chief technology officer of Tenstorrent.

One of the top chip architects of our time, Keller has had an impactful career. He has held high-profile roles at AMD (where he designed the Zen cores that helped the company compete in datacenters once more), Tesla and Apple. Keller joined AI chip startup Tenstorrent earlier this year following two years as senior vice president of Intel’s silicon engineering group.

Hi Jim, congrats on your new position as CTO & President of Tenstorrent and being named an HPCwire Person to Watch for the second time! Tell us about your role at Tenstorrent, your areas of responsibility, and what drew you to the company.

Thank you for this opportunity.

As CTO, I’m working on new technology at Tenstorrent. Following our roadmap, we have a chip (Grayskull) currently starting production. We are powering up on our second-generation part and designing our 3rd and 4th generation of processors as we speak. I’m spending my time working on all of these parts and system designs around them.

As President, I’ve been working with our growing team on business strategy. We’ve gained significant traction with various companies, system builders and their customers, which we can now start translating into revenue.

I was the first investor at Tenstorrent. Ljubisa Bajic (Tenstorrent Founder and CEO) and I go way back. We worked together at AMD and I was always impressed by his approach to building AI silicon. He knows how GPUs work, how the software works, and he also knows the math behind AI, which is a rare combination. That’s why I was interested in investing with him.

Personally, I think the AI revolution is bigger than the Internet. Joining Tenstorrent is a great way for me to contribute to it, and so far it’s been super fun.

With so many startups engaged in designing and commercializing AI silicon, what sets Tenstorrent apart?

There are a few different things to consider. First, and it took us a while to realize this, you have to get right all the basics at a very deep level: memory, compute and network bandwidth together with programmability.

We’ve talked to a number of customers who are frustrated about the current state of AI silicon at its core.

The second thing I really like is the approach to the software. It begins with a unique compiler and software strategy, with hardware designed around it properly.

Some AI chip companies build chips with lots of GFLOPS or TFLOPS, and then they design the software later.

But Tenstorrent has always been different. We build hardware in collaboration with software right from the start.

The original software team consists of people who worked at Altera on FPGA compilers and CAD tools, which are both very complicated problems; we have people from AI and also people who work on HPC computers. There’s a big presence of talent in Toronto from companies and institutions like Intel, Nvidia, AMD and the University of Toronto.

How does the Tenstorrent approach differ in terms of architecture, and in combination of software and hardware. What is “Software 2.0” and how is it important?

What sets Tenstorrent apart is networking, data transformation and math engines of the software stack that work in sync with the hardware.

When you look at the Tenstorrent processor, it looks like an array of math processors, which is pretty common. There’s actually a real matrix multiplier and convolutional engine, so you don’t have to write programs to emulate that kind of math. The Tenstorrent engine does it naturally. It makes the number of programs you have to write for high performance lower because it runs the AI idioms of matrix multiply and convolution natively.

Then there are two units we call “Unpacker” and “Packer”, which are data transformation engines. Rather than writing programs to move bytes around, we have hardware that does it in a very straightforward way and presents a common data format into the math engine, which simplifies the programming.

And finally, networking is built in the Tenstorrent technology from the ground up. When all compute engines do their work, they have to send data somewhere – they send the data packet to the other engine.
We use the same on-chip and off-chip protocol to connect multiple chips together.

The first time I heard about Software 2.0 it was coined by Andrei Karpathy, who is the director of AI and autopilot at Tesla.

His idea was that we’re going from a world where you write programs to modify data to where you build neural networks and then program them with data to do the things you want. So modern computers are literally programmed with data.

It means a very different way of thinking about programming in many places where AI has had so much success. I think in the Software 2.0 future, 90% of computing will be done that way.

There will always be some computing that runs standard C programs but more and more of the actual cycles will be done in AI hardware running what we think of as Software 2.0.

What is the status of Grayskull and Wormhole and what markets and use cases do these chips address?

We’ve started our first production run of Grayskull, which we’re sampling to our customers. Our chip goes on a PCIe card and we have 75 W, 150 W and 300 W form factors. People can buy and plug them into their server infrastructure. We’ve released our inference software, and in a month or so, we are going to release training software. It’s built for a broad variety of AI applications, both training and inference.

Wormhole is our 2nd generation part that is going to take Tenstorrent to the next level because it has native networking between chips and lets us scale from a single chip to many chip systems just using our own network. This greatly improves bandwidth between chips and lowers the cost of building a system.

What excites you most about being a computer architect right now?

I’m sort of amazed by this but I’ve been building and designing computers for 40 years. The complexity of the computers that we build today is so far past what we did or even considered hard 40 years ago.

The reason we can build these computers is that modern tools and software have gotten so much better. You can think of an idea, write down RTL, synthesize it and build it into a chip with a really small team.

People at one point thought there’d be so many transistors and things would be so complicated we wouldn’t be able to build silicon because it’d be too expensive. But the opposite is true. Tenstorrent built Grayskull and Wormhole as a very small team of really great people. They took a very clever approach to modularity and design. We have a relatively small number of units that we put together to make a very complex chip. The amount of change I’ve seen in the last 5 or 10 years of computer design is probably greater than the previous 20.

We’ve been through a lot of revolutions. I think the AI revolution is going to be the biggest one so far.

Outside the professional sphere, what activities, hobbies or travel destinations do you enjoy in your free time?

I like to be active and fairly physical – I kitesurf, snowboard. I like to run and workout. I find it’s almost meditative, especially when I’m working on a hard problem. I get the problem loaded up in my head and I go run or snowboard for four hours. Somehow or other, it sorts itself out.

I like to travel. I went to Egypt with my kids a couple of years ago, it was great. I went to Serbia last year, we had a really great time there before Serbia got shut down due to the pandemic. I often go to Hawaii to surf, and I really enjoy the beach. The last year has been tough on travel so we’ll see about next year.

Keller is one of 14 HPCwire People to Watch for 2021. You can read the interviews with the other honorees at this link.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from its predecessors, including the red-hot H100 and A100 GPUs. Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. While Nvidia may not spring to mind when thinking of the quant Read more…

2024 Winter Classic: Meet the HPE Mentors

March 18, 2024

The latest installment of the 2024 Winter Classic Studio Update Show features our interview with the HPE mentor team who introduced our student teams to the joys (and potential sorrows) of the HPL (LINPACK) and accompany Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the field was normalized for boys in 1969 when the Apollo 11 missi Read more…

Apple Buys DarwinAI Deepening its AI Push According to Report

March 14, 2024

Apple has purchased Canadian AI startup DarwinAI according to a Bloomberg report today. Apparently the deal was done early this year but still hasn’t been publicly announced according to the report. Apple is preparing Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimization algorithms to iteratively refine their parameters until Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. Wh Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the fi Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimizat Read more…

PASQAL Issues Roadmap to 10,000 Qubits in 2026 and Fault Tolerance in 2028

March 13, 2024

Paris-based PASQAL, a developer of neutral atom-based quantum computers, yesterday issued a roadmap for delivering systems with 10,000 physical qubits in 2026 a Read more…

India Is an AI Powerhouse Waiting to Happen, but Challenges Await

March 12, 2024

The Indian government is pushing full speed ahead to make the country an attractive technology base, especially in the hot fields of AI and semiconductors, but Read more…

Charles Tahan Exits National Quantum Coordination Office

March 12, 2024

(March 1, 2024) My first official day at the White House Office of Science and Technology Policy (OSTP) was June 15, 2020, during the depths of the COVID-19 loc Read more…

AI Bias In the Spotlight On International Women’s Day

March 11, 2024

What impact does AI bias have on women and girls? What can people do to increase female participation in the AI field? These are some of the questions the tech Read more…

Alibaba Shuts Down its Quantum Computing Effort

November 30, 2023

In case you missed it, China’s e-commerce giant Alibaba has shut down its quantum computing research effort. It’s not entirely clear what drove the change. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Analyst Panel Says Take the Quantum Computing Plunge Now…

November 27, 2023

Should you start exploring quantum computing? Yes, said a panel of analysts convened at Tabor Communications HPC and AI on Wall Street conference earlier this y Read more…

DoD Takes a Long View of Quantum Computing

December 19, 2023

Given the large sums tied to expensive weapon systems – think $100-million-plus per F-35 fighter – it’s easy to forget the U.S. Department of Defense is a Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Intel’s Server and PC Chip Development Will Blur After 2025

January 15, 2024

Intel's dealing with much more than chip rivals breathing down its neck; it is simultaneously integrating a bevy of new technologies such as chiplets, artificia Read more…

Baidu Exits Quantum, Closely Following Alibaba’s Earlier Move

January 5, 2024

Reuters reported this week that Baidu, China’s giant e-commerce and services provider, is exiting the quantum computing development arena. Reuters reported � Read more…

Leading Solution Providers

Contributors

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Training of 1-Trillion Parameter Scientific AI Begins

November 13, 2023

A US national lab has started training a massive AI brain that could ultimately become the must-have computing resource for scientific researchers. Argonne N Read more…

Shutterstock 1179408610

Google Addresses the Mysteries of Its Hypercomputer 

December 28, 2023

When Google launched its Hypercomputer earlier this month (December 2023), the first reaction was, "Say what?" It turns out that the Hypercomputer is Google's t Read more…

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Shutterstock 1606064203

Meta’s Zuckerberg Puts Its AI Future in the Hands of 600,000 GPUs

January 25, 2024

In under two minutes, Meta's CEO, Mark Zuckerberg, laid out the company's AI plans, which included a plan to build an artificial intelligence system with the eq Read more…

Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn't an apt description, so it settled on Hypercomputer Read more…

China Is All In on a RISC-V Future

January 8, 2024

The state of RISC-V in China was discussed in a recent report released by the Jamestown Foundation, a Washington, D.C.-based think tank. The report, entitled "E Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire