Must See TV: IBM Watson Heads for Jeopardy Showdown

By Michael Feldman

February 9, 2011

Next week the IBM supercomputer known as “Watson” will take on two of the most accomplished Jeopardy players of all time, Ken Jennings and Brad Rutter, in a three-game match starting on February 14. If Watson manages to best the humans, it will represent the most important advance in machine intelligence since IBM’s “Deep Blue” beat chess grandmaster Garry Kasparov in 1997. But this time around, the company also plans to make a business case for the technology. Trivial pursuit this is not.

And impressive technology it is. On the hardware side, Watson is comprised of 90 Power 750 servers, 16 TB of memory and 4 TB of disk storage, all housed in a relatively compact ten racks. The 750 is IBM’s elite Power7-based server targeted for high-end enterprise analytics. (The Power 755 is geared toward high performance technical computing and differs only marginally in CPU speed, memory capacity, and storage options.) Although the enterprise version can be ordered with 1 to 4 sockets of 6-core or 8-core Power7 chips, Watson is maxed out with the 4-socket, 8-core configuration using the top bin 3.55 GHz processors.

The 360 Power7 chips that make up Watson’s brain represent IBM’s best and brightest processor technology. Each Power7 is capable of over 500 GB/second of aggregate bandwidth, making it particularly adept at manipulating data at high speeds. FLOPS-wise, a 3.55 GHz Power7 delivers 218 Linpack gigaflops. For comparison, the POWER2 SC processor, which was the chip that powered cyber-chessmaster Deep Blue, managed a paltry 0.48 gigaflops, with the whole machine delivering a mere 11.4 Linpack gigaflops.

But FLOPS are not the real story here. Watson’s question-answering software presumably makes little use of floating-point number crunching. To deal with the game scenario, the system had to be endowed with a rather advanced version of natural language processing. But according to David Ferrucci, principal investigator for the project, it goes far beyond language smarts. The software system, called DeepQA, also incorporates machine learning, knowledge representation, and deep analytics.

Even so, the whole application rests on first understanding the Jeopardy clues, which, because they employ colloquialisms and often obscure references, can be challenging even for humans. That’s why this is such a good test case for natural language processing. Ferrucci says the ability to understand language is destined to become a very important aspect of computers. “It has to be that way,” he says. “We just cant imagine a future without it.”

But it’s the analysis component that we associate with real “intelligence.” The approach here reflects the open domain nature of the problem. According to Ferrucci, it wouldn’t have made sense to simply construct a database corresponding to possible Jeopardy clues. Such a model would have supported only a small fraction of the possible topics available to Jeopardy. Rather their approach was to use “as is” information sources — encyclopedias, dictionaries, thesauri, plays, books, etc. — and make the correlations dynamically.

The trick of course is to do all the processing in real-time. Contestants, at least the successful ones, need to provide an answer in just a few seconds. When the software was run on a lone 2.6 GHz CPU, it took around 2 hours to process a typical Jeopardy clue — not a very practical implementation. But when they parallelized the algorithms across the 2,880-core Watson, they were able to cut the processing time from a couple of hours to between 2 and 6 seconds.

Even at that, Watson doesn’t just spit out the answers. It forms hypotheses based on the evidence it finds and scores them at various confidence levels. Watson is programmed not to buzz in until it reaches a confidence of at least 50 percent, although this parameter can be self-adjusted depending on the game situation.

To accomplish all this, DeepQA employs an ensemble of algorithms — about a million lines of code — to gather and score the evidence. These include temporal reasoning algorithms to correlate times with events, statistical paraphrasing algorithms to evaluate semantic context, and geospatial reasoning to correlate locations.

It can also dynamically form associations, both in training and at game time, to connect disparate ideas. For example it can learn that inventors can patent information or that officials can submit resignations. Watson also shifts the weight it assigns to different algorithms based on which ones are delivering the more accurate correlations. This aspect of machine learning allows Watson to get “smarter” the more it plays the game.

The DeepQA programmers have also been refining the algorithms themselves over the past several years. In 2007, Watson could only answer a small fraction of Jeopardy clues with reasonable confidence and even at that, was only correct 47 percent of the time. When forced to answer the majority of the clues, like a grand champion would, it could only answer 15 percent correctly. By IBM’s own admission, Watson was playing “terrible.” The highest performing Jeopardy grand champions, like Jennings and Rutter, typically buzz in on 70 to 80 percent of the entries and give the correct answer 85 to 95 percent of time.

By 2010 Watson started playing at that level. Ferrucci says that while the system can’t buzz in on every question, it can now answer the vast majority of them in competitive time. “We can compete with grand champions in terms of precision, in terms of confidence, and in terms of speed,” he says.

In dozens of practice rounds against former Jeopardy champs, the computer was beating the humans with a 65 percent win rate. Watson also prevailed in a 15-question round against Jennings and Rutter in early January of this year. See the performance below.

None of this is a guarantee that Watson will prevail next week. But even if the machine just makes a decent showing, IBM will have pulled off quite possibly the best product placement in television history. Open domain question answering is not only one of the Holy Grails of artificial intelligence but has enormous potential for commercial applications. In areas as disparate as healthcare, tech support, business intelligence, security and finance, this type of platform could change those businesses irrevocably. John Kelly, senior vice president and director of IBM Research, boasts, “We’re going to revolutionize industries at a level that has never been done before.”

In the case of healthcare, it’s not a huge leap to imagine “expert” question answering systems helping doctors with medical diagnosis. A differential diagnosis is not much different from what Watson does when it analyzes a Jeopardy clue. Before it replaces Dr. House, though, the machine will have to prove itself in the game show arena.

If Jennings and Rutter defeat the supercomputer this time around, IBM will almost certainly ask for a rematch, as it did when Deep Blue initially lost its first chess match with Kasparov in 1996. The engineers will keep stroking the code and retraining the computer until Watson is truly unbeatable. Eventually the machine will prevail.

—–

For a broader discussion on this topic between the author and InterSect360 Research CEO Addison Snell, download this week’s HPCwire Soundbite podcast.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from its predecessors, including the red-hot H100 and A100 GPUs. Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. While Nvidia may not spring to mind when thinking of the quant Read more…

2024 Winter Classic: Meet the HPE Mentors

March 18, 2024

The latest installment of the 2024 Winter Classic Studio Update Show features our interview with the HPE mentor team who introduced our student teams to the joys (and potential sorrows) of the HPL (LINPACK) and accompany Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the field was normalized for boys in 1969 when the Apollo 11 missi Read more…

Apple Buys DarwinAI Deepening its AI Push According to Report

March 14, 2024

Apple has purchased Canadian AI startup DarwinAI according to a Bloomberg report today. Apparently the deal was done early this year but still hasn’t been publicly announced according to the report. Apple is preparing Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimization algorithms to iteratively refine their parameters until Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. Wh Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the fi Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimizat Read more…

PASQAL Issues Roadmap to 10,000 Qubits in 2026 and Fault Tolerance in 2028

March 13, 2024

Paris-based PASQAL, a developer of neutral atom-based quantum computers, yesterday issued a roadmap for delivering systems with 10,000 physical qubits in 2026 a Read more…

India Is an AI Powerhouse Waiting to Happen, but Challenges Await

March 12, 2024

The Indian government is pushing full speed ahead to make the country an attractive technology base, especially in the hot fields of AI and semiconductors, but Read more…

Charles Tahan Exits National Quantum Coordination Office

March 12, 2024

(March 1, 2024) My first official day at the White House Office of Science and Technology Policy (OSTP) was June 15, 2020, during the depths of the COVID-19 loc Read more…

AI Bias In the Spotlight On International Women’s Day

March 11, 2024

What impact does AI bias have on women and girls? What can people do to increase female participation in the AI field? These are some of the questions the tech Read more…

Alibaba Shuts Down its Quantum Computing Effort

November 30, 2023

In case you missed it, China’s e-commerce giant Alibaba has shut down its quantum computing research effort. It’s not entirely clear what drove the change. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Analyst Panel Says Take the Quantum Computing Plunge Now…

November 27, 2023

Should you start exploring quantum computing? Yes, said a panel of analysts convened at Tabor Communications HPC and AI on Wall Street conference earlier this y Read more…

DoD Takes a Long View of Quantum Computing

December 19, 2023

Given the large sums tied to expensive weapon systems – think $100-million-plus per F-35 fighter – it’s easy to forget the U.S. Department of Defense is a Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Intel’s Server and PC Chip Development Will Blur After 2025

January 15, 2024

Intel's dealing with much more than chip rivals breathing down its neck; it is simultaneously integrating a bevy of new technologies such as chiplets, artificia Read more…

Baidu Exits Quantum, Closely Following Alibaba’s Earlier Move

January 5, 2024

Reuters reported this week that Baidu, China’s giant e-commerce and services provider, is exiting the quantum computing development arena. Reuters reported � Read more…

Leading Solution Providers

Contributors

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Training of 1-Trillion Parameter Scientific AI Begins

November 13, 2023

A US national lab has started training a massive AI brain that could ultimately become the must-have computing resource for scientific researchers. Argonne N Read more…

Shutterstock 1179408610

Google Addresses the Mysteries of Its Hypercomputer 

December 28, 2023

When Google launched its Hypercomputer earlier this month (December 2023), the first reaction was, "Say what?" It turns out that the Hypercomputer is Google's t Read more…

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Shutterstock 1606064203

Meta’s Zuckerberg Puts Its AI Future in the Hands of 600,000 GPUs

January 25, 2024

In under two minutes, Meta's CEO, Mark Zuckerberg, laid out the company's AI plans, which included a plan to build an artificial intelligence system with the eq Read more…

Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn't an apt description, so it settled on Hypercomputer Read more…

China Is All In on a RISC-V Future

January 8, 2024

The state of RISC-V in China was discussed in a recent report released by the Jamestown Foundation, a Washington, D.C.-based think tank. The report, entitled "E Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire