February 16, 2011

Supercomputer Pummels Jeopardy Champs

Michael Feldman

Jeopardy whizzes Ken Jennings and Brad Rutter got thrashed by IBM Watson on national television Tuesday night in the second round of the man vs. machine exhibition match. The computer decisively won the first game of the match (which began on Monday night) with a total of $35,734. Rutter ended with $10,400, while Jennings managed just $4,800.

In the second round of the game under Double Jeopardy, Watson was able to rack up the numbers twice as fast and only faltered a few times. On the first answer, the machine nabbed $2,000, and never looked back. Jennings and Rutter could only look on and watch their swifter silicon-based counterpart beat them to the buzzer.

Jennings, in particular, appeared ready to answer a number of times, but only after Watson had already buzzed in. Toward the end of the match, both humans were squirming at their respective podiums as Watson built a commanding double digit lead.

The machine’s algorithmic-driven personality was on display at times, adding a touch of humor to the broadcast. When Watson hit the first Daily Double, instead of betting a nice round number, it wagered $6,435. The audience chuckled and Jeopardy host Alex Trebek deadpanned, “I won’t ask.” Watson answered the Daily Double correctly, much to the delight of Watson project lead David Ferrucci and the rest of the IBM’ers in the crowd.

When the computer hit the next Daily Double, Watson had some trouble processing the clue: THE ANCIENT “LION OF NIMRUD” WENT MISSING FROM THIS CITY’S NATIONAL MUSEUM IN 2003, ALONG WITH A LOT OF OTHER STUFF. Its top rated answer, Baghdad, had only a 32 percent confidence level — below its normal 50 percent threshold. I’ll take a guess,” Watson responded. “What is Baghdad?” Which was correct.

One of the more impressive displays of Watson’s algorithmic smarts was revealed by the clue, IT CAN MEAN TO DEVELOP GRADUALLY IN THE MIND OR TO CARRY DURING PREGNANCY, to which Watson correctly answered, “What is gestate?” The system had managed to break apart the clue into two distinct phrases and then have the intelligence to correlate them.

The computer’s only real embarrassing gaffe came near the end of the match in the Final Jeopardy round. The category was U.S. Cities and the clue was ITS LARGEST AIRPORT IS NAMED FOR A WORLD WAR II HERO; ITS SECOND LARGEST FOR A WORLD WAR II BATTLE. Oddly enough, Watson responded with the name of a Canadian city, “What is Toronto?” Apparently, the IBM engineers need to tweak the geography algorithms a bit.

Both Jennings and Rutter had responded correctly (“What is Chicago?”), wagering nearly all of their meager earnings. That ended Jennings and Rutter’s night with $4,800 and $10,400 respectively. But it wasn’t nearly enough. Watson had bet only $957 on Final Jeopardy, finishing with $35,734. Game over.

Here is the first part of last night’s show. Jump to 4:48 if you want to skip the IBM Watson infomercial (although honestly, it’s pretty slick) and other introductory fluff:

To watch the second half of the show, go here.