June 9, 2014

Chat Bot Passes Turing Test

Tiffany Trader
Eugene Turing Test

A computer program has successfully passed the Turing test by pretending to be a 13-year-old Ukranian boy named Eugene Goostman. During a series of five-minute long keyboard-based conversations, the chat bot convinced 33 percent of the judges that it was human. The results are just above the 30 percent mark, commonly cited as the passing threshold for the eponymous test, devised by famed mathematician Alan Turing over six decades ago.

The competition was organized by the University of Reading and conducted at the Royal Society in London. The date of the event, June 7, 2014, marked the 60th anniversary of Turing’s death and nearly six months after his post-humous royal pardon.

As outlined in Alan Turing’s 1950 paper, the test was designed to assess a machine’s ability to exhibit intelligence indistinguishable from that of a human. In its basic form, a human judge asks the computer a series of questions. If the judge cannot distinguish the computer from a human interlocutor, then the machine is said to have passed the test.

Until today, no computer program had successfully passed the 30 percent confidence threshold, although several, including Elbot and ELIZA, came pretty close.

Eugene’s developers, Vladimir Veselov and Eugene Demchenko, attribute the success to the program’s personality (a know-it-all teenager) and to the dialog system that is capable of handling both vague and direct questions.

“Our main idea was that he can claim that he knows anything, but his age also makes it perfectly reasonable that he doesn’t know everything. We spent a lot of time developing a character with a believable personality,” said Veselov. “This year we improved the ‘dialog controller’ which makes the conversation far more human-like when compared to programs that just answer questions. Going forward we plan to make Eugene smarter and continue working on improving what we refer to as ‘conversation logic’.”

Some of Eugene’s responses betray his “artificial” nature. For example, when asked how he felt about the accomplishment, Eugene remarked: “I feel about beating the turing test in quite convenient way. Nothing original.”

The announcement from the University of Reading describes the program as a supercomputer, but it really doesn’t take any sort of specialized hardware to run a chat bot. Real supercomputers are making breakthroughs that benefit humanity and expand our horizons. The kind of AI that Turing envisioned, something akin to actual human intelligence that shows creativity and broad problem-solving ability, has been cast aside in favor of more specialized intelligence tests, like winning a chess match or a television game show. Today’s breed of AI programs (Watson, Siri, etc.) can’t, as Gary Markus puts it, “come close to doing what any bright, real teenager can do: watch an episode of ‘The Simpsons,’ and tell us when to laugh.”

Share This