Hot-Buzzer Topic: Did IBM's Watson Have an Advantage on 'Jeopardy!'?

Updated

You could say Jeopardy! contestant Brad Rutter's buttons have been pushed, if not buzzed.

As the dust settles following the closely watched, man-against-machine Jeopardy! contest, a debate has begun over whether the IBM supercomputer Watson had an advantage over Rutter and the other human competitor, Ken Jennings, while buzzing-in answers to Jeopardy! clues.

According to an article on the Kurzweil web site, IBM (IBM) researchers say it was the humans who had the advantage, since they could anticipate the buzzer light going off and could respond even faster than Watson. But during an interview with DailyFinance, Rutter says the IBM folks are acting more like "politicians than scientists" by making such comments. "Technically their comments are true," Rutter says, "but highly misleading."

Unhuman Reflexes

Rutter racked up a record $3.26 million in game winnings over his Jeopardy! career, and is well-known for his speed in buzzing-in answers. But he says human contestants can rarely hit the button as soon as the light comes on, whereas Watson is consistently fast.

Sponsored Links

"It's hard to be in that zone," Rutter says, noting Jeopardy! players who jump the gun in anticipation of the light often end up being shut out for a brief moment. "Watson doesn't have to worry about being shut because he doesn't anticipate."

Jennings, who won the most games in the show's history, expressed similar sentiments in a New York Daily News Op-Ed piece: "The reflexes of even a very good human player will vary slightly," he wrote, "but not Watson's. If it knows the answer, it makes the perfect buzz. Every single time. And it's hard to win if you can't buzz."

Going into the event, Rutter says he wasn't sure what to fully expect. While watching "sparring" videos Watson had with other Jeopardy! contestants, he could see the supercomputer was winning roughly 70% of the time. While he sensed the biggest advantage for Watson would be its buzzing ability, Rutter also thought he would prove a strong rival.


On Day 1 of the contest, Rutter managed to keep pace with the supercomputer. Watson and Rutter ended the day tied with $5,000 each in winnings.

"We had one more practice round before the game and I felt good," he recalled. "I hit my stride. I got the first clue and got it right but then Watson got hot and locked us out through the first commercial break. It was amazing to see how fast he is when he's confident. But after the commercial break, he got a few wrong and had some hiccups."

No Ebb and Flow in the Game

Although Rutter gained a substantial lead, Watson recovered toward the end of the round to tie. Rutter initially thought Day 1 of the event was similar to a typical Jeopardy! program -- where there is an ebb and flow as the game progresses, with contestants pulling ahead and falling behind.

Day 2, however, changed all that. Rutter says a number of the categories randomly selected that day tended to favor Watson's sweet spot: concrete, fact-based clues rather than abstract concepts and short clues. Watson dominated game play on the second day -- landing a number of Daily Doubles that allowed him to dramatically boost its score.

"Once Watson got in the lead, the Daily Doubles became even more important to find," Rutter says, noting his usual strategy is to rack up some winnings before seeking out the Daily Doubles, in order to try for a bigger bang with his bucks.

"If you have zero dollars and get a Daily Double, that's still zero dollars," he says. Rutter observed that Watson was on the prowl for Daily Doubles even though it had only several hundred dollars. That strategy appeared to ensure the computer's lead while keeping the other two contestants from later boosting their scores.

Luck is Still a Factor


Rutter says he knew Day 3 would be an uphill battle -- and notes that Jennings was fortunate to hit a Daily Double that put him back in the game. But in the end, Rutter placed third, Jennings second and Watson first.

"I felt I put in a pretty good showing," Rutter says. "With Jeopardy!, there is usually a lot of luck involved. A couple little breaks here and there can make a big difference."

And while CBS (CBS) got high television ratings for the event, Rutter doesn't expect computers to replace humans on game shows.

"I think the ratings bump came from the novelty factor. I think over time people would get bored watching computers," Rutter says, adding wryly that "Watson never jumped up or down if he got it right."

Get info on stocks mentioned in this article:

Advertisement