In Second AI-Human Showdown, IBM's Debater Outwitted

IBM's Project Debater, unveiled last month in a debate-style event in San Francisco, was developed at the company's Israeli research center

Lilach Baumer and Yoav Stoler 11:0204.07.18

TAGS:

In June, IBM unveiled a project six years in the making—an artificial intelligence system called Project Debater that draws on extensive data banks to construct logical, convincing arguments. During the event, held in IBM offices in San Francisco, the AI took on two human debaters. In both cases the audience thought it to be worse at delivery, but better at conveying knowledge, awarding it the win in the second debate. On Tuesday, IBM held a second event, showcasing the system to a crowd of journalists in Israel, and this time the AI fared slightly worse.

For daily updates, subscribe to our newsletter by clicking here.

Though the first time a computer won against a human was in 1956—when MANIAC defeated a novice player at a simplified chess game—it was IBM that marked a watershed moment with Deep Blue, the chess-playing computer that beat world champion Garry Kasparov in 1997. After rolling out Watson, a question-answering computer initially developed to answer Jeopardy questions, in 2011, IBM moved on to the next challenge—a computer capable of holding its own in a debate against humans.

Noam Slonim (left) and Ranit Aharonov. Photo: Or Kaplan

Project Debator was developed in IBM's research center in Israel, its largest research center outside the U.S. AI scientist Noam Slonim, who suggested the project as IBM's next challenge after Watson, is the principal researcher. AI scientist Ranit Aharonov is the project's manager.

The system has three main capabilities. The first is data-driven speech writing and delivery—the system has access to over 300 million scientific and press articles, as well as other data. The second is the ability to listen to speech and identify key statements and claims. The third is the ability to model dilemmas, enabling it to construct principled arguments. According to IBM, the system is not trained on the topics it debates, and its arguments are not scripted. At this point, the debates are performed according to a curated list of topics, and extra-sensitive topics such as abortion rights are avoided.

On Tuesday, Debater faced off against two experienced Israeli debaters. Yaar Bach, who headed the debating society at Israeli academic institute IDC Herzliya for two years as a student, challenged the system on the subject of mass surveillance. Hayah Goldlist Eichler, who represented Israel at numerous national and international tournaments and also worked as an advisor on the project, debated genetic engineering.

Speaking against the use of mass surveillance methods, the AI identified the crux of the matter as the conflict between the right for privacy and the need to prevent terrorism and maintain public safety. The system did not only marshal concrete arguments, but also provided counterarguments after Bach said his piece. Its performance, however, wasn't without faults, as its attempts at joking were not always appropriate, and it repeated some points, such as the matter of privacy, without always expanding or adding new information. In the second debate round, the system overly focused on environmental aspects.

The system and Bach tied, but in the second round the win went to Goldlist Eichler, who succeeded in changing the minds of more people in the audience, despite scoring lower on knowledge.