A replacement for the Turing test has been proposed. The original test, in which a computer program tries to fool a human judge into thinking it’s human during a five-minute text-only conversation, has been criticized because the central task of devising a false identity is not part of intelligence, and because some conversations may require relatively little intelligent reasoning.

The new test would be based on so-called Winograd schemas, devised by Stanford computer scientist Terry Winograd in 1972. Here’s the classic example:

The city councilmen refused the demonstrators a permit because they [feared/advocated] violence.

If the word feared is used, to whom does they refer, the councilmen or the demonstrators? What if we change feared to advocated? You know the answers to these questions because you have a practical understanding of anxious councilmen. Computers find the task more difficult because it requires not only natural language processing and commonsense reasoning but a working knowledge of the real world.

“Our WS [Winograd schemas] challenge does not allow a subject to hide behind a smokescreen of verbal tricks, playfulness, or canned responses,” wrote University of Toronto computer scientist Hector Levesque in proposing the contest in 2014. “Assuming a subject is willing to take a WS test at all, much will be learned quite unambiguously about the subject in a few minutes.”

In July 2014 Nuance Communications announced that it will sponsor an annual Winograd Schema Challenge, with a prize of $25,000 for the computer that best matches human performance. The first competition will be held at the 2016 International Joint Conference on Artificial Intelligence, July 9-15 in New York City.

Here’s another possibility: Two Dartmouth professors have proposed a Turing Test in Creative Arts, in which “we ask if machines are capable of generating sonnets, short stories, or dance music that is indistinguishable from human-generated works, though perhaps not yet so advanced as Shakespeare, O. Henry or Daft Punk.” The results of that competition will be announced May 18 at Dartmouth’s Digital Arts Exposition.

(Thanks, Kristján and Sharon.)