You remember the Final Jeopardy! in the first round?
US cities category was on the board, and Watson - I admit, indicating his doubt (??? why a male personality? forgive me, his name is masculine) with the rings circling way at the bottom of "his face" and a bunch of question marks - attempted Toronto as the answer.
Well, after his impressive debut this was shocking.
Little did it help - I would say, at first sight it made it worse - that David Ferrucci explained there were some Toronto cities also in the US. At second sight, to the scientific mind it really mattered, but for the ordinary audience it might looked like marketing-explaining a product bug.
Well, let us not forget that Watson is a machine and also a scientific experiment. Or case study. Or worload-optimized, special purpose system. In that regard, his mistake is important and fruitful. So is Ferrucci's explanation.
Let me elaborate on this a bit.
If Watson had simply won without any mistakes one would assume he really "knew" what he had been asked, and what it meant. In the light of this "Toronto" and some other guesses he made (though he did not reach the confidence level needed to press the button (to attempt to answer), the response candidates were visible throughout the entire game) we now he had little clue about what was going on.
And let's emphasize, Ferrucci also said Watson did not "think" and "understand" as humans do, instead, it can process human language.
And that is a significant difference, much like how birds and airplanes differ.
Can we use airplanes to fly? No doubt.
Now, with Watson's mistakes we try to understand what and why he can "understand" (process precisely), and where he is completely clueless.
It occurs, that during the training process Watson learns not only semantics but also some general knowledge which is used to disambiguate possible meanings. People do the same when we understand what we are told.
Still, there is a long way to go.
Not only with Watson - which (who) is amazing - but in machine natural language processing in general.
I have no idea when (if at all) machines will admire poetry, humour and riddles.
