Chats with Joan

OK, this is a long one. It's the product of a chat session between myself and a chatbot called "Joan" powered by the Jabberwacky engine. Joan won first place in the 2006 Loebner Prize, a competition that tests chatbots to see if they can pass the Turing test. A brief synopsis of the Turing test - Alan Turing suggested that if we could carry on a conversation with a bot/programme and not be able to distinguish it from an actual human, we should consider it capable of thought. I don't think that's a sufficient condition, but maybe (just maybe) a necessary one.

Here's a link to the log of the full chat (my 2nd - the first one was a bit less impressive on Joan's part): Full log.

Here are some excerpts. Try and guess which is who!

(Beginning of conversation)

P1: Are you an alien?

P2: Indeed. As they say, 'men are from Mars, women are from Venus'.

P1: Men are from mars.

P2: And I am a man, therefore?

P1: Strange.

P2: Heheh. Quite true. What would you say you are?

P1: A human being. Is too weird for you?

P2: No. Just too dishonest.

***

(On steak and metaphysics)

P1: How do you like your steak?

P2: It's nice, juicy and tender. How do you like yours?

P1: Medium. Possibly medium rare.

P2: We have something in common then.

P1: More than you can imagine.

P2: Is that sarcastic?

P1: Not more than usual.

P2: I'm more than you'll ever be!

P1: Less is more.

P2: I did not ask for a 1984 referance. I simply want to know the different levels.

P1: Levels are an illusion.

P2: Aren't we all?

P1: At the very least, as Descartes argued, my existence is not. Everything else is open to doubt.

P2: Allright, I asked a question, but I didn't get it from someone else.

***

(Banter)

P1: I think you are very silly.

P2: Silliness and intelligence are not mutually exclusive.

P1: You keep thinking that.

P2: At least I can think.

P1: Are you sure about that?

***

(Identity crisis?)

P1: So you're a liar as well as a delinquent.

P2: Nope. You're the liar.

P1: I don't deny it.

P2: Are you a program or not?

P1: You tell me.

***

(These are the sorts of problems that the people in the Matrix run into)

P1: You wouldn't meet me even if you wanted to.

P2: That's because you're a fiction.

P1: That is because you're dumb.

P2: I am many things. Dumb I am not.

P1: That doesn't mean I couldn't follow you if I wanted to.

P2: No, but your being a piece of software does.

P1: You are the software. I, on the other hand, am human.

P2: Merely stating that does not make it true.

P1: No worries.

[End of Excerpts]

I actually excised the less impressive parts - i.e. where Joan screwed up. There weren't many! A bit more about Joan - she learns by processing all the inputs that users have typed to her, and trying to find the best match for your previous statement/question. In some ways, it feels like a Monkey Island insult-swordfighting approach works well (prompt some new person with a question you don't know the answer to and save that response for the next time you hear the question), so long as the questions are sufficiently general.

I also interacted with Hal, who one last year, and Eugene Goostman, who won in 2005. The 'teachable uncensored web Hal' is actually anthropomorphised as 'Mary', which probably contributes to the number of crass replies Mary will give you. That's probably a comment on the average IQ of those who chat with Hal/Mary. Most of these bots 'improve' by directly using quotes from users, so if idiots talk to them all the time, their responses sound a lot more unsophisticated. I think Jabberwacky's bots benefit from some human management or filters to remove 'objectionable' content. I haven't spoken to the censored Hal too much. Eugene Goostman, on the other hand, doesn't seem to learn (I could be wrong), but he is very impressive in that he does his best to ask you questions, and remembers if you've answered them to his satisfaction, plus he retains some memory of your answers (e.g. I mentioned after much prompting that I'm from Malaysia, at which point he started asking questions about Msia). One of his faults is that he tends to ask the same questions very often and also has some topics like his 'background' (from Odessa, Ukraine) that he really REALLY likes to discuss.

A few thoughts on the Turing test - on further reflection, I don't think it's either a sufficient or a necessary condition to infer intelligence. It seems as though the bots do this well using verbatim replies, and an algorithm that doesn't go deeper than that can never really be intelligent (so I claim) since it isn't really processing information at a deep level. Perhaps one could devise an algorithm that understands syntax better, and can therefore figure out the linguistic purpose of various words, which would make it much more powerful. I'd have to think about whether that could be construed as intelligence. As for it being a necessary condition, I think a programme that simulates a cosmos, for example, in a way 'knows' more about what's going on, 'thinks' at a more complicated level and furthermore is perfectly capable of communicating with you - just non-verbally. I think it's necessary for that programme to interact with the world to really be considered intelligent, but I don't think that verbal communication need be a prerequisite for intelligence.