Chatting with virtual characters
First I talked to Eliza on Athena's Emacs.
Next I talked to a few natural language programs written at Industry Canada by a group that develops information retrieval technologies including Conversational Hypertext Access Technology (CHAT) and TIPS.
Almost immediately, as my very first chat with Eliza got underway, the program's limited understanding was revealed. I had the impulse to "dumb-down" my comments so that Eliza could best interpret them. I had this same impulse with all the other chatbots I tried out. This is akin to the way we talk to young children. Complex language is sacrificed in order to feel like we're getting through.
During my first chat with Eliza, I tried to use my own natural language, and to talk about real situations. This led to quite a few nonsensical exchanges, because Eliza either wildly misinterpreted remarks, or was unable to interpret them altogether. This was a truly entertaining experience. I was crying laughing.
Before my second chat with Eliza I read Weizenbaum, and got a sense of how the program interprets each comment. During that chat I assumed the role of a depressed person, and constructed short simple sentences that I hoped Eliza would be able to interpret. The results were inconsistent, and the chat was much less entertaining than the first.
In all of my chats, the highest degree of verisimilitude was reached at the beginnings and ends of conversations. By nature these are formulaic, and the programs had decent scripts for handling them. In any given chat, the program's limitations were hit upon pretty quickly, and manifested as an absurd response to a misinterpreted remark or question. None of the programs seemed to learn much in the conversations, except for memorizing a few strings.
Overall I'd say that the real ability level of these programs to interpret truly natural language is pretty limited. In each program the personality of the chattor is defined according to some fairly rigid caricature. This automatically narrows the scope of acceptible conversation to such a degree that it can no longer be called natural. This seems like something of a cheap trick to me. Some examples of these caricature's traits are:
The latter two programs soon began to feel like text adventure games, where only certain phrases can elicit results, and where there is a mystery to be solved, or a task to be accomplished. In particular, talking with Maur the dragon was very frustrating, because if I didn't figure out exactly the right things to say I got incinerated. This happened about 8 times, after which I lost patience and left.One of the more satisfying chats was with a Sex Ed advise dispensing program, which didn't claim to be a person, or even a personality. The program would interpret my questions, and answer them neutrally, but in detail. This chat was only frustrating when the program misinterpreted me, and gave detailed answers to questions it incorrectly assumed I was asking. This problem could be addressed by building a 'clarifying' step into the program. If I ask a question that it rates as difficult-to-interpret, it could ask me a clarifying question.
In sum, the most enjoyable chats I experienced were hilariously absurd, rather than realistic.