Theory of mind


Published on

HICSS 2008 talk at Persistent Conversation track. This talk discusses Theory of Mind and how it relates to HCI and conversation.

Published in: Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Theory of mind

  1. 1. “ Theory of Mind” tasks, Human-Computer Interaction, Persistent Conversation & Flickr John C. Thomas IBM T. J. Watson Research Center January 7, 2008 HICSS, Kona Hawaii Persistent Conversation Workshop
  2. 2. “Theory of Mind” <ul><li>Of interest to comparative psychologists, anthropologists, developmental psychologist, AI </li></ul><ul><li>Can an entity “imagine” what it is like for another intelligent entity? (Do they have a theory of the mind of others?) </li></ul>
  3. 3. Typical Theory of Mind Task <ul><li>Tom watches Susan hide a LAN cable under the table. Susan leaves. </li></ul><ul><li>Later, Tom sees John come in and move the LAN cable behind the drapes at the back of the room. </li></ul><ul><li>Now, Susan returns. We ask Tom where he thinks Susan will look for the cable. </li></ul>
  4. 4. General interest is a “capacity” question: who has it? Who does not? <ul><li>Apparently, human children do not “have it” till they are 4 or 5 years old. </li></ul><ul><li>Apparently, only humans have this ability although earlier tests suggested perhaps dolphins and apes have it. </li></ul><ul><li>Smart adult humans with Asperger’s Syndrome apparently do not have this ability. </li></ul><ul><li>But even “normal” adult humans have limited capacity…I think that Tom believes that Susan wishes that Wendy would have forgotten about Tom’s requests for John to inform Susan about Wendy’s desire for longer talks. Huh? </li></ul><ul><li>In contrast, the ability to follow linear causal sequences seems basically unlimited. </li></ul>
  5. 5. A different set of questions: <ul><li>When or under what conditions are people more likely to exhibit this ability? </li></ul><ul><li>What does this tell us, in general, about HCI? About conversation? </li></ul><ul><li>How can we use this to analyze a particular system like Flickr or a particular conversation within Flickr? </li></ul>
  6. 6. ToM and conversation <ul><li>First degree ToM probably necessary for “true” conversation between A and B. </li></ul><ul><li>Second degree needed for C to listen to & understand a simple story about A & B’s conversation about the physical world. </li></ul><ul><li>Third degree needed for D to understand a story about how A tried to use ambiguity to trick B into thinking something untrue about C. </li></ul><ul><li>Fourth degree needed for E to write a story that intentionally misleads the audience D into believing that A was trying to use ambiguity to trick B into thinking something untrue about C. </li></ul>
  7. 7. “Theory of Mind” and HCI <ul><li>Designing good systems for a single user requires the designers to imagine the user’s tasks, capabilities, context and motivations --- all different from theirs </li></ul><ul><li>One common failing is understanding that what is obvious to the project team who have worked on something for months is not obvious to someone just beginning to use the system </li></ul><ul><li>In debates about immigration policy, many native speakers of English seem completely oblivious to how difficult acquiring it can be for non-native adults </li></ul>
  8. 8. Social Computing and ToM <ul><li>The designer’s task is now more difficult because they must also understand and support the users and their “theories of mind” --- issues such as these arise: </li></ul><ul><ul><li>Trust </li></ul></ul><ul><ul><li>Reputation </li></ul></ul><ul><ul><li>Plausible deniability </li></ul></ul><ul><ul><li>Social translucence </li></ul></ul>
  9. 9. Flickr profile <ul><li>“This is me.” </li></ul><ul><li>“This is how I want (these) people to see me.” </li></ul><ul><li>“This is how I want (these) people to imagine that I want to be seen.” </li></ul>
  10. 10. Flickr Picture <ul><li>This is how it is. </li></ul><ul><li>This is how I see it. </li></ul><ul><li>This is how I want other people to see it. </li></ul><ul><li>This is how I want people to see me (e.g., professionalism, choice of subject) </li></ul>
  11. 11. Flickr conversations about… <ul><li>A picture </li></ul><ul><li>A person’s set of pictures. </li></ul><ul><li>The pictures of all participants around a topic. </li></ul><ul><li>The definition of what is in and what is out of this topic. </li></ul><ul><li>How the topic could have/should have been better defined. </li></ul>
  12. 12. An interesting example because <ul><li>Despite the person A setting up the topic or contest trying to be “clear” rules a priori </li></ul><ul><li>Almost always debate about what should and should not be in or out. </li></ul><ul><li>Furthermore, almost never this reaction: </li></ul><ul><ul><li>“ Oh, now I see what your referents are.” </li></ul></ul><ul><ul><li>Rather: A: “Well I meant this!” B: “Well, I meant this!” C: “I agree with A because…” D: “I agree with B because…” </li></ul></ul>
  13. 13. Odd because, there is probably no way in a finite set of words to specify what is “in” and “out” <ul><li>Example topics: </li></ul><ul><ul><li>Something in which the subject is black </li></ul></ul><ul><ul><li>A photograph with error or mistake as a theme </li></ul></ul><ul><li>What about something “obvious” and “objective” like a picture of a tree? </li></ul>
  14. 14. Is this a tree?
  15. 15. How about this?
  16. 16. Or these?
  17. 17. Discussion fragments about “The aesthetics of failure” <ul><li>I just quickly scanned the pool and i d say more than 1/2 the pics don t belong. Please people make an effort .. </li></ul><ul><li>You mean - some people are even failing to adhere to the guidelines? That's real dedication! </li></ul><ul><li>There was a discussion a while ago on this topic, and I think the ultimate observation was that while failing to moderate the group would be in line with the meta-ideology of the group, it would also produce a meaningless pile of photos without relevant context, negating the idea of failure beyond &quot;we don't keep this clean.&quot; </li></ul><ul><li>If &quot;failing to moderate&quot; is &quot;in line&quot;, then mustn't there be a posted photo of that particular failing? Is this group for a discussion on failure or is it based in photography? A photo of a pond, by itself, is not a photo of failure (and there are ponds here)... a photo of an accidental pond in the living room of a fishing tycoon because the roof caved in during a storm... flanked by fish tanks... ...possible candidate...? </li></ul><ul><li>what guidelines??? You've failed to post them.... </li></ul>
  18. 18. Further discussion points… <ul><li>May I, as a philosopher specialising in the field of aesthetics, point out that the Wikipedia definition of aesthetics is somewhat limited and the last sentence complete crap. How can there be any aesthetics if it is all &quot;a matter of personal taste&quot;? </li></ul><ul><li>I just joined this group. I looked around, but I cannot seem to find the rules for posting. I guess you could say I failed to find them. (is that part of the plan?) Little help? </li></ul><ul><li>still unclear - are we posting photos of failures or failed photographs? I hope the former as it's much more interesting </li></ul><ul><li>@GustavoG -- Thank you for the promotion to moderator. I've just re-read this discussion thread, and it seems that we got off topic. Could you, or any of the other admins, please post a set of guidelines? It seems pointless to talk about what the guidelines should be, with this many people -- somebody needs to take the forefront. I think, though, that if we had a set of guidelines to debate, refining them with this many points of view would make for an interesting discussion. </li></ul>
  19. 19. More comments on “aesthetics of failure” <ul><li>d_boon_147 says: My last photo was, ( as far as I can tell, not accepted to the pool. I've re-entered it into the pool, and I was wondering whether I might post a justification here, see if the admin might change their mind. If you're going to start talking about failure and whether a photo fits within the guidelines (lest we forget &quot;Brakdoewn, slippgae, entrpy, accidetn, gl!tczhes&quot;), then you should know that I've thought a lot about this stuff. It's not an offhand thing to put a photo in this pool, not thinking about whether it fits. On the one hand, the photograph is the result of an unintended technical and real-life mistake - the subject moving too fast, and with too weird a situation of light, for the camera to process - hence fractures, glitching, the failure to produce the intended picture. On the one hand you can relate this to certain processes in electronic music (Autechre/Oval's use of tech-scrambling, noise musicians' use of battered equipment/circuit-bending), and on the other you can relate it to certain ideas in art and philosophy (aleatory techniques, Maurice Blanchot's work on the 'art of the disaster'). I've worked with intended distortion in the past (my Nightlights set), using it to open up these ideas, by ruptures in the technological, so surely it's even more in line with pool guidelines if the result is unintended. Hoping you'll actually include it. Posted 2 months ago. ( permalink ) </li></ul>
  20. 20. What ultimately is “in” or “out” is a the result of a social process…disguised as a logical one <ul><li>About Fortunate Mistake </li></ul><ul><li>A group for posting and/or discussing photos which were a &quot;fortunate mistake&quot;, or &quot;fortunate accident&quot; -- something where you specifically didn't capture what (or how) you intended, but where you like the results. (with a light sprinkling of the usual rules -- left vague for now, and hopefully indefinitely.) </li></ul>
  21. 21. Discussion about fortunate mistakes <ul><li>Would this include Photoshop mistakes? </li></ul><ul><li> LarryB     says: </li></ul><ul><li>I ask this because at least once I've made an error adjusting brightness or contrast or other basic adjustments (NOT filters, etc) and gotten a nice surprise. For example: Where I accidentally jogged the contrast slider all the way to the right. Originally posted at 5:47PM, 14 February 2006 EST ( permalink ) LarryB edited this topic 23 months ago. </li></ul><ul><li>lindes     says: </li></ul><ul><li>Sure... I'm thinking it should start out as a photograph, but other than that, probably pretty much anything (that's a &quot;mistake&quot; that you like the results of) goes. Posted 23 months ago. ( permalink ) </li></ul>
  22. 22. Another example discussion about a group re. mistakes <ul><li>Three Hundred Thwarts (and all) -- The Contest </li></ul><ul><li> therese flanagan     says: </li></ul><ul><li>There will be guidelines, okay, rules, but they'll come later. Once we hit 295 photos I'll post the contest rules and we'll let the game begin! Posted at 7:57PM, 19 September 2005 EST ( permalink ) </li></ul>
  23. 23. Conclusion: <ul><li>“ Theory of Mind” is not something people either “have” or “don’t have”… </li></ul><ul><li>People have some ability to “see” how the world looks through the eyes of others </li></ul><ul><li>Easiest to do in the spatial domain (hidden object) </li></ul><ul><li>Harder to do in the temporal domain (learning English; understanding how a system seems to users) </li></ul><ul><li>Harder yet to do in the conceptual domain (how do definitions in words relate to sets of photos?) </li></ul><ul><li>There may be tools or training or representations to help… </li></ul><ul><li>Being “good” at ToM may be a critical skill to allow people to pull scams, outsmart the competition, “win” at war or love, be an effective orator or writer…yet, when and how is it taught? </li></ul>