Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Developing a Multimodal Transcription to account for Interaction in 3D Virtual Worlds: the 3M method

1,140 views

Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

Developing a Multimodal Transcription to account for Interaction in 3D Virtual Worlds: the 3M method

  1. 1. Developing a Multimodal Transcription to account for Interaction in 3D Virtual Worlds: the 3M method Joan Tomàs Pujolà Cristina Palomeque
  2. 2. 3M transcription M ULTI- M ODAL M UVE 3
  3. 3. index <ul><li>MUVE </li></ul><ul><li>Multimodality </li></ul><ul><li>Multimodal Transcription tradition </li></ul><ul><li>3M Transcription </li></ul><ul><li>Future directions </li></ul>
  4. 4. 3D muves Social dimension Interaction: environment, objects, avatars Immersive experience Multimodal communication
  5. 5. senses <ul><li>authenticity </li></ul>presence immersiveness creativity gaming
  6. 6. interact <ul><li>interactivity (n) – interactive (adj) </li></ul>interaction (n) – interactional (adj)
  7. 7. channels <ul><li>Written </li></ul><ul><ul><li>instant message (IM) </li></ul></ul><ul><ul><li>local chat </li></ul></ul><ul><ul><li>notecard </li></ul></ul><ul><li>Oral </li></ul><ul><ul><li>voice call </li></ul></ul><ul><ul><li>voice chat </li></ul></ul><ul><li>synchronous asynchronous </li></ul>multitasking
  8. 8. video example <ul><li>Second Life </li></ul><ul><li>Complexity of communication </li></ul><ul><ul><li>multi-channels </li></ul></ul><ul><ul><li>interface </li></ul></ul><ul><ul><li>virtual paralanguage </li></ul></ul>
  9. 9. aim <ul><li>How can interaction be transcribed in a learning-teaching context in a muve ? </li></ul>
  10. 10. multimodal transcriptions <ul><li>difficulties: </li></ul><ul><ul><li>sequential organisation of data </li></ul></ul><ul><ul><li>represent simultaneous multimodal phenomena (interrelation btwn the modes) </li></ul></ul><ul><ul><li>how to add a spatial dimension </li></ul></ul>
  11. 11. multimodal transcriptions <ul><li>Hampel and Hauck, 2006 </li></ul><ul><li>Baldry and Thibault, 2006 </li></ul><ul><li>Norris, 2006 </li></ul><ul><li>Hampel and Hauck, 2007 </li></ul><ul><li>Goodwin, 2007 </li></ul><ul><li>Source: Flewitt, R., et al, What are multidata and transcription? In Jewitt, C. 2009. The Routledge Hanbook of Multimodal Analysis </li></ul>
  12. 15. <ul><li>Macro level (sequence map) </li></ul><ul><ul><li>Session </li></ul></ul><ul><li>Micro level </li></ul><ul><li>(magnifying glass) </li></ul>transcription levels
  13. 16. macro level
  14. 17. multimodal transcription
  15. 18. 3M transcription
  16. 19. tagging system TYPES Body posture <bp> Sitting <bps> Laying <bpl> Kneeling <kpl> Standing (default, not tagged) Kinesics <k> <ul><li>Walk <kw> </li></ul><ul><li>Run <kr> </li></ul><ul><li>Jump <kj> </li></ul><ul><li>Laugh <kl> </li></ul><ul><li>Dance <kd> </li></ul>Proxemics <p> Facing <pf> Backing <pb> Haptics <h> Touching <ht> Pointing <hp> Chronemics <c> Wating <cw> Short / medium / long pause waiting <cspw>, <cmpw>, <clpw> Thinking <ct> Short / medium / long pause thinking <cspt>, <cmpt>, <clpt>
  17. 20. future directions Transcription on the web - trying to visualize the multiplicity of channels Video streaming - trying to visualize the spatial dimension
  18. 21. 3M M UVE of a m ultimodality to describe the m ultimodal transcription We are pursuing a

×