Developing a Multimodal Transcription to account for Interaction in 3D Virtual Worlds: the 3M method

1,015 views
905 views

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,015
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Virtual worlds in learning are becoming increasingly popular and there is an extended belief that VW can be used as VLEs basically for 4 reasons. VW allow for multimodal communication: voice chat, text chat, private and public. There is a strong social dimension as users are represented by avatars. The residents of VW are usually quite friendly and willing to help newcomers. Apart from interacting with other avatars, you can interact with objects (e.g. when you open a door) or with the environment itself (you can contribute to create the environment). It is an immersive experience: the environment really makes you feel like if you really were in that place.
  • Developing a Multimodal Transcription to account for Interaction in 3D Virtual Worlds: the 3M method

    1. 1. Developing a Multimodal Transcription to account for Interaction in 3D Virtual Worlds: the 3M method Joan Tomàs Pujolà Cristina Palomeque
    2. 2. 3M transcription M ULTI- M ODAL M UVE 3
    3. 3. index <ul><li>MUVE </li></ul><ul><li>Multimodality </li></ul><ul><li>Multimodal Transcription tradition </li></ul><ul><li>3M Transcription </li></ul><ul><li>Future directions </li></ul>
    4. 4. 3D muves Social dimension Interaction: environment, objects, avatars Immersive experience Multimodal communication
    5. 5. senses <ul><li>authenticity </li></ul>presence immersiveness creativity gaming
    6. 6. interact <ul><li>interactivity (n) – interactive (adj) </li></ul>interaction (n) – interactional (adj)
    7. 7. channels <ul><li>Written </li></ul><ul><ul><li>instant message (IM) </li></ul></ul><ul><ul><li>local chat </li></ul></ul><ul><ul><li>notecard </li></ul></ul><ul><li>Oral </li></ul><ul><ul><li>voice call </li></ul></ul><ul><ul><li>voice chat </li></ul></ul><ul><li>synchronous asynchronous </li></ul>multitasking
    8. 8. video example <ul><li>Second Life </li></ul><ul><li>Complexity of communication </li></ul><ul><ul><li>multi-channels </li></ul></ul><ul><ul><li>interface </li></ul></ul><ul><ul><li>virtual paralanguage </li></ul></ul>
    9. 9. aim <ul><li>How can interaction be transcribed in a learning-teaching context in a muve ? </li></ul>
    10. 10. multimodal transcriptions <ul><li>difficulties: </li></ul><ul><ul><li>sequential organisation of data </li></ul></ul><ul><ul><li>represent simultaneous multimodal phenomena (interrelation btwn the modes) </li></ul></ul><ul><ul><li>how to add a spatial dimension </li></ul></ul>
    11. 11. multimodal transcriptions <ul><li>Hampel and Hauck, 2006 </li></ul><ul><li>Baldry and Thibault, 2006 </li></ul><ul><li>Norris, 2006 </li></ul><ul><li>Hampel and Hauck, 2007 </li></ul><ul><li>Goodwin, 2007 </li></ul><ul><li>Source: Flewitt, R., et al, What are multidata and transcription? In Jewitt, C. 2009. The Routledge Hanbook of Multimodal Analysis </li></ul>
    12. 15. <ul><li>Macro level (sequence map) </li></ul><ul><ul><li>Session </li></ul></ul><ul><li>Micro level </li></ul><ul><li>(magnifying glass) </li></ul>transcription levels
    13. 16. macro level
    14. 17. multimodal transcription
    15. 18. 3M transcription
    16. 19. tagging system TYPES Body posture <bp> Sitting <bps> Laying <bpl> Kneeling <kpl> Standing (default, not tagged) Kinesics <k> <ul><li>Walk <kw> </li></ul><ul><li>Run <kr> </li></ul><ul><li>Jump <kj> </li></ul><ul><li>Laugh <kl> </li></ul><ul><li>Dance <kd> </li></ul>Proxemics <p> Facing <pf> Backing <pb> Haptics <h> Touching <ht> Pointing <hp> Chronemics <c> Wating <cw> Short / medium / long pause waiting <cspw>, <cmpw>, <clpw> Thinking <ct> Short / medium / long pause thinking <cspt>, <cmpt>, <clpt>
    17. 20. future directions Transcription on the web - trying to visualize the multiplicity of channels Video streaming - trying to visualize the spatial dimension
    18. 21. 3M M UVE of a m ultimodality to describe the m ultimodal transcription We are pursuing a

    ×