Remzi Ateş Gürşimşek  Public defense of research for the degree of Doctor of PhilosophyDepartment of Communication, Busine...
1. Introduction2. Theoretical framework     Virtual worlds, social semiotics, design research3. Methodology and Analytical...
Second Life provides its users withvirtual places and artifacts as a virtual world.
BUT..How can the world’s residents communicate by designing?What do these                         (Mediated)       (Repres...
BUT..How can the world’s residents communicate by designing?What do these                       (Mediated)       (Represen...
VIRTUAL        WORLDS  SOCIAL       DESIGNSEMIOTICS     RESEARCH
Virtual Worlds as                           PlacesAvatars, Presence &                         Co-design and   Co-presence ...
VIRTUALSemiotic resources        WORLDSContext of SituationMeaning Potentials                    SOCIAL           DESIGN  ...
VIRTUAL              WORLDS                              Affordances and                            constraints in design ...
How do the VW users co-producemultimodal    meaning     potentials   invirtual places and artifacts throughcollaborative d...
 Avatars, presence and co-presence(Bartle 2004, Gordon 2008, Damer 1998, 2009,Schroeder 1995, 2006, 2011) Spatial practi...
 Phenomenology and the experiential perspective:  Place as ”organized world of meaning”(Tuan 1977, Turner and Turner 2004...
 Social semiotics focuses on sign systems and semiotic  resources in particular contexts of situation, through  which the...
 Mode: socially and culturally shaped semiotic  resources for meaning-making Multimodal cohesion: orchestration of sign ...
 Design translates rhetor’s intentions into  semiotically shaped material Discourse shapes the choice of design, but  ch...
 Design as a ”a reflective conversation with the situation”(Cross 2007 [1990]) Role of intuition, emotion and aesthetic ...
 ”Individual insights” and ”momentary collective  processes”(Hargadon and Becky 2006) Emergence of innovative solutions ...
 Co-presence and connected presence in computer-supported   collaborative design(Schroeder 2011, Koutsabatsis et al. 2012...
Virtual wordsAvatars, co-presence and collaborative place-making in a 3D virtual studio                  environment with ...
   Virtual places and artifacts in SL as multimodal data       Participatory design observation       Ethnographic obse...
• The three meta-functions in multimodal theory of  space and place (Stenglin 2008, 2009, O’Toole 1994,  2004) describe th...
The “nexus analysis” or MDA perspective to social semiotics considers:        The social actors and relations        The...
 Capturing power and agency with multimodal analysis Interpreting the analysis of contexts and contents of  design throu...
 A design team and a        An organic ally formed    Four design teams, 15  research team               group as desig...
 Backgrounds, skills and purposes of the co-  designers (i.e. Experience, motivation) Professionalism, the amateur spiri...
 Movements of humans and avatars in  real and virtual places, as well as online  and offline media (i.e. Google, Blender,...
Second Life   Collaboration              Tools                                          (Co-)design                       ...
 Division of tasks and collaborative  place-making in groups (i.e. Seats) Heterochronous, real-time or  asynchronous, co...
 Both inworld and offline    Mostly inworld             Both inworld and offline  processes                    processe...
Rank-scale/ Function   EXPERIENTIAL                               INTERPERSONAL                                           ...
 The analysis of Experiential meaning  potentials focuses on presence, movement  and practical functionality in virtual p...
 The analysis of Interpersonal meaning  potentials focuses on the visual language  and representation of social presence....
 The analysis of Textual meaning potentials  focuses on structure, organization and  multimodal orchestration in design.M...
3D representations       Users as co-designers of SL                                                                   Des...
 Analyzing virtual worlds as places    Meaning potentials in virtual places are defined by not only visualcharacteristics...
 Analyzing virtual worlds as places Analysis of multimodal design with nexus analysis framework    Meaning as multimodal...
 Analyzing virtual worlds as places Analysis of multimodal design with nexus analysis framework Design as construction ...
   Analyzing virtual worlds as places   Analysis of multimodal design with nexus analysis framework   Design as constru...
 Implications for platform and content developers         Affordances for multimodal design         Collaboration is no...
 Implications for platform and content developers Implications for designers       Affordances for multimodal design   ...
 New technologies    Emerging virtual world platforms    VR and mobile augmented reality    3D content on the Web New...
Multimodal Semiotics and Collaborative Design in Virtual Worlds (PhD Presentation)
Multimodal Semiotics and Collaborative Design in Virtual Worlds (PhD Presentation)
Upcoming SlideShare
Loading in …5
×

Multimodal Semiotics and Collaborative Design in Virtual Worlds (PhD Presentation)

1,720 views
1,600 views

Published on

Multimodal Semiotics and Collaborative Design: A Social Semiotic Approach to the Co-production of Virtual Places and Artifacts in Second Life
by
Remzi Ateş Gürşimşek

Public defense of research for the degree of Doctor of Philosophy
Department of Communication, Business and Information Technologies
Roskilde University, Denmark 2012

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,720
On SlideShare
0
From Embeds
0
Number of Embeds
14
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Multimodal Semiotics and Collaborative Design in Virtual Worlds (PhD Presentation)

  1. 1. Remzi Ateş Gürşimşek Public defense of research for the degree of Doctor of PhilosophyDepartment of Communication, Business and Information Technologies Roskilde University, Denmark 2012
  2. 2. 1. Introduction2. Theoretical framework Virtual worlds, social semiotics, design research3. Methodology and Analytical Framework Case-studies, data production, analysis4. The Analysis Social contexts and multimodal semiotics5. Conclusions and Discussions Implications, limitations, further directions
  3. 3. Second Life provides its users withvirtual places and artifacts as a virtual world.
  4. 4. BUT..How can the world’s residents communicate by designing?What do these (Mediated) (Representation of) • 3D Presence Space (Avatar-based) (Sense of) • Social Co-presence Place (Content) (Design) • User-generated Co-production Tools virtual worlds mean to their makers?
  5. 5. BUT..How can the world’s residents communicate by designing?What do these (Mediated) (Representation of) • 3D Presence Space (Avatar-based) (Sense of) • Social Co-presence Place (Content) (Design) • User-generated Co-production SO BEGAN MY QUEST FORTools virtual worlds mean to their makers?
  6. 6. VIRTUAL WORLDS SOCIAL DESIGNSEMIOTICS RESEARCH
  7. 7. Virtual Worlds as PlacesAvatars, Presence & Co-design and Co-presence VIRTUAL Co-creation in VWs WORLDS SOCIAL DESIGN SEMIOTICS RESEARCH
  8. 8. VIRTUALSemiotic resources WORLDSContext of SituationMeaning Potentials SOCIAL DESIGN SEMIOTICS RESEARCH Multimodality and Mediated DiscourseMultimodal Discourse Analysis
  9. 9. VIRTUAL WORLDS Affordances and constraints in design SOCIAL DESIGN SEMIOTICS RESEARCHThe (collaborative) Designerly ways of design process knowing and doing
  10. 10. How do the VW users co-producemultimodal meaning potentials invirtual places and artifacts throughcollaborative design, as exemplified bythe social semiotic analysis of the threecase studies in SL?
  11. 11.  Avatars, presence and co-presence(Bartle 2004, Gordon 2008, Damer 1998, 2009,Schroeder 1995, 2006, 2011) Spatial practices and space/place models in design of collaborative virtual systems(Dourish 2006, Schroeder 2011). Spatial technologies of representation, verbal and non-verbal communication(Ondrejka 2005, Boellstoff 2008, Damer 1998, 2009) Place metaphors, and representation of sense of place in VWs(Bartle 2004, Prasolova-Førland 2008, Taylor 2009).
  12. 12.  Phenomenology and the experiential perspective: Place as ”organized world of meaning”(Tuan 1977, Turner and Turner 2004) Space and place are built on social relations which frame interactive behavior in collaborative virtual environments(Harrison and Dourish 1996) Design of a virtual place as architectural problem(Bridges and Charitos 1997) Architectural space and cognitive/perceptual and physical characteristics of sense of place in built space(Rasmussen 1959, O’Toole 1994, 2004, Najafi et al 2011)
  13. 13.  Social semiotics focuses on sign systems and semiotic resources in particular contexts of situation, through which they present various semiotic potentials for social interaction (Hodge & Kress 1988, van Leeuwen 2005). Semiotic Resources: systems of meaning at people’s disposal Context of Situation: social interactions and relations in specific contexts in which modes are shaped Meaning Potentials: resourcefulness and aptness of the modes in relation to the semiotic requirements of the context of situation.
  14. 14.  Mode: socially and culturally shaped semiotic resources for meaning-making Multimodal cohesion: orchestration of sign systems as multimodal arrangements to present meaning potentials The critique of multimodal approach says: • Modes are neither pre-determined set of rules nor they can be framed by the 5 senses (Pink 2011) • Nature of the social is dynamic and problematic (Latour 2005) • Power, social roles and agency defines the post-modernist view of signs (Poynton 1993)
  15. 15.  Design translates rhetor’s intentions into semiotically shaped material Discourse shapes the choice of design, but choice of design in turn shapes the discourse(Kress 2010) Design = de + signare(Krippendorff 1998) Design as a basic human activity to shape and bring relational order to the lived environment(Papanek 1984, Buchanan 2001a, Heskett 2002) Affordances as meaning and action potentials that are co-contructed through interactions of people and objects(McGenere and Ho 2000, Chemero 2003, Norman 2008)
  16. 16.  Design as a ”a reflective conversation with the situation”(Cross 2007 [1990]) Role of intuition, emotion and aesthetic judgement in design decisions(Simonsen et al 2010) ”lll-defined” or ”wicked” problems of design thinking(Cross 2007 [1982], Buchanan 1992) The problem and the solution in design as ”mutually constructive”, ”emergent” notions(Wisser 2006)
  17. 17.  ”Individual insights” and ”momentary collective processes”(Hargadon and Becky 2006) Emergence of innovative solutions or unexpected outcomes as dialogic process(Grossen 2008, 2010) A ”problem-processing activity” with complex problems and distributed knowledge(Arias et al. 2000, Julier 2008) ”Thinking together apart” and”objects to think with” in distributed design environments(Larsson 2003)
  18. 18.  Co-presence and connected presence in computer-supported collaborative design(Schroeder 2011, Koutsabatsis et al. 2012) Physically remote but virtually co-located avatars in virtual design studios turn space to inhabitated place(Maher and Simoff 2000, Reffat et al. 2008, Weiley and Pisan 2008, Gu et al. 2011) The social creativity of residents as user-generated content , co-creation , and user-driven innovation and as exploitation of voluntary labor(Ondrejka 2005, Pearce 2006, Fischer and Giaccardi 2007, Bonsu and Darmody 2008,Kohler et al. 2009)
  19. 19. Virtual wordsAvatars, co-presence and collaborative place-making in a 3D virtual studio environment with distributed actors Social semiotics and multimodality Social construction of meaning potentials by using modes as semiotic resources in a particular context Design research Designing affordances for both meaningful and functional places as a communicative practice
  20. 20.  Virtual places and artifacts in SL as multimodal data  Participatory design observation  Ethnographic observations with co-designers  Interviews and focus groupsImages of 3D environments and video captures of designer interactions in SL, interviewtranscripts , field notes and online archives
  21. 21. • The three meta-functions in multimodal theory of space and place (Stenglin 2008, 2009, O’Toole 1994, 2004) describe the:  Experiential  Interpersonal  Textual meaning potentials.. Nathan Winters, Architecture Is Elementary, Salt Lake City, UT: Peregrine Smith Books, 1986
  22. 22. The “nexus analysis” or MDA perspective to social semiotics considers: The social actors and relations The mediational means and affordances The collaborative design processes Multimodal semiotics in places and artifacts As analytical units.. Multimodal Analysis and Mediated Discourse Analysis Analyzing both the products and the social contexts of co-design
  23. 23.  Capturing power and agency with multimodal analysis Interpreting the analysis of contexts and contents of design through multiple viewpoints on meaning Generalizations from case study findings Questions about identity, truthfulness and participation
  24. 24.  A design team and a  An organic ally formed  Four design teams, 15 research team group as design team students Professional and  Team and hierarchy  Educational context and amateur participants based on inworld student responsibilities A professional contract friendship  Amateur and first-timer Roles and relations  Professional and  Roles and relations defined amateur participants evolved by learning and Decision –making based  Roles and relations interest on research priorities evolved over time
  25. 25.  Backgrounds, skills and purposes of the co- designers (i.e. Experience, motivation) Professionalism, the amateur spirit and being a ”builder” in SL (i.e. Outsourcing) Individual participations and the social context (i.e. Access to resources) Power relations, organization, decision- making and hierarchy in group formation (i.e. Being the ”chief” of design)
  26. 26.  Movements of humans and avatars in real and virtual places, as well as online and offline media (i.e. Google, Blender, Paper) Affordances and contraints of Second Life determine interactions with both:  The user-interface and design tools  Social affordances of the world The socio-technical environment of semiotic resources shapes not only the processes but also the products of design (i.e. Prim-based thinking)
  27. 27. Second Life Collaboration Tools (Co-)design ExternalMEDIATIONAL Avatars 3D Objects MEANS Material Scripts, textures Resources Inspiration Form Visual language
  28. 28.  Division of tasks and collaborative place-making in groups (i.e. Seats) Heterochronous, real-time or asynchronous, collaboration (i.e. The workshop groups) Collective practices or working alone (i.e. Costume bazaar and the ‘pixel landscape’)
  29. 29.  Both inworld and offline  Mostly inworld  Both inworld and offline processes processes processes Both real-time and  Both real-time and  Both real-time and asynchronous working asynchronous working asynchronous working Tasks are divided or  Tasks are divided as  Tasks are divided by outsourced parts of design interest and skill  In another sim for  Metrotopia’s Sandbox On a workplace 900mt building above Metrotopia  Via various other online  Via e-mail and Google’s media, including e-mail Via e-mail and file collaboration platform and MSN Messenger transfer
  30. 30. Rank-scale/ Function EXPERIENTIAL INTERPERSONAL TEXTUAL Practical Functions: Genre and Style (research/education/content- Metaphoricity Modes as textual elements sharing/community-building) Social Presence Textures and colors Imagined use-contextsSIM AS Size and Ratio (Avatars in Virtual Space) Light and shadows Public/Private Spaces Direction: Verticality Opacity/VisibilityVIRTUAL PLACE Entry Point Mood Rhythm and contrast in shapes and forms – Map/Layout(Building) Modality Division/organization of space Flow of Movement Color Open/closed spaces (Open or Closed spaces) Cues for Navigation/Orientation Field of Vision / Relation to environment Relation to neighbors Functional and non-functional areas Visibility Intertextuality (reference/mimicry/contrast) Orientation HeightELEVATIONS & Sub-functions Elevations in relation to each other Spaciousness Access Elevations in relation to ground-planeDIVISIONS Materials/Texture Functional elevations Color and texture Separation of Groups (Privacy) Visibility / Accessibility(Floor) Movements between levels Degree and permanence of spatial Organization Visual Links between floors/areas Setting/layout Functions of specific placesINTERACTION Modality Scale Types of interaction (user-object, user- Color Open/closed spacesSPACES user) Foregrounding of functions Orientation Relation to outside and other spaces Type of places (activity, community or(Room) Social Presence Spatial connectors content oriented) Focus of attention Style Interactivity Modality Structural elements (prims) Functions of virtual objects:ARTIFACTS Color Textures/maps Providing information(Element) Relevance to environment Scripts and Animations Social interaction Affordances Positioning Decoration Stylistic coherenceINFORMATION Interactivity Modality Functions of surfaces Relevance Information as textureSURFACES Types of information: (still image, Affordances Placement of surfaces Interaction with surfaces
  31. 31.  The analysis of Experiential meaning potentials focuses on presence, movement and practical functionality in virtual places.METROTOPIA PAL WORKSHOPResearch and A social news Narrativeteaching purposes channel for music construction Design hasn’tDesign concept and Design has changed changed afterfunction defined throught the years submissionSim designed as a Sim designed as Small-scale objectscity with interactive open space with and interactivebuildings music zones areas Visitors are guidedSim divided into by the orange ---regions by streets pathway
  32. 32.  The analysis of Interpersonal meaning potentials focuses on the visual language and representation of social presence.METROTOPIA PAL WORKSHOPSuperhero genre Based on ”SL-like”conventions superheroinesTropes in an urbanmetropolitan ”Pixel-landscape” Mixture of narrativelandscapeConsistency with Visualizing music Mythologies andmovies and games and rhythm popular cultureSandbox and Fight Seating areas, the The dance floorClub Dojo club and live stage
  33. 33.  The analysis of Textual meaning potentials focuses on structure, organization and multimodal orchestration in design.METROTOPIA PAL WORKSHOPDivided into streets, Divided by zones Elevated designelevated by and pathways, elementsbuildings elevated ”Sl-like” Prims and texturePrims and textures Prims and textures based modularto signify buildings for signification conceptLogo, notecards, Logo, notecards,posters, maps, posters, maps, Logo and postersteleportation teleportationInteractivity Sound Metaphor
  34. 34. 3D representations Users as co-designers of SL Design andof space, and buildexperiential togethersemiotics of place How do the VW users co-produce In particular multimodal meaning potentials in contexts, by using the virtual places and artifacts through semiotic resources collaborative design, as exemplified by the social semiotic analysis of the three case studies in SL? As designIn a virtual elements,design includingstudio, As a user-driven platform for objects,by using co-design of interconnected textures andavatars virtual places. scripts.
  35. 35.  Analyzing virtual worlds as places Meaning potentials in virtual places are defined by not only visualcharacteristics, but also experiential potentials and affordances for avatar interaction.
  36. 36.  Analyzing virtual worlds as places Analysis of multimodal design with nexus analysis framework Meaning as multimodal texts places are defined by not onlyaction Places potentials in virtual + place-making as mediated visualcharacteristics, but also experiential potentials and affordances for avatar interaction. Content Context + relations
  37. 37.  Analyzing virtual worlds as places Analysis of multimodal design with nexus analysis framework Design as construction of a sense of place Meaning as multimodal texts places are defined by not onlyaction Places potentials in virtual + place-making as mediated visualcharacteristics, but also experiential potentials and affordances for avatar Designers use the available material and semiotic resources in SL to signify interaction.mediated presence in three dimensional space and co-presence in virtual places. Content Context + relations
  38. 38.  Analyzing virtual worlds as places Analysis of multimodal design with nexus analysis framework Design as construction of a sense of place Affordances in virtual places and artifacts Meaning as multimodal texts places are defined by not onlyaction Places potentials in virtual + place-making as mediated visualcharacteristics, but also experiential potentials and affordances for avatar Designers use the available materialform semiotic resources in SL experience. Design in SL orchestrates meaning, and and function for avatar to signify interaction.mediated presence the socio-technical environmentco-presence in virtual places. in in three dimensional space and facilitates co-production of AffordancesContent Context + relations virtual places as multimodal discourses.
  39. 39.  Implications for platform and content developers  Affordances for multimodal design  Collaboration is not always “working together”  Allow movement between media  Expect various actors and contexts  Balance ‘complexity of tools’ vs. ‘freedom to create’  Know that limitations are open-ended, give people some space.
  40. 40.  Implications for platform and content developers Implications for designers  Affordances for multimodal design  Collaboration is not always “working together”  Real-time testing and evaluation of ideas  Allow movement between media  Design by walking around  Expect various actors and contexts  Social affordances  Balance ‘complexity of tools’ vs. ‘freedom to create’  Potentials for participatory design and hands-on learning  Know that limitations are open-ended, give people some space.  Professional limitations and the learning curve
  41. 41.  New technologies  Emerging virtual world platforms  VR and mobile augmented reality  3D content on the Web New social contexts  Semiotic analysis of visitor experience  Specific user groups, i.e. professional designers/architects New methods and perspectives  Participatory design and user-driven innovation

×