Recommended
More Related Content
What's hot
What's hot (20)
Similar to Better Together: Building Social Immersive Technologies
Similar to Better Together: Building Social Immersive Technologies (20)
More from Liv Erickson
More from Liv Erickson (6)
Recently uploaded
Recently uploaded (20)
Better Together: Building Social Immersive Technologies
- 1. Better Together Building Social Experiences for Immersive Technologies @MissLiviRose
- 3. This is a talk about technology
- 4. This is a talk about technology people
- 5. Social: relating to society or its organization.
- 6. Social: Communication + Community
- 7. Social Technologies are already widely used
- 10. Synchronous Asynchronous SymmetricAsymmetric Same tool, different times Different tools, same time Same tool, same time Different tools, different time
- 11. SocialMedia
- 12. VideoGames
- 13. Do we always understand when our technology is social?
- 14. Social “Secondary” Technologies “Transportation network” “Transportation network” “Marketplace and hospitality” “Video sharing” “Business directory service”
- 15. Social “Secondary” Technologies “Word processor” “Password manager”
- 16. So Let’s Talk Immersive Tech Virtual reality, augmented reality, mixed reality
- 17. Immersive: providing information or stimulation for a number of senses
- 18. Virtual Reality (VR) Replacing your physical environment with a digital one
- 19. A Few VR Terms 3 Degrees of Freedom 6 Degrees of Freedom Untethered
- 20. Augmented Reality (AR) Adding digital objects to your physical environment
- 23. Types of AR Content •Mixed reality (MR) – often used to describe head mounted AR technology that has the digital world interact with the physical world •Heads up displays – adding UI menus to physical objects, like your car windshield •Marker-based AR – uses images or QR codes to tell the device when to display a piece of digital content
- 24. But… why?
- 26. UserExpression
- 27. Recall&Impact
- 28. Examples How people are using Social AR & VR tech today
- 30. hubs.mozilla.com
- 33. BlueSkyPaint
- 36. Building Social VR & AR Considerations & Challenges
- 37. Establishing a shared base ‘reality’ • Anchor points • Private vs. public content • Practical for asymmetric content… necessary for safety in virtual reality
- 38. Establishing a shared base ‘reality’ • Relay information from the headsets of different users – relative position from tracking cameras, head rotation, controller positions • Keep a shared environmental state synchronized between people • Assign physics ownership of objects
- 39. Laws of Physics •How closely do you want to mimic real-world physics? •What happens when two users apply conflicting physical actions? •Can your users fly? •What happens when a user sticks their head through a wall? •Are physics really necessary?
- 40. Teleportation • How does controller movement impact trajectory? • What do other people see when you’re teleporting? • What kind of effects does the user see? • Which buttons cause the teleportation start and end actions? • Where are you allowed to teleport to?
- 41. Teleportation
- 46. “I don’t see my avatar as a vehicle of self-expression. I see it as a tool to be recognized, or to be anonymous.” Jessica Outlaw
- 48. Identity and Personal Information • What can information tracked by virtual and augmented reality sensors reveal about you? • What sort of sensors can be added to these experiences in the future? • Eye tracking has benefits, but can give a lot away Put the user in control of this information!
- 49. Personal space
- 50. The Handshake Problem • How do you handle voluntary interactions around touch that require controlling the other user’s body? • What do you do if one user doesn’t have hand controllers? • What if one person has a VR body, and one is seeing them in AR? Technical challenges aside, the handshake problem represents a social consideration for immersive environments – knowing the difference between wanted and unwanted contact is a challenging one for software to determine.
- 51. People are the hard part
- 52. Impact of Immersive Technologies • Things that feel more realistic can be positive or negative • Access to the technology will impact who can benefit from them and what types of content is made • Democratization of knowledge and access to experts is changing our educational processes • We get to push the boundaries of human-computer interaction to entirely new cognitive levels
- 53. Impact of Immersive Technologies • There is a lot of information that could be exposed about our actions and behaviors • Fundamental questions of assumed privacy and longevity of our communication channels are changing • You’re (probably) putting cameras on your face
- 54. (Future) Impact of Immersive Technologies • Wide-scale adoption can offset carbon emissions related to travel, especially in training and educational scenarios • Facial tracking and reconstruction can be combined with photorealistic digital bodies will force questions about identity use
- 55. (Future) Impact of Immersive Technologies • Wide-scale adoption can offset carbon emissions related to travel, especially in training and educational scenarios • Facial tracking and reconstruction can be combined with photorealistic digital bodies will force questions about identity use If used and created with care, virtual and augmented reality can remove the barriers of geographic distribution to create a healthier and more equitable global ecosystem.
- 56. Looking Ahead • Increased availability of AR features in social applications • Wider adoption of VR technologies • More integrations that work with platforms we already use • Acceptance of dynamic identities • Greater attention to privacy and ethical impact
- 57. Getting Started (The Social Side) 0. Recognize that you already can contribute 1. Identify a community that aligns with your values – or create one 2. Establish a relationship based on what you can offer and the community’s needs 3. Be an active participant – whatever that means to you and the community 4. Communicate! 5. Learn, contribute, and repeat
- 58. Getting Started (The Tech Side) 0. Recognize that you already can do this – no headset required 1. Watch the talks that happened at #FullStackCon on the immersive web How to Build X in 3D – A short Introduction to Babylon.js: http://bit.ly/2xFmSfu Building Immersive Worlds with Mixed Reality: bit.ly/32veaPg Getting the Web to Speak: Using Polly and three.js to Create a Realistic Avatar: bit.ly/2xJ3IW6 Using the Web to bring the Internet of Things to High End Augmented Reality: bit.ly/32dkACi Building an ARt Gallery in the Browser: http://bit.ly/2Sd9IA0
- 59. “Technology’s come a long way – and can do a lot of things – but it can’t feel.” Jessie Wisdom
Editor's Notes
- Hey everyone, thank you – I’m incredibly excited to be here because, as you might be able to tell from my talk, I am extremely interested in exploring the ways that technology intersects with social experiences, and tech conferences are definitely a part of that.
- My name is Liv, I’m an OSS virtual reality developer. I’ve been working across VR and AR technology for almost 5 years. I experienced virtual reality for the first time in 2015 – I had started to hear more about developments happening with hardware for VR headsets, and the growing community of other people who were drawn to the opportunities that immersive technology presented. It was not only eye opening to step inside of a digital world and experience truly being immersed in the creation of someone else’s imagination – it was the first time since moving across the country to California that I had begun to find community. I’ve been building virtual and augmented reality applications ever since trying the first version of the Oculus Rift, back in 2015, and over the past two years, I’ve focused exclusively on building open, social applications for 3D worlds using virtual reality. But, the VR and AR elements aside, this is also a very personal talk for me to give and the first time I’m giving it, because technology – specifically AR and VR technology – has given me the framework and opportunity for me to learn how to more effectively communicate, because I’m actually naturally quite socially awkward and have a lot of trouble processing and following along with spoken communication. Working in this area has changed my focus and understanding of what technology could truly enable.
- This is a talk about technology. It’s a talk about technologies very near and dear to my heart – technologies that I’ve lived, worked, and breathed for the better part of my career. It’s about technology still in early stages of development, but with a lot of potential to change the way that we are able to communication. It’s about technology that you look through, or into, and yes – that means that sometimes, it’s technology you wear on your face.
- But it’s also a talk about people, because every day, the technology that we build impacts people in different ways. It’s a talk about the past, present, and future of virtual and augmented reality technology, and the social interactions that have led to the normalization of 3D worlds and social experiences. We’ll take a look at the ways that technology has changed the way that we interact socially, what it means to build social, immersive technology applications, and the challenges that lie in working in this space.
- Let’s break this talk apart, starting with ‘Social Experiences’. For us as humans, social connectivity is a powerful part of who we are. It defines our lives, perhaps more than anything else we do. Our ability to communicate and connect with one another, to form deep relationships, motivates our behaviors and actions starting from birth. I’m going to break this into two more concrete components: Communication, and community.
- It isn’t a coincidence that ‘community’ and ‘communication’ share a root – because it is with communication that we form relationships and subsequently, communities. We’ve organized here, this week, as a result of the relationships that we have with each other in the larger development community. We likely all have different motivations for what brought us here, but there are common threads in that we chose to be here to learn and grow. I know that I have. While we might not communicate the same way as one another, at the core of what we do, quite a lot of our time is spent communicating with others – being social. Technology has always facilitated communication in different ways, starting with early forms of communication like painting and writing, leading into the telegram and telephone, email, text messages, social media, video chats, and now, into virtual and augmented reality. Using technologies available today, we can translate between languages in almost real-time and have the power to change the way that we convey information between one another. Communication is inherently a social technology, and it’s important to recognize how communication is not restricted to verbal speaking. We communicate ideas, feelings, thoughts. We communicate to drive action in our friend groups, to find empathy in a challenging time that we’re facing, to educate one another about knowledge we have that we don’t – we communicate to create social bonds that surpass what we are able to fully experience as a single individual. We hear the term ‘socializing’ sometimes used to describe the act of being social, often through the lens of entertainment. We are social for entertainment purposes, sharing positive and enjoyable experiences with others we care about, or seeking out other people who share our experience of being entertained by a movie, game, TV show, book, art piece, poem, song – and we look for ways to bond over that entertainment. We’re probably also learning, creating, or exploring the ideas that entertainment sparks within us, and for many, there is an inherent drive to find connection with others around those things. Perhaps without thinking about it, we may already have a conscious model of what ‘social’ means – but I’m going to dive into a few different types of social experiences that we have to frame the conversation about the technologies that are built around facilitating those interactions.
- I’m first going to talk about the types of social communication and then give some examples of social technologies we use today. People have a lot of different notions about what it means for a technology to be social. There’s also a few concepts that I think related to the different types of social technologies that are important to address, like – how and when we do it.
- One of the ways that we communicate is synchronously – in fact, I’d guess that this is probably one of the first things that comes to mind when people think about social communication. Technology enables a lot of real-time communication: we can speak to each other in conversations, trading off who is saying things (or, in some cases, literally talking at the same time, but I’d argue that makes the communication part a bit more difficult). Video and phone calls also allow for synchronous communication and socialization – it’s all happening at roughly the same point in time.
- Let’s talk then, about the opposite of synchronous communication, async. Probably familiar with this, but let’s apply the terms to how we talk to one another. This tends to be the side of things that people don’t really consider as much when they think about some types of social technologies, because without technology, it’s far harder to socially communicate asynchronously. We did that by learning how to write things down and read, so that generations could communicate asynchronously, granted, only in one direction. It’s a bit out of the scope of this talk to cover how we can build technologies to communicate back to the past. And these aren’t discrete terms – but they give you a sense of how we can start to set social expectation around communication. Text messaging is a really good example of a tool that fluctuates between synchronous and async communication – and there are definitely errors that can occur in how we relate to one another if different people in the conversation are viewing their communication from one end or the other. Who here has ever had an experience where you’ve either gotten frustrated by or been the one frustrating someone for not getting back to a text message quickly enough?
- Okay, so we’ve tackled the element of timing in our conversations. Great. Let’s talk about the tools, and why that matters too. I’ll introduce the terms ‘Symmetric’ and ‘Asymmetric’. These are important qualifiers when we think about building social technologies, because they change the social dynamics of experiences and communities. Symmetric communication interaction occur when all involved parties have the same tools and methods available to communicate equally with others. Asymmetric communication describes cases where methods of communicating are not equal. Synchronous, symmetric: talking on the phone, and we’re both using voice communication. Talking via facetime, where we both have cameras on Asynchronous, symmetric: text messages, where we’re both using text, gifs, emojis, but we’re not immediately expecting responses in real time. Email. Voice mail. Synchronous Asymmetric: I’m live streaming, and I can speak with video and chat, but you can only send messages or hearts Asynchronous Asymmetric: I send you emails, you reply in text messages. I post a video and communicate through my video, and you leave me comments. And again, these aren’t ‘hard and fast’ categories. I could describe me, on stage, giving this talk to you, from two different lenses – It could be considered synchronous, asymmetric communication, because you all are answering questions by raising your hands to indicate assent, while I’m speaking, and even if you chose to speak, I have a tool that you don’t, a Microphone and visual aids. This doesn’t even speak to all of the _social_ power that people may carry with them in a conversation, because there are a lot of unspoken rules there, too. However, I could also describe it as asynchronous, symmetric communication when we get to the Q&A part, because I won’t know what you want to ask, and you’ll have a microphone too. It’s natural for us to move between each of these elements, so let’s look at some of those further.
- Social Media. We’re all probably familiar with some form of social media and experienced the growth in popularity of asynchronous, often asymmetric communication platforms. To exclude these from a conversation about social immersive technologies ignores a lot of lessons in community and relationship building. Over the past several decades, especially with wide-scale adoption of smartphones, we’ve become even more accustomed to the differences in communication and availability of communities that aren’t geographically bound.
- Video Games Videos games are often associated with virtual reality, because 3D online games are some of the closest predecessors of what we usually think of when we think of VR technology. Talk to World of Warcraft, forming guilds, events, voice chat, embodiment.
- Lyft and Uber: social interactions (or not) between passengers and drivers. A lack of infrastructure around the social elements leads to people pretending to be deaf to avoid conversations, or getting taken to places they don’t belong, or getting letters to their addresses later from drivers. AirBnB facilitates social interactions with assumed trust in the platform between hosts and guests, but has little in the way of support that they can offer outside of legal battles and monetization efforts if and when things go poorly. <Tell story> YouTube – I called this out earlier, because I think it’s a bit more obvious than some of the first couple of examples, but it’s important to note the they still first and foremost bill themselves as a Video Sharing network, which is interesting because it’s also now a community platform and entertainment site where you’re invited to join Youtuber communities, purchase things from them, and communicate with them through comments. And again, many of the community management features fall short, because the social element isn’t as embedded into the product as, perhaps, it should be. Same with Yelp – the reviews and comments facilitate a lot of very one-directional communications, and it brings a largely unfederated social experience into something like looking up a restaurant. Which is interesting – it’s something that we do with friends – but a lot of that underlying trust that has to exist to guide social interactions exists in a bit of a closed off architecture that Yelp owns.
- But what about a few things that are a bit bigger? Kids – using Google docs 1 password: something that should not be social… until it is.
- Social immersive tech, pre-VR/AR (world of warcraft, social media)
- Talk to the loosely structured definition of ‘immersive’, it can really be most anything that takes over a lot of you retention and makes you feel like you’re in a new place or experiencing something different; it doesn’t specifically have to be VR/AR/ etc. but that’s what we’ll focus on today
- Social immersive tech, pre-VR/AR (world of warcraft, social media)
- 1) Explain 3 degrees of freedom, and
- Social immersive tech, pre-VR/AR (world of warcraft, social media)
- Social immersive tech, pre-VR/AR (world of warcraft, social media)
- Being social with equal footing – to an extent. VR and AR doesn’t fix the challenges with equal access to infrastructure around power and connectivity (again, a larger talk that brings in a lot of social and geopolitical considerations), but it can start to bridge connections for areas that do have those infrastructures. Talk about getting access to global experts from anywhere with access, how you can personalize business or communication, reduce travel costs. Games platforms and social media have begun to facilitate that, now we start to lay the groundwork for actually bringing you to a different location in real time.
- User expression. You can make some guesses off of how I’m feeling, but digital embodiments give us a far greater surface for expressing ourselves. And, in a lot of social interactive experiences, you can use not just your body or online profiles to express things, but entire worlds.
- Studies show that having participants complete tasks around empathetic behaviors in VR has a tendency to last longer in how long they demonstrate empathetic characteristics. You’re also able to do things that you can’t do easily alone – like practice giving a talk to a bunch of people.
- Social immersive tech, pre-VR/AR (world of warcraft, social media)
- Remote co-working and organizational committees. How many of you participate in OSS, have been a remote attendee of a meeting, or have been in a meeting with a remote attendee? Immersive technologies allow us to facilitate different types of meetings that rely heavily on expression and interaction – say, brainstorming or collaborative design sessions – and give people a different set of capabilities than they would have working in a 2 dimensional medium.
- Virtual meetups and events. This is the product I work on at Mozilla, Hubs, which aims to integrate 3D worlds into work flows that are familiar. It’s a 3D web application. You can screen share, video share, bring in emojis and gifs and images and videos, and chat – either from the world itself, or via connected chat platforms. Platforms like this facilitate all four types of communication: Synchronous, symmetric through voice or text chat when we’re all in the same space Syncrhonous asymmetric, if I’m watching a stream and chatting instead of being present with a virtual body in the space Asynchrnous symmetric if we leave content around the world and visit at different times. (Lead into Where thoughts go)
- Watching things together – virtual cinemas or even less casual, things like
- Asynchronous, symmetric social.
- AR apps where some of the content itself is social, but people are social even outside of the app mechanics by playing individually, together.
- One of the most fun examples of async game play
- Social immersive tech, pre-VR/AR (world of warcraft, social media)
- Explain how you need to think about the types of content you want to create – how much of physical reality does the user need to know about or experience? What are the things you expect different people to know about the scene? Is everyone supposed to see the same thing, or do they have different realities? Talk to the types of application where it makes sense, and where this might not Last point: You need to know where your wall is in the virtual world even if there’s no virtual wall there
- When you create a social experience, you need to give users some kind of agency over their digital representation of themselves in a space. A talk on what comprises identity and how users can express themselves could cover an entire separate keynote, but for this specific component, I’ll speak to the implications of the virtual bodies that users may choose from in a social experience. If you’ve ever played a video game, you may be familiar with the process of character customization – and some virtual and augmented reality applications use a variety of mix and match parts for creating the digital bodies for players – but immersive technologies, which allow the embodiment of these forms – can lead to users having different cognitive models for how they relate to the body that they’ve chosen. One area that this can come up frequently in is hands – it’s actually quite a strange experience to look down at your hands and see something different than you’re expecting. I’ve had times where I’ve put on a VR headset and grabbed my controllers and I’m immediately taken out of the experience because I have these very detailed masculine, large, hairy hands. It’s jarring and completely at odds with what I’ve expected, and I certainly wouldn’t feel like myself interacting with other people in a body like that! But, creating fully featured character creators is a technical challenge, and it may not be entirely appropriate in all scenarios, nor is it likely to completely satisfy everyone who is using your application. So, in many instances, platforms could instead choose to have some set of preconfigured defaults, and in other cases require that users create their own characters using a different tool and that they use those. So let’s say we have an application that has some default choices for bodies. One of the design considerations here is that you have a diverse and inclusive set of designs to choose from - because you want users to be able to find something that they feel comfortable using as a stand-in for their own physical self.
- It’s important to note, though, that the types of bodies that users choose isn’t just about one’s own identity – it’s also something that signals and communications clues about that person’s identity to the other people who are in the virtual spaces around them. If you’ve spent time on social media or playing online games, you are possibly already familiar with the fact that users tend to bring their assumptions and biases into the virtual spaces that they inhabit, which means that avatar selection can – and, unfortunately, does – lead to harassment and abuse in these social spaces if the community isn’t managed effectively.
- Post by Jessica outlaw – are people paying enough attention to how their choice of representation is exposing different elements of their identity? Which was super interesting because the author (a social scientist who researches social VR) said: “I don’t see my avatar as a vehicle of self-expression. I see it as a tool to be recognized, or to be anonymous.”
- Talk about giving away identity information and the choices that I make with each of these. I’m almost immediately recognizable, anyone who has seen a photo of me might recognize this as me, in a virtual space– assuming I’m the one actually using it. An abstracted out version of myself hints at (but doesn’t confirm) my appearance. Robots are more anonymous, but can still be identifiable and can still choose which parts of my identity that I’m giving away.
- There are other parts of our identity that we expose to varying degrees when we’re in virtual spaces. Just by speaking, people might make assumptions about gender, geographic location or ethnicity. They might be able to match it to videos that you have online, or recognize you. The higher-fidelity tracking of a system that you have, the more information that you’ll expose about your movement or lack thereof – maybe I can see that you’re sitting on a couch, or walking around your room. Perhaps other users can see that you’re using a 3DoF headset to access a space, instead of a 6DoF headset, and make assumptions about your financial status. Headsets that have eye tracking enabled could expose that information to others in the space – which is something that at small scales, we’re quite used to – you can tell roughly where I’m looking right now – but this information can give highly personal information away as to my interest in a space if you can tell I’m staring off in the distance because my robot isn’t looking at you. There’s also a lot of room for development and exploration into services that mask these things, or the degree to which a platform might choose to make those decisions on behalf of the user. Eye contact gives a lot of benefits in a social situation, but even in our physical world, different users have different capacities and preferences for making direct eye contact. For me, it’s quite difficult, so I might want a social platform to make choices about where to move my eyes. Other people might feel differently – these are all open questions that we’ll need to solve for.
- VR and AR introduce even more ‘human’ into the idea of ‘human computer interaction’ and there’s a lot to consider from that!
- I came across this quote earlier in the week and I think that it sums up social technologies, especially empathy and presence-driving technologies like virtual and augmented reality, incredibly well. Technology’s come a long way – and can do a lot of things – but it can’t feel. We feel, and we feel through the tools and software and platforms we use. We feel excited when we walk into a larger-than-life world with our friends from the other side of the country, and we feel upset when we are the target of aggression or anger by someone else, even when we’re not experiencing that face to face. It’s important, as creators of technologies by and for humans, that we pay close attention to the impact of those technologies, especially the impact and responsibility that comes with connecting people. Because, great power, great responsibility.