Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to UX for XR: Creating User Flows for Mixed Reality
Similar to UX for XR: Creating User Flows for Mixed Reality (20)
Recently uploaded
Recently uploaded (20)
UX for XR: Creating User Flows for Mixed Reality
- 1. UX for Mixed Reality: Creating User Flows Lillian Warner New York University Tandon School of Engineering Integrated Digital Media Program Seattle Interactive Conference 2017
- 2. Who am I?
- 3. HoloGlass: NYU UX/Microsoft Project
- 4. HoloGlass: 3D Art Portfolio
- 6. This is Mixed Reality Image from Iron Man 2
- 7. This is also Mixed Reality Image via Microsoft
- 8. Mixed Reality is Still Being Defined “The best way to predict the future is to invent it.” Ray Bradbury’s “The Veldt”
- 9. Traditional Web/Mobile User Flow
- 10. • Voice Interaction • 360 Sound • 3D Objects • Scaling, re-sizing, and placing objects • Gesturing • Video • Spatial Mapping • Gaze Input • Cortana Input Features Many Components to Design For
- 11. Mixed Reality User Flow: Scene, Not Screen Reality Sequence
- 12. What Designing Can Feel Like
- 13. Ideal End Result
- 14. Our Process
- 16. Where to Start? 1. Research 2. Brainstorming 3. Protoyping 4. Keep Track of the Details
- 17. 1. Research
- 18. Microsoft “The user is the camera.” Image via Microsoft
- 19. Mixed Reality Design Lab GitHub Image via Microsoft
- 20. Interaction Fundamentals Image via Microsoft
- 22. Snapshot
- 23. Medium
- 24. 2. Brainstorming
- 25. Encourage All Kinds of Ideas
- 26. Storyboarding
- 27. 3. Prototyping
- 28. Act it Out
- 30. Consider Web & Mobile Prototyping Tools
- 31. Marvel Prototype
- 32. Marvel Prototype
- 34. 4. Keep Track of Everything
- 35. Spreadsheets Default Page Physical Space User Input (options available) Interaction Holographic Form Sound Big Web Place web on wall or ground Voice Command: Name of artist; Gesture: Airtap to select artist airtap, voice command 3D Web of artist Some sort of cool hover sound; default instructional audio: “To select an artwork, airtap it. Airtap and hold to move the artwork around. Drag the artwork with your fingers to enlarge it. Double airtap to go back” Artist Web Place web on wall or ground Voice Command: Information (artist bio plays). Gesture: Airtap name for artist bio; Gesture: email biography, contact info, or exhibition history airtap, voice command 3D Web of artworks No default audio. Artist Bio plays when selected Painting Place painting on wall or have tag along Gesture: Tap info icon; Gesture: tap Process icon; Gesture: double airtap to go back one step; Gesture: airtap for transcript to be emailed airtap, rotate, adjust size Painting No default audio. Artwork info plays when info is selected; Process audio plays when process is selected Sculpture Place sculpture on ground or have tag along Gesture: Tap info icon; Gesture: tap Process icon; Gesture: double airtap to go back one step; Gesture: airtap for transcript to be emailed airtap, rotate, adjust size Sculpture No default audio. Artwork info plays when info is selected; Process audio plays when process is selected Interactive Installation place cube on table or floor Gesture: Tap info icon; Gesture: airtap cube to experience the installation; Gesture: tap Process icon; Gesture: double airtap to go back one step; Gesture: airtap for transcript to be emailed airtap, rotate, adjust size Holographic rain drops/responsive vertical cord/3d cube of experience Instructional audio by default plays: "To enter the installation, airtap. Walk around to experience the installation. To exit, say 'exit.'" Artwork info plays when info is selected; Process audio plays when process is selected 360 Video place video still on wall Gesture: Tap info icon; Gesture: airtap play button to play video; Gesture: tap Process icon; Gesture: double airtap to go back one step; Gesture: airtap for transcript to be emailed airtap, rotate, adjust size Immersive video Instructional Audio by default plays: "To play the video, airtap the play button. To pause the video, say 'pause.'" Artwork info plays when info is selected; Process audio plays when process is selected Info Page Physical Space User Input (options available) Interaction Holographic Form Sound
- 36. End Result: Reality Sequence
- 37. Describe What Is Happening Scene Input Options Audio User Input Icon Key
- 38. Scene
- 39. Input Options
- 40. ”To make a selection, gaze at the target or say ’Select.’ To move an artwork, air tap and hold. To adjust an artwork’s size, air tap, gaze, and drag. To email a transcription of the audio recording, say ’Email.’” Audio: What Does the User Hear?
- 41. User selects Artist 1 via air tap > > Previous Scene Next Scene Input Selection (User Action)
- 45. Why Now? Why Mixed Reality?
- 46. 1. Industry Growth Image via McKinsey
- 47. 2. XR Design Skills are in Need
- 48. 3. XR is Good for Designers • VUI • AR • VR • Game Dev. • 3D Modeling • Sound Design
- 49. Other Research
- 50. Context Map Created by Shannon Holloway Shannon.Holloway@nyu.edu
- 51. Relational Context: UX Equation Created by Adelle Lin Adelle.Lin@nyu.edu
- 52. Created by Adelle Lin Adelle.Lin@nyu.edu
Editor's Notes
- Last year in one of my classes, I worked with a small group to design and build an application for the HoloLens. Our app is called HoloGlass, and it is a 3D holographic art portfolio, primarily aimed at curators at museums. With our app, users can view and interact with artwork. They can place the artwork in their real world environments, share their world with other users, scale the artwork so it is true to size, and learn about the artist and artwork in an interactive way. This project was my first experience designing for mixed reality, and it was great. -experimental mindset
- Today, I’m going to be talking about mixed reality specifically. Some people call it augmented reality, but for the purposes of this talk, I’m going to stick with mixed reality. Just so we are all on the same page, mixed reality is somewhere between the real world and virtual reality. Mixed reality allows users to place digital objects in their physical worlds and interact with them. It’s different from mobile augmented reality in that it’s a little more comprehensive and can be more immersive. A headset is required, but the user’s vision isn’t fully occluded like it is in virtual reality.
- To me, this scene from Iron Man is mixed reality.
- …and so is this demonstration of someone wearing the HoloLens. The person in this video is video chatting, placing digital objects in her real world environment, and viewing multiple applications at one time. There’s a lot going on
- Field is exciting and still being defined No rules except 1: stay away from 2D assets if possible -not often that a whole new medium gets introduced
- -key piece of documentation that all stakeholders view -design team solely responsible -roadmap -have only one input Creating user flows are a key part of the UX design process. They’re an important deliverable for any product. User flows not only help solidify the product and product’s goals, but they’re also a key way in which designers communicate with developers, product managers, and even clients. User flows help all the stakeholders get on the same page. Communicating user flows is a responsibility of the design team, and it’s important to get user flows right in order to allow developers and product managers to move forward with their tasks. As UX designers, we are all probably very familiar with user flows for web and mobile products like the one here. But as voice-based platforms and augmented and virtual reality devices become more popular, the types of apps we are designing are going to change--and so are our design practices. The way we communicate our design ideas to others is going to change.
- All of these things are tools to help you let the user know what to do next. A good MR experience should incorporate a lot, if not all of these.
- But at the end of our process, we were able to deliver clear and compelling user flows for a variety of interactions/functions within our app. These documents were really helpful to our developers, and they were also crucial in allowing us to explain our final concept before we had actually built it.
- When you’re in the thick of the design process, it can feel like this. Everything component is spread out, it feels messy, and there’s sometimes a sense that there’s no cohesion at all.
- But at the end of your process, you can go back and put together something beautiful like this. Whether it’s in an amazing final product that launches or excellent documentation that you’re handing over to product managers or developers, you’re likely producing something elegant, clean, crisp, and functional.
- Apply animation When my group and I designed our hololens app, we had a lot going on. It felt like the messy kitchen.
- But at the end of our process, we were able to deliver clear and compelling user flows for a variety of interactions/functions within our app. These documents were really helpful to our developers, and they were also crucial in allowing us to explain our final concept before we had actually built it.
- As I mentioned earlier, designing for mixed reality can be really intimidating at first because there are so many different things going on. It can be stressful to begin.
- As I mentioned earlier, designing for mixed reality can be really intimidating at first because there are so many different things going on. It can be stressful to begin.
- Luckily, there’s already a significant amount of documentation out there. Interaction design fundamentals Mixed reality principles
- Table of elements
- 8ninth’s table of elements was especially useful to our team as we explained our proof of concept because the elements gave us the ability to take a snapshot of a mixed reality scene and explain everything going on in a simple and elegant way.
- So much good thinking going on on Medium every day.
- As I mentioned earlier, designing for mixed reality can be really intimidating at first because there are so many different things going on. It can be stressful to begin.
- -write down all your ideas and then see if theyre good for the medium later during low-fi prototyping.
- After researching and just being aware of the various resources out there, I found it really effective to just start writing things out on paper without worrying if it was messy. Again, because of the variety of components happening in any given MR scene, sketching and writing everything down gave me a sense of understanding the scope of each scene. FIGURE OUT HOW FAR AWAY OBJECTS ARE FROM THE USER
- The most important thing we did. Can do sound, interactions, field of view, etc. Our world is 3D and a mixed reality experience should be 3D. Limited though. Sensory experience. The only form of prototyping
- Use props and tools-–doesn’t need to be fancy. Sensory experience. The only form of prototyping . One takeaway was with the field of view—we realized we didn’t want to clutter our scenes, and the only way we were able to make that design decision was by using the sandwich box to approximate the hololens. Make clear that we didn’t have the hololens the whole time
- 2D prototyping isn’t perfect, but this is a more polished way of refining storyboarding. The downside to this is that there is often not enough room to showcase text and there aren’t effective ways of showing animations or interactions aside from clicking. But in terms of pulling your product’s user journey together, it can still provide a good foundation.
- Storyboard but more detailed and enhanced
- Incredibly important. This is the most detailed thing we did while designing our hololens app, and we referenced it constantly. Pairing this with any of the visual elements of our design was incredibly helpful in communicating a specific scene.
- And lastly, of course, any good user flow should show what’s happening at any given moment. This is clean and simplified, but can be a lot more comprehensive. One drawback to this format is that it takes up a lot of space on the page and it doesn’t have the traditional information architecture flow style that complicated products often have. However, to deliver a super comprehensive flow you could put together a thicker packet of flows in this format, and add additional content, like HQ renders, for your developers as you see fit. But in general, we found that this streamlined flow hit all the main points that we needed to communicate to our developers and other people interested in the project.
- Object distance from user
- Goldman Sachs says the combined markets of AR and VR will be valued at 95 billion by 2025 (McKinsey) -Some people are predicting that 2019 will be the year of the mixed reality headset. Microsoft is likely releasing a new version of the HoloLens -META, ODG, and MIRA have received hundreds of millions of dollars in investments this past year -Magic Leap, which is also building a headset, is currently valued at 4.5 billion dollars. -Big companies are starting to introduce mixed reality into their workflows. BMW, Citi, NASA -Can help save companies money during prototyping processes in product development cycles -McKinsey report: 2017: demand for freelancers with VR development and design skills grew higher than any other
- -Big companies are starting to introduce mixed reality into their workflows. BMW, Citi, NASA -Can help save companies money during prototyping processes in product development cycles- -McKinsey report: 2017: demand for freelancers with VR development and design skills grew higher than any other -Doing all this for the first time All of these different types of realities are only becoming more widely used. Companies need people who can design and build for these platforms. More and more companies are also going to be testing out these new technologies for the first time. Communication of design goals is going to be more important than ever for companies doing this for the first time. Which is why it will be really useful to be able to create great user flows for your team.
- MR incorporates features found in voice apps, virtual reality, and augmented reality. So when you’re learning how to design for mixed reality, you’re also inadvertently learning how to design for those mediums as well. The wide variety of features that MR provides is what makes it seem messy and complicated at first, but it’s also what makes it the most exciting and interesting. When you’re designing for mixed reality, you’re also learning how to design voice interfaces, you’re learning about ux design for vr and ar products too.
- Maybe add picture of makerspace?