Recommended
More Related Content
What's hot
What's hot (6)
Similar to Virtual Reality: Immersive Experiences with Real-world Applications | Jill Knight
Similar to Virtual Reality: Immersive Experiences with Real-world Applications | Jill Knight (20)
More from LavaConConference
More from LavaConConference (20)
Recently uploaded
Recently uploaded (20)
Virtual Reality: Immersive Experiences with Real-world Applications | Jill Knight
- 1. 1 | Copyright © 2018 Deloitte Development LLC. All rights reserved. Virtual Reality: Immersive Experiences with Real-world Applications
- 2. Agenda 1 2 3 4 5 6 Introduction What is VR? The Development Pipeline The Paradigm Shift Questions VR Demo
- 3. Introduction Jill Knight • Interactive Media & Game Design @ WPI 2013 • 3D & Experience Designer @ Deloitte Digital (Digital Reality DxR Group) since 2015 • Working in VR/AR since 2016 • Second generation working in technical communication/learning in technology
- 4. 4 | Copyright © 2018 Deloitte Development LLC. All rights reserved.
- 11. The Hardware Spectrum 6ft x 6ft min 12ft x 12ft recommended
- 15. 15 | Copyright © 2018 Deloitte Development LLC. All rights reserved.
- 16. 16 | Copyright © 2018 Deloitte Development LLC. All rights reserved.
- 17. 17 | Copyright © 2018 Deloitte Development LLC. All rights reserved.
- 18. 18 | Copyright © 2018 Deloitte Development LLC. All rights reserved.
- 19. 19 | Copyright © 2018 Deloitte Development LLC. All rights reserved.
- 20. 20 | Copyright © 2018 Deloitte Development LLC. All rights reserved.
- 21. VR isn’t a universal solution.
- 22. 22 | Copyright © 2018 Deloitte Development LLC. All rights reserved.
- 23. 23 | Copyright © 2018 Deloitte Development LLC. All rights reserved.
- 24. Client Team Team Structure Engineer Lead Eng. Producer Exp. Designer Lead Exp. 3D Artist Lead 3D Vis. Designer Lead Vis.
- 25. Development Environment Engineer Exp. Designer 3D Artist Vis. Designer
- 28. Discovery Design Develop Deploy
- 30. Discovery (need stock image)
- 59. Questions?
- 60. jiknight@deloitte.com @jill3d Thank you! Jill Knight 3D Artist/Experience Designer
Editor's Notes
- Virtual Reality is the cutting edge of gaming entertainment, with emerging technology that immerses players in new worlds like nothing we've seen before. But VR is increasingly being used for real-world applications in a wide variety of industries, not just for games. In this session, we'll look at how VR enables us to create training and educational experiences that are engaging, impactful, and address the challenges of employee training in innovative ways. From concept to implementation, we'll talk about the development of virtual reality experiences, including multi-disciplinary team roles, needs assessment, planning, graphic and textual content development, and deployment. We'll also explore how the platforms of virtual reality and augmented reality have created a paradigm shift in designing user experiences, and the interesting challenges presented by these new technologies.
- Introduction: Jill Knight, 3D Artist and Experience Designer Wears a lot of hats Experience on projects a variety of different VR platforms Mom: Interplay of UI and UX after this session!
- Virtual Reality Definition: Interactive computer experience taking place within a simulated (virtual) environment Umbrella term we use for simulated experiences that make use of a head-mounted display that takes up your entire field of view Entirely obscures the real world
- Cannot see the real world when you are in a virtual reality simulated environment – for that reason, it is immersive: you feel embedded in and transported to that simulated space The power of immersion allows us to do things with this technology that aren’t possible in other platforms/mediums Let’s talk about VR as a platform: broad range of hardware solutions, hardware itself plays a major role in the kind of experience you can build
- Lower-fidelity platforms: Lower resolution Lower level of interaction (often none – passive 360 video or 3D experiences) Lower price point/barrier for entry Relies on other hardware for visuals(mobile phones) Examples include Google Cardboard and Gear VR, made for use with modern phones
- Middle-fidelity platforms: Standalone hardware, not reliant on smartphone Better resolution/graphic performance Controller input, some controller tracking in space (spatial relationship between headset and controller, rotation and location in space) Middle price point Examples include Oculus Go!
- Lower- and middle-fidelity experiences are often seated or standing - stationary. No tracking in real-world space – if you move IRL, you do not see the environment in the headset reflect that Because of this: motion sickness is an issue (think roller coaster seated experiences) Still 360 view, but stationary Even when you have controller support for some interactions, lack of tracking can create motion sick experiences (flying squirrel suit)
- Higher-fidelity platforms: Higher resolution Tracked controllers with multiple inputs – can perform more complex interactions Headset tracking – understands your movement in real-world space within the field of view of the sensors Higher price point Requires high-performance PC/laptop that drives the graphics processing Equipment on slide: HMD Hand controller Stands on either side are sensors that track movement in real world space Examples include HTC Vive, Oculus Rift, PlayStation VR
- Higher-fidelity experiences we call “room space” – wall or stand-mounted sensors tracks the user’s movement in real-world space, either 180 or 360-degrees Minimum footprint for Vive is 6x6, max is actually about 19x19 Have to be within line of sight of both lighthouse sensors Lighthouses are IR/optical
- Games for entertainment: Job Simulator!
- Games for entertainment: authoring tools like Google’s Tilt Brush! (More creative/exploration that game with mechanics/goals)
- Serious games/simulators for education and training in many different industries! This space is GROWING
- Why VR? Why is VR a good platform for training and education?
- Immersion: real-world scale provides learning in context to actual tasks – the power of immersion to create a lifelike simulation New technology appealing to a younger generation of workers
- - Good for situations where training is otherwise prohibitively dangerous, difficult, or costly
- Good for procedure training and reactive simulations – limitlessly repeatable for the learner, where repetition increases retention of information Reset of simulation can be faster than reset of other kinds of training, as well as reducing time of real-world tasks to increase repetitions as well
- Good use cases include the things just shown: Equipment training Procedure training Medical (surgical/situation training)
- Good use cases include the things just shown: Tool creation! High-tech manufacturing workflow (Factory 5) Reduce cost of iterative design workflows – moving away from expensive and time-consuming real-world sculpted models to VR mockups
- Not the right platform for every learning solution/experience Justification for VR – has to be a good use case/add value to the learning experience There are bad use cases…
- Bad use cases: Something that cannot be replicated in VR (kneading bread) No value add for being in a cool platform(time sheets/taxes) So when we have the right use case….how do we actually develop for the VR platform?
- Before the how… let’s talk about the who
- Producer: Team project manager Runs sprint planning, internal meeting cadences, tracks deliverable dates etc Interfaces with client/client team to plan meetings and relay questions from developers Development team: Engineer Experience/Game Designer 3D Artist Visual Designer - Leads from each discipline have small bucket of hours for oversight – feedback on individual work as well as the direction of the project as a whole with regards to their discipline
- Software and tools user by individual disciplines - Each role uses different software programs to author their individual content, but as you can see, everyone has a hand in implementing their assets/content into….
- Unity – our game engine of choice! - Where all of the assets created by each team member comes together Marrying the 3D and 2D visuals with the functionality/mechanics – where we make it all work All team members have at least some familiarity in implementing their work in the engine
- Agile-ish workflow: using JIRA to track sprints and individual tasks/tickets assigned to each team member Our development process is agile plus video game development (alpha, beta, gold) – sort of waterfall by necessity (client contracts)
- Four main steps of project: - Discovery - Design - Development - Deployment
- Linde Project Brief: Company that manufactures and delivers liquid gas to clients Lots of challenges in training new drivers on how to deliver gas: Long training period that takes experienced drivers out of the field (10 weeks) to bring new drivers up to speed Actual gas delivery done infrequently in training, and is a time-consuming process (half a dozen times across training period) – (1.5hrs total) Procedure for delivering gas has 40 STEPS – retention of info was low Important to standardize procedure for safety reasons – cryogenic gas! (Found even experienced drivers were not doing the steps in the same/exactly correct order)
- Discovery: Understand user needs Identifying technology requirements and constraints Establish medium: pre-rendered vs real time, delivery platform
- Justification of VR – is virtual reality the right solution platform? If it is, which level of hardware is appropriate for how the experience will be deployed?
- Discovery artifacts: Design specs for technology (we like to develop and iteratively test on the same hardware that will be used for final deployment) Reference video and photo, at the least for scoping purposes (field trip to client site – video, photo, audio reference) Notes from user interviews and other client stakeholders around the problem we are trying to solve
- Design: Scripts & Storyboards Gathering additional visual references Creating concept art to inform look & feel, design of specific characters or spaces Building an asset list and timeline for development cycle
- Design Artifacts: User flows/storyboard/scripts Project timelines Asset lists Concepts/mockups of both 2D and 3D elements
- Develop: Creating assets defined by design process Work towards minimum viable product – game is playable at every deliverable stage Iteration based on feedback from internal groups and clients
- Iteration – we do a lot of it! Development process built around having feedback cycles that include: in-person client demos/playthroughs with stakeholders Giving the build to clients for testing with end users (on Linde – drivers and driver trainers) Creating tasks in future sprints based on consolidated feedback Reworking features based on feedback
- First deliverable – alpha – is a vertical slice of the finished experience: Prototype of major functionality/features and visual components, but not every single feature and not polished – again, prototype! Methods of interaction Overall layout of environment/space In Linde, this was the first 8 steps out of the 40 step procedure
- Early valve turning prototype that did not work with physics as expected Blocking out physical space based on footprint constraints from the client
- Beta : Major functionality implemented, prototyping refined Most major visual components are in Items that needed addressing from first round of feedback have been incorporated Still needs an iterative polish pass Lighting is correct for a cloudy day, pipes have correct metal assignments – steel, plastic, brass
- Beta : Implementing animations and additional interactions (hose!)
- Feedback!! Left: feedback from client – about specific steps but also overall items of note that testers called out – awesome client gave thorough feedback, though it took some time to parse, consolidate, prioritize, and assign to an owner Right: my notes on specific items of feedback based on watching user testing and debrief after client stakeholder conversation ** really important to build time into the schedule for these client feedback cycles, both in terms of how long it takes a client to provide feedback when they get a deliverable, and to incorporate feedback into already existing workloads
- Gold: the final deliverable! Thoroughly tested for bugs, intuitive interactions, feature completion and performance goals All features and visuals completely implemented Polish pass at the end of the dev cycle
- Polish items include: Effects like x-ray and steam/particles, frost buildup on the pipes, polished animations Refining the materials – brass looks like brass, steel looks like steel etc
- Also final implementation of menu systems/front matter at the start of the experience, as well as after-action report that has all of the metrics hooked up on the back end.
- User testing! We did user testing throughout – gave builds to the clients to test with their end users as well as on our side, with our colleagues/willing test subjects - Looking for intuitiveness of interactions, real-world scale, readability, as well as bugs/show-stopping issues
- A note on FIDELITY: On some projects (like with Linde), important to have true-to-life fidelity so that the learner has clear expectations/reference point in the VR training Learning in context – that context has to be recognizable to the real-life work they will be doing Got feedback through the project around things that would “pull” users out of the experience – things that would break immersion that needed to be addressed This is why reference is so important!
- Deploy: User testing for feedback before final build Ensure client-side technology is ready for deployment Continuous quality control and iteration even after deployment
- Final delivery!
- Why is VR so different and challenging to develop for?
- Methods of interaction for VR are totally different from other mediums/platforms And a lot of VR controllers at this point are different from each other! Not a lot of clear parallels between other devices and VR controllers in terms of how you use them as input methods (besides the wii?) – really foreign for first-time users who have ZERO gaming experience Designing for “intuitive” interactions aren’t easy – ex: turning valves in Linde was a real challenge Reducing “friction” in user interactions so that they are intuitive means a lot of WORK for designers – not always intuitive from the get go!
- Traditional design standards often don’t apply, or are more complex in VR: Text/UI treatment has additional considerations – readability and distance, not always being in the player’s POV “Heads-up display” or overlay user interface elements present in other apps and traditional video games don’t work in VR! (reticles, health, etc) Often what feels most good in VR are interactions that mirror real-life physical actions such as buttons, levels, turning handles – not gestures such as tapping or swiping Human-computer interaction problems that just don’t exist on other platforms: fatigue of holding your arms up, etc!
- Developing with new technology can be challenging Software updates mid-project – keeping up with changes in functionality on the toolsets as well as hardware Compatibility of different toolsets can be difficult Limitations of the hardware and software itself (literal cables breaking!)
- Long-term viability of the hardware: Will this company still exist and make this product in 2 years? In 5? How long does the hardware itself last? How to we plan to support/service the technology long-term, or enable our clients to do so?
- Not a lot of established industry best practices (especially when we first started building for VR) Many problems still haven’t been solved – which is exciting and also difficult at the same time
- A lot of people want it, but don’t necessarily have a good use case – and the hardware choice shouldn’t be driving the solution, the solution should be dictating the hardware choice!
- Audience familiarity vs reluctance Attractive to a new generation of the workforce who grew up with video games and technology Reluctance in older generations and folks who have never tried it before, or who have had bad experiences around motion sickness and VR in the past
- As developers, we are our own worst QA! If your audience has never used VR before, need to test with folks who have never used VR before ….which means they cannot be used to test for “first user” intuitiveness more than once! The more you are in VR, the less you can objectively test it – especially when you know what the design and goals are of the experience/intention behind the design choices
- Wrap up – the technology of VR and new medium that it creates makes for compelling learning experiences and other real-world applications outside of games for entertainment