Mixed Reality Interfaces and Product Management
•Download as PPTX, PDF•
1 like•1,007 views
Slides Vikas Batra recently used in his discussion w/ mentees of The Product Mentor. Synopsis: In this talk Vikas will share recent developments in the field of Virtual Reality(VR) and Augmented Reality (AR) . Share use-cases on how AR is being used by enterprises to help you identify how you could use it to gain competitive advantage in your market. The Product Mentor is a program designed to pair Product Mentors and Mentees from around the World, across all industries, from start-up to enterprise, guided by the fundamental goals…Better Decisions. Better Products. Better Product People. Throughout the program, each mentor leads a conversation in an area of their expertise that is live streamed and available to both mentee and the broader product community. http://TheProductMentor.com
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Mixed Reality Interfaces and Product Management
Similar to Mixed Reality Interfaces and Product Management (20)
More from Jeremy Horn
More from Jeremy Horn (20)
Recently uploaded
Recently uploaded (20)
Mixed Reality Interfaces and Product Management
- 1. Mixed Reality and Product Management Vikas Batra Feb 2017
- 2. Product Management Role Define Product... Viable Valuable UsableFeasible 2
- 3. The Objective Augmented Reality and Virtual Reality Examine Competitive Advantage ? Address New Market/Customer Segment ? Process of Assessment 3
- 4. Spatial User Interface New Compute Interface Redefines human interaction with Physical Objects and Physical Space 1. 3-Dimensional, Computer-generated, Graphical Spatial View 1. augments actual physical view (Augmented Reality), or 2. eliminates actual physical view (Virtual Reality) 2. Hands Free. Controlled by hands gestures, head/body movements 4
- 5. Spatial User Interface Source Microsoft 5
- 6. Compute Interface Evolution Command Line Interface Graphical User Interface Touch Screen Voice Interface Spatial User Interface 2D Screen – Text 2D Screen – Graphical 2D Screen – Graphical Speakers 3D Spatial Projection Keyboard Keyboard Mouce Touch Screen Mic Camera. InfraRed Sensors Key press Mouse move, clicks Swipe, Pinch Speak Hand gestures Head/Body movement MS–DOS Windows, Mac Smartphones, Tablets, etc. Echo, Siri, Google Assistant Head Mounted Display, Smart Glasses 6
- 8. Virtual Reality - Phone Based8
- 9. Virtual Reality – Head Mounted Display9
- 10. Virtual Reality – Gaming Console Source: Sony Sony PlayStation VR 10
- 11. Virtual Reality – PC Based Source: Facebook/Oculus/Endgadet Facebook Oculus Development Platform Facebook Oculus Rift and Touch 11
- 12. Creating Virtual Reality Content Capture Store Edit Transmit Playback Nokia Ozo 360 Camera Source: Nokia 12
- 14. On TV14
- 15. On Mobile Phone Lets see it in action 15
- 16. On Mobile Phone - Tango Depth Sensing Motion Tracking Google Tango Application Development Platform Lenovo Pahb2 Pro 16
- 17. On Mobile Phone – Tango See Before you Buy Wayfair View App on Tango 17
- 18. Visualization in 3D Space Visualize Wi-Fi Signal Coverage Signal Mapper App on Tango By Signal Garden 18
- 19. On Smart Glasses Microsoft HoloLens ODG R-8 Smart Glasses 19
- 20. Optics Holographic WaveguideCurved CombinerFlat Combiner Display Lens Computer Graphics Actual Physical View 20
- 21. Smart Glasses 40+ Smart Glasses AugmentedReality.org ww.arglassesbuyersguide.com Source: AugmentedReality.org 21
- 22. Software Recognize and Track Environment Define AR Experiences Execute AR Experiences Integration with Existing Systems 50+ AR Authoring Platforms/SDKs Comparison at Social Compare http://socialcompare.com/en/comparison/augmented-reality-sdks 22
- 24. The Big Picture24 Internet of Things Artificial Intelligence/ Analytics Humans Data Automated Control Data Manual Control ‘Pull’ ‘ Contextually Relevant Push’
- 25. The Promise Augmented Reality 3-Dimensional, Augmented Views (Audio-Video) Hands Free Operation - Input/output from hand gestures, voice, camera, sensors … Larger, adjustable display area Greater Comfort, Always with you. ‘Contextually Relevant Push’ Virtual Reality Immersive Experiences Significantly better end-user experience compared to Smartphone/PC/Tablets 25
- 26. The Challenges Technical Optics Weight Battery Life Bandwidth/Delay Business Price Content Application Platforms and Eco-system 26
- 27. Trends ~4B VC funding since 2014 Big Players Miniaturization Optics Lower Prices 5G Wireless Networks WiGig/802.11ad 27
- 29. Use Cases Areas Physical Objects Physical Space Immersive Experiences 29
- 30. Use Cases – Physical Objects Design Assemble Warehouse/Logistics Operate/Use Maintain Support. I See What You See Survey Education/Training Execute a Procedure 30
- 31. Use Cases – Physical Spaces Architecture/Interior Design Navigation Venue Experiences e.g. Museums See Before You Buy 3-D Visualization in Space Language Translation … 31
- 32. Uses Cases – Immersive Experiences Gaming Social Virtual Experience Movies Theme Parks Live Events – Every Viewer on Prime Seat Virtual Tours Before to travel Real Estate … Holoportation 32
- 33. What will you define? 33
- 34. The Process34
- 35. Assessment Process Define Concept Identify and Capture Value •End Customers, Organization, Partners Usability Viability • Customer ROI/Time • Impact to organization (e.g. CBA, Users, Usage Time) • Partners (CBA) Risks Assess Concept Refine Value, Usability, Viability, and Risks Prototype •Objective •Deliverable •Cost/Effort •Time/Schedule Develop Prototype Refine Value, Usability, Viability and Risks Product Development •Scope •Cost/Effort •Time/Schedule Go To Market, Support etc. Perform Assessment? Develop Prototype? Develop Product? 35
- 36. Questions ?36
- 37. Back Up37
- 38. Example Use Cases 3D Virtual Design Design prior to development Visualize prior to purchase Execute Procedure. Hands-free, 3D AR, Audio assisted Physical Object: Install, Operate, Maintain Example ; Boeing Example ; Nokia Small Cell Surgical/Medical Procedure Pick and Pack Surveys, Navigation Always On Self/Environment Monitoring, Compliance Non intrusive Alerting, Audio output Contextual Situation Awareness Push Contextually Relevant Information IOT Enabled Machine Visibility and Control Multi party conference I see what you see Remote Expert, Customer Support, etc. 3D Non-real time Video for Training Example Contextual Marketing Larger Viewing Area, Non intrusive Contextually relevant Holo-portation. Real Time Communication 3D Virtual Travel 3D Immersive Entertainment …. 38
- 39. See before you buy Wayfair View App on Tango 39
- 40. Design SketchUp Viewer Trimble Inc. 40 Meta 2 Glasses Meta
- 41. Execute a Procedure AR Car Repair Solution Continental Corporation 41 Hands Free Instructions in the Field Epson/Atheer
- 44. Education HoloAnatomy by Case Western Reserve University HoloHeart by Made In Point GmbH 44
- 45. Navigation Aircraft Navigation Aero Glass Indoor Navigation to the shelf/aisle Aisle 411 - 45
- 49. Virtual Keyboard/Key pad Augumenta Interaction Platform HoloLens 3D3 Keyboard 49
- 50. Live Events50 Live Events – US Open Golf NextVR
- 52. Technology Components Optics (Waveguide, Curved Surface Combiner) Gesture Recognition and Control Voice Recognition and Control Depth Sensing Inertial Measurement Unit (IMU) Context Aware – Computing 3D Video/Audio, Capture, Editing, and Transmission Augmented Reality Authoring Software Platform 52
- 53. Enterprise AR Value Chain Nokia Internal Use. Confidential. Components • Qualcomm • TI • Intel • Nvidia • AMD •… • Samsung • Lumus • Rochester Optical •…. Smart Glasses • ODG • Intel/Recon • Epson • Vuzix • Daqri (Helmet) • Google • Microsoft • Meta • Lenovo • MagicLeap Connectivity • Nokia, Ericsson, Huawei • Cisco, Brocade/Ruckus, HPE/Aruba.. Applications Development • APX Labs/UpSkill • PTC • Atheer • Wikitude • Augumenta • Daqri, • Pristine • UBIMax • Incontext Solutions • Surgical Theater • … System Integration • IBM • Accenture • HP •… • Nokia Services ? Operations & Management • AirWatch by vmware • Augmate •Nokia Impact (MDM)?
- 54. End User Experience At Home Source Microsoft 54
- 55. End User Experience Source: Microsoft HoloLens 55
- 56. End User Experience Source: Microsoft HoloLens Everywhere Else Source ODG, 56
- 57. Questions to Ask Can it enable meeting existing customer needs better ? Can it enable meeting an unmet customer need? Are there new customer segments that can now become attractive? Can it improve monetization ? E.g. moving subscriber from free to premium offering? Can it enable higher Productivity? Can it enable higher Quality? Can it enable greater Safety? Can it help with Compliance requirements? Directly from VR HMD/AR Smart Glasses and Technology building blocks 57
- 58. Resources www.arglassesbuyersguide.com/ www.augmentedworldexpo.com/ Augmented Reality for Enterprise Alliance (AREA) www.thearea.org/ Karl Guttag on Technology www.kguttag.com Depth Sensing Technologies Comparison http://www.ti.com/lsds/ti/sensing-products/optical- sensors/3d-time-of-flight-sensors-technology- comparison.page 58
Editor's Notes
- We need to define something that is Valuable, Usable, Feasible and Viable. Valuable: Deliver Value, benefit or utility to the customer. Usable: The end-consumer is able to get that value easily and emotionally satisfying way. Feasible: Feasible given all our constraints around resources, funding, internal capabilities, external capabilities and so on... Feasible not just for our organization but also for all partners, who are involved in delivering that value. All three combined should be viable. Viable for the customers, our organization, and partners e.g. channel partners. This combination keep on changing. As new Technologies make feasible what was earlier not feasible, they Improve usability. As certain needs are met certain new one emerge making products more or less valuable. The viability changes are our capabilities improves or prices drop. As Product managers our challenge is to stay on top of them.
- The objective on the talk is share developments, underlying technology components and some use cases in the space of AR/VR. What this is making feasible and what impact it has on usability. For each us as Product Managers, to examine whether this can add or enhance our current offerings. Can it enables us to creates a competitive advantage in current market or enable us to enter new markets/customer segment by meeting unmet customer needs? This is Product Mentor talk. Talk briefly about the Process of making that assessment. It that applies to any new technology. So what is Augmented Reality/VR
- A big component of this is a New Compute Interface. That redefines… I call this Spatial User Interface In your current actual physical view, that your eyes getting, when you are watching this on your PC/Mac or mobile. The PC/mobile is a 2D screen in your total physical view. The 2nd key trait is Hands free. And it looks something like this
- This is how the interface looks like. To understand it a little better….. Lets look at the Compute Interface Evolution
- Lets look at the Compute Interface Evolution Note that A single device can and usually does support multiple interfaces. Augmented Reality is not just Spatial User Interface but it also includes applying voice interface, A 3D Spatial voice and GUI in space. Lets understand the components behind this and how it is coming together. I will start with simple case and build it up. Lets start with Virtual Reality.
- Lets start with the simpler virtual reality.
- To get to 3D we need to two images one for each eye. Stereoscopic 3D You need for two separate views one for each eye. Lets see it in action. You tube 360. glasses icon. Show pan and tilt, movements. Head Tracking –– Three sensors gyroscope, accelerometer and compass/magnetometer Notice the forward or backward motion is not tracked in this case. You need bring it close to your eyes to eliminate the actual view. And so you need lenses to zoom it. Stop Sharing.. To take to the next level by Show the lens/GearVR The Display moves with your head… you get 360 degree view To get a good experience you need high resolution and you need to refresh the frames. A minimum frame rate of around 60 frames per second is needed to users feeling sick. The current crop of VR headsets go way beyond this - Oculus is capable of 90fps, for instance, Sony's PlayStation VR manages 120fps. Computation intensive for CPU and GPU => Lot of power and heat, battery life.. Start Sharing..
- Describe in own words.. Switch to camera
- One way to solve this is to connect to a more powerful processor. A PC or gaming console. Add controller. More immersive experience. The variation are smoother. The Resolution is higher, the Frame refresh rate is high (Sony Play Station can do 120 fps) 9 LED lights to track head movement Challenge: Tethered to Cable: Cost. Sony Recent brought the price down to $400.
- Oculus Touch is a set of wireless controllers, designed to make you feel like you're using your own hands in VR. There is also a matrix of sensors on each controller to detect gestures such as pointing and waving. Eye tracking It's not available An infrared sensor monitor's your eyes inside the headset so FOVE knows where your eyes are looking in virtual reality. The main advantage of this is to make depth of field more realistic. And address the bandwidth challenge. Position Tracking: Oculus Development Platform : Develop not just games, but also Apps, Social VR Experiences wherein individuals can create their Virtual Avatars and interact with others in Virtual Room. They can together talk, share a whiteboard, view a video or play 3D games all in the virtual room.
- Content A host of players are developing solutions and innovations in this space. Large studio like Walt Disney to mom and Pop videography that record events like weddings are looking at creating content in 3D.
- Now lets look at the more complex augmented reality.
- It has been around from some time. We have seen it all. Describe in own words. The idea is to augment the actual view with computer generated graphics. For those of you outside of US, not familiar with the game … the orange and blue lines This is simple case where there is one single POV where the camera is.
- Pokemon Go What is really happening is Anchoring/tracking to an object? A dollar bill in this case. Take the feed of the camera and super-impose the image on an anchor. This is called the Marker based tracking. Recognizing an image the app knows before hand. Here notice that motion tracking is achieved due to the marker. There is another approach Called Markerless Tracking but we not quite there which is I did on my Samsumg Note. Google is taking this forward with Tango.
- Depth Sensing computer vision to enable mobile devices to detect their position relative to the world around them without using GPS or other external signals. by illuminating an area with modulated IR light. By measuring the phase change of the reflected signal the distance can be accurately determined for every pixel in the sensor creating a 3D depth map It store this map. Lenovo Phab 2 Pro and the Asus Zenfone AR. Priced : $500 not more expensive An example of this.. Range is few meters upto 10-15 meters. And usually in indoor scenario. Time of Flight ToF Depth Sensor 5Hz (time of flight camera). 3D time of flight operates by illuminating an area with modulated IR light. By measuring the phase change of the reflected signal the distance can be accurately determined for every pixel in the sensor creating a 3D depth map of the subject or scene.
- It gives motion tracking and remembers it.
- It is in 3D if you move but it still on 2D Screen in your view. We need to a get to where the entire physical view is augmented.
- Enter Smart glasses. Form factor Now you have two images, one for each eye to get 3D. Head tracking to see the computer generated graphics from different POVs. They have full on board compute resources. Diplay sensors, battery, network connectivity. There is no cable. They also include Sensors 6 DOF tracking. All other Sensors?? ODG R-9 enterprise $1800, R-8 $1000. Picture is R-8 ODG did not have gesture control but a track pad on the glasses, unlike the Hololens.
- How is it coming together. It is coming together by Optics Unlike in the phone where the camera stream is augmented here the actucal view is augmented. MagicLeap. basic tradeoffs between resolution, field of view, eye box, image quality, hardware weight/fit, aesthetic form factor, FOV human eyes: 200x100 degree FOV Flat: Google Glass gets 13 degree FOV, and Epson BT-300 gets 23 degree FOV with 1280x720 resolution. Pros: Light, small, relatively affordable ($500–700ish). waveguide hologram to progressively extract a collimated image guided by total internal reflection (TIR) in a waveguide pipe. Pros: Potentially better FOV and resolution on medium sized devices. Cons: Expensive (estimated $3k+), technology still actively in development, maybe black magic. Cons: Very limited FOV and display, difficult to improve. Curved: Meta: “almost 90 degree FOV” and a 2560x1440 pixel display split between both eyes. Pros: High FOV and resolution, relatively affordable ($900ish). Cons: Bulky, low angular resolution, material quality risk
- Even without this there are at There are at least 40 different models of models across about 10 vendors? These are mainstream at least not yet… for various reasons.
- 25+ AR Platforms with varying levels of functionality. For example, in our example of the stool, we need to look up the details and order it, Software Challenges: Inconsistent buttons What does pinch, swipe mean? Variations across Scaling and managing smart glasses. Most play in android. Application authoring tools, Tools Content : CAD designs, Object Recognition. So After doing all this….
- We are not quite there yet but that is where we moving towards. Lets look at where it sits in the big picture…
- Consumer vs. Enterprise play.
- Here are some of he challenges. For taking them mainstream and for wide spread adoption. If you notice most of these challenges while critical for consumer adoption are not so crucial for Business adoption. AR is getting traction on enterprises. I believe that AR will continue to get adopted by Enterprises. AR adopted will be driven by enterprises. While VR is likely to be driven largely by Gaming and Events.
- However there are a lot of smart people with lot of money are working to overcome these challenges. They are in fact competing to solve this before others. E.g. ~ 4B venture funding has gone into the space over the last 3 years. + Investments from large players 100s of patents are filled every year over the last three years. Google Tango/Day Dream, Facebook: Oculus Microsoft : Speculate has invested over 1B on Hololens with Wave guide Optics. Apple: Rumored to launch AR in 2017. Amazon: Filed application to FCC to test new devices. On the Price do not let the price hold you back. Can you get glasses that meet your specific needs at the required price. What can you get on existing phones? Can you offer it by providing the glasses yourself? One can argue that why now we can wait till it matures. Sure. The benefit of defining it now. Is to assess what incremental value it can add. How exactly you can advantage of the usability improvements. And even it turns out it is not feasible or viable. You have a view of what needs to change to make it feasible/viable. And then you can track and as soon as it meet. You can get into the market get the first move advantage. And that is one possible way to get Competitive advantage.
- Execute a procedure.. There are uses cases in the entire physical object life cycle. Not just machines, but could also be organs e.g. doing a surgery.
- Navigation While driving – BMW use the windshield of the car as the combiner. It makes some of the challenges in smart glasses form factor like weight, battery life .. irrelevant. It is not just normal driving …. Think of assistance it can provide during poor visibility conditions like fog. Depth sensing to assist a Fireman to guide in smoke. Example of aircraft assistance. Shop 411.
- The list is by no mean exhaustive. I still keep on learning and thinking about new use cases…. Hopefully, this triggered some creative juices for you to define your own uses for your own organizations.
- Valuable Not just to customers but also to your organization, your partners e.g. channel partners. Usable by end consumers Feasible: Not just technical but also business. Feasible given your constraints around resources, funding, capabilities your and if you need partners then those of your partners. They keep on changing as new Technologies make feasible what was earlier not feasible. Improve usability. As certain needs are met certain new one emerge or become more or less valuable. To share developments in the space of Mixed Reality AR/VR. What is this making feasible and what impact it has on usability. For us as Product Managers. to determine if this add/enhance our offerings; enables us to creates a competitive advantage in product market mix or enable us to enter new markets/customer segment. Enables us to meet unmet customer needs?
- IMU A combination of Accelerometer, gyroscope and compass. Provide 6 degree of motion to track the head movement, position The gyroscopes measures 360 degree rotation, accelerometers measure movement along the XYZ axes and magnetometers can determine orientation towards a magnetic field.
- More Pictures ?
- More Pictures ?
- More Pictures ?
- See if we can add more questions. Marketing use cases? If multiple use cases emerge then you have to prioritize. Document the use case. Look for the right Smart Glasses Look for the right AR platform. Is this something that is mature now or you need to wait and monitor? Is there anything you can do to be prepared. Back end systems? Can ipad