Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Forge - DevCon 2016: Introduction to building for HoloLens


Published on

Dona Sarkar, Microsoft

Microsoft HoloLens is the world's first untethered holographic computer, mixing holograms into your real world. Developing holographic apps uses the Universal Windows Platform. All holographic apps are Universal Windows apps, and all Universal Windows apps can be made to run on Microsoft HoloLens. With Windows 10 and familiarity with middleware tools like Unity, you can start building holographic experiences today. You will learn about the six fundamental building blocks for mixed reality holographic apps:
• World coordinates
• Gaze input
• Gesture input
• Voice input
• Spatial sound
• Spatial mapping

Published in: Technology
  • Be the first to comment

Forge - DevCon 2016: Introduction to building for HoloLens

  1. 1. You Had Me at HoloWorld Dona Sarkar Principal Product Person
  2. 2. 1994-2004 tech
  3. 3. 2005-2015 tech
  4. 4. 2016-???
  5. 5. the future is screenless
  6. 6. we are analog creatures
  7. 7. Human Environment Machine Tech
  8. 8. HoloLens: Windows 10 computer
  9. 9. design. explore. communicate.
  10. 10. HoloLens blends interactive holographic content and your physical environment.
  11. 11. “What are holograms good for?”
  12. 12. experiences and information embedded in your environment
  13. 13. Transportive Storytelling transportive storytelling
  14. 14. exploration and collaboration
  15. 15. Making and seeing 3D in 3D
  16. 16. Complement existing devices and workflows
  17. 17. “What are holograms not good for?”
  18. 18. Challenges in holographic design INDOORFOCUS BATTERYLIFE NEARCLIPPLANE ARMFATIGUE 
  19. 19. Universal Windows Platform Tools Ecosystem Windows Store Windows Holographic Unity DirectX Visual Studio Emulator
  20. 20. Build with Windows Holographic Voice Commands Spatial Mapping Gesture Input Spatial Sound World Coordinates Gaze Input
  21. 21. • used for targeting and intent • uses position and orientation of your user's head, not their eyes, to determine their gaze vector • laser pointer from between the eyes • as the user looks around app intersects this ray, both with its own holograms and with the spatial mapping mesh to determine what virtual or real-world object your user may be looking at. • implement a cursor in HoloLens that represents the user's gazeGaze Input
  22. 22. • allows interaction with hologram with hands or clicker • gestures recognized by tracking position of either or both hands that are visible to the device • hands: ready state + pressed state • clicker • within the gesture frame • 2 main gestures: • Air-tap – activate a hologram • Bloom – return to Shell • app can recognize more by combining presses and releases with movement of your hand or using a clicker Gesture Input
  23. 23. • users can gaze at a hologram and speak your command • voice input is powered by the same engine that supports speech in all other Universal Windows Apps. • even without specifically adding voice support to your app, your users can activate your holograms simply by saying "select“ • “Hey Cortana” will enable you to do a multitude of things: check weather, check battery, take a picture, etc. • Manipulation commands: • Face me • Bigger | Enhance • Smaller Voice Commands
  24. 24. • holograms appear visually where the user's gaze is pointing but sound can come from all directions; above, below, behind, to the side, etc. • use this feature to draw attention to an object that might not currently be in the user's view • user can perceive sounds to be emanating from a source in the mixed-reality world. • Walk toward it and away • Behind the user • sounds should come from realistic places. Eg: bird chirp from the sky not the groundSpatial Sound
  25. 25. • detailed representation of real-world surfaces in the environment • apps align with user expectations by providing familiar real-world behaviors and interactions. • Placement • Occlusion • Physics Spatial Mapping
  26. 26. - xyz coordinate system - relative to the real world: 1 meter in the real world is 1 meter in your app - used to stabilize your hologram in a realistic way World Coordinates
  27. 27. Did you know… - You can port any Windows 10 PC or Mobile app to HoloLens as a 2D app? - You can see & record what the HoloLens is seeing by using the Windows Device Portal on a PC? Great way to do demos for a large group. - You can take a screenshot or a video - “Cortana, take a picture” - “Cortana, start recording” …and immediately share on social - The sharing experience (multiple people seeing the same hologram) can be used out of the box from the HoloToolkit - People feel like holograms are real - If you place a holographic elephant in the room, people will walk around it, not through it - If you are in a shared experience and others are shooting holographic projectiles at you, you’ll duck
  28. 28. Gaze Input Gesture Input Voice Input Spatial Anchors Spatial Mapping Spatial Sound Cursor prefabs Sharing prefab Spatial prefabs Utilities Shaders Tests Holograms Materials Models Scripts Textures Shaders App Experience – you build HoloToolkit – you get open source for this Windows Holographic – we made this
  29. 29. • Documentation • Forums • Academy • Open Source
  30. 30. Dona Sarkar