Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Similar to Tony Parisi (Unity): Face the Future - Computing in an Augmented World
Similar to Tony Parisi (Unity): Face the Future - Computing in an Augmented World (20)
More from AugmentedWorldExpo
More from AugmentedWorldExpo (20)
Recently uploaded
Recently uploaded (20)
Tony Parisi (Unity): Face the Future - Computing in an Augmented World
- 2. FACE THE FUTURE: COMPUTING IN AN AUGMENTED WORLD Tony Parisi Global Head of VR/AR, Unity 3D Technologies @auradeluxe #AWE2017
- 3. Tony Parisi Head of VR/AR Strategy Unity Technologies Face the Future: Computing in an Augmented World tonyp@unity3d.com @auradeluxe
- 5. in·ter·face ˈin(t)ərˌfās/ noun 1. a point where two systems, subjects, organizations, etc., meet and interact. "the interface between accountancy and the law" 2. COMPUTING a device or program enabling a user to communicate with a computer. verb 1. interact with (another system, person, organization, etc.). "his goal is to get people interfacing with each other" 2. COMPUTING connect with (another computer or piece of equipment) by an interface.
- 8. Overthe past several decades, every time people madecomputers work more like we do – everytime weremoveda layer of abstraction between us and them –computers became more broadly accessible, useful, and valuable to us. -- ClayBavor, VP of Virtual andAugmentedRealityat Google
- 12. 38% of the top 1000 Free mobile games 5B mobile app installs Q1 2017 2B unique mobile devices 58% of VR/AR developers USE UNITY >2/3 of ALL VR/AR Content RELEASED
- 13. 82% of Vuforia-based AR apps 91% of hololens development
- 19. Software is Eatingthe World - MarcAndreessen, 2011
- 24. Thedigital world will soon beenmeshed enough with the physical world that our‘reality’ will bethe transparency modethat we choose. - Monika Bielskyte, Creative Strategist, Future Prototyper
- 28. Tony Parisi Head of VR/AR Strategy Unity Technologies Face the Future: Computing in an Augmented World tonyp@unity3d.com @auradeluxe
Editor's Notes
- Good morning. I’m Tony Parisi, Global Head of VR/AR Strategy at Unity Technologies. I’m excited to talk to you about Unity as the underlying layer of all immersive computing - which is the future of computing. IT’S ALL GOING 3D.
- We live in a 3D world. People move, think and experience in three dimensions. But for decades, human-computer interface has been… less than that. As computing continues to get woven deeper into our daily lives, in many ways, interface hasn’t caught up. We still operate our computing devices-- or perhaps more accurately, they operate us-- at some remove from how we work within the real world.
- From punch cards to command lines to GUIs to touch screens, the computer industry has seen a steady progression in usability; but there remain levels of abstraction compared to how we interact with our surroundings and each other on the physical plane.
- Today, we peer into an endless sea of information via tiny portholes… flip through pages and tabs linearly while trying maintain a mental map of things and their connections of much higher than one dimension … and remember and share moments from the world *around* us by capturing only the photons in *front* of us.
- Tomorrow, information will be displayed where and when we need it, its essence and connections laid bare to see... and whole experiences will be captured and shared. We are about to escape the tyranny of the rectangle… and information itself will achieve six degrees of freedom, in a virtual sensorium that maps the web onto the real world and brings the real world into the web.
- With the advent of immersive technology we are approaching the next step change in human-computer interaction. As HCI goes 3D, some truly staggering things are going to happen. Doubtless this transformation to an immersive computing landscape is going to make digital technology, in Clay Bavor’s words, “more broadly accessible, useful, and valuable to us.” We’re here today because we all know that immersive computing represents the next platform. But most of us can’t really imagine the ways it is going to disrupt our daily lives, any more than back in the day we could have imagined the Internet disrupting brick and mortar retail, or smart phones upending the transportation and hospitality industries. We’ll explore some of the possibilities momentarily. But first, let’s ask ourselves why now? Real time 3D is nothing new, and we’ve been tinkering with virtual and augmented reality hardware for decades. Why do we think now is the time to collectively invest so much energy as an industry?
- 3D graphics is nearly as old as the computer itself, tracing its roots back to the 1960s. It has been used in applications spanning engineering, education, training, architecture, finance, sales and marketing, as well as gaming and entertainment. Even the first immersive VR systems were demonstrated over fifty years ago. Historically, 3D applications relied on high-end computer systems and expensive software. But that has changed in the last decade. 3D processing hardware is now shipped in every computer and mobile device, with the consumer smartphone of today possessing more graphics power than the professional workstation of a decade ago.
- Look around: a lot of our media is already 3D—though until recently it’s been presented on flat screens. Animated films are created from computer-generated 3D images. Video games, whether running on dedicated consoles or mobile phones, are typically rendered in 3D. And even the news has gone 3D: the sight of a CNN analyst meandering through a virtual set, comically awkward a few years ago, has become an accepted part of the broadcast milieu as cable channels vie for increasing attention in a 24-hour news cycle. But until very recently, 3D was still lurking at the margins. Other than professional design software, the only widespread use of 3D has been in video games.
- In my opinion, that’s because historically, 3D has been a luxury for most applications. The benefits of real time interfaces on flat screens were simply never worth it, other than for niches in design, engineering and filmmaking. Gaming is the one exception-- the only segment of the computer industry that has been able to solve simultaneously for the high cost of producing killer content, and monetizing it at scale. Well, all of this changes with XR. Inexpensive stereo displays, holographic optics, binaural audio, 360 cameras, volumetric capture, and computer vision, combined with the previous innovations in mobility, low-latency networking and location services, are enabling applications even the wildest of us 3D dreamers couldn’t have even dreamt about a decade before. And with immersive hardware, we have no choice but to render in 3D. The entire graphics pipeline is set up for it. We have a 360 degree space in which to work. Even if we only display textured rectangles in that space, we need to lay them out using 3D coordinates. But of course we can do so much better that that: we can create virtual spaces and display virtual objects. We can organize information the way we want, using spatial memory and spatial reasoning to get the most out of it. And all this content can be blended seamlessly with the real world around us. Beyond cheap hardware, we also now have a generation of content creators savvy in 3D development– thanks to the video games, design and VFX industries– and most importantly, an interactive generation who expects-- nay demands-- this. Today’s young kids are tomorrow’s XR natives. With XR, real time 3D is not a sideshow; it the main attraction. With XR, there’s nowhere to go but in.
- While much of the focus today is on the innovations on the hardware side, software, content and services are where most of the value will be created over time. I’m sure most of you are familiar with my company, Unity. We occupy a unique position in this new industry. For those that aren’t fully aware of us: Unity is the creator of a flexible and high-performance end-to-end development platform used to create rich interactive 2D, 3D, VR and AR experiences. Our graphics engine and full-featured editor enable the development of beautiful games or apps and easily bring them to multiple platforms: mobile devices, home entertainment systems, personal computers, and embedded systems. We also offer additional solutions and services including Unity Ads, Unity Analytics, Unity Asset Store, Unity Cloud Build, Unity Collaborate, Unity Connect and Unity Certification, all focused on helping our to developers succeed. By the numbers, here are some amazing stats: CLICK 38% of the top 1,000 free mobile games are Made With Unity; CLICK Unity games triggered 5B app installs last quarter alone, CLICK across 2B unique mobile devices. When it comes to VR and AR, CLICK around 58% of developers surveyed are Unity users, and CLICK over two thirds of all VR/AR content released through known channels is Made With Unity. I’m sure you’re familiar with many of the VR games like Job Simulator and popular VR experiences like TiltBrush and new Cinematic fare like Baobab’s ASTEROIDS. And Facebook Spaces, the company’s new VR social application, recently shown at F8, and Unity’s Vision Summit.
- Add to this that CLICK over 80% of Vuforia AR applications, and CLICK 91% of all Hololens mixed reality is made with our software… you can see that Unity has become the Foundation of VR/AR development. So, how did we get here? Over the last couple of years, we invested heavily in supporting VR and AR hardware platforms, forging deep partnerships with the major HMD providers. But we couldn’t have even gotten into that position without first having built a solid foundation for our business. Unity was founded one three core principles, to which we remain committed to this day: Democratize development - everyone can bring their creations to life, including a new wave of creators in AR and VR – many of whom do not have game development experience Solving hard problems – supporting almost every platform on the planet, and continuing to invest in the best graphics and performance Make developers successful – making a profit is hard… but we’re proud to be able to say that in 2016, we paid out more to developers than we've made ourselves
- Understanding Unity’s position in the ecosystem, now let’s look at some of the great XR being made with Unity, in particular augmented and mixed reality that is pushing the envelope on user interface. For brands: Rewind Studios created Flight Deck, an application that will allowed to combine HoloLens with 2D screens to create a whole new type of viewing experience for the Red Bull Air Race. In the HoloLens, the user can actually put the race course on a coffee table or likely even the floor — for a real bird's eye view — and watch in real time as the pilots make their maneuvers. This piece envisions both brand experience and live sports in a whole new way.
- For work: ScopeAR “puts KNOWLEDGE WHERE YOU NEED IT”... they created Remote AR, a telepresence collaboration tool that allows remote field technicians to connect to experts via live video, audio, and a powerful annotation toolset that is augmented to ‘lock’ onto the real world with augmented reality. Scope used Unity to create and manage 3D objects, while being cross platform and supporting many different variations of devices, from phones and tablets to AR glasses to desktops. And taking advantage of Unity's robust 3D rendering and plugin architecture they built an entire real time audio-video stack along with the capability to remotely synchronize 3D data (meshes from Tango or Hololens for example, or 3D drawings using line renderers, or complicated CAD models).
- Nexus Studios took outdoor adventure to a new level, by creating an AR experience for The Gruffalo, the best-selling picture book. Families follow clues on an interactive trail in the English forest, tracking signs of their favourite characters. These clues lead to different Augmented Reality markers – footprint signposts,specific to each one of the characters. When the visitor aims their device at the footprint marker, a short animation of that character is triggered, blending in with its natural surroundings. The Gruffalo experience shows how we can use AR to turn the world into a canvas for art and storytelling.
- SwapBots are collectable & customisable toys that are brought to life by a smartphone or tablet using augmented reality. There are hundreds of combinations of the physical toy, as you swap parts, and endless single- and multi-user game play possibilities. Mekamon are multi-functional, connected battlebots with augmented reality capabilities. Control them with your smartphone and see power ups and explosions using AR. Fight hostile aliens in AR, play arcade games or engage in warfare with other bots.
- Finally, I think we’re all excited about the possibilities for education. MyLab provides augmented reality educational content for Hololens… And MergeVR’s Merge Cube is a physical cube that, when viewed through a standard smartphone and the MergeVR headset, displays animated, interactive education content. Wild.
- Even if with all this great stuff going on, you’re still skeptical, well I’m here to say it’s time to suspend that disbelief. It’s fair to think of the last few years of XR as one big, expensive, global proof of concept. We’re still experimenting with hardware and tools, formats and form factors. We’re still learning the right way to do a lot of things, and the best applications of immersive computing. In fact, we’re just getting started. But the pace of development doesn’t seem to be letting up. Massive, ongoing investments are being made in this space - PTC/Vuforia, FB, Snap, Google, MS, and startups like Meta, Magic Leap, Improbable. (Work in “Camera as platform” meme… bringing the power of AR to billions of smartphone owners today.) So this is happening. And more innovation is right around the corner. A few years back, Marc Andreessen famously said that software is eating the world. At the time, he didn't intend it to mean the actual world-- he meant information, communications, business processes. But the actual world-- well, that comes next.
- Google Tango is about to hit the mainstream. Asus and Lenovo are now showing their new Tango-powered phones, with built-in fisheye camera, depth sensor and infrared sensor. Combine this with powerful software from Vuforia and Unity, and we can blend the real and virtual worlds seamlessly.
- And mapping the interior - what Google Maps and StreetView did for the outside SLAM (Simultaneous Localization and Mapping) does for the inside. F8 Facebook (SLAM, object recognition, 3D effects, style transfer and so on) Google calls it WorldSense Vuforia has Smart Terrain Get explicit directions within a store or other property. And not just physical objects, but heat mapping your wifi coverage, airflow within a room, and more...
- But not just mapping the world - ingesting it. In just a few seconds, this Tango-enabled phone scanned a room interior in crude form. With several more minutes’ effort, you can get a high quality scan, bring it into Unity and publish a Daydream app based on real physical environments. All from a consumer-grade smartphone. With the help of some nifty new smartphone hardware, software is now poised to-- literally-- eat the world.
- Of course it’s not only about getting the right tools in the hands of both the consumers and pros; it’s also about infrastructure: a global information system that allows anyone to publish and share content and applications. Also known as the World Wide Web. At Google I/O, the company demonstrated a development version of Chrome that extends WebVR-- the API that connects web browsers to VR hardware-- into augmented reality. WebAR brings the phone’s camera and Tango-style sensing capabilities into the browser, which means that for many use cases, you would no longer need to download a specialized app to recognize markers, scan the environment and overlay information: the world becomes your QR code. And find cool virtual stuff nearby, because someone shared a link in their social feed.
- This is all the great stuff that is happening right now. Now let’s play it forward a few years. Take it as a given that immersive computing becomes the dominant paradigm: virtual worlds, virtual places, virtual objects commingling with physical space, all presented in familiar, intuitive real-world metaphors, accessible with simple commands and discoverable online. Gesture input and voice, of course. Untethered and fully mobile. Lightweight glasses or even contact lenses. These all seem obvious… once we’ve taken the plunge into immersion. What’s really interesting is what comes after that, when other computing innovations converge with XR. AI for endlessly variant NPCs in large scale worlds and never-ending stories; deep learning to enable a malleable environment that responds to our every whim. Integrate IoT and we can control stuff in the real world-- with our very minds, or as close as we can get.
- Last year, designer and filmmaker Keiichi Matsuda released a thought-provoking short film called Hyper-Reality, depicting some of the possibilities of an immersive, augmented digital world. As computing goes immersive, every inch of available virtual real estate will be up for sale for manufacturers and marketers to hawk their wares. The history of advertising is, after all, the history of a rise in production value. And with XR we so much more production value at our disposal: 3D graphics, animation, 3D sound, and haptic inputs, to name a few. This could make our future world a brand-rich candyland with myriad choices...
- Or a hot mess - swipe left to reset your life, your credit history, your social status… as these technologies that promise to bring us closer to our world and each other instead trap us in a Super Sad True Love Story of our own making.
- Of course it’s too early to tell how this will all shake out. I’m hopeful, if only because we’ve been here before with other disruptive technology. One of the safeguards against some of these more negative outcomes is democratization of the platform. By enabling the independent creator, we can level the playing field, and not everything has to be paid for by big companies. A 15 year old kid could become a future YouTube star in XR… or develop cancer detection tools… or create the next XR artistic masterpiece. Though the early returns are incredibly promising, they’re just that: early. We don’t know which devices and form factors will win; which will be the dominant formats or killer apps. But we do know that the world is going immersive. Investments made now in immersive computing-- in learning the tools of the trade like Unity, in understanding immersive design, in adding meaningful layers to the real world, in short, in building interfaces for people-- will pay off many times over in the future. Because that future is 3D.
- Thank you - and enjoy AWE!