Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Context is King: AR, AI, Salience, and the Constant Next Scenario


Published on

Clark Dodsworth’s AREvent talk, Santa Clara, CA June 3, 2010: "Context is King: AR, AI, Saience, and the Constant Next Scenario" Mostly about smartphone AR as a gateway to context aware computing becoming indispensible.

Published in: Technology
  • Be the first to comment

Context is King: AR, AI, Salience, and the Constant Next Scenario

  1. 1. Context is King<br />AR,-Salience and the<br />Constant Next-Scenario<br />
  2. 2. Process of tool design<br /> Empathic pragmatismBroad-based tool knowledge / pattern recognition / intuitive<br />Perspective“Product as tool as interface” Tool should fit you. <br />WorkUser-centered product strategy, dev, design.<br />
  3. 3. “Ambient Intelligence” strategy Philips, ‘98<br />all photos: Philips<br />© 2002 Elmo Diederiks<br />
  4. 4. Hands-up displays<br />Commercial helmet-mounted displays<br />AR Timeline<br />Efficientpower usage<br />Dedicated tools begin:<br />Commercial aviation<br />1st car HUD:Cutlass Supreme <br />ARToolkitUW HITLab<br />Rear-view <br />mirrors<br />Useful smart- phone AR<br />Sailboat <br />tell-tales<br />US DoD air-combat testing<br />“Augmentedenvironments,” sensor nets <br />1stjetfighter <br />heads-up display<br />Nokia’s MARA <br />Now<br />20??<br />~1900<br />1958<br />1970s<br />1974-8<br />1988<br />1992+<br />1999<br />2006<br />Prehistory,soon after invention of the sail.<br />Horse<br />carriages<br />The goal<br />Tom Caudell &David Mizell<br />“UbiComp”:Mark Weiser<br />Sutherland/Sproull, ‘68: 1st HMDs<br />
  5. 5. AR Phase One current status<br />Smartphone ecosystem.<br />Hardware & software platforms exist.<br />Infrastructure issues: huge. <br />Power use issues: huge.<br />Privacy/security issues: gigantic.<br />Interoperability issues: roadblocks.<br />ButPhase One AR is just part of transition to mobile broadband as our primary platform.<br />
  6. 6. AR Phase Two<br />Smartphone ecosystem: <br />------The platform for<br />)<br />(<br />Pervasive computing<br />Context awareness<br />Semantic Web<br />Ubicomp & sensornets<br />Ambient Intelligence <br />Things that Think<br /> last.<br />
  7. 7. Alfred North Whitehead-(1861-1947)First augmented reality theorist<br />1: “Civilization advances by extending the number of important operations which we can perform without thinking of them.”<br />----------------------------...AR shouldn’t increase cognitive load<br />2: “The art of progress is to preserve order amid change and to preserve change amid order.” ...AR: contextually dynamic data management & display<br />3: We think in generalities, but we live in detail.” <br />---------------------------...AR tasks: managing detail<br />
  8. 8. Next Steps: AR Phase Two<br />“It’s all about long-term, sustaining relationships.” <br /> -Alan Kay (2009), on the computer industry trend toward a service-based model.<br /><ul><li>Phase Two AR: long-term, sustaining,-contextually-dynamic, hyperpersonalized-relationships between users & info services.
  9. 9. The available data streams must be far better managed. </li></ul> ...It will just happen first on smartphones. <br />
  10. 10. AR Phase Two<br />Adaptively deliver info, highly personalized via n dimensions of user states, with high space/time precision, without spiking your cognitive load.<br /> Augmented Context + Information Salience = Value<br />
  11. 11. AR Phase Two<br />Mobility <br /> • Diversity of use cases / user.<br /> • User attention = scarce resource.<br /> Tool should learn the user.<br /> The learning is contextual: context becomes king.<br /> • Collect & analyze user’s paths through location, time, tasks, people, queries, interests, decisions...<br /> • Accretes<br /> • Deliver predictive, scenario-driven support.<br />Result: Adaptive context analysis manages data delivery: What, when, how much, what modality, what priority. <br />-------------------------------------------------------Especially when not-to.<br />
  12. 12. Phase Two<br />Context Is King<br />Realtime, context-driven mobile broadband services.<br /> “The software layer that learns you.”<br /> • Realtime data delivery (live monitoring of processes & states).<br /> • Constant predictive analytics, contextual filtering/salience eval. <br /> • Semantic Web:better,-partly-automated-search.<br /> • Hyperpersonalized database creation, web services.<br />• Digital assistants (Sense Networks, Siri) learn you over time.<br />
  13. 13. Phase Two Software<br />•  Allostatic control of data delivery: only the data needed, when needed. <br />• Constant autodiscovery of data feeds, sensor feeds.<br />• Constant markerless 2D & 3D feature I.D. and object recognition. <br />• Motion analysis + evaluation linked to the object recognition.<br />• Indoor position mapping: multiple methods.<br />• Nuanced gestural interpretation + integration with voice.<br />• Constant audio + visual awareness with flagging & interpretation.<br />
  14. 14. Phase Two Hardware<br /><ul><li>Near-field RFID
  15. 15. Constant, wide-field visual recording + auto-tagging
  16. 16. Constant markerless-object I.D. in 2D-&-3D-+-auto-tagging
  17. 17. Constant audio recording + auto-tagging
  18. 18. Galileo-grade location (+ dead reckoning, incl. gyros)
  19. 19. Ultrasonic& IR-sensors for position sensing, spatialanalysis,modeling, mapping, contextualizing
  20. 20. Laser projector + 3D object geometry awareness: </li></ul>-------------“shape-adaptive projection”-for-object-augmentation<br />
  21. 21. Phase Two Example Market<br />Entertainment<br /> First rule of theme parks:<br /> You bring the fun in with you - your friends-& family. <br /> • The attractions you pay for aren’t the experience.<br /> • Now, a new type of friend.<br />For as long as you want.<br />c. Lynne LaCascia, all rights reserved<br />All images © Chimera Design 2007-2008<br />
  22. 22. Phase Two Example Market<br />Entertainment<br />Any licensed character: a useful buddy, a delivery method on top of context-aware, locational, hyper-personalized services. <br />The character behavior, persona, & animation: Character Skin.<br />
  23. 23. Dubailand AR project ’07-’08 • Convert concept to smartphone AR + lots of hidden sensors & projectors.• Live & scripted personalized entertainment via licensed characters, + services for hotel, leisure, retail, restaurant, transit,-and-follow visitors home.<br />© Chimera Design 2007-2008<br />Marvel Superheroes park<br />Part of Dubailand model:<br />c. Lynne LaCascia, all rights reserved<br />
  24. 24. Dubailand AR project ’07-‘08 <br /><ul><li> Storytelling
  25. 25. Extension of attractions
  26. 26. Personal guide/friend
  27. 27. Contextuallycustomizedexperiences
  28. 28. Interaction with park elements
  29. 29. Wayfinding
  30. 30. Drill-down for information
  31. 31. Social networking: local & global
  32. 32. Database: each user’s interests
  33. 33. Game play on device & with others
  34. 34. Marketing: retail opportunities
  35. 35. Long-Term Relationship: “follow the visitor home”</li></ul>All images © Chimera Design 2007-2008<br />
  36. 36. Phase Two Biological Model - Consciousness<br />Key functions of consciousness:<br /> 1) Constant, integrated analysis & evaluation of surroundings<br /> 2) Dynamically judge salience of each aspect of your current situation (state) in context of your goals/needs<br /> 3) Constantly construct scenarios for next step<br />Parallel function of AR2: assess, select & deliver salient data <br /> • “Contextual evaluation drives contextual augmentation” • “Software ‘awareness’ augments user’s awareness”<br />History + current state = basis of decision-making for next state<br />
  37. 37. Phase Two Biological Model<br />The brain’s Default Mode Network<br /> • One of the hardest-working systems in the brain<br /> • Active only when the brain is notfocused on a particular task<br /> • 2 linked brain areas:<br /> Empathy re: intent of others,-+ your own state awareness<br /> Personal memories + visualization of future scenarios<br />State & intent awareness<br />Scenario modeling<br />
  38. 38. Default Mode Network of the brain<br />Posterior <br />cingulate <br />cortex, with <br />the precuneus:<br />accessing<br />personal <br />memories, <br />visualizing<br />oneself in <br />scenarios.<br />Medial <br />prefrontal <br />cortex:<br />Imagining & <br />modeling-the mindset of others.<br />> Who you are, what’s relevant now, and very-near-future scenarios to choose from<br />Brain photo: Omikron/photoresearchers Network overlay: Olaf Sporns/Indiana Univ. (modified by J. Korenblat) <br />
  39. 39. Phase Two Implementation<br />Brain Default Mode Network: constantly considers what to do next<br />> Observe states/intents, visualize-scenarios/outcomes<br /> > Scenario creation supports each next decision<br />Phase Two mobile broadband services assess, prioritize, & select data support decisions<br /> • Realtime data mgmt. helps preserve-focus<br /> • Inverse Content Management System (CMS) ...the filter part<br /> • Adaptive B2B & B2C service optimization ...the revenue part<br />
  40. 40. Facets of Phase Two AR<br />Efficiency & revenue from contextual salience<br />Location Based-Services<br />Constant personal scenario support<br />Autodiscovery:net services &sensor data<br />Integrated sensor,gesture, & voice<br />Hyper-personalization,relationship mgmt.<br />Predictive analytics<br />
  41. 41. Facets of Phase Two AR<br />Efficiency & revenue from contextual salience<br />Normal Everyday Life<br />Data interchange standards.<br />Artificial intelligence permeates web svcs.<br />Context <br />awareness.<br />User control of personal data.<br />Constant personal scenario support<br />Autodiscovery:net services &sensor data<br />Integrated sensor,gesture, & voice<br />Hyper-personalization,relationship mgmt.<br />Predictive analytics<br />Better interfaces matter.<br />Probabilistic data delivery. <br />Activity awareness & modeling.<br />Adaptive filtering of realtime data.<br />Resource conservation. <br />Manage the info overload.<br />The opposite of advertising.<br />Sensor datastream analytics.<br />Qualcomm<br />My6sense<br />Aloqua, Xiam<br />Your Name Here<br />Canesta, Apple/Siri Softkinetic, Nuance<br />Corventis Pachube<br />Sense Networks<br />© Clark Dodsworth<br />
  42. 42.
  43. 43.<br />end OA logo<br />Contact for .pptx & speaker notes.<br />
  44. 44. Credits<br /><ul><li>First use of term, “context awareness: Schilit, B., Theimer, M. Disseminating Active Map Information to Mobile Hosts. IEEE Network, 8(5) (1994) 22-32
  45. 45. Schilit, Bill.; Adams, Norman; Want, Roy. Context-Aware Computing Applications. 1st International Workshop on Mobile Computing Systems and Applications. (1994) 85-90
  46. 46. Default Mode Network paper: Gusnard, DA, Raichle, ME: “Searching for a baseline: the resting human brain from the perspective of functional imaging.” Nature Reviews Neuroscience, 2001</li></ul>• “iLamps: Geometrically Aware and Self-Configuring Projectors,” Rascar, et al. 2003<br /><ul><li>Commercial aviation HUD image courtesy Todd Lapin, cc
  47. 47. Photo of sail’s tell-tales courtesy Bill Gracey, cc
  48. 48. Modern helmet-mounted see-through display courtesy Rockwell-Collins: "SIM EYE SR100-A”
  49. 49. Theme park illustrations © Chimera Design, 2007-2009
  50. 50. Photo of Dubailand model © Lynne LaCascia, 2009
  51. 51. Photo of Alfred North Whitehead from</li>