Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

[2C3]Developing context-aware applications


Published on

DEVIEW 2014 [2C3]Developing context-aware applications

Published in: Technology
  • A professional Paper writing services can alleviate your stress in writing a successful paper and take the pressure off you to hand it in on time. Check out, please ⇒ ⇐
    Are you sure you want to  Yes  No
    Your message goes here
  • Great beautiful deep interesting intelligent presentation. Thank you.
    Are you sure you want to  Yes  No
    Your message goes here

[2C3]Developing context-aware applications

  1. 1. Developing context-aware applications Marsal Gavaldà Expect Labs @MarsalGavalda
  2. 2. Anticipatory computing is transforming the way we find information Set reminder - Launch calendar app - Create new reminder - Enter flight details Check flight status - Launch web browser - Go to airline site - Enter flight number Check traffic - Launch web browser - Go to map / traffic site - Enter current location - Enter airport address Today, we find information Tomorrow, information finds us
  3. 3. Anticipatory computing relies on context awareness Source: “Entourage” by HBO
  4. 4. Source: Our Mobile Planet by Google Smartphone adoption (2013)
  5. 5. Mobile devices capture context via many sensors Source: Samsung Source: - Cameras - Microphones - Cellular receiver - Wi-Fi receiver - GPS receiver - Gyroscope - Thermometer - Barometer - …
  6. 6. Backend systems infer user situation, activity, intent, mood Sources: GigaOM, Robin Labs
  7. 7. Responsive design Source: Mashable
  8. 8. Contextual design
  9. 9. Recent technology advances Speech Recognition - Deep learning (deep / recurrent ANNs) - Ultra large language models - Dynamic speaker adaptation - Massive datasets (108s of users) Computer Vision - Deep learning - Massive datasets Language Understanding - Deep learning - Knowledge graphs Source: Facebook Source: Stanford University
  10. 10. Knowledge graphs From disembodied strings to grounded entities • Yahoo! 10 M entities, 30 M properties, 10 M connections • Microsoft 300 M entities, 800 M connections • Google 570 M entities, 18 B properties and connections • Wikipedia 4 M entities • Freebase 40 M topics, 2 B facts • Factual 66 M local businesses and POIs in 50 countries • LinkedIn 225 M people • Facebook 1.15 B people Cf. • Cyc 239 K concepts, 2 M facts • OpenCyc 6 K concepts, 60 K facts Source: Yahoo
  11. 11. Dynamic activation of the knowledge graph TIME Continuous user context hayes valley palo alto north beach cow hollow I really want to see that new movie with Ben Affleck It is the one about the Iran Hostage Crisis You have to see that video of the Today Show doing the Harlem Shake I am going to meet Raymond at Goat Hill Pizza at noon It is near the Comstock Saloon I am planning to go whitewater rafting in the Grand Canyon The Black Keys were on the Colbert Report last night It is near the Comstock Saloon Rolling Context Window Dynamic entity graph (~10M entities) things I recently wrote or said restaurants near North Beach places in the Bay area topics related to things I recently read current my friends, colleagues events and recent contacts links that my friends have recently shared Human Knowledge (~50B entities) 5B people 1B places 1B products 100M interests 100M events 1B media 2008 (1M entities) 2010 (10M entities) 2014 (500M entities) 2016 (10B entities) 5B domain-specific
  12. 12. The knowledge graph enables anchored NLP “I saw the man on the hill with the telescope” Source: Deniz Yuret
  13. 13. Voice 10% of Baidu search queries are done with voice today. In five years, it’ll be 50% ” Andrew Ng
  14. 14. Types of voice-driven applications Question & Answer "What is the capital of California?" "Who directed Citizen Kane?" Command & Control "Call Jenny's work phone." "Turn up the heat to 72 degrees." Content Discovery "Is there a good Japanese restaurant near Union Square?" "Show me all the James Bond movies with Roger Moore." Performing Tasks "Make a reservation for two at Kama tomorrow at 8pm." "Book me on a flight to JFK on Saturday afternoon." Dictation "Send a text to Jenny saying…" "Send the following email to Joe…" Passive Listening "…have you seen that video of the Russian meteor…" "…I’m thinking of getting a pair of red Kobe 9 sneakers…"
  15. 15. Anatomy of a voice interaction 1. Speech recognition 2. Natural language understanding type: restaurant category: Italian location: San Francisco cost: $, $$ filter: good for kids ”It’d be nice to find an inexpensive Italian restaurant in San Francisco that is good for kids.” 3. Search ranking & filtering 4. Real-time visualization of results Candidate 1: Buca di Beppo [confidence: 0.91] Candidate 2: La Traviata [confidence: 0.82] Candidate 3: Ragazza [confidence: 0.80] Candidate 4: Sotto Mare [confidence: 0.76] …
  16. 16. The MindMeld platform generate a continuously changing model of user intent based on long-1