Towards the Semantic Concierge

        Deborah L. McGuinness
    Tetherless World Senior Constellation Chair
   Professor of Computer and Cognitive Science
    Rensselaer Polytechnic Institute, Troy, NY
   & CEO McGuinness Associates, Latham, NY


          OpenTravel 2012 Advisory Forum   April 11, 2012
Background
– Semantic Technologies – technological support for encoding
  meaning in a form computers can understand and
  manipulate – are maturing and increasing in usage
– Computational encodings of meaning can be used to help
  integrate, validate, filter,…. Essentially to make smarter,
  more context-aware applications
– This can provide competitive advantage in todays
  increasingly networked and competitive environments
– Motivating Vision
– Semantic Web intro through examples: Ontologies, Mobile
  Advisor, Linked Data
– Discussion

Delivered by semantic web guru and road warrior…
Scenario 1: Real
              Time Travel Change
Change and the disrupted traveler
   • Weather or mechanical issues mean leaving a traveler
     stranded at an intermediate destination
   • Default coping strategy is not desirable to traveler (i.e., local
     intermediate hotel and transportation the next day)
   • Transportation to an alternative airport somewhat near
     desired destination along with confirmed new last one way
     leg (e.g. one way car rental available at the arrival time)
     would be vastly superior
Scenario 1 Information
                      Requirements
• Location:
   – Current airport
   – Destination airport (and/or final destination)
   – Airports within x hour driving radius
• Schedules
   – Flights to alternative airports in correct time period
• Connections to other ecosystem partners
   – Other transportation options to final destination (train, bus, car with
     a local starting point)
   – Other transportation options from alternative destination to final
     destination
   – Critical parameters: availability, one-way rental, operating hours for
     pickup/drop off
Much of this is freely available and with passenger data
including loyalty memberships, this could be connected easily
Scenario 2: Advance
                       Travel purchase
• Known departure and destination airport
• With air loyalty program information, could offer air approp. options
  (confirmed upgrades, lay back seat, reclining seat, discounted
  business vs. full fare economy….)
• With destination address and hotel loyalty programs, could offer hotel
  rooms near address with loyalty brands (including benefits of loyalty
  program customer tier – e.g., upgrades, discounts, etc.)
• With car loyalty programs, could offer car options

• If additional tickets purchased, could offer larger car
• If with spouse (often in air systems), could offer more leisure
  packages
• If connected to restaurant booking (foursquare, open table, …) or
  preferences, could offer dinner bookings
• If connected to online calendar (e.g., Google), could pick up
  conference information
Semantic Web: Introduction
             through examples

• Semantic Sommelier: Mobile Context Aware Semantic Wine
  Advisor
• SemantAqua: Semantically-Aware information aggregator &
  visualizer

There are many more….
• PopSciGrid – aimed at Preventable Cancers
• DARPA Personal Assistant that Learns -> SIRI
• IARPA Analyst Workbench -> Watson
• Home Theatre Advisor Configurator
Notes
• Examples are from other domains because travel domains
  have not been built out to the same level. Most travel
  examples are at a syntactic level (or light weight semantics)
  such as aggregators or natural language interfaces with
  some semantics (such as Siri) but less about actually
  “thinking” or acting as a personal assistant and more about
  finding information

• One take home message after these examples will be that
  the time is now to build the same kind of applications
  described in the next few slides in travel…. Creating
  customized semantic concierges
Semantically-enabled advisors
utilize:
      • Ontologies
      • Reasoning
      • Social
      • Mobile
      • Provenance
      • Context

Patton & McGuinness.et. al
tw.rpi.edu/web/project/Wineagent
Semantic
            Sommelier
Previous versions used ontologies
to infer descriptions of wines for
meals and query for wines
New version uses
  Context: GPS location, local
  restaurants and wine lists, user
  preferences
  Social input: Twitter, Facebook, Wiki,
  mobile, …
Source variability in quality,
contradictions exist,
Maintenance is an issue… however
new models emerging
SemantEco/SemantAqua
• Enable/Empower citizens &
  scientists to explore pollution
  sites, facilities, regulations,
  and health impacts along with 5                                                          4
  provenance.                                           2               3
• Demonstrates semantic
  monitoring possibilities.
• Map presentation of analysis                                  1
• Explanations and                        http://was.tw.rpi.edu/swqp/map.html and
                                          http://aquarius.tw.rpi.edu/projects/semantaqua
  Provenance available
           1.   Map view of analyzed results
           2.   Explanation of pollution
           3.   Possible health effect of contaminant (from EPA)
           4.   Filtering by facet to select type of data
           5.   Link for reporting problems
SemantAqua Workflow

                         Publish


CSV2RDF4LOD
   Direct                              visualize

   derive      archive               derive




                           Archive


 CSV2RDF4LOD
   Enhance                                         12
Why did I present wine and
                        water applications?
• Wine advisor shows semantic technology in action making
  actionable recommendations
• Water application shows a “typical” semantic integration web
  3.0 application

• Both of these styles are needed for a semantic concierge and
  these features are realizable today

• Next – they depend on
   –   Semantic web stack
   –   Data availability – linked data cloud is growing
   –   Ontologies
   –   Semantic methodologies
   –   Understandable and Actionable applications
Foundations: Web Layer Cake

                                                    Visualization APIs
                                                           S2S
                                                        Govt Data
   Inference Web, Proof
   Markup Language, W3C                                                      Inference Web IW Trust,
   Provenance Working                                                        Air + Trust
   group formal model,
   W3C incubator group,                                                          DL, KIF, CL, N3Logic
   …
                                                                      Ontology repositories
OWL 1 & 2 WG Edited main OWL                                          (ontolinguag),
   Docs, quick reference,                                             Ontology Evolution env:
   OWL profiles (OWL RL),                                             Chimaera,
  Earlier languages: DAML,                                            Semantic eScience
      DAML+OIL, Classic                                               Ontologies, MANY other ontologie
                                                                                  RIF WG
                                                                           AIR accountability tool
  SPARQL WG, earlier QL –
  OWL-QL, Classic’ QL, …
                                                                              Govt metadata search
                                                                              Linked Open Govt Data

                  SPARQL to Xquery translator   RDFS materialization
                                                (Billion triple winner)      Transparent Accountable
                                                                             Datamining Initiative (TAM
Foundations: The Tetherless World
            Constellation Linked Open Government
                          Data Portal




  Convert      TWC LOGD
                               Query/
                               Access
               LOGD                       Community Portal
              SPARQL            • RDF
              Endpoint          • RSS
                                • JSON
Create                          • XML
                                • HTML
                                • CSV
                                •…


             Enhance

                                         Data.gov deployment
                                                     16
Ontology Spectrum
          Thesauri
         “narrower                                       Formal Frames General
Catalog/   term”                                          is-a (properties) Logical
ID        relation                                                     constraints

     Terms/                          Informal                      Formal          Disjoint-
                                                                           Value ness,
    glossary                            is-a                      instance
                                                                           Restrs. Inverse,
                                                                                                        part-of…



 From 99 AAAI panel with Gruninger, Lehmann, McGuinness, Uschold, Welty. , 2000 Dagstuhl talk by McGuinness
Originally developed for VSTO, now in SSIII, SESDI, SESF, OOI …

                                                                    The Virtual Solar-Terrestrial
Observatory: A Deployed Semantic Web Application Case Study for Scientific Research. Proc. 19
Conf. on Innovative Applications of Artificial Intelligence (IAAI-07),
                http://www.vsto.org
Inference Web: Making Data Transparent and
                            Actionable Using Semantic Technologies

• How and when does it make sense to use smart system results & how do we
  interact with them?




                                                       (Mobile)
      Knowledge                                       Intelligent
 Provenance in Virtual
                                                        Agents      NSF Interops:
    Observatories                                                   SONET
                                                                    SSIII – Sea Ice
                                           Intelligence Analyst
                                                   Tools




                                          Hypothesis
                                        Investigation /
                                        Policy Advisors
                                                                           19
Back to Travel

Existing mobile and web site applications allow online
browsing, status checks, mobile access, with purchase
options

However they often
• Are not well connected into travel ecosystems
• Do not use sensor-based context – e.g., GPS
• Are not connected to user context – previous queries, and
  actions, google calendar, loyalty connections, status
  levels etc.
Remember Scenario 1 and 2
•   Semantic Technologies: ready for use
•
                      The Semantic Web
    Tools & tutorials available; deep apps
                          enables…
   future planning may benefit from
   consultants
•   • New models of intelligent services
    Context-aware, semantic
  apps are the future

    •   E-commerce solutions
    •   M-commerce
    •   Web assistants
    •   …

    New forms of web assistants/agents that act on a
          human’s behalf requiring less from humans
          and their communication devices…
Questions?


dlm @ cs . rpi . edu

What would you like from your semantic
travel concierge ?



Acknowledgements: Thanks to Opentravel and Thematix for
motivating discussions.
Extra

20120411 travelalliancemcguinnessfinal

  • 1.
    Towards the SemanticConcierge Deborah L. McGuinness Tetherless World Senior Constellation Chair Professor of Computer and Cognitive Science Rensselaer Polytechnic Institute, Troy, NY & CEO McGuinness Associates, Latham, NY OpenTravel 2012 Advisory Forum April 11, 2012
  • 2.
    Background – Semantic Technologies– technological support for encoding meaning in a form computers can understand and manipulate – are maturing and increasing in usage – Computational encodings of meaning can be used to help integrate, validate, filter,…. Essentially to make smarter, more context-aware applications – This can provide competitive advantage in todays increasingly networked and competitive environments – Motivating Vision – Semantic Web intro through examples: Ontologies, Mobile Advisor, Linked Data – Discussion Delivered by semantic web guru and road warrior…
  • 4.
    Scenario 1: Real Time Travel Change Change and the disrupted traveler • Weather or mechanical issues mean leaving a traveler stranded at an intermediate destination • Default coping strategy is not desirable to traveler (i.e., local intermediate hotel and transportation the next day) • Transportation to an alternative airport somewhat near desired destination along with confirmed new last one way leg (e.g. one way car rental available at the arrival time) would be vastly superior
  • 5.
    Scenario 1 Information Requirements • Location: – Current airport – Destination airport (and/or final destination) – Airports within x hour driving radius • Schedules – Flights to alternative airports in correct time period • Connections to other ecosystem partners – Other transportation options to final destination (train, bus, car with a local starting point) – Other transportation options from alternative destination to final destination – Critical parameters: availability, one-way rental, operating hours for pickup/drop off Much of this is freely available and with passenger data including loyalty memberships, this could be connected easily
  • 6.
    Scenario 2: Advance Travel purchase • Known departure and destination airport • With air loyalty program information, could offer air approp. options (confirmed upgrades, lay back seat, reclining seat, discounted business vs. full fare economy….) • With destination address and hotel loyalty programs, could offer hotel rooms near address with loyalty brands (including benefits of loyalty program customer tier – e.g., upgrades, discounts, etc.) • With car loyalty programs, could offer car options • If additional tickets purchased, could offer larger car • If with spouse (often in air systems), could offer more leisure packages • If connected to restaurant booking (foursquare, open table, …) or preferences, could offer dinner bookings • If connected to online calendar (e.g., Google), could pick up conference information
  • 7.
    Semantic Web: Introduction through examples • Semantic Sommelier: Mobile Context Aware Semantic Wine Advisor • SemantAqua: Semantically-Aware information aggregator & visualizer There are many more…. • PopSciGrid – aimed at Preventable Cancers • DARPA Personal Assistant that Learns -> SIRI • IARPA Analyst Workbench -> Watson • Home Theatre Advisor Configurator
  • 8.
    Notes • Examples arefrom other domains because travel domains have not been built out to the same level. Most travel examples are at a syntactic level (or light weight semantics) such as aggregators or natural language interfaces with some semantics (such as Siri) but less about actually “thinking” or acting as a personal assistant and more about finding information • One take home message after these examples will be that the time is now to build the same kind of applications described in the next few slides in travel…. Creating customized semantic concierges
  • 9.
    Semantically-enabled advisors utilize: • Ontologies • Reasoning • Social • Mobile • Provenance • Context Patton & McGuinness.et. al tw.rpi.edu/web/project/Wineagent
  • 10.
    Semantic Sommelier Previous versions used ontologies to infer descriptions of wines for meals and query for wines New version uses Context: GPS location, local restaurants and wine lists, user preferences Social input: Twitter, Facebook, Wiki, mobile, … Source variability in quality, contradictions exist, Maintenance is an issue… however new models emerging
  • 11.
    SemantEco/SemantAqua • Enable/Empower citizens& scientists to explore pollution sites, facilities, regulations, and health impacts along with 5 4 provenance. 2 3 • Demonstrates semantic monitoring possibilities. • Map presentation of analysis 1 • Explanations and http://was.tw.rpi.edu/swqp/map.html and http://aquarius.tw.rpi.edu/projects/semantaqua Provenance available 1. Map view of analyzed results 2. Explanation of pollution 3. Possible health effect of contaminant (from EPA) 4. Filtering by facet to select type of data 5. Link for reporting problems
  • 12.
    SemantAqua Workflow Publish CSV2RDF4LOD Direct visualize derive archive derive Archive CSV2RDF4LOD Enhance 12
  • 13.
    Why did Ipresent wine and water applications? • Wine advisor shows semantic technology in action making actionable recommendations • Water application shows a “typical” semantic integration web 3.0 application • Both of these styles are needed for a semantic concierge and these features are realizable today • Next – they depend on – Semantic web stack – Data availability – linked data cloud is growing – Ontologies – Semantic methodologies – Understandable and Actionable applications
  • 14.
    Foundations: Web LayerCake Visualization APIs S2S Govt Data Inference Web, Proof Markup Language, W3C Inference Web IW Trust, Provenance Working Air + Trust group formal model, W3C incubator group, DL, KIF, CL, N3Logic … Ontology repositories OWL 1 & 2 WG Edited main OWL (ontolinguag), Docs, quick reference, Ontology Evolution env: OWL profiles (OWL RL), Chimaera, Earlier languages: DAML, Semantic eScience DAML+OIL, Classic Ontologies, MANY other ontologie RIF WG AIR accountability tool SPARQL WG, earlier QL – OWL-QL, Classic’ QL, … Govt metadata search Linked Open Govt Data SPARQL to Xquery translator RDFS materialization (Billion triple winner) Transparent Accountable Datamining Initiative (TAM
  • 16.
    Foundations: The TetherlessWorld Constellation Linked Open Government Data Portal Convert TWC LOGD Query/ Access LOGD Community Portal SPARQL • RDF Endpoint • RSS • JSON Create • XML • HTML • CSV •… Enhance Data.gov deployment 16
  • 17.
    Ontology Spectrum Thesauri “narrower Formal Frames General Catalog/ term” is-a (properties) Logical ID relation constraints Terms/ Informal Formal Disjoint- Value ness, glossary is-a instance Restrs. Inverse, part-of… From 99 AAAI panel with Gruninger, Lehmann, McGuinness, Uschold, Welty. , 2000 Dagstuhl talk by McGuinness
  • 18.
    Originally developed forVSTO, now in SSIII, SESDI, SESF, OOI … The Virtual Solar-Terrestrial Observatory: A Deployed Semantic Web Application Case Study for Scientific Research. Proc. 19 Conf. on Innovative Applications of Artificial Intelligence (IAAI-07), http://www.vsto.org
  • 19.
    Inference Web: MakingData Transparent and Actionable Using Semantic Technologies • How and when does it make sense to use smart system results & how do we interact with them? (Mobile) Knowledge Intelligent Provenance in Virtual Agents NSF Interops: Observatories SONET SSIII – Sea Ice Intelligence Analyst Tools Hypothesis Investigation / Policy Advisors 19
  • 20.
    Back to Travel Existingmobile and web site applications allow online browsing, status checks, mobile access, with purchase options However they often • Are not well connected into travel ecosystems • Do not use sensor-based context – e.g., GPS • Are not connected to user context – previous queries, and actions, google calendar, loyalty connections, status levels etc.
  • 21.
  • 22.
    Semantic Technologies: ready for use • The Semantic Web Tools & tutorials available; deep apps enables… future planning may benefit from consultants • • New models of intelligent services Context-aware, semantic apps are the future • E-commerce solutions • M-commerce • Web assistants • … New forms of web assistants/agents that act on a human’s behalf requiring less from humans and their communication devices…
  • 23.
    Questions? dlm @ cs. rpi . edu What would you like from your semantic travel concierge ? Acknowledgements: Thanks to Opentravel and Thematix for motivating discussions.
  • 24.