Grupo de Procesado de Datos y Simulación
                                                          ETSI de Telecomunicación
                                                  Universidad Politécnica de Madrid



                                     PERSEO: a system to
                   Personalize the Environment Response
                       through Smart phonEs and Objects
                                                               SENAmI 2013
 5th International Workshop on Smart Environments and Ambient Intelligence


                                                          (presented by Josué Iglesias)

Luca Bergesio, Íñigo Marquínez, Ana M. Bernardos, Juan A. Besada, José R. Casar
                                              luca.bergesio@grpss.ssr.upm.es
contents

                  introduction and motivation
                  interaction method fundamentals
                  ‘smart night table’ use case
                                    functionalities
                                    architecture & implementation
                                    example
                                    video-demonstration
                  discussion and future works

Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es   1/8
contents

                  introduction and motivation
                  interaction method fundamentals
                  ‘smart night table’ use case
                                    functionalities
                                    architecture & implementation
                                    example
                                    video-demonstration
                  discussion and future works

Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es   1/8
introduction and motivation
    smart
    space




Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es   1/8
introduction and motivation
    smart
    space




                                                smart
                                                objects




Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es   1/8
introduction and motivation
                                                                            smart space behaviour
    smart                                                                          personalization
    space




                                                smart
                                                objects




Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es      1/8
introduction and motivation
                                                                              smart space behaviour
    smart                                                                            personalization
    space




                                                               focus
                                                                general approach
                                                             mobile-instrumented
                                                smart         interaction method
                                                objects
                                                                particular implementation
                                                             device-objects
                                                              coordination for content
                                                              representation &
                                                              management


Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es        1/8
contents

                 
                  introduction and motivation
                  interaction method fundamentals
                  ‘smart night table’ use case
                                    functionalities
                                    architecture & implementation
                                    example
                                    video-demonstration
                  discussion and future works

Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es   2/8
interaction method fundamentals



mobile-instrumented
concept
trigger
orchestrator




Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es   2/8
interaction method fundamentals



mobile-instrumented
concept
trigger
orchestrator




                                       physical-based interaction
                                       maintaining traditional interaction
                                       paradigm



Smart Environments and Ambient Intelligence – SENAmI 2013              luca.bergesio@grpss.ssr.upm.es   2/8
interaction method fundamentals



mobile-instrumented
concept
trigger
orchestrator




                                       mobile device – smart object
                                       coordinated response
                                       logical state of mobile device


                                       physical-based interaction
                                       maintaining traditional interaction
                                       paradigm



Smart Environments and Ambient Intelligence – SENAmI 2013               luca.bergesio@grpss.ssr.upm.es   2/8
interaction method fundamentals



mobile-instrumented
concept
trigger
orchestrator
                                       device-hosted
                                       content sharing
                                       audio, video, pictures, etc.

                                       mobile device – smart object
                                       coordinated response
                                       logical state of mobile device


                                       physical-based interaction
                                       maintaining traditional interaction
                                       paradigm



Smart Environments and Ambient Intelligence – SENAmI 2013               luca.bergesio@grpss.ssr.upm.es   2/8
interaction method fundamentals



mobile-instrumented                    multiple & heterogeneous
concept                                smart objects interfaces
trigger                                TV, Hi-Fi, tablet, PC, photo frame, etc.
orchestrator
                                       device-hosted
                                       content sharing
                                       audio, video, pictures, etc.

                                       mobile device – smart object
                                       coordinated response
                                       logical state of mobile device


                                       physical-based interaction
                                       maintaining traditional interaction
                                       paradigm



Smart Environments and Ambient Intelligence – SENAmI 2013               luca.bergesio@grpss.ssr.upm.es   2/8
interaction method fundamentals


                                       rule-based reasoning
                                       ECA model
                                       Events – Conditions – Actions


mobile-instrumented                    multiple & heterogeneous
concept                                smart objects interfaces
trigger                                TV, Hi-Fi, tablet, PC, photo frame, etc.
orchestrator
                                       device-hosted
                                       content sharing
                                       audio, video, pictures, etc.

                                       mobile device – smart object
                                       coordinated response
                                       logical state of mobile device


                                       physical-based interaction
                                       maintaining traditional interaction
                                       paradigm



Smart Environments and Ambient Intelligence – SENAmI 2013               luca.bergesio@grpss.ssr.upm.es   2/8
interaction method fundamentals
               context
                 user                  context-responsive
          environment                  context analysis before executing
                  etc.                 actions


                                       rule-based reasoning
                                       ECA model
                                       Events – Conditions – Actions


mobile-instrumented                    multiple & heterogeneous
concept                                smart objects interfaces
trigger                                TV, Hi-Fi, tablet, PC, photo frame, etc.
orchestrator
                                       device-hosted
                                       content sharing
                                       audio, video, pictures, etc.

                                       mobile device – smart object
                                       coordinated response
                                       logical state of mobile device


                                       physical-based interaction
                                       maintaining traditional interaction
                                       paradigm



Smart Environments and Ambient Intelligence – SENAmI 2013               luca.bergesio@grpss.ssr.upm.es   2/8
contents

                 
                  introduction and motivation
                 
                  interaction method fundamentals
                  ‘smart night table’ use case
                                    functionalities
                                    architecture & implementation
                                    example
                                    video-demonstration
                  discussion and future works

Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es   3/8
‘smart night table’ use case
                                                                    functionalities

                                                                                 hi-fi audio




                                                                                 digital photo frame




                                                                                 television




     ‘smart
night table’                                                                     tablet

Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es             3/8
‘smart night table’ use case
                                                                    functionalities

                                                                                 hi-fi audio




            Voice (call)


                                                                                 digital photo frame
Photos                       Music




                           Internet
   Videos




                                                                                 television




     ‘smart
night table’                                                                     tablet

Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es             3/8
‘smart night table’ use case
                                                                       functionalities

                                      Voice (call)     Music
                                                                                    hi-fi audio




            Voice (call)
                                                     Photos
                                                                                    digital photo frame
Photos                       Music




                           Internet
   Videos




                                                     Videos

                                                                                    television




     ‘smart
                                                 Internet
night table’                                                                        tablet

Smart Environments and Ambient Intelligence – SENAmI 2013      luca.bergesio@grpss.ssr.upm.es             3/8
contents

                 
                  introduction and motivation
                 
                  interaction method fundamentals
                  ‘smart night table’ use case
                    
                     functionalities
                                    architecture & implementation
                                    example
                                    video-demonstration
                  discussion and future works

Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es   4/8
‘smart night table’ use case
                                         architecture & implementation

                                      Voice (call)     Music
                                                                                    hi-fi audio




            Voice (call)
                                                     Photos
                                                                                    digital photo frame
Photos                       Music




                           Internet
   Videos




                                                     Videos

                                                                                    television




     ‘smart
                                                 Internet
night table’                                                                        tablet

Smart Environments and Ambient Intelligence – SENAmI 2013      luca.bergesio@grpss.ssr.upm.es             4/8
‘smart night table’ use case
                                                      architecture & implementation

                                      Android
                                       service
                                                   Voice (call)     Music
                                                                                                 hi-fi audio




            Voice (call)
                                                                  Photos
                                                                                                 digital photo frame
Photos                       Music




                                                 NFC tag
                           Internet
   Videos




                                                                  Videos

                                                                                                 television




     ‘smart
                                                              Internet
night table’                                                                                     tablet

Smart Environments and Ambient Intelligence – SENAmI 2013                   luca.bergesio@grpss.ssr.upm.es             4/8
‘smart night table’ use case
                                                      architecture & implementation

                                      Android
                                       service
                                                   Voice (call)     Music
                                                                                                                       hi-fi audio




            Voice (call)
                                                                      
                                                                  Photos Airplay Mirroring (Apple)
                                                                       Intel Wireless Display (WiDi)                  digital photo frame
                                                                       Wireless HD
                             Music
Photos
                                                                       Miracast

                                                 NFC tag              DLNA
                           Internet
   Videos
                                                                      (DMS) Digital Media Server
                                                                      (DMR) Digital Media Renderer
                                                                  Videos
                                                                      (DMC) Digital Media Controller
                                                                                                                       television
                                                                             maturity
                                                                             interoperability
                                                                              (S.O., comm. tech., brands, etc.)
                                                                             certified devices
                                                                             SDK available
     ‘smart
                                                              Internet
night table’                                                                                                           tablet

Smart Environments and Ambient Intelligence – SENAmI 2013                                         luca.bergesio@grpss.ssr.upm.es             4/8
‘smart night table’ use case
                                                             architecture & implementation
                                                                                                                            DLNA Renderer
                                             Android                                                                                (DMR)
                                              service                                                                      ID: Renderer003
                                                          Voice (call)     Music
                                                                                                                           hi-fi audio
                                       DLNA Renderer
                                               (DMR)
                                      ID: Renderer001                                        DLNA Controller                DLNA Renderer
                                                                                                     (DMC)                          (DMR)
                                         DLNA Server                                                                       ID: Renderer004
            Voice (call)
                                             M-DMS                                      RESTful API
                                                                         Photos
                                                                                                                           digital photo frame
Photos                       Music




                                                        NFC tag              DLNA                                           DLNA Renderer
   Videos
                           Internet                                                                                                 (DMR)
                                                                             (DMS) Digital Media Server                    ID: Renderer002
                                                                             (DMR) Digital Media Renderer
                                                                         Videos
                                                                             (DMC) Digital Media Controller
                                                                                                                           television




     ‘smart
                                                                     Internet
night table’                                                                                                               tablet

Smart Environments and Ambient Intelligence – SENAmI 2013                                             luca.bergesio@grpss.ssr.upm.es             4/8
‘smart night table’ use case
                                                             architecture & implementation
                                                                                                                          Bluetooth


                                                                                                                      DLNA Renderer
                                             Android                                                                          (DMR)
                                              service                                                                ID: Renderer003
                                                          Voice (call)     Music
                                                                                                                     hi-fi audio
                                       DLNA Renderer
                                               (DMR)
                                      ID: Renderer001                                  DLNA Controller                DLNA Renderer
                                                                                               (DMC)                          (DMR)
                                         DLNA Server                                                                 ID: Renderer004
            Voice (call)
                                             M-DMS                                 RESTful API
                                                                         Photos
                                                                                                                     digital photo frame
Photos                       Music




                                                        NFC tag                                                       DLNA Renderer
   Videos
                           Internet                                                                                           (DMR)
                                                                                                                     ID: Renderer002

                                                                         Videos

                                                                                                                     television


                                                                                                                             sockets

     ‘smart
                                                                     Internet
night table’                                                                                                         tablet

Smart Environments and Ambient Intelligence – SENAmI 2013                                       luca.bergesio@grpss.ssr.upm.es             4/8
‘smart night table’ use case
                                                             architecture & implementation
                                                                                                                          Bluetooth


                                                                                                                      DLNA Renderer
                                             Android                                                                          (DMR)
                                              service                                                                ID: Renderer003
                                                          Voice (call)     Music
                                                                                                                     hi-fi audio
                                       DLNA Renderer
                                               (DMR)
                                      ID: Renderer001                                  DLNA Controller                DLNA Renderer
                                                                                               (DMC)                          (DMR)
                                         DLNA Server                                                                 ID: Renderer004
            Voice (call)
                                             M-DMS                                 RESTful API
                                                                         Photos
                                                                                                                     digital photo frame
Photos                       Music




                                                        NFC tag                                                       DLNA Renderer
   Videos
                           Internet                                                                                           (DMR)
                                                                                                                     ID: Renderer002

                                                                         Videos

                                                                                                                     television


                                                                                                                             sockets

     ‘smart
                                                                     Internet
night table’                                                                                                         tablet

Smart Environments and Ambient Intelligence – SENAmI 2013                                       luca.bergesio@grpss.ssr.upm.es             4/8
contents

                 
                  introduction and motivation
                 
                  interaction method fundamentals
                  ‘smart night table’ use case
                    
                     functionalities
                    
                     architecture & implementation
                                    example
                                    video-demonstration
                  discussion and future works

Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es   5/8
‘smart night table’ use case
                                                                                                                      example
                                                                                                                          Bluetooth


                                                                                                                      DLNA Renderer
                                             Android                                                                          (DMR)
                                              service                                                                ID: Renderer003
                                                          Voice (call)     Music
                                                                                                                     hi-fi audio
                                       DLNA Renderer
                                               (DMR)
                                      ID: Renderer001                                  DLNA Controller                DLNA Renderer
                                                                                               (DMC)                          (DMR)
                                         DLNA Server                                                                 ID: Renderer004
            Voice (call)
                                             M-DMS                                 RESTful API
                                                                         Photos
                                                                                                                     digital photo frame
Photos                       Music
                                                1
  Videos
                           Internet
                                                        NFC tag                                                       DLNA Renderer
                                                                                                                              (DMR)
                                                                                                                     ID: Renderer002

                                                                         Videos

                                                                                                                     television


                                                                                                                             sockets

     ‘smart
                                                                     Internet
night table’                                                                                                         tablet

Smart Environments and Ambient Intelligence – SENAmI 2013                                       luca.bergesio@grpss.ssr.upm.es             5/8
‘smart night table’ use case
                                                                                                                      example
     SELECT render                                                                                                        Bluetooth

                                                          2 GET renderers
         3                                                                                                            DLNA Renderer
                                                                                                                              (DMR)
                                             Android
                                              service                                                                ID: Renderer003
                                                          Voice (call)     Music
                                                                                                                     hi-fi audio
                                       DLNA Renderer
                                               (DMR)
                                      ID: Renderer001                                  DLNA Controller                DLNA Renderer
                                                                                               (DMC)                          (DMR)
                                         DLNA Server                                                                 ID: Renderer004
            Voice (call)
                                             M-DMS                                 RESTful API
                                                                         Photos
                                                                                                                     digital photo frame
Photos                       Music
                                                1
  Videos
                           Internet
                                                        NFC tag                                                       DLNA Renderer
                                                                                                                              (DMR)
                                                                                                                     ID: Renderer002

                                                                         Videos

                                                                                                                     television


                                                                                                                             sockets

     ‘smart
                                                                     Internet
night table’                                                                                                         tablet

Smart Environments and Ambient Intelligence – SENAmI 2013                                       luca.bergesio@grpss.ssr.upm.es             5/8
‘smart night table’ use case
                                                                                                                       example
                                                                                                                           Bluetooth

                                                           2
         3                                                                                                             DLNA Renderer
                                                                                                                               (DMR)
                                              Android
                                               service                                                                ID: Renderer003
                                                           Voice (call)     Music
                                                                                                                      hi-fi audio
                                        DLNA Renderer      4 STOP 5
                                                             render
                                                (DMR)
                                       ID: Renderer001                                  DLNA Controller                DLNA Renderer
                                                                                                (DMC)                          (DMR)
                                          DLNA Server                                                                 ID: Renderer004
             Voice (call)
                                              M-DMS                                 RESTful API
                                                                          Photos
                                                                                                                      digital photo frame
Photos                        Music
                                                 1
   Videos
                            Internet
                                                         NFC tag                                                       DLNA Renderer
                                                                                                                               (DMR)
                                                                                                                      ID: Renderer002

                                                                          Videos

                                                                                                                      television


                                                                                                                              sockets

     ‘smart
                                                                      Internet
night table’                                                                                                          tablet

Smart Environments and Ambient Intelligence – SENAmI 2013                                        luca.bergesio@grpss.ssr.upm.es             5/8
‘smart night table’ use case
                                                                                                                       example
                                                                                                                           Bluetooth

                                                           6 START playing
         3                                                                                                             DLNA Renderer
                                                                                                                               (DMR)
                                              Android
                                               service                                                                ID: Renderer003
                                                           Voice (call)     Music
                                                                                                                      hi-fi audio
                                        DLNA Renderer      4                  5
                                                (DMR)
                                       ID: Renderer001                                  DLNA Controller                DLNA Renderer
                                                                                                (DMC)                          (DMR)
                                          DLNA Server                                                                 ID: Renderer004
             Voice (call)
                                              M-DMS                                 RESTful API
                                                                          Photos
                                                                                                                      digital photo frame
Photos                        Music
                                                 1                                  7 START playing
   Videos
                            Internet
                                                         NFC tag                                                       DLNA Renderer
                                                                                                                               (DMR)
                                                                                                                      ID: Renderer002

                                                                          Videos

                                                                                                                      television


                                                                                                                              sockets

     ‘smart
                                                                      Internet
night table’                                                                                                          tablet

Smart Environments and Ambient Intelligence – SENAmI 2013                                        luca.bergesio@grpss.ssr.upm.es             5/8
‘smart night table’ use case
                                                                                                                             example
                                                                                                                                 Bluetooth

                                                           6
         3                                                                                                                   DLNA Renderer
                                                                                                                                     (DMR)
                                              Android
                                               service                                                                      ID: Renderer003
                                                           Voice (call)     Music
                                                                                                                            hi-fi audio
                                        DLNA Renderer      4                  5
                                                (DMR)
                                       ID: Renderer001                                        DLNA Controller                DLNA Renderer
                                                                                                      (DMC)                          (DMR)
                                          DLNA Server                                                                       ID: Renderer004
             Voice (call)
                                              M-DMS                                       RESTful API
                                                                          Photos
                                                                                                                            digital photo frame
Photos                        Music
                                                 1                                       7
   Videos
                            Internet
                                                         NFC tag                                                             DLNA Renderer
                                                                                                                                     (DMR)
                                                                                                                            ID: Renderer002
                                                                                                                                              
                                                                                                                                              Videos



                                                                          Videos    8 (continuous streaming)
                                                                                                                            television


                                                                                                                                    sockets

     ‘smart
                                                                      Internet
night table’                                                                                                                tablet

Smart Environments and Ambient Intelligence – SENAmI 2013                                              luca.bergesio@grpss.ssr.upm.es                  5/8
contents

                 
                  introduction and motivation
                 
                  interaction method fundamentals
                 
                  ‘smart night table’ use case
                    
                     functionalities
                    
                     architecture & implementation
                    
                     example
                                    video-demonstration
                  discussion and future works

Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es   6/8
‘smart night table’ use case
                                                        video-demonstration




Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es   6/8
contents

                 
                  introduction and motivation
                 
                  interaction method fundamentals
                 
                  ‘smart night table’ use case
                    
                     functionalities
                    
                     architecture & implementation
                    
                     example
                    
                     video-demonstration
                  discussion and future works

Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es   7/8
discussion and future works
              smart space behaviour personalization
              ⁷   Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
                  objects through mobile devices. Proc. of the 9th ACM International Symposium on
                  Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011




Smart Environments and Ambient Intelligence – SENAmI 2013                    luca.bergesio@grpss.ssr.upm.es    7/8
discussion and future works
              smart space behaviour personalization
              ⁷   Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
                  objects through mobile devices. Proc. of the 9th ACM International Symposium on
                  Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011


               smart objects orchestration architecture
                  mobile-centred




Smart Environments and Ambient Intelligence – SENAmI 2013                    luca.bergesio@grpss.ssr.upm.es    7/8
discussion and future works
              smart space behaviour personalization
              ⁷   Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
                  objects through mobile devices. Proc. of the 9th ACM International Symposium on
                  Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011


               smart objects orchestration architecture
                  mobile-centred

               mobile-instrumented interaction method
                  mobile media content/services ↔interface device transfer




Smart Environments and Ambient Intelligence – SENAmI 2013                    luca.bergesio@grpss.ssr.upm.es    7/8
discussion and future works
              smart space behaviour personalization
              ⁷   Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
                  objects through mobile devices. Proc. of the 9th ACM International Symposium on
                  Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011


               smart objects orchestration architecture
                  mobile-centred

               mobile-instrumented interaction method
                  mobile media content/services ↔interface device transfer

               specific use case
                  ‘smart night table’




Smart Environments and Ambient Intelligence – SENAmI 2013                    luca.bergesio@grpss.ssr.upm.es    7/8
discussion and future works
              smart space behaviour personalization
              ⁷   Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
                  objects through mobile devices. Proc. of the 9th ACM International Symposium on
                  Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011


               smart objects orchestration architecture
                  mobile-centred

               mobile-instrumented interaction method
                  mobile media content/services ↔interface device transfer

               specific use case
                  ‘smart night table’


  future extensions
   extending the set of interaction actions to perform
    (generic approach)




Smart Environments and Ambient Intelligence – SENAmI 2013                    luca.bergesio@grpss.ssr.upm.es    7/8
discussion and future works
              smart space behaviour personalization
              ⁷   Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
                  objects through mobile devices. Proc. of the 9th ACM International Symposium on
                  Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011


               smart objects orchestration architecture
                  mobile-centred

               mobile-instrumented interaction method
                  mobile media content/services ↔interface device transfer

               specific use case
                  ‘smart night table’


  future extensions
   extending the set of interaction actions to perform
    (generic approach)

   interface device interaction
    e.g., mobile-device movement detection, Kinect, etc.



Smart Environments and Ambient Intelligence – SENAmI 2013                    luca.bergesio@grpss.ssr.upm.es    7/8
discussion and future works
              smart space behaviour personalization
              ⁷   Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
                  objects through mobile devices. Proc. of the 9th ACM International Symposium on
                  Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011


               smart objects orchestration architecture
                  mobile-centred

               mobile-instrumented interaction method
                  mobile media content/services ↔interface device transfer

               specific use case
                  ‘smart night table’


  future extensions
   extending the set of interaction actions to perform
    (generic approach)

   interface device interaction
    e.g., mobile-device movement detection, Kinect, etc.

   context-responsiveness enhancements

Smart Environments and Ambient Intelligence – SENAmI 2013                    luca.bergesio@grpss.ssr.upm.es    7/8
any question?




Smart Environments and Ambient Intelligence – SENAmI 2013   luca.bergesio@grpss.ssr.upm.es   8/8

[SENAmI'13] PERSEO - a system to personalize the environment response through smart phones and objects

  • 1.
    Grupo de Procesadode Datos y Simulación ETSI de Telecomunicación Universidad Politécnica de Madrid PERSEO: a system to Personalize the Environment Response through Smart phonEs and Objects SENAmI 2013 5th International Workshop on Smart Environments and Ambient Intelligence (presented by Josué Iglesias) Luca Bergesio, Íñigo Marquínez, Ana M. Bernardos, Juan A. Besada, José R. Casar luca.bergesio@grpss.ssr.upm.es
  • 2.
    contents  introduction and motivation  interaction method fundamentals  ‘smart night table’ use case  functionalities  architecture & implementation  example  video-demonstration  discussion and future works Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 1/8
  • 3.
    contents  introduction and motivation  interaction method fundamentals  ‘smart night table’ use case  functionalities  architecture & implementation  example  video-demonstration  discussion and future works Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 1/8
  • 4.
    introduction and motivation smart space Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 1/8
  • 5.
    introduction and motivation smart space smart objects Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 1/8
  • 6.
    introduction and motivation smart space behaviour smart personalization space smart objects Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 1/8
  • 7.
    introduction and motivation smart space behaviour smart personalization space focus general approach  mobile-instrumented smart interaction method objects particular implementation  device-objects coordination for content representation & management Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 1/8
  • 8.
    contents   introduction and motivation  interaction method fundamentals  ‘smart night table’ use case  functionalities  architecture & implementation  example  video-demonstration  discussion and future works Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8
  • 9.
    interaction method fundamentals mobile-instrumented concept trigger orchestrator SmartEnvironments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8
  • 10.
    interaction method fundamentals mobile-instrumented concept trigger orchestrator physical-based interaction maintaining traditional interaction paradigm Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8
  • 11.
    interaction method fundamentals mobile-instrumented concept trigger orchestrator mobile device – smart object coordinated response logical state of mobile device physical-based interaction maintaining traditional interaction paradigm Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8
  • 12.
    interaction method fundamentals mobile-instrumented concept trigger orchestrator device-hosted content sharing audio, video, pictures, etc. mobile device – smart object coordinated response logical state of mobile device physical-based interaction maintaining traditional interaction paradigm Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8
  • 13.
    interaction method fundamentals mobile-instrumented multiple & heterogeneous concept smart objects interfaces trigger TV, Hi-Fi, tablet, PC, photo frame, etc. orchestrator device-hosted content sharing audio, video, pictures, etc. mobile device – smart object coordinated response logical state of mobile device physical-based interaction maintaining traditional interaction paradigm Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8
  • 14.
    interaction method fundamentals rule-based reasoning ECA model Events – Conditions – Actions mobile-instrumented multiple & heterogeneous concept smart objects interfaces trigger TV, Hi-Fi, tablet, PC, photo frame, etc. orchestrator device-hosted content sharing audio, video, pictures, etc. mobile device – smart object coordinated response logical state of mobile device physical-based interaction maintaining traditional interaction paradigm Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8
  • 15.
    interaction method fundamentals context user context-responsive environment context analysis before executing etc. actions rule-based reasoning ECA model Events – Conditions – Actions mobile-instrumented multiple & heterogeneous concept smart objects interfaces trigger TV, Hi-Fi, tablet, PC, photo frame, etc. orchestrator device-hosted content sharing audio, video, pictures, etc. mobile device – smart object coordinated response logical state of mobile device physical-based interaction maintaining traditional interaction paradigm Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8
  • 16.
    contents   introduction and motivation   interaction method fundamentals  ‘smart night table’ use case  functionalities  architecture & implementation  example  video-demonstration  discussion and future works Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 3/8
  • 17.
    ‘smart night table’use case functionalities hi-fi audio digital photo frame television ‘smart night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 3/8
  • 18.
    ‘smart night table’use case functionalities hi-fi audio Voice (call) digital photo frame Photos Music Internet Videos television ‘smart night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 3/8
  • 19.
    ‘smart night table’use case functionalities Voice (call) Music hi-fi audio Voice (call) Photos digital photo frame Photos Music Internet Videos Videos television ‘smart Internet night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 3/8
  • 20.
    contents   introduction and motivation   interaction method fundamentals  ‘smart night table’ use case   functionalities  architecture & implementation  example  video-demonstration  discussion and future works Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8
  • 21.
    ‘smart night table’use case architecture & implementation Voice (call) Music hi-fi audio Voice (call) Photos digital photo frame Photos Music Internet Videos Videos television ‘smart Internet night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8
  • 22.
    ‘smart night table’use case architecture & implementation Android service Voice (call) Music hi-fi audio Voice (call) Photos digital photo frame Photos Music NFC tag Internet Videos Videos television ‘smart Internet night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8
  • 23.
    ‘smart night table’use case architecture & implementation Android service Voice (call) Music hi-fi audio Voice (call)  Photos Airplay Mirroring (Apple)  Intel Wireless Display (WiDi) digital photo frame  Wireless HD Music Photos  Miracast NFC tag DLNA Internet Videos (DMS) Digital Media Server (DMR) Digital Media Renderer Videos (DMC) Digital Media Controller television  maturity  interoperability (S.O., comm. tech., brands, etc.)  certified devices  SDK available ‘smart Internet night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8
  • 24.
    ‘smart night table’use case architecture & implementation DLNA Renderer Android (DMR) service ID: Renderer003 Voice (call) Music hi-fi audio DLNA Renderer (DMR) ID: Renderer001 DLNA Controller DLNA Renderer (DMC) (DMR) DLNA Server ID: Renderer004 Voice (call) M-DMS RESTful API Photos digital photo frame Photos Music NFC tag DLNA DLNA Renderer Videos Internet (DMR) (DMS) Digital Media Server ID: Renderer002 (DMR) Digital Media Renderer Videos (DMC) Digital Media Controller television ‘smart Internet night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8
  • 25.
    ‘smart night table’use case architecture & implementation Bluetooth DLNA Renderer Android (DMR) service ID: Renderer003 Voice (call) Music hi-fi audio DLNA Renderer (DMR) ID: Renderer001 DLNA Controller DLNA Renderer (DMC) (DMR) DLNA Server ID: Renderer004 Voice (call) M-DMS RESTful API Photos digital photo frame Photos Music NFC tag DLNA Renderer Videos Internet (DMR) ID: Renderer002 Videos television sockets ‘smart Internet night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8
  • 26.
    ‘smart night table’use case architecture & implementation Bluetooth DLNA Renderer Android (DMR) service ID: Renderer003 Voice (call) Music hi-fi audio DLNA Renderer (DMR) ID: Renderer001 DLNA Controller DLNA Renderer (DMC) (DMR) DLNA Server ID: Renderer004 Voice (call) M-DMS RESTful API Photos digital photo frame Photos Music NFC tag DLNA Renderer Videos Internet (DMR) ID: Renderer002 Videos television sockets ‘smart Internet night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8
  • 27.
    contents   introduction and motivation   interaction method fundamentals  ‘smart night table’ use case   functionalities   architecture & implementation  example  video-demonstration  discussion and future works Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 5/8
  • 28.
    ‘smart night table’use case example Bluetooth DLNA Renderer Android (DMR) service ID: Renderer003 Voice (call) Music hi-fi audio DLNA Renderer (DMR) ID: Renderer001 DLNA Controller DLNA Renderer (DMC) (DMR) DLNA Server ID: Renderer004 Voice (call) M-DMS RESTful API Photos digital photo frame Photos Music 1  Videos Internet NFC tag DLNA Renderer (DMR) ID: Renderer002 Videos television sockets ‘smart Internet night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 5/8
  • 29.
    ‘smart night table’use case example SELECT render Bluetooth 2 GET renderers 3 DLNA Renderer (DMR) Android service ID: Renderer003 Voice (call) Music hi-fi audio DLNA Renderer (DMR) ID: Renderer001 DLNA Controller DLNA Renderer (DMC) (DMR) DLNA Server ID: Renderer004 Voice (call) M-DMS RESTful API Photos digital photo frame Photos Music 1  Videos Internet NFC tag DLNA Renderer (DMR) ID: Renderer002 Videos television sockets ‘smart Internet night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 5/8
  • 30.
    ‘smart night table’use case example Bluetooth 2 3 DLNA Renderer (DMR) Android service ID: Renderer003 Voice (call) Music hi-fi audio DLNA Renderer 4 STOP 5 render (DMR) ID: Renderer001 DLNA Controller DLNA Renderer (DMC) (DMR) DLNA Server ID: Renderer004 Voice (call) M-DMS RESTful API Photos digital photo frame Photos Music 1  Videos Internet NFC tag DLNA Renderer (DMR) ID: Renderer002 Videos television sockets ‘smart Internet night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 5/8
  • 31.
    ‘smart night table’use case example Bluetooth 6 START playing 3 DLNA Renderer (DMR) Android service ID: Renderer003 Voice (call) Music hi-fi audio DLNA Renderer 4 5 (DMR) ID: Renderer001 DLNA Controller DLNA Renderer (DMC) (DMR) DLNA Server ID: Renderer004 Voice (call) M-DMS RESTful API Photos digital photo frame Photos Music 1 7 START playing  Videos Internet NFC tag DLNA Renderer (DMR) ID: Renderer002 Videos television sockets ‘smart Internet night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 5/8
  • 32.
    ‘smart night table’use case example Bluetooth 6 3 DLNA Renderer (DMR) Android service ID: Renderer003 Voice (call) Music hi-fi audio DLNA Renderer 4 5 (DMR) ID: Renderer001 DLNA Controller DLNA Renderer (DMC) (DMR) DLNA Server ID: Renderer004 Voice (call) M-DMS RESTful API Photos digital photo frame Photos Music 1 7  Videos Internet NFC tag DLNA Renderer (DMR) ID: Renderer002  Videos Videos 8 (continuous streaming) television sockets ‘smart Internet night table’ tablet Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 5/8
  • 33.
    contents   introduction and motivation   interaction method fundamentals   ‘smart night table’ use case   functionalities   architecture & implementation   example  video-demonstration  discussion and future works Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 6/8
  • 34.
    ‘smart night table’use case video-demonstration Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 6/8
  • 35.
    contents   introduction and motivation   interaction method fundamentals   ‘smart night table’ use case   functionalities   architecture & implementation   example   video-demonstration  discussion and future works Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8
  • 36.
    discussion and futureworks smart space behaviour personalization ⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart objects through mobile devices. Proc. of the 9th ACM International Symposium on Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011 Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8
  • 37.
    discussion and futureworks smart space behaviour personalization ⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart objects through mobile devices. Proc. of the 9th ACM International Symposium on Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011  smart objects orchestration architecture mobile-centred Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8
  • 38.
    discussion and futureworks smart space behaviour personalization ⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart objects through mobile devices. Proc. of the 9th ACM International Symposium on Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011  smart objects orchestration architecture mobile-centred  mobile-instrumented interaction method mobile media content/services ↔interface device transfer Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8
  • 39.
    discussion and futureworks smart space behaviour personalization ⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart objects through mobile devices. Proc. of the 9th ACM International Symposium on Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011  smart objects orchestration architecture mobile-centred  mobile-instrumented interaction method mobile media content/services ↔interface device transfer  specific use case ‘smart night table’ Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8
  • 40.
    discussion and futureworks smart space behaviour personalization ⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart objects through mobile devices. Proc. of the 9th ACM International Symposium on Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011  smart objects orchestration architecture mobile-centred  mobile-instrumented interaction method mobile media content/services ↔interface device transfer  specific use case ‘smart night table’ future extensions  extending the set of interaction actions to perform (generic approach) Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8
  • 41.
    discussion and futureworks smart space behaviour personalization ⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart objects through mobile devices. Proc. of the 9th ACM International Symposium on Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011  smart objects orchestration architecture mobile-centred  mobile-instrumented interaction method mobile media content/services ↔interface device transfer  specific use case ‘smart night table’ future extensions  extending the set of interaction actions to perform (generic approach)  interface device interaction e.g., mobile-device movement detection, Kinect, etc. Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8
  • 42.
    discussion and futureworks smart space behaviour personalization ⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart objects through mobile devices. Proc. of the 9th ACM International Symposium on Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011  smart objects orchestration architecture mobile-centred  mobile-instrumented interaction method mobile media content/services ↔interface device transfer  specific use case ‘smart night table’ future extensions  extending the set of interaction actions to perform (generic approach)  interface device interaction e.g., mobile-device movement detection, Kinect, etc.  context-responsiveness enhancements Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8
  • 43.
    any question? Smart Environmentsand Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 8/8