RelateGateways An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services Master Thesis, Dominique Guinard, Fribourg 2007 Supervisors: Hans Gellersen, Denis Lalanne, Rolf Ingold.

Context Lancaster, Northern UK: Pop: 30’000 (100’000 when including the sheeps). Infolab 21: 250 IT researchers. Ubicomp Group. / 35

Lancaster, Northern UK:

Pop: 30’000

(100’000 when including the sheeps).

Infolab 21:

250 IT researchers.

Ubicomp Group.

Today’s Menu… Introduction, Aim E2ESD Model Spatial Discovery Video Network and Service Discovery Invocation and Interoperability Cross-device Interactions Evaluation Conclusion / 35

Introduction, Aim

E2ESD Model

Spatial Discovery

Video

Network and Service Discovery

Invocation and Interoperability

Cross-device Interactions

Evaluation

Conclusion

Introduction: Bob’s Adventure Bob, researcher in biology at Unifr talk at Lancs. Bob needs to print his presentation. W here is the printer? / 35 ? T - 30 minutes

Bob, researcher in biology at Unifr talk at Lancs.

Bob needs to print his presentation.

W here is the printer?

Introduction: Bob’s Adventure Bob finds someone who knows someone who met someone who knows where the printer is located! Hem, Bob feels less cool. / 35 ? T - 10 minutes

Bob finds someone who knows someone who met someone who knows where the printer is located!

Hem, Bob feels less cool.

Introduction: Bob’s Adventure Bob now needs: To find the printer’s drivers. Install the printer. Get the right to access it. Find out about the printer’s properties and accepted formats. Etc… … print the document! Arg, Bob doesn’t feel cool anymore! / 35 T - 30 seconds

Bob now needs:

To find the printer’s drivers.

Install the printer.

Get the right to access it.

Find out about the printer’s properties and accepted formats.

Etc…

… print the document!

Arg, Bob doesn’t feel cool anymore!

Introduction: Summary Mobile users can benefit from access to pervasive services. Network and service discovery technologies facilitate spontaneous connections. However, these approaches are not user centric: Difficult for users to identify services; Lack of simple and natural interaction techniques: => Lack of spontaneity in interaction… / 35 Introduction

Mobile users can benefit from access to pervasive services.

Network and service discovery technologies facilitate spontaneous connections.

However, these approaches are not user centric:

Difficult for users to identify services;

Lack of simple and natural interaction techniques:

=> Lack of spontaneity in interaction…

Aim Using the mobile device to: Identify services available in the user’s immediate environment. Consume the services in a natural and standard manner. Need to: Include users in the discovery process. Address identification, discovery and invocation. / 35 Introduction

Using the mobile device to:

Identify services available in the user’s immediate environment.

Consume the services in a natural and standard manner.

Need to:

Include users in the discovery process.

Address identification, discovery and invocation.

Today’s Menu… Introduction, Aim E2ESD Model Spatial Discovery Video Network and Service Discovery Invocation and Interoperability Cross-device Interactions Evaluation Conclusion / 35

Introduction, Aim

E2ESD Model

Spatial Discovery

Video

Network and Service Discovery

Invocation and Interoperability

Cross-device Interactions

Evaluation

Conclusion

End-to-End Service Discovery Model (E2ESD) / 35 E2ESD Model

Layer 1: Spatial Discovery / 35

Gateways User Interface Widgets as access points to the services: Gateways. Users visually discover the services: Mapping the user’s view of his environment on the mobile desktop. UI as a compass. / 35 Spatial Discovery

Widgets as access points to the services:

Gateways.

Users visually discover the services:

Mapping the user’s view of his environment on the mobile desktop.

UI as a compass.

Implementation Small Java Swing windows representing the service providers. At the screen periphery, integrated to the desktop. Two interaction modes: Drag-and-Drop. Click. / 35 Spatial Discovery

Small Java Swing windows representing the service providers.

At the screen periphery, integrated to the desktop.

Two interaction modes:

Drag-and-Drop.

Click.

Spatial Context Spatial context initialy delivered to the mobile client by a Wizard of Oz interface. Introducing Relate: EU-founded project. Ad-hoc sensor network. Providing relative positioning. Achieved: first extend to provide real-time positioning data to the compass UI. / 35 Spatial Discovery

Spatial context initialy delivered to the mobile client by a Wizard of Oz interface.

Introducing Relate:

EU-founded project.

Ad-hoc sensor network.

Providing relative positioning.

Achieved: first extend to provide real-time positioning data to the compass UI.

Spatial Context: Deployement To provide the user interface with spatial context we need: A USB Dongle/Brick on the mobile device. An autonomous Dot on each service provider. / 35 Spatial Discovery

To provide the user interface with spatial context we need:

A USB Dongle/Brick on the mobile device.

An autonomous Dot on each service provider.

Video Demonstration / 35

Layer 2: Network & Service Discovery / 35

Extending Relate: Before / 35 Network Discovery

Extending Relate: After / 35 Network Discovery

Layer 3: Invocation and Interoperability / 35 Interoperability

Modeling the Services A service is composed of: A ServiceProvider, enclosing the service logic. A ServiceRequester containing enough information to invoke the service. 2 types of services: Push service (can be invoked using a Universal Requester). Pull-and-Push service. / 35 Interoperability

A service is composed of:

A ServiceProvider, enclosing the service logic.

A ServiceRequester containing enough information to invoke the service.

2 types of services:

Push service (can be invoked using a Universal Requester).

Pull-and-Push service.

« Plug and Play » Invocation The semantics of Pull-and-Push Services is unknown to the mobile client before discovery. Packets of Mobile Code (ServiceRequesters, descriptions, icon, etc.) are downloaded and dynamically loaded on the mobile device. / 35 Interoperability

The semantics of Pull-and-Push Services is unknown to the mobile client before discovery.

Packets of Mobile Code (ServiceRequesters, descriptions, icon, etc.) are downloaded and dynamically loaded on the mobile device.

Cross-Device Interactions Using the computing power as a service. Use-cases: Collaborative tasks Cross-device interactions for single user. Extending the EBL toolkit: Cooperation with UCL (Université Catholique de Louvain la Neuve, HCI Lab). / 35 Interoperability

Using the computing power as a service.

Use-cases:

Collaborative tasks

Cross-device interactions for single user.

Extending the EBL toolkit:

Cooperation with UCL (Université Catholique de Louvain la Neuve, HCI Lab).

Today’s Menu… Introduction, Aim E2ESD Model Spatial Discovery Video Network and Service Discovery Invocation and Interoperability Cross-device Interactions Evaluation Conclusion / 35

Introduction, Aim

E2ESD Model

Spatial Discovery

Video

Network and Service Discovery

Invocation and Interoperability

Cross-device Interactions

Evaluation

Conclusion

Settings Test run in Lancaster, formative user study in Munich. Total of 20 users. 3 “service enabled” devices within a large office. / 35 Evaluation

Test run in Lancaster, formative user study in Munich.

Total of 20 users.

3 “service enabled” devices within a large office.

Qualitative Results Most cited benefits: No installation, no configuration: saves time. Ease of interaction with the services: drag and drop. Dynamic spatial arrangement of the gateways: making the UI more natural, especially useful in unknown places. Suggested a number of UI improvements. And services to implement. / 35 Evaluation

Most cited benefits:

No installation, no configuration: saves time.

Ease of interaction with the services: drag and drop.

Dynamic spatial arrangement of the gateways: making the UI more natural, especially useful in unknown places.

Suggested a number of UI improvements. And services to implement.

Today’s Menu… Introduction, Aim E2ESD Model Spatial Discovery Video Network and Service Discovery Invocation and Interoperability Cross-device Interactions Evaluation Conclusion / 35

Introduction, Aim

E2ESD Model

Spatial Discovery

Video

Network and Service Discovery

Invocation and Interoperability

Cross-device Interactions

Evaluation

Conclusion

Bootstraping Spontaneous Mobile Spatial Interactions Implementation of the E2ESD bundled into a single, runnable application offering: A spatial user interface « toolkit » (MVC based + contextual rules-engine). A Network and Service Discovery system. An Invocation and Interoperability system. An architecture for prototyping pervasive services. Various simulation and debugging tools. A framework (SOA) supporting the rapid prototyping of mobile spatial interactions. / 35 Conclusion

Implementation of the E2ESD bundled into a single, runnable application offering:

A spatial user interface « toolkit » (MVC based + contextual rules-engine).

A Network and Service Discovery system.

An Invocation and Interoperability system.

An architecture for prototyping pervasive services.

Various simulation and debugging tools.

A framework (SOA) supporting the rapid prototyping of mobile spatial interactions.

Quantitative Outputs Dissemination 3 accepted workshop papers: MSI @ CHI 07, Permid @ Pervasive 07, SensorNet 07 2 conference papers submited Ubicomp 07 (Demo Paper), LoCA 07 Prototyping Framework: ~16’000 lines of code. ~160 classes (to consolidate!). To few hours of sleep… / 35 Conclusion

Dissemination

3 accepted workshop papers:

MSI @ CHI 07, Permid @ Pervasive 07, SensorNet 07

2 conference papers submited

Ubicomp 07 (Demo Paper), LoCA 07

Prototyping Framework:

~16’000 lines of code.

~160 classes (to consolidate!).

To few hours of sleep…

Open Questions Scalability of the application: In terms of user interface. In terms of prototyping framework. User study is formative: Need for a comparative study as well. Need to evaluate the framework and its use for the prototyping of mobile spatial applications. Security concerns: Spontaneity is nice but it leaves a number of doors open for attackers. / 35 Conclusion

Scalability of the application:

In terms of user interface.

In terms of prototyping framework.

User study is formative:

Need for a comparative study as well.

Need to evaluate the framework and its use for the prototyping of mobile spatial applications.

Security concerns:

Spontaneity is nice but it leaves a number of doors open for attackers.

Questions ? Thanks for your attention…. Project’s homepage: http://ubicomp.lancs.ac.uk/relategateways Contact: [email_address] / 35 Conclusion

Thanks for your attention….

Project’s homepage: http://ubicomp.lancs.ac.uk/relategateways

Contact: [email_address]