Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Distributing User
Interfaces
4th Workshop on Distributed User Interfaces and Multimodal Interaction
14th International Con...
Index
 Introduction
 Motivation
 Characterizing the Distribution of User Interfaces.
 User Interface Distribution Capa...
Why distributing user interfaces is
important?
Motivation
User interfaces
Multi-purpose mobile devices
User interfaces
Multi-purpose stationary devices
User interfaces
Specific devices
User Interface EcoSystems
See on deviceNotification
User interface
L. Terrenghi, A. Quigley y A. Dix, «A taxonomy for and ...
Distributed User Interfaces
vs.
Distributing User Interfaces
The User Interface Distribution Dichotomy
Case 1: Distributing the article section
of the Web Page
 Imagine this scenario…
 Suppose that you are browsing informat...
Case 1: Distributing the article section
of the Web Page
 When you arrive home you find your flatmates watching the Smart...
Case 1: Distributing the article section
of the Web Page
 To quickly share this information with them, you “transfer” the...
Case 2: Distributing a menu section of a
Web Page
 You are projecting a Web page on the Wall to show event participants
i...
Case 2: Distributing a menu section of a
Web Page
 As you want to control pages to be displayed by the projector remotely...
Case 3: Continuity
 You are filling a Web form using your Laptop, but you have to catch the train!
 “Transfer” (Distribu...
Characterization of the User Interface
Distribution
 How do we describe these characteristics on an Ecosystem of User Int...
How can we characterize the distribution
capabilities of user interfaces?
Metamodel defined in ECORE
enriched with OCL
Ped...
Graphical UI Distribution Model Editor
• Technology
• Eclipse Plugin
• EMF
• GMF
• Characteristics
• Graphical DSL
• Model...
Platform
 The combination of Hardware and OS that supports the user interface
 Tables / Smart phone running Android or i...
Interaction object
 The Interaction Object plays the same role as the Abstract Interaction Object
defined by the Cameleon...
Hosting
 Defines a relationship between two Interaction Objects (Host and Guest)
 It represents that the Guest Interacti...
Interaction Container
Interaction Component
 The Hosting relationship defines Interaction Objects as
 Interaction Contai...
Implementation
Interaction Surface
 Defines a relationship between an Interaction Container and a Platform
 It represent...
Interaction dependency
 Defines a relationship between two Interaction Surfaces (Master and Slave)
 It represents that t...
Paint .NET
Interaction Components
Paint .NET
Interaction Containers and Hostings
Paint .NET
Platforms, Implementations, Interaction
Surfaces and Interaction Dependencies
User Interface Distribution Capabilities
 Divisible
 A UI System is Divisible iff exists an Interaction Object that can ...
User Interface Distribution States
 We define the User Interface Distribution State as the organization of all the
Intera...
Reflexive UI Distribution Model Editor
Video
Quiz
 Capabilities
Divisible
Distributable
 Distribution States
Unified
Divided
Single Platform
Distributed
M. Manca y F...
Paint .NET
 Capabilities
Divisible
Distributable
 Distribution States
Unified
Divided
Single Platform
Distributed
GIMP 2.7
 Capabilities
Divisible
Distributable
 Distribution States
Unified
Divided
Single Platform
Distributed
WallShare
 Capabilities
Divisible
Distributable
 Distribution States
Unified
Divided
Single Platform
Distributed
!
Pedro...
RFID Panels
 Capabilities
Divisible
Distributable
 Distribution States
Unified
Divided
Single Platform
Distributed
Ricar...
Think about it…
Keep thinking …
See on deviceNotification
User interface
IF Distributed UI is an State
AND
IF Distributable is a Capability
THEN
DUI ≠ DeUI
Therefore, when we talk about Distribut...
Proxywork : Distributable User
Interfaces for the Web
 Proxywork allows users to distribute the user interface components...
Proxywork Primitives
 Connect / Disconnect (Device)
 Rename
 Clone
 Copy
 Migrate
Video
Copy Clone Migrate
Proxywork Limitations
 The Web page should be “well formed”
 Although Proxywork is able to distribute any component; the...
Conclusions
 Defines a metamodel to characterize the distribution of User Interfaces
 Graphical Model Editor to build an...
Future Work
 Distributable User Interfaces and Task modeling
 Adapting Distributable User Interfaces
 Controlling Distr...
Thank you!
Ricardo Tesoriero ricardo.Tesoriero@uclm.es
José A. Gallud jose.gallud@uclm.es
Pedro González Villanueva pedrog...
Upcoming SlideShare
Loading in …5
×

Distributing user interfaces. 4th Distributed User Interfaces Workshop on 14th International Conference on Web Engineering ICWE 2014

823 views

Published on

Presentation of "Distributing User Interfaces" as invited talk by Prof. Dr. Ricardo Tesoriero on the 4th Workshop on Distributed User Interfaces in the 14th International Conference on Web Engineering ICWE 2014

  • Be the first to comment

  • Be the first to like this

Distributing user interfaces. 4th Distributed User Interfaces Workshop on 14th International Conference on Web Engineering ICWE 2014

  1. 1. Distributing User Interfaces 4th Workshop on Distributed User Interfaces and Multimodal Interaction 14th International Conference on Web Engineering ICWE 2014 Prof. Dr. Ricardo Tesoriero ISE Research Group University of Castilla-La Mancha
  2. 2. Index  Introduction  Motivation  Characterizing the Distribution of User Interfaces.  User Interface Distribution Capabilities  User Interface Distribution States  Proxywork  Proxywork Distribution Primitives  Conclusions  Future Work. Open Issues
  3. 3. Why distributing user interfaces is important? Motivation
  4. 4. User interfaces Multi-purpose mobile devices
  5. 5. User interfaces Multi-purpose stationary devices
  6. 6. User interfaces Specific devices
  7. 7. User Interface EcoSystems See on deviceNotification User interface L. Terrenghi, A. Quigley y A. Dix, «A taxonomy for and analysis of multi-person-display ecosystems,» Personal Ubiquitous Comput., vol. 13, nº 8, pp. 583-598, 2009.
  8. 8. Distributed User Interfaces vs. Distributing User Interfaces The User Interface Distribution Dichotomy
  9. 9. Case 1: Distributing the article section of the Web Page  Imagine this scenario…  Suppose that you are browsing information on the Internet using your Smartphone  You go to your favorite Web Site and you start reading an interesting article while you are on the bus, train or metro on your way home
  10. 10. Case 1: Distributing the article section of the Web Page  When you arrive home you find your flatmates watching the Smart TV  As you tell them about the article you have read, they suddenly got interested in it.
  11. 11. Case 1: Distributing the article section of the Web Page  To quickly share this information with them, you “transfer” the article from the Smartphone to the SmartTV  You DISTRIBUTE the article section of the Web page from the Smartphone to the Smart TV
  12. 12. Case 2: Distributing a menu section of a Web Page  You are projecting a Web page on the Wall to show event participants information  You use the laptop keyboard and pad to get the information
  13. 13. Case 2: Distributing a menu section of a Web Page  As you want to control pages to be displayed by the projector remotely, you “transfer” the menu section to the Smartphone
  14. 14. Case 3: Continuity  You are filling a Web form using your Laptop, but you have to catch the train!  “Transfer” (Distribute) the Web form of the Web Page to the Smartphone (including the information you have already entered when using the Laptop)
  15. 15. Characterization of the User Interface Distribution  How do we describe these characteristics on an Ecosystem of User Interfaces?  There are many models to describe the User Interface. However, most of them (i.e. AUI Model of the CRF or IFML) employ a Tree based structure where Interaction Objects have a single parent, if any (root).  How can we characterize the user interface distribution if a component can be hosted by more than one container? G. Calvary, J. Coutaz, L. Bouillon, M. Florins, Q. Limbourg, L. Marucci, F. Paternò, C. Santoro, N. Souchon, D. Thevenin y J. Vanderdonckt, «The CAMELEON reference framework,» Deliverable 1.1, CAMELEON Project, 03 Septiembre 2002. [On Line]. Available: http://www.w3.org/2005/Incubator/model-based- ui/wiki/Cameleon_reference_framework. [last access: 12/06/13]. OMG. IFML: The Interaction Flow Modeling Language. URL= http://www.ifml.org/
  16. 16. How can we characterize the distribution capabilities of user interfaces? Metamodel defined in ECORE enriched with OCL Pedro G. Villanueva, Ricardo Tesoriero, José A. Gallud. Revisiting the Concept of Distributed User Interfaces. Distributed User Interfaces: Usability and Collaboration. Springer. Human–Computer Interaction Series 2013, ISBN 978-1-4471-5499- 0, pp 1-15.
  17. 17. Graphical UI Distribution Model Editor • Technology • Eclipse Plugin • EMF • GMF • Characteristics • Graphical DSL • Model Validation • UI Distribution Capabilities Video
  18. 18. Platform  The combination of Hardware and OS that supports the user interface  Tables / Smart phone running Android or iOS  Laptops running Windows or Linux  Etc.  What about the Web?  The Web Browser is considered as part of the platform  Two Web Browsers running in the same machine are two different platforms  Two Web Browser Tabs (even in separate Windows) are the same platform  What about multiple monitors  Monitors are considered devices that are connected to the same platform  N monitors connected to the same computer belong to a single platform
  19. 19. Interaction object  The Interaction Object plays the same role as the Abstract Interaction Object defined by the Cameleon framework.  On concrete implementations they may play the role of Windows, Panels, Text Fields, Buttons, etc. J. M. Vanderdonckt y F. Bodart, «Encapsulating knowledge for intelligent automatic interaction objects selection,» de INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems, Amsterdam, 1993. G. Calvary, J. Coutaz, L. Bouillon, M. Florins, Q. Limbourg, L. Marucci, F. Paternò, C. Santoro, N. Souchon, D. Thevenin y J. Vanderdonckt, «The CAMELEON reference framework,» Deliverable 1.1, CAMELEON Project, 03 Septiembre 2002. [On Line]. Available: http://www.w3.org/2005/Incubator/model-based- ui/wiki/Cameleon_reference_framework. [last access: 12/06/13].
  20. 20. Hosting  Defines a relationship between two Interaction Objects (Host and Guest)  It represents that the Guest Interaction Object can be hosted in the Host Interaction Object during the execution of the User Interface  A Guest Interaction Object can be hosted in more than one Host Interaction Object  All Guest Interaction Objects must be hosted in at least one Host Interaction Object during the User Interface lifetime.  The host may change during the User Interface lifetime
  21. 21. Interaction Container Interaction Component  The Hosting relationship defines Interaction Objects as  Interaction Containers  Panels, Layouts, Menus, Submenus, Tables, etc.  Interaction Components  Buttons, Labels, Entry Fields, Menu Items, RFID Tags  If an Interaction Object does not host any Interaction Object then it is an Interaction Component  If an Interaction Object hosts another Interaction Object then it “mutates” into an Interaction Container  Both Interaction Components and Interaction Container must be contained in another Interaction Object
  22. 22. Implementation Interaction Surface  Defines a relationship between an Interaction Container and a Platform  It represents that the Interaction Container is supported by a Platform  An Interaction Container is implemented by at most one Platform  An Interaction Container which is implemented by Platform “mutates” into an Interaction Surface  An Interaction Surface might not be hosted on any other Interaction Object.  Windows, Activities, RFID Panels, Pages, Views
  23. 23. Interaction dependency  Defines a relationship between two Interaction Surfaces (Master and Slave)  It represents that the lifetime of the Slave Interaction Surface depends on the lifetime of the Master Interaction Surface  Floating Toolbars depend on the Main Window
  24. 24. Paint .NET Interaction Components
  25. 25. Paint .NET Interaction Containers and Hostings
  26. 26. Paint .NET Platforms, Implementations, Interaction Surfaces and Interaction Dependencies
  27. 27. User Interface Distribution Capabilities  Divisible  A UI System is Divisible iff exists an Interaction Object that can be hosted on more than one Interaction Surface  Distributable  A UI System is Distributable iff exists at least one Interaction Object that can be hosted on at least two Interaction Surfaces implemented on different Platforms
  28. 28. User Interface Distribution States  We define the User Interface Distribution State as the organization of all the Interaction Objects that are part of a UI System at a given instant in time.  Unified State: A UISystem reaches the Unified State iff all Interaction Objects are hosted on the same Interaction Surface at a given time  Divided State: A UISystem reaches the Divided State iff it has at least two Interaction Surfaces which host at least one Interaction Object each at a given time.  Distributed State: A UI System reaches the Distributed State iff it defines at least two Interaction Surfaces that are hosted on different Platforms. As Interaction Surfaces are Interaction Contaners, they host at least one Interaction Component each at a given time.  Single Platform State: A UI System reaches the Single Platform State iff it all Interaction Objects that are part of the UI System are hosted on the a set of Interaction Surfaces that share the same Platform.
  29. 29. Reflexive UI Distribution Model Editor Video
  30. 30. Quiz  Capabilities Divisible Distributable  Distribution States Unified Divided Single Platform Distributed M. Manca y F. Paternò, «Distributed User Interfaces with MARIA,» de Distributed User Interfaces 2011 (DUI 2011), CHI 2011 Workshop, Vancouver, BC, Canada, 2011.
  31. 31. Paint .NET  Capabilities Divisible Distributable  Distribution States Unified Divided Single Platform Distributed
  32. 32. GIMP 2.7  Capabilities Divisible Distributable  Distribution States Unified Divided Single Platform Distributed
  33. 33. WallShare  Capabilities Divisible Distributable  Distribution States Unified Divided Single Platform Distributed ! Pedro González Villanueva, Ricardo Tesoriero, José A. Gallud: Multi-pointer and collaborative system for mobile devices. Mobile HCI 2010: 435-438. Pedro González Villanueva, José A. Gallud, Ricardo Tesoriero: WallShare: a multi-pointer system for portable devices. AVI 2010: 416
  34. 34. RFID Panels  Capabilities Divisible Distributable  Distribution States Unified Divided Single Platform Distributed Ricardo Tesoriero, Pedro G. Villanueva, Habib Moussa Fardoun, Gabriel Sebastian. “Distributed User Interfaces in Public Spaces using RFID-based Panels”. International Journal of Human-Computer Studies. ISSN: 1071-5819. Volume 72. Issue 1. Pages: 111–125. January 2014. Pedro G. Villanueva, Ricardo Tesoriero, José A. Gallud, Abdulrahman H. Altalhi. “A Framework to Develop Web Applications Based on RFID Panels”. International Journal of Universal Computer Science. ISSN: 0948-695x. Volume 19. Issue 12. Pages: 1792-1807. June 2013. !
  35. 35. Think about it…
  36. 36. Keep thinking … See on deviceNotification User interface
  37. 37. IF Distributed UI is an State AND IF Distributable is a Capability THEN DUI ≠ DeUI Therefore, when we talk about Distributed User Interfaces, we are really talking about Distributable User Interfaces!
  38. 38. Proxywork : Distributable User Interfaces for the Web  Proxywork allows users to distribute the user interface components of Web applications among a set of displays.  The distribution is controlled by the user through a set of primitives (i.e. show, hide, copy, move, etc.) attached to Web page components.  These primitives are automatically attached to Web page components on runtime by the Proxywork Web proxy.  Therefore, Web pages do not require any extra information to be distributed among different displays. Pedro González Villanueva, Ricardo Tesoriero and José. A. Gallud. Distributing web components in a display ecosystem using Proxywork. BCS-HCI '13. Proceedings of the 27th International BCS Human Computer Interaction Conference, Brunel University, London, UK, 9-13 September 2013, art. 28. URL=http://dl.acm.org/citation.cfm?id=2578048 Pedro González Villanueva. Distributable User Interfaces. PhD Thesis. University of Castilla-La Mancha. 2014
  39. 39. Proxywork Primitives  Connect / Disconnect (Device)  Rename  Clone  Copy  Migrate Video Copy Clone Migrate
  40. 40. Proxywork Limitations  The Web page should be “well formed”  Although Proxywork is able to distribute any component; the distribution is limited to specific Tags (DIVs)  HTTPS  User Interface Adaptation  Cross domain communication  Etc.
  41. 41. Conclusions  Defines a metamodel to characterize the distribution of User Interfaces  Graphical Model Editor to build and validate User Interface Distribution models  Reflexive Model Editor that characterizes User Interfaces  Set the difference between User Interface capabilities and states to redefine the DUI concept as DeUI  Proxywork transforms Web Pages into Distributable User Interfaces
  42. 42. Future Work  Distributable User Interfaces and Task modeling  Adapting Distributable User Interfaces  Controlling Distributable User Interfaces  Etc.
  43. 43. Thank you! Ricardo Tesoriero ricardo.Tesoriero@uclm.es José A. Gallud jose.gallud@uclm.es Pedro González Villanueva pedrogovi@uclm.es

×