Successfully reported this slideshow.
1. Juni 2011<br />Model-based Design of Interactions that can bridge RealitiesThe Augmented “Drag-and-Drop”Sebastian Feuer...
The Augmented “Drag-and-Drop”<br />2<br />1. Juni 2011<br />Basic Question<br />How to model Interactions that span Modes ...
The Augmented “Drag-and-Drop<br />3<br />1. Juni 2011<br />Which Media ?<br />http://flic.kr/p/6A7D3E<br />
The Augmented “Drag-and-Drop”<br />4<br />1. Juni 2011<br />Which Modes?<br />http://flic.kr/p/7sZ6ho<br />http://flic.kr/...
The Augmented “Drag-and-Drop<br />5<br />1. Juni 2011<br />How ?<br />Device ; Modality<br />
The Augmented “Drag-and-Drop<br />6<br />1. Juni 2011<br />Why modeling?<br />One specification that<br />is declarative a...
The Augmented “Drag-and-Drop<br />7<br />1. Juni 2011<br />Use Case: <br />Augmented Drag and Drop<br />Buying Furniture <...
Video: Connecting the Web with AR<br />The Augmented “Drag-and-Drop<br />8<br />1. Juni 2011<br />VIDEO<br />Mouse-based D...
The Augmented “Drag-and-Drop<br />9<br />1. Juni 2011<br />How to model?<br />Models<br />Interactors(Abstract & Concrete ...
The Augmented “Drag-and-Drop<br />10<br />1. Juni 2011<br />Abstract Media Model<br />
The Augmented “Drag-and-Drop<br />11<br />1. Juni 2011<br />Abstract Media Model<br />Single Choice<br />Aggregates a set ...
The Augmented “Drag-and-Drop<br />12<br />1. Juni 2011<br />Abstract Behavior Model<br />
Design of Multimodal Interaction<br />13<br />1. Juni 2011<br />Mode Model<br />(Example: Mouse)<br />
The Augmented “Drag-and-Drop<br />14<br />1. Juni 2011<br />Multimodal Mapping<br />The Drag-and-Drop<br />
The Augmented “Drag-and-Drop<br />15<br />1. Juni 2011<br />Conclusions & Future Work<br />What’s the advantage?<br />Deta...
Gesture-based Drag and Drop Control <br />The Augmented “Drag-and-Drop<br />16<br />1. Juni 2011<br />Cam<br />Jessica <br...
Gesture-based Drag and Drop Control <br />Design of Multimodal Interaction<br />17<br />1. Juni 2011<br />VIDEO<br />Gestu...
The Augmented “Drag-and-Drop<br />18<br />1. Juni 2011<br />The End …<br />Thank you for your attention!<br />Your questio...
The Augmented “Drag-and-Drop<br />19<br />1. Juni 2011<br />Abstract Model<br />Reference – Object or Property of an Objec...
The Augmented “Drag-and-Drop<br />20<br />1. Juni 2011<br />Multimodal Mapping<br />(Combining Mode with Media)<br />
Upcoming SlideShare
Loading in …5
×

Model-based Design of Interactions that can bridge Realities – The Augmented Drag-and-Drop

1,323 views

Published on

The development of Augmented Reality (AR)
applications is still driven by development at source-code level.
Although recent approaches focus to standardize AR
functionality, to our knowledge a declarative and model-driven
design (MDD) has not been applied for AR development so far.
MDD approaches have been successfully applied to model user
interfaces for a wide spectrum of modes (such as speech-
command interfaces, or remote controls) and media such as
HTML, 3D, and smart phone interfaces for instance. In this
paper we propose a MDD approach for modeling seamless
interaction between Web and AR interfaces. Therefore we
implemented a prototype of a web furniture shop that supports
arranging furniture in an augmented reality to prove our
approach. We then present our approach of modeling a reality
spanning Drag-and-Drop interaction between a 2D browser
and an AR scene. Finally, we discuss the issues that we were
confronted with to support reality spanning interaction like
switching interaction modes and coordinate systems and
present limitations that we have experienced with using other
control modes, like a Wii-Controller or glove-based gesture
detection.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Model-based Design of Interactions that can bridge Realities – The Augmented Drag-and-Drop

  1. 1. 1. Juni 2011<br />Model-based Design of Interactions that can bridge RealitiesThe Augmented “Drag-and-Drop”Sebastian Feuerstack, Allan C. M. de Oliveira, Regina B. Araujo<br />Universidade Federal de São CarlosDepartamento de Computacão<br />
  2. 2. The Augmented “Drag-and-Drop”<br />2<br />1. Juni 2011<br />Basic Question<br />How to model Interactions that span Modes and Media ?<br />
  3. 3. The Augmented “Drag-and-Drop<br />3<br />1. Juni 2011<br />Which Media ?<br />http://flic.kr/p/6A7D3E<br />
  4. 4. The Augmented “Drag-and-Drop”<br />4<br />1. Juni 2011<br />Which Modes?<br />http://flic.kr/p/7sZ6ho<br />http://flic.kr/p/6eeLwg<br />
  5. 5. The Augmented “Drag-and-Drop<br />5<br />1. Juni 2011<br />How ?<br />Device ; Modality<br />
  6. 6. The Augmented “Drag-and-Drop<br />6<br />1. Juni 2011<br />Why modeling?<br />One specification that<br />is declarative and exact to discuss & execute<br />Is located between Tool and Code<br />can be easily adapted or enhanced to different forms of interaction <br />
  7. 7. The Augmented “Drag-and-Drop<br />7<br />1. Juni 2011<br />Use Case: <br />Augmented Drag and Drop<br />Buying Furniture <br />Shopping Cart<br />Drag and Drop<br />Into my environment<br />
  8. 8. Video: Connecting the Web with AR<br />The Augmented “Drag-and-Drop<br />8<br />1. Juni 2011<br />VIDEO<br />Mouse-based Drag-and-Drop<br />
  9. 9. The Augmented “Drag-and-Drop<br />9<br />1. Juni 2011<br />How to model?<br />Models<br />Interactors(Abstract & Concrete Media, Mode) <br />Static: class diagramm, Behavior: state-chart<br />Mappings Mode-To-Media Synchronization<br />Custom Notation <br />
  10. 10. The Augmented “Drag-and-Drop<br />10<br />1. Juni 2011<br />Abstract Media Model<br />
  11. 11. The Augmented “Drag-and-Drop<br />11<br />1. Juni 2011<br />Abstract Media Model<br />Single Choice<br />Aggregates a set of Entities from that only one can be chosen at a time<br />Examples: Direction (leftorright), Shopping Cart<br />Further Properties: <br />Output tothe User<br />Container -> Aggregation,<br />Discrete,<br />thatare Inputs<br />Contains Single Choice Elements<br />
  12. 12. The Augmented “Drag-and-Drop<br />12<br />1. Juni 2011<br />Abstract Behavior Model<br />
  13. 13. Design of Multimodal Interaction<br />13<br />1. Juni 2011<br />Mode Model<br />(Example: Mouse)<br />
  14. 14. The Augmented “Drag-and-Drop<br />14<br />1. Juni 2011<br />Multimodal Mapping<br />The Drag-and-Drop<br />
  15. 15. The Augmented “Drag-and-Drop<br />15<br />1. Juni 2011<br />Conclusions & Future Work<br />What’s the advantage?<br />Detailed, declarative Modeling of Interactions (Media, Mode)<br />We can combine Modes & Media and bridge Realities<br />What`s still missing is a tool<br />What`s the advantage???<br />Lets see a demo with different Modes<br />
  16. 16. Gesture-based Drag and Drop Control <br />The Augmented “Drag-and-Drop<br />16<br />1. Juni 2011<br />Cam<br />Jessica <br />Pointing<br />Cam<br />Command<br />Monitor<br />
  17. 17. Gesture-based Drag and Drop Control <br />Design of Multimodal Interaction<br />17<br />1. Juni 2011<br />VIDEO<br />Gesture-based Drag-and-Drop<br />
  18. 18. The Augmented “Drag-and-Drop<br />18<br />1. Juni 2011<br />The End …<br />Thank you for your attention!<br />Your questions please …<br />
  19. 19. The Augmented “Drag-and-Drop<br />19<br />1. Juni 2011<br />Abstract Model<br />Reference – Object or Property of an Object to unambigously identify it<br />Examples: Name, Color, Number, Shape, Coordinate<br />Further Properties: <br />Discrete -> Distinguishable<br />Input -> bythe User <br />
  20. 20. The Augmented “Drag-and-Drop<br />20<br />1. Juni 2011<br />Multimodal Mapping<br />(Combining Mode with Media)<br />

×