User Interface Engineering Team
CLIPS-IMAG Laboratory, University of Grenoble




Mixed Reality:
A Model of Mixed Interact...
Introduction: Mixed Systems	



•  Merging
  Physical and
  Digital
  Worlds	

•  In many
  application
  fields	





    ...
Introduction: Designers Needs

 A framework for :	


 Understanding and capitalizing on
                                  ...
Outline

•  Mixed Interaction Model
•  Descriptive Power
•  Generative and Comparative Powers
•  Conclusion and Future Wor...
Outline

•  Mixed Interaction Model
  •  Mixed object

  •  Mixed interaction

•  Descriptive Power
•  Generative and Comp...
Example




          5
Mixed Interaction Model: Mixed Object


                    mixed object	





          tool	

                     task ...
Mixed Interaction Model: Mixed Object

   physical object	

             mixed object	

              digital object	


  ...
Mixed Interaction Model: Mixed Object


  •  Linking modality	

                                  mixed object	



       ...
Mixed Interaction Model: Mixed Object




                                   mixed drawing	




                     camer...
Mixed Interaction Model: Mixed Object




                                        a music bottle	





                   ...
Mixed Interaction Model: Mixed Object
                               mixed object	


                physical 	

         ...
Mixed Interaction Model: Mixed Interaction



                                 tool	

                                    ...
Mixed Interaction Model: Mixed Interaction




 •  Interaction modality m = (d, l)	





                                 ...
Mixed Interaction Model: Mixed Interaction

                                  mixed tool	


                       input	
...
Mixed Interaction Model: Mixed Interaction




                                         15
Mixed Interaction Model: Mixed Interaction

                        mixed tool	


                                        ...
Outline

•  Mixed Interaction Model
•  Descriptive Power
•  Generative and Comparative Powers
•  Conclusion and Future Wor...
Descriptive Power



•  Schemes for classification and better
  understanding of mixed systems




                       ...
Outline

•  Mixed Interaction Model
•  Descriptive Power
•  Generative and Comparative Powers
•  Conclusion and Future Wor...
RAZZLE: Case Study




                     20
RAZZLE: Case Study




                                       mixed piece	


                   location	

   numeric	

  ...
Generative Power
               mixed tool	





       ?	

          ?	

      ?	





                                  ...
RAZZLE: Gestural Modality




                            23
RAZZLE: Gestural Modality
                                 mixed tool	


                                                 ...
RAZZLE: Tactile Modality




                           25
RAZZLE: Tactile Modality
                                    mixed tool	


                                               ...
Outline

•  Mixed Interaction Model
•  Descriptive Power
•  Generative and Comparative Powers
•  Conclusion and Future Wor...
Comparative Power

• Criteria within the model for assessing
multiple design alternatives
  •  Observability

  •  Spatial...
Comparative Power: Observability

                                       mixed tool	


                            input	
...
Comparative power: Observability
                                  mixed tool	


                                         ...
Comparative Power: Observability
                                    mixed tool	


                                       ...
Comparative Power: Spatial Continuity

                                       mixed tool	


                            in...
Comparative Power: Spatial Continuity
                                           mixed tool	


                           ...
Comparative Power: Spatial Continuity
                                            mixed tool	


                          ...
Outline

•  Mixed Interaction Model
•  Descriptive Power
•  Generative and Comparative Powers
•  Conclusion and Future Wor...
Conclusion


•  Definition of a Mixed Interaction Model which	

   •  Capitalizes existing approaches to describe mixed
   ...
Future Work




•  Improving comparative power by identifying more
  criteria	





                                      ...
Future Work

•  Providing a development environment
AVI06 mixed reality coutrix
Upcoming SlideShare
Loading in...5
×

AVI06 mixed reality coutrix

194

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
194
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

AVI06 mixed reality coutrix

  1. 1. User Interface Engineering Team CLIPS-IMAG Laboratory, University of Grenoble Mixed Reality: A Model of Mixed Interaction Céline Coutrix, Laurence Nigay
  2. 2. Introduction: Mixed Systems •  Merging Physical and Digital Worlds •  In many application fields 1
  3. 3. Introduction: Designers Needs A framework for : Understanding and capitalizing on Descriptive power existing Mixed Systems Creating designs Generative power Assessing multiple design Comparative power alternatives and increasing usability 2
  4. 4. Outline •  Mixed Interaction Model •  Descriptive Power •  Generative and Comparative Powers •  Conclusion and Future Work 3
  5. 5. Outline •  Mixed Interaction Model •  Mixed object •  Mixed interaction •  Descriptive Power •  Generative and Comparative Powers •  Conclusion and Future Work 4
  6. 6. Example 5
  7. 7. Mixed Interaction Model: Mixed Object mixed object tool task object 6
  8. 8. Mixed Interaction Model: Mixed Object physical object mixed object digital object physical physical digital digital properties properties properties ? properties •  Interaction modality m = (d, l) •  speech = (microphone, natural language) 7
  9. 9. Mixed Interaction Model: Mixed Object •  Linking modality mixed object input acquired physical input device language data physical digital properties properties output generated output device physical language data 8
  10. 10. Mixed Interaction Model: Mixed Object mixed drawing camera image computer vision physical properties shapes projector image selection 9
  11. 11. Mixed Interaction Model: Mixed Object a music bottle lighting pitch and music physical screen colored volume part properties table sequence analysis loud audio identity speakers sequence 10
  12. 12. Mixed Interaction Model: Mixed Object mixed object physical digital properties properties Linking modalities tool task object 11
  13. 13. Mixed Interaction Model: Mixed Interaction tool elementary task action response reaction task object feedback 12
  14. 14. Mixed Interaction Model: Mixed Interaction •  Interaction modality m = (d, l) 13
  15. 15. Mixed Interaction Model: Mixed Interaction mixed tool input acquired input device physical language data physical digital properties properties generated output output physical language device data interaction interaction output input language language task object 14
  16. 16. Mixed Interaction Model: Mixed Interaction 15
  17. 17. Mixed Interaction Model: Mixed Interaction mixed tool computer camera image button mixed tool vision pressed paper word, button place on drawing translation drawing 16
  18. 18. Outline •  Mixed Interaction Model •  Descriptive Power •  Generative and Comparative Powers •  Conclusion and Future Work 17
  19. 19. Descriptive Power •  Schemes for classification and better understanding of mixed systems 18
  20. 20. Outline •  Mixed Interaction Model •  Descriptive Power •  Generative and Comparative Powers •  Conclusion and Future Work 19
  21. 21. RAZZLE: Case Study 20
  22. 22. RAZZLE: Case Study mixed piece location numeric geometric sensor values transforma° Identifier physical orienta° numeric geometric Location properties sensor values transforma° Orientation head 3D mounted image projection display 21
  23. 23. Generative Power mixed tool ? ? ? ? ? 22
  24. 24. RAZZLE: Gestural Modality 23
  25. 25. RAZZLE: Gestural Modality mixed tool computer camera image vision physical recognized properties gestures translation 24
  26. 26. RAZZLE: Tactile Modality 25
  27. 27. RAZZLE: Tactile Modality mixed tool input touchpad data language physical click properties translation 26
  28. 28. Outline •  Mixed Interaction Model •  Descriptive Power •  Generative and Comparative Powers •  Conclusion and Future Work 27
  29. 29. Comparative Power • Criteria within the model for assessing multiple design alternatives •  Observability •  Spatial continuity 28
  30. 30. Comparative Power: Observability mixed tool input acquired input device physical language data physical digital properties properties generated output output physical language device data interaction interaction output input language language task object input acquired input device physical language data physical digital properties properties generated output output physical language device data 29
  31. 31. Comparative power: Observability mixed tool input camera image language physical recognized properties gestures translation 30
  32. 32. Comparative Power: Observability mixed tool input touchpad data language physical click properties translation 31
  33. 33. Comparative Power: Spatial Continuity mixed tool input acquired input device physical language data physical digital properties properties generated output output physical language device data interaction interaction output input language language task object input acquired input device physical language data physical digital properties properties generated output output physical language device data 32
  34. 34. Comparative Power: Spatial Continuity mixed tool computer camera image vision physical recognized properties gestures translation mixed piece location numeric geometric sensor values transforma° Identifier physical orienta° numeric geometric Location properties sensor values transforma° Orientation head 3D mounted image projection display 33
  35. 35. Comparative Power: Spatial Continuity mixed tool input touchpad data language physical click properties translation mixed piece location numeric geometric sensor values transforma° Identifier physical orienta° numeric geometric Location properties sensor values transforma° Orientation head 3D mounted image projection display 34
  36. 36. Outline •  Mixed Interaction Model •  Descriptive Power •  Generative and Comparative Powers •  Conclusion and Future Work 35
  37. 37. Conclusion •  Definition of a Mixed Interaction Model which •  Capitalizes existing approaches to describe mixed systems •  Allows designers to create new interaction modalities •  Allows them to choose between multiple design alternatives 36
  38. 38. Future Work •  Improving comparative power by identifying more criteria 37
  39. 39. Future Work •  Providing a development environment

×