1) The document presents OP, a programming toolkit that allows designers to conceptualize and prototype mixed physical-digital objects based on an interaction model. 2) OP includes software blocks that implement the concepts of the interaction model, like digital properties, linking modalities, languages, and devices. 3) The toolkit allows designers to systematically explore different aspects of mixed objects, like physical properties, input/output modalities, and compositions of links. This helps narrow the design space.

1. OP: A Novel Programming Model
for Integrated Design and Prototyping
of Mixed Objects
Céline Coutrix, Laurence Nigay
Laboratoire d’Informatique de Grenoble, France
1

2. Context
• Mixed Reality Interfaces
2

3. Problem
• Prototyping mixed physical-digital interfaces:
Different problems
- Lower technical difficulties
- Enable new users
- Adequacy to the design problem
3

4. Problem
• The design problem
[Bill Buxton. 2007. Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann
Publishers Inc., San Francisco, CA, USA]
4

5. Problem
• The design problem
[Bill Buxton. 2007. Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann
Publishers Inc., San Francisco, CA, USA]
5

6. Problem
• The design problem
[Bill Buxton. 2007. Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann
Publishers Inc., San Francisco, CA, USA]
6

7. Problem
• The design problem
[Bill Buxton. 2007. Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann
Publishers Inc., San Francisco, CA, USA]
7

8. Problem
• The design problem: Elaboration
Conceptual Practical
activity activity
[Bill Buxton. 2007. Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann
Publishers Inc., San Francisco, CA, USA]
8

9. Problem
• The design problem: Elaboration
Interaction Prototyping
Models toolkits
[Bill Buxton. 2007. Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann
Publishers Inc., San Francisco, CA, USA]
9

10. Problem
• Gap between interaction models and
prototyping toolkits
Interaction Prototyping
Models toolkits
- Slows down the elaboration of ideas
10

11. Problem
• Lower technical difficulties
• Enable new users
• Adequacy to the design problem
11

12. Contribution
• OP (Object Prototyping) Toolkit
- Library of software elements
• Materializing the concepts of an
interaction model
• Using Qt signals and slots mechanism
Mixed Interaction
OP toolkit
Model
12

13. OP
• Underlying interaction model:
The Mixed Interaction Model
• Why this one?
- Has been compared to others and
proved interesting
- Not easily used though, like the others
13

14. OP
• Concepts implemented by OP
- Core concept: the mixed (physical-digital)
object
14

15. OP
• Concepts implemented by OP
- Core concept: the mixed (physical-digital)
object
15

16. OP
• Concepts implemented by OP
Digital Properties is exposed to light
Physical Properties shape, colors, weight, markers, etc.
16

17. OP
• Concepts implemented by OP
Digital Properties is exposed to light
Linking Modalities
Physical Properties shape, colors, weight, markers, etc.
17

18. OP
• Concepts implemented by OP
Digital Properties is exposed to light
Linking Languages threshold impulse
Linking Devices light
sensor
LED
Physical Properties shape, colors, weight, markers, etc.
18

19. OP
• Concepts implemented by OP
Digital Properties is exposed to light
Linking Languages threshold impulse
Linking Devices light
sensor
LED
Physical Properties shape, colors, weight, markers, etc.
19

20. OP
• Concepts implemented by OP
Digital Properties
Composition of modalities Complementarity
Linking Languages
Linking Devices
Physical Properties shape, colors, weight, markers, etc.
20

21. OP
• Concepts implemented by OP
Digital Properties is exposed to light
Composition of modalities
threshold impulse
Linking Languages
light
LED
sensor
Linking Devices
Physical Properties shape, colors, weight, markers, etc.
21

22. OP
• Software elementary blocks for:
generic
Digital Properties digital
property
Composition of modalities complementarity ...
Linking Languages
ARToolKit
threshold computer
vision
...
Phidget Interface-
Linking Devices camera sensor/
controller
Z sensor/
controller ...
Physical Properties
22

23. OP
• Inserting the use of an interaction model
into design practice: example
23

24. 24

25. 25

26. 26

27. is exposed to light
threshold impulse
light
LED
sensor
shape, colors, weight,
markers, etc.
27

28. is exposed to light
threshold impulse
light
LED
sensor
shape, colors, weight,
markers, etc.
28

29. is exposed to light
threshold impulse
light
LED
sensor
shape, colors, weight,
markers, etc.
29

30. is exposed to light
threshold impulse
light
LED
sensor
shape, colors, weight,
markers, etc.
30

31. is exposed to light
threshold impulse
light
LED
sensor
shape, colors, weight,
markers, etc.
31

32. is exposed to light
threshold impulse
light
LED
sensor
shape, colors, weight,
markers, etc.
32

33. is exposed to light
threshold impulse
light
LED
sensor
shape, colors, weight,
markers, etc.
33

34. 34

35. is exposed to light
threshold impulse
light
LED
sensor
shape, colors, weight,
markers, etc.
35

36. is exposed to light
threshold impulse
light
LED
sensor
shape, colors, weight,
markers, etc.
36

37. is exposed to light
threshold impulse
light
LED
sensor
shape, colors, weight,
markers, etc.
37

38. is exposed to light
threshold impulse
light
LED
sensor
shape, colors, weight,
markers, etc.
38

39. is exposed to light
threshold impulse
light
LED
sensor
shape, colors, weight,
markers, etc.
39

40. is exposed to light
threshold impulse
light
LED
sensor
shape, colors, weight,
markers, etc.
40

41. OP
41

42. Benefits
• For design: Systematic exploration of the
design space
- Physical properties
- Linking devices
- Linking languages
- Composition of linking modalities
- Digital Properties
42

43. Benefits
• Exploration of input linking devices
- Phidgets →Interface-Z
43

44. Benefits
• Exploration of input linking devices
- Phidgets →Interface-Z
44

45. Benefits
• Exploration of input linking languages
- Threshold
45

46. Benefits
• Exploration of input linking languages
- Threshold + Repetition
46

47. Benefits
• Exploration of input linking languages
- Threshold + Repetition
47

48. Benefits
• Exploration of output linking modality
- Impulse + LED
48

49. Benefits
• Exploration of output linking modality
- Impulse + sound
49

50. Benefits
• Exploration of output linking language
- Repetition + Impulse
50

51. Conclusion
• For design
- Designers can explore a mixed object
similarly to the underlying Mixed
Interaction Model
Mixed Interaction
OP toolkit
Model
51

52. Future Work
• Lower technical difficulties
• Adequacy to the design problem
• Enable new users
52

53. Future Work
• Enable new users
53

54. OP: A Novel Programming Model
for Integrated Design and Prototyping
of Mixed Objects
Céline Coutrix, Laurence Nigay
Laboratoire d’Informatique de Grenoble, France
54