This paper proposes user-customizable passive control widgets, called MagGetz, which enable tangible interaction on and around mobile devices without requiring power or wireless connections. This is achieved by tracking and ana-lyzing the magnetic field generated by controllers attached on and around the device through a single magnetometer, which is commonly integrated in smartphones today. The proposed method provides users with a broader interaction area, customization input layouts, richer physical clues, and higher input expressiveness without the need for hardware modifications. We have presented a software toolkit and several applications using MagGetz.

1. MagGetz
Customizable Passive TangibleControllers on and around Conventional Mobile Devices
Presenter: S.J. Hwang
Ph.D. Student | 6thSemester

2. MOTIVATION
Physical objects in real life provide users with intuitive tactile clues through its physical artifacts

3. MOTIVATION

4. MOTIVATION
Lacks of tactile clues

5. Occlusioncaused by fingers
MOTIVATION

6. PREVIOUS WORKS
by adding other types of tangible objects
Researchers have attempt to increase this input space …
by expanding the interaction area

7. PREVIOUS WORKS
Tangible Interactions for Mobile Devices

8. PREVIOUS WORKS
Tangible Interactions for Mobile Devices: Using capacitive markers
CapStonesand ZebraWidgets(Chan et al., CHI’12)
CapWidgets
(Kratzetet al., CHIEA’11)
TUIC
(Yu et al., CHI’11)
Tangible Remote Controllers(Jansen et al., CHI’ 12)
Cilpon Gadgets
(Yu et al., UIST’11)

9. PREVIOUS WORKS
Portico
(Avrahamiet al., UIST’11)
Camera-based Tangible Controller(Kato, mobileHCI‘09)
Vision-based force sensor
(Kato, mobileHCI’ 12)
Tangible Joysticks
Physical tokens
Tangible Interactions for Mobile Devices: Using cameras

10. PREVIOUS WORKS
Technique (Author, Year)
Sensor used (property)
Limitations
P/A
(tangibles)
Cilpon Gadgets
(Yu et al., 2011)
Touchscreen(spatial)
Confinedwithin the device
Invisible when it is untouched
Passive
TUIC
(Yu et al., 2011)
Touchscreen
(spatial, frequency)
Confinedwithin the touchscreen
Occlusion problem
Need battery (active)
Invisible when untouched(passive)
Passive
/ Active
CapWidgets
(Kratzet al.,2011)
Touchscreen(spatial)
Occlusion problem
Confinedwithin the touchscreen
Invisible when untouched
Passive
Tangible Remote Controllers (Jansen et al., 2012)
CapStones
(Chanet al., 2012)
Camera-based Tangible Controller (Katoet al., 2009)
Camera (marker)
Controllers are fixed and limited
Need light
Passive
Vision based force sensor
(Sato et al., 2012)
Portico
(Avrahami et al.,2011)
Camera (color)
Bulky
Need light
Previous techniques to enable tangible interactions on mobile devices

11. PREVIOUS WORKS
Expanding the interaction area

12. PREVIOUS WORKS
Cheap;
Small;
Passive;
Permanent;
Detectable;

13. PREVIOUS WORKS
Magnetically Driven Input for Mobile Devices
Abracadabra
(Harrison et al., 2009)
MagiWrite
(Ketabdaret al., 2010)
Digital Music Performance (Ketabdaret al., 2011)
Nenya
(Ashbrooket al., 2011)
MagiTact
(Ketabdaret al., 2011)
GaussSense
(Liang et al., 2012)

14. PREVIOUS WORKS
Technique (Author, Year)
Task
Limitation
Abracadabra
(Harrison et al., 2009)
accurate selections for small screens
No tactile clue
MagiWrite
(Ketabdar et al., 2010)
writing system
No tactile clue
Digital Music Performance
(Ketabdaret al., 2011)
musical performance
No tactile clue
Nenya
(Ashbrooket al., 2011)
Menuselection
Need additional mechanicalhardware.
GaussSense
(Liang et al., 2012)
Pen interaction
Need additional mechanicalhardware.
(a board with 192 magnetic sensors and USB connection).
Malfunctioning for ferromagnetic materials.
Magnetically Driven Input Techniques

15. OUR METHOD : MAGGETZ
+
passive magnets
2. expand input area to avoid occlusions.
+
a magnetometer
3. using a magnetometer that already installed on current mobile devices.
1. provide richer tactile clues.
Multiple control widgets

16. OUR METHOD : MAGGETZ

17. IMPLEMENTATION

18. IMPLEMENTATION
The type and location of a magnet for each widget

19. BACKGROUND
The location of a magnet and visualization of magnetic fields for each control widgets.
Push button
Toggle
Switch
Slider
Joystick
Rotational
Wheel

20. OUR METHOD
Software
-
WedevelopedaMagGetztoolkitand3Dvisualizer(OpenGL) ontheAndroid4.0.4.(SHV-E160S,SamsungGalaxyNote).
-
Theinertialthree-axismagnetometersensesthemagneticfieldatasamplingfrequencyof100Hz.
-
Usedafixedsizebuffertoreducesamplingnoise.
Thisprocessgeneratesa3Dvectorthatreflectsthevariationsofthemagneticfieldaroundthedevice.

21. OUR METHOD
Software –linear type

22. OUR METHOD
Software –linear type
Wherep1isalastreferencepoint, p0isafirstreferencepoint,andx’isavectorthatnewpointxprojectedontop0p1.
x’

23. OUR METHOD
Software –curved type
c

24. OUR METHOD
Software –curved type
wherepiisithreferencepointfromstartingpointp0,kisasequenceofthereferencepointclosesttoanewmagneticinputpoint.
pk

25. OUR METHOD
Software –2D plane type
center
East
West
South
North

26. OUR METHOD
Software –2D plane type
Where pcis a center point, pkis a vector that is closest to a new input point xamong vectors between a center point pcand directional reference points (p0~pn), pknis the last point of vector pkstarting from pk0(=pc), and x’kis a vector that new point xprojected onto pcpkn. The direction of a joystick can be obtained by applying kof pkto equation

27. OUR METHOD
Software –multiple widgets
Button a
Button b

28. OUR METHOD
Software –multiple widgets
Button a
Button b

29. OUR METHOD
Software –multiple widgets
Button a
Button b

30. OUR METHOD
Software –multiple widgets
Button a
Button b

31. OUR METHOD
Software –multiple widgets
where xis a new magnetic point according to the user’s input, pais a reference point of button awith maximum pressure, pbis a maximum point of button b,and p0is a reference point of button aand bwith minimum pressure.

32. OUR METHOD
Software –multiple widgets
Two buttons
Three buttons
Joystick + Button

33. APPLICATONS

34. LIMITATION
-
The magnetism involved might damage the objects
-
Occasionally reports a large offset in the data or a stuck pointing in one direction
However, magnets do not affect flash memories on modern mobile devices or IC chips on credit cards.
This technical glitch, however, will be undoubtedly solved as sensor technology improves.

35. CONCLUSION
-
We have presented a MagGetztoolkit and various applications using it, which provide users with a wider interaction area, higher input expressiveness, and a customizable layout of richer tactile clues.
-
MagGetzdo not need power, a wireless connection, or the hardware modification of mobile devices.
-
We believe that MagGetztoolkit can be successfully integrated for conventional mobile devices.
-
For the future work, we plan to design different shapes and forms of input control widgets (e.g., deformable, stretchable, or elastic properties).

36. THANK YOU