1. Lisa Anthony, PhD
lanthony@cise.ufl.edu
X equals
4 liters…
Pictures from flickr.com by Creative Commons License
Turbo
boost!

2. Lisa Anthony, PhD
lanthony@cise.ufl.edu
X equals
4 liters…
Pictures from flickr.com by Creative Commons License
Turbo
boost!

3. @drlanthony
Research Approach
 What are “natural user interactions”?
Why NUIs for Children
Touchscreen Interaction Projects
Future Research
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 3

4. @drlanthony
Research Approach
 What are “natural user interactions”?
Why NUIs for Children
Touchscreen Interaction Projects
Future Research
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 4

5. VIDEO LINK:
https://www.youtube.com/watch?v=6ECVHsUVuJg
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD @drlanthony 5

6. speech
intent
touch
vision
gesture
motion
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD @drlanthony 6

7. natural interaction
doesn’t interfere
with the user’s task
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD @drlanthony 7

8. @drlanthony
Understanding
user needs and
capabilities
Developing
new intelligent
algorithms
Design the
algorithms
informed by
user behavior
Design the
interfaces
informed by
tech strengths
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 8

9. Computer science
Machine learning
Statistics
…
@drlanthony
Understanding
user needs and
capabilities
Developing
new intelligent
algorithms
Design the
algorithms
informed by
user behavior
Design the
interfaces
informed by
tech strengths
Psychology
Social science
Ergonomics
…
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 9

10. speech
intent
touch
vision
gesture
motion
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD @drlanthony 10

11. @drlanthony
Research Approach
 What are “natural user interactions”?
Why NUIs for Children
Touchscreen Interaction Projects
Future Research
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 11

12. kids are
hands-on and
learn by doing
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD @drlanthony 12

13. kids apply real-world
metaphors
to technology
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD @drlanthony 13

14. VIDEO LINK:
https://www.youtube.com/watch?v=aXV-yaFmQNk
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD @drlanthony 14

15. Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD @drlanthony 15

16. LETTERS SHAPES
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 16
@drlanthony
A
E
Q
Δ
O
+
drawn to scale

17. LETTERS SHAPES
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 17
@drlanthony
A
E
Q
Δ
O
+
drawn to scale
Adult Child Adult Child

18. X equals
4 liters…
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD @drlanthony 18

19. @drlanthony
Research Approach
 What are “natural user interactions”?
Why NUIs for Children
Touchscreen Interaction Projects
Future Research
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 19

20. 1. Understanding how kids use touchscreens now
2. Understanding patterns in how kids make touch gestures
3. Understanding when and why current touch gesture
@drlanthony
recognition approaches don’t work
4. Designing and developing new algorithms for touch
gestures
5. Designing and developing tools to help developers analyze
input behavior patterns and choose the best gestures /
recognizers for their specific context
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 20

21. Touch cloud
Gesture
@drlanthony
“Mobile Touch and Gesture Interaction for Children”
https://mtagic.wordpress.com/
Funding support: NSF HCC Small
 NSF CISE #IIS-1218395/IIS-1433228 and IIS-1218664
Collaborators:
 Quincy Brown (Bowie State University)
 Students from UMBC, Bowie State, and UF
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 21

22. 4 studies with kids and adults (S1^, S2*, S3*, S4-ongoing)
^ Brown, Q. and Anthony, L. 2012. Toward Comparing the Touchscreen Interaction Patterns of Kids and Adults. Proc. ACM SIGCHI Workshop on
Educational Software, Interfaces and Technology (EIST’2012), Austin, TX, 05-06 May 2012.
* Anthony, L., Brown, Q., Nias, J., Tate, B., and Mohan, S. 2012. Interaction and Recognition Challenges in Interpreting Children’s Touch and
Gesture Input on Mobile Devices. Proc. ACM Conf. on Interactive Tabletops and Surfaces (ITS’2012), Cambridge, MA, 14 Nov 2012, p.225-234.
@drlanthony
 49 kids (ages 7-17), 36 adults (ages 18+)
Study 2
setting
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 22

23. Touch Task
Study 2
Gesture Task
Study 2
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD @drlanthony 23

24. Blue triangles: adults
Red squares: children
Black outline: target
Phone
screen
outline
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD @drlanthony 24

25. Study 2a (ITS 2012) Study 2b (JPUC 2014)
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
7 to 10
11 to 13
14 to 17
Adults 18+
Very Small (0.125") Small (0.25") Medium (0.375") Large (0.5")
Average Proportion of Misses
Target Size (inches)
Average proportion of misses by target size for adults and children over all targets.
Error bars show 95% confidence interval.
** significant at the p<0.01 level * significant at the p<0.05 level
@drlanthony
1. Children miss more targets than adults**
2. Smallest targets most challenging*
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Adults
Children
Very Small (0.125") Small (0.25") Medium (0.375") Large (0.5")
Average Proportion of Misses
Target Size (inches)
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 25

26. Children
Adults
Study 1 (EIST 2012)
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD @drlanthony 26

27. Visual Feedback
(children)
No Visual Feedback
(children)
Study 2 (IDC 2013)
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD @drlanthony 27

28. Adults
Children
Study 2a (ITS 2012)
<60% 60-65% 65-70% 70-75% 75-80% 80-85% 85-90% 90-95% 95-100%
Recognition Accuracy
Histogram of recognition accuracy distribution
for gestures drawn by children and by adults.
** significant at the p<0.01 level
6
5
4
3
2
1
0
Number of Participants
^ Anthony, L. and Wobbrock, J.O. 2010. A Lightweight Multistroke Recognizer for User Interface Prototypes. Proc. Graphics Interface (GI’2010),
Ottawa, Canada, 02 Jun 2010, p.245-252.
@drlanthony
1. Kids make gestures differently than adults
 S1: kids make bigger gestures**
 S1: kids make gestures with more strokes**
2. Kids’ gestures are recognized^
less accurately than adults’**
 S2: correlated to age**
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 28

29. Design guidelines app
@drlanthony
Demonstrate design recommendations in
an open-source app as a tool for developers
Kids make gestures and touch targets
differently than do adults: we can design
and develop for this
Next steps:
 Understanding younger kids’ touch and
gesture patterns
 Understanding and designing contextualized
interactions
 Developing child-specific recognizers
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 29

30. Kids’ gestures are recognized less accurately than adults’ by
modern recognizers designed for adults
Study 2b (JPUC 2014) Best
performer
on kids
aged 7 to 10
is still only
at 84%
@drlanthony
 Even when trained on kids gestures!
100.0%
90.0%
80.0%
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
7 to 10 11 to 13 14 to 17 Adults 18+
Recognition Accuracy
Participant Age Groups
User-Independent TabletPC User-Independent $P
User-Dependent $N-Protractor User-Dependent $P
Recognition performance by age group for each
of the four recognizer configurations tested.
Error bars indicate the 95% confidence interval.
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 30

31. @drlanthony
Research Approach
 What are “natural user interactions”?
Why NUIs for Children
Touchscreen Interaction Projects
Future Research
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 31

32. 1. Understanding how kids use whole-body interaction now
2. Understanding patterns in how kids move and make hand
@drlanthony
and body gestures
3. Understanding when and why current hand and body
gesture recognition approaches don’t work
4. Designing and developing new algorithms for hand and
body gestures
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 32

33. @drlanthony
Education
Fitness
Games
Multimodal Algebra Equation Solving
(PhD thesis, Carnegie Mellon, 2008)
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 33

34. Image from PufferFish Ltd.
Investigating Public Engagement with Science
@drlanthony
Education
Fitness
Games
(new project)
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 34

35. @drlanthony
Education
Fitness
Games
Kinect Games for Exercise
http://kinectforexercise.wordpress.com
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 35

36. Acknowledgments:
National Science Foundation,
Pittsburgh Science of Learning
Center, University of Florida,
University of Maryland
Baltimore County
@drlanthony
Lisa Anthony, PhD
 lanthony@cise.ufl.edu @drlanthony
 http://lisa-anthony.com and http://init.cise.ufl.edu
speech
intent
touch
vision
gesture
motion
Understanding, Designing, and Developing Natural User Interactions for Children / Lisa Anthony, PhD 36