The document discusses innovative interactive technologies designed for children, focusing on tangible and gesture-based interactions in educational contexts, particularly for children with disabilities. It highlights various case studies demonstrating the application of these technologies to enhance learning, communication, and engagement for children with cognitive and motor impairments. Additionally, it emphasizes the importance of customizing approaches and tools to meet the unique needs of each child.

1. Interactive technologies for
children: new frontiers
Franca Garzotto
Matteo Valoriani
Dipartimento di Elettronica ed Informazione – HOC
Lab
Politecnico di Milano

2. Outline
Who we are
Children and Technology
New frontiers:
- Tangible Interaction
- Touchless Gesture Based Interaction

3. Who we are
HOC Lab
A multidisciplinary research lab at the Dept. of
Electronics and Information at Politecnico di Milano
Aplication domain:
• e-learning
• e-culture
• e-tourism
Technologies: web, mobile devices, tangibles,
gesture-based
http://hoc.elet.polimi.it/hoc/

4. Children and technology
• A huge amount of digital technologies for
children
• Mostly for entertainment, education, and
learning
• Children are a stimulating and wide target
of users for current and new technologies

5. Example 1: tangible interaction
• physical representation of data and
functions
• manipulation of digitally enriched
physical objects
• Embodiment and embeddedness in real
space

6. I/O Brush- MIT Media Lab

7. I/O Brush- MIT Media Lab

8. I/O Brush- MIT Media Lab
Input and Control Device: Brush

9. I/O Brush- MIT Media Lab
Input: coloured balls
Input and Control Device: Brush

10. Paper-based Multimedia
as Learning Tool for
Disabled Children
Tangible Interaction: Case

11. Goal
To support learning
in the school context
for children with severe cognitive,
linguistic and motor disabilities
in the long term and in a sustainable way

12. Profile of our disabled children
• Severe impairments in body
movement and muscle coordination
• Limited cognitive functions
• They understands from 40 to 70% of
speech and non verbal communication
(i.e., conventional gestures and face
expressions),
• They are not able to speak
• They normally communicate through
face expressions, body movements,
sounds, PCSs and portable VOCAs

13. Requirements on the UX
• Use of PCS based Communication
• Customization (“each child is unique”)
• Tangible Interaction
• Use of familiar, situated material
• Consolidation
• Enagagement
• Concentration and Timing

14. Requirements on Technology
• Open
• Flexible
• Easily adaptable
• “Construction Kit” and “End User Programming” paradigms

15. tangible paper based visual content
RFID technology
multimedia resources
sound, voice, music
images or videos (on the screen or on the wall)
Physical visual content:
- representations of concepts to learn
(e.g., PCSs, story elements, …)
- affordances for interacting with digital
material

16. Scenario 1: Enriching the
Goal: Recognition of known concepts; Learning new
PCS symbols
Materials:
• Videos of school mates’ actions
• PCS tagged and untagged action cards and
phrases; School mates’ tagged pictures; class
poster
Activity:
• Looking at videos corresponding to school mate
images
• Given a PCS phrase, recognize which school mate
performs the corresponing action
• Tag the class poster with proper PCS action
symbols

17. Scenario 2: Understanding more complex
communication structures (storytelling)
Goals: Recognition of story scenes;
building associations (scenes ->PCS phrases);
aggregating scenes
Materials:
• Story animations (whole story + single scenes)
• PCS tagged and untagged phrases;
• tagged story poster; story “path” poster
Activity:
• Interact with the story (scenes) poster
• Match a PCS phrase with the corresponding scene
• Build the story poster

18. Scenario 3: Building “My” story
Goal: Understanding temporal concepts
(“before”, “after”); develop elementary
narrative skills
Materials:
• Videos of child’s activities at school
• Tagged pictures of child’s activities
• Tagged PCS symbols
Activity: story building
• Put images in the right temporal order
• Build a proper PCS phrase for each moment
• Associate phrases to images

19. Engaging the whole class
Non-disabled children: experience co-designers
• Identifying what their disables mates like more
• Starring actors in videos
• Content authors (images, pictures, sounds)

20. Case study 2:
the element
project

21. 60M people in the world
70% ∼3 x
increase in last Australian

22. new kinect-based, playful
learning application for
autistic children with innovative
cloud remote assistance

23. problem & goals
Autism Spectrum Disorders (ASDs) is very large and
comprises different symptoms

24. problem & goals
Autism Spectrum Disorders (ASDs) is very large and
comprises different symptoms
interest motion communication relation

25. problem & goals
Autism Spectrum Disorders (ASDs) is very large and
comprises different symptoms
1
create
interest and
motivation
interest motion communication relation

26. problem & goals
Autism Spectrum Disorders (ASDs) is very large and
comprises different symptoms
1 2
create foster
interest and physical
motivation
interest motion
activity communication relation

27. problem & goals
Autism Spectrum Disorders (ASDs) is very large and
comprises different symptoms
1 2 3
create foster improve
interest and physical communication
motivation
interest motion
activity communication
skills relation

28. problem & goals
Autism Spectrum Disorders (ASDs) is very large and
comprises different symptoms
1 2 3 4
create foster improve promote social
interest and physical communication interaction thru
motivation
interest motion
activity communication
skills teamwork
relation

29. problem & goals
Autism Spectrum Disorders (ASDs) is very large and
comprises different symptoms
technology can’t heal, but can help
people bypass disability effects
1 2 3 4
create foster improve promote social
interest and physical communication interaction thru
motivation
interest motion
activity communication
skills teamwork
relation

30. our solution

31. our solution
customization

32. our solution
customization
therapy centers

33. our solution
customization
therapy centers @home therapy

34. our solution
customization remote therapy
therapy centers @home therapy

35. game patterns
different activities to address very different aspects
of autism
Sequential Quiz Music Interaction
games games games games

36. game patterns
different activities to address very different aspects
of autism
Focused on
teaching a
sequence of actions
Sequential Quiz Music Interaction
games games games games

37. game patterns
different activities to address very different aspects
of autism
Flexible and adaptable to many teaching
activities
Sequential Quiz Music Interaction
games games games games

38. game patterns
different activities to address very different aspects
of autism
Leverage the
emotion and the
involvement of
music
Sequential Quiz Music Interaction
games games games games

39. game patterns
different activities to address very different aspects
of autism
Focused on
communication and
social interaction
Sequential Quiz Music Interaction
games games games games

40. personalization
standard
AAC different
elements levels
different
contents
and tasks

41. journal & reporting
teachers can insert private notes about child
response and/or comments for the parents
parents can view these notes on a personal web journal

42. remote therapy

43. remote therapy
remote remote remote remote
supervision communication customization interaction

44. User testing

45. User testing

46. Thanks for your
attention
Franca Garzotto
Matteo Valoriani
Dipartimento di Elettronica ed Informazione – HOC Lab
Politecnico di Milano
[garzotto, valoriani@polimi.it]