영유아 발달 증진을 위한 모바일 시스템 디자인

영유아 발달 증진을 위한 모바일 시스템 디자인
KAIST
School of Computing
Ph.D. Student
SEOKWOO SONG
2018. 3. 12.

BebeCODE:
Collaborative Baby Development Tracking System
)() ( ) ) )
PUBLICATION
ACM International Conference on Human Factors in Computing Systems (CHI), 2018

MOTIVATION
§ (~60 months) Tracking of developmental milestone is
important for baby
– Early detection and early intervention can show better results.
§ Paper-based solution (parent-report) is available, but busy
parents often forget to enter the information.
그림 아동발달검사

MOTIVATION
§ (~60 months) Tracking of developmental milestone is
important for baby
– Early detection and early intervention can show better results.
§ Paper-based solution (parent-report) is available, but busy
parents often forget to enter the information.
그림 검사 기간

MOTIVATION
K-DST검사 영역
• 대근육운동
• 소근육운동
• 인지
• 언어
• 사회성
• 자조

RELATED WORK
§ Baby step [CHI 09]
§ Baby steps – twitter [CHI 14]

Motivation
§ (~ 60 months) Tracking developmental milestone is
important for baby
– Early detection of developmental delay and early intervention can
show better results.
§ Paper-based solution (parent-report) is available, but
busy parents often forget to enter the information.
§ Individual parent’s observation makes different result
in parental report of baby development.

PRELIMINARY STUDY
§ Interviews
) )
)
()
4561 04 6 35
4561 12 26 728 6
205 65
6 42 5
0 2 2 042
5812 3 28 2 042
42 7
42 7
42 7

PRELIMINARY STUDY
§ Interviews (semi-structured)
) )
)
()
C
) 2 A 1 5 A
) 2 23 3 3
A 31 A
, 3 A
3 3 3 2A 1 31 5 1 2 3 GA 23 3 3
(2 53A 3 AG 1 1 2 3 23 3 3
AA3AA 3
A 1 A 3 3 A AC 31 3 3 1 2 1
1 2 A 3 5 1 1 5 33 2 A 3 3 A
AC 23 A 2 3 23 3 3 ?C3A AI .
A 3 11C 3 3 3 A 53 3 C
3 1 2GA 23 3 3 1 3 32 3 1 I
3 5 3 2 2 C 1 2 A .

PRELIMINARY STUDY
) )
)
()
,B
2 2B6 D62 6B
. 6B82BD6 D62 6B
0 F6B2 6D 6F6 6 D 2 286 6 D D 6 B 2 2B6 6 D6B
0 B6 6 2D G D B6 I 2B6 D
0 ) B 6 6B 2 D2 D G D B6 I 2B6 D
6 B6 D 2 ( B 6B 2 G D B6 G 36DD6B G 23 D
B6 I 6F6 6 D D 2 D 6 B 2B6 D 6 2BD 6B 2
D6BF 6G G D 2B6 D D 6 B D 6 D 23 D D 6 B B6 I
23 D K 1
6F6B 2B6 6B 2 6 36B 36 2 6 D 6 2B6 D G
B6 6 D D2 D 6 2 8 23 D D 6 B B6 6F6 2D6 2D 8 D K 1

PRELIMINARY STUDY
( . .1
. (
( . )
,
1B8 (
1B8
. B8
). ( . 1 ( . )(
F 3 4 7 3 B 1 38 4 4 B 2 BE B8 1B8
1 4 B8 B8
I) 31 G E 4 B B8 38 431
4 31 G B1 1 38 4 4 B B B 1 4 B B8 8 B1 1 E B8 G
38 4
I.G E 4 B 7 1 41 7 B B B8 38 4 38 1 3 1 3
2 B 4

PRELIMINARY STUDY
( . .2
. (
( . )
. 2 8
. 2 8
2 8
. 2 1 332 2 2 72 3 71 . 2
. ( 2 ). . ( .2 2 ( . ) ) 2
- 472 . 2 2 .
332 2 2 772
332 2 2 . 3 4 . 2 4

DESIGN IMPLICATIONS
§ Both parents have to participate in assessing child
development
§ Help parents resolve their disagreements
§ A less disruptive way to ask a third person about
child development.
§ Long-term way to continuously track child development
[Babysteps 09]

OVERALL USAGE DIAGRAM
View result
True
Father Mother
False
Compare responses and
identify disagreements
Reach
consensus
Next set of development
questions for child’s age
Timeout algorithm
(for questions that child
could not do)
questionsDevelopmental
questions
Answer
developmental
questions
questions
Answer
developmental
questions
· Chatting
· Sharing image/video
· Asking a third person

BEBECODE SYSTEM DESIGN
View result
True
Father Mother
False
Reach
consensus
Timeout algorithm
could not do)
questions
Answer
developmental
questions
questions
Answer
developmental
questions
· Chatting

View result
True
Father Mother
False
Reach
consensus
Timeout algorithm
could not do)
questions
Answer
developmental
questions
questions
Answer
developmental
questions
· Chatting
Disagreement
• ,
- ,
- , ,
•
- ,
- , ,
• ,
- ,
/ , -

View result
True
Father Mother
False
Reach
consensus
Timeout algorithm
could not do)
questions
Answer
developmental
questions
questions
Answer
developmental
questions
· Chatting
Disagreement
optional

TIMEOUT ALGORITHM
View result
True
Father Mother
False
Reach
consensus
Timeout algorithm
could not do)
questions
Answer
developmental
questions
questions
Answer
developmental
questions
· Chatting
Ex)
2018 March 12 :
Father answered “child can’t walk
up and down stairs alone”.
Remove father’s answer after N days

Progress Status Progress Graph

BEBECODE EXPERIMENT
§ Participants
#
) )
( )
# )
( ) ( ) ()
199 68 A 8A . 8 A 8A 0
3C 78 C . 8 A 8A
0
/
2 5 6 99 6 4 . 8 A 8A
0
/
2 5 6 99 6 4 . 8 A 8A 0 /
199 68 A 8A . 8 A 8A 0
4A 8 C8A . 8 A 8A
2 5 6 99 6 4 . 8 A 8A /
2 5 6 99 6 4 2 5 6 99 6 4
3C 78 C
3C 78 C
199 68 A 8A 3C 78 C /
2 5 6 99 6 4 2 5 6 99 6 4
199 68 A 8A 199 68 A 8A

BEBECODE EXPERIMENT RESULTS
§ Overall Usage Pattern

§ Methods used to resolve
the disagreement
Chatting /
conversation
76%
Image / video
sharing
8%
Asking
a third person
10%
Etc
6%
Remembered what
he/she had
forgotten
59%
Simply followed
the spouse
23%
Discuss and
consensus
18%
§ Reasons why the parent’s
changed their opinion

§ Experience in Resolving Disagreements
A. Chat
08 ) / , 00/ ) . / , , . , 0 8 ,8/ ,// 0
0 / , 00 08 ( (
(,.0 1,.0 , 8 / . 8 , -0 0 8.0 ) ,
,- 0 . 8 ., 0 1, 0 ,8/ 0.0 0/ 0, 0 /0 1
100/-,. (

B. Image/Video sharing
)9 ) - - - 9 ) - - - 5 9 - )9 - 9 .
9 - ) )9 9 )5 - 9 ) -5
9 9-
( - - 9 5 ) ) - - - 5. -5 - )
- - 5 9 ) ) - )5 9 ) - -9 )5 - 9

C. Asking a third person
) . , 5- . - , . ,. .
.- . . , . -.
( . . ., 55 .5 5 ., . . - .
., . .

§ Notifications for Continuous Tracking
(Timeout algorithm)
F, A 1 4 1 1
A A A 1 1A A - A A
0 A . ( A 1 11 0 A
A A A A A )
!
F 1 44 1 A A 4 A
0 A A A 4 A A 0 A
A 4 0
A ) !

§ Increased interest in children
- - , -.9 4 , - -. . 4. , 9.
9 .- .9 . 4 , - 4 9. , . 9 .
-. . 4. . (
) . .9 ..4.- . 4 9. .9. .- ,
. . 9 . 9. (

§ Improving communication between parents
( , ,- - -5 . 5 - - - -
2 - , -, - 5 -
,- - - , 2 )- --,
- -, - --2-5, - 5 2- 2 -
. - -5
)- , - - - - - -5 - 5,
- ,-5 - 5 - - - 5 - , --, 5

BEBECODE EXTENSTION
§ Other assessment based on parent-report

TalkLIME:
Mobile System Intervention to Improve Parent-Child
Interaction for Children with Language Delay
PUBLICATION
( 6 6 6 ( 1 A 6A 2 06 6 ( 6 )(,

INTRODUCTION
§ Language development has a significant impact on our social
relationships and academic achievements.
§ Language development during the childhood can have
lasting impact on the speech.
§ As a result, early identifying language delays and
proper treatment is important.

INTRODUCTION
§ Language Treatment
– Direct therapy : Speech-Language Pathologist’s treatment
– Indirect therapy : environment change, parent-training, …
37/42
§ Recent studies showed that parent-training is effective to
improve language development of children.
.

PARENT-TRAINING
§ Home-based intervention that is administered by the
parents of children.
§ However, the parents feels difficult to apply the strategies to
their children in the home by themselves.
38/42
Language
therapist
Examples of guideline)
Spend 10-15 minutes each day with their children
(parent-child interaction session)
Talk more slowly
Wait for the child to talk back
Talk in short and simple sentences
…
? ?

CONTENTS
§ Motivation / Background
à TalkLIME (Talk Less Is MorE, UBICOMP ’16)
– Preliminary Study
– System Design
– Methodology
– Results
– Discussion
§ Future Work
39/42

OVERALL STUDY PROCEDURE
§ Phase 1
– Preliminary Studies
§ Phase 2
– TalkLIME system implementation
§ Phase 3
– User Study
Feedback
Study
Data
Collection
Parent Study
System Design
Pre-test
Intervention
Post-test
Phase 1
Phase 2
Phase 3
40/42

PARENT STUDY
§ Goal : understand the parent’s difficulties
in practicing parent-child interactions at
home
§ Participants : 10 parents (10 females)
(had experienced parent-training
before)
§ Methods : face-to-face interview for 30
minutes
§ Main Questions
ü Were parent-training helpful?
ü Did you practice parent-child interactions
continuously at home? If not, what were the
difficulties?
ü Did you ever stopped or gave up during a
parent-child session at home? If so, why?
41/42
Feedback
Study
Data
Collection
Parent Study
System Design
Pre-test
Intervention
Post-test
Phase1Phase2Phase3

§ Difficult to know whether they were
practicing well
§ Difficult to stay motivated during the
interaction
ü Short-term : children with language delay
rarely provide any reaction to their parents
which demotivates parents.
ü Long-term : they felt that child’s language
development is not improving.
Feedback
Study
Data
Collection
Parent Study
System Design
Pre-test
Intervention
Post-test
Phase1Phase2Phase3
42/42
PARENT STUDY RESULT

practicing well
→ Provide real time feedback
interaction
→ Provide a timer
→ Provide statistical graph
Feedback
Study
Data
Collection
Parent Study
System Design
Pre-test
Intervention
Post-test
Phase1Phase2Phase3
43/42
DESIGN IMPLICATIONS
Motivating parents > Perfect real-time feedback

practicing well
interaction
→ Provide a timer
Feedback
Study
Data
Collection
Parent Study
System Design
Pre-test
Intervention
Post-test
Phase1Phase2Phase3
44/42
DESIGN IMPLICATIONS

FEEDBACK TYPE STUDY
Feedback
Study
Data
Collection
Parent Study
System Design
Pre-test
Intervention
Post-test
Phase1Phase2Phase3
§ Goal : understand parent’s preference in
how feedback is provide to them during
parent-child interaction session.
§ Participants : 9 parents (9 female)
(have child 2~5 years old)
§ Methods : feedback type experience
45/42
Speak
more slowly
Speak
more slowly
Type I
Post-feedback
Type II
Screen
Type III
Sound

Speak
more slowly
Speak
more slowly
3 ( ( 3 3
33( )( 0
Post-session
feedback
Selected feedbacks
• Speak more slowly
• Wait for the child’s speaking
• Do not interrupt the child’s speaking
• Speake more simple and respond to the child
10% incorrect feedback 46/42
SLP gave proper feedback
-
Type I Type II Type III
Time
10 minute 20 minute 30 minute
FEEDBACK TYPE STUDY

-
- -
-
→ Less disruptive and error tolerant
47/42
FEEDBACK TYPE STUDY RESULTS

practicing well
interaction
→ Provide a timer
Feedback
Study
Data
Collection
Parent Study
System Design
Pre-test
Intervention
Post-test
Phase1Phase2Phase3
48/42
DESIGN IMPLICATIONS

STATISTICAL DATA COLLECTION
Feedback
Study
Data
Collection
Parent Study
System Design
Pre-test
Intervention
Post-test
Phase1Phase2Phase3
§ Goal : measuring the average number of
utterances of children without language delays
§ Participants : 119 children without language
delays
§ Methods : Played with the child to collect
spontaneous utterance
49/42This study was conducted from Ewha-womans university.

§ Real time feedback
– The number of utterance
• Important metric in child language development
– The initiation ratio of utterance
• The number of initiation utterances / The number of utterance
• Sign of improvement in language development
50/42
SYSTEM DESIGN

51/42
SYSTEM DESIGN
(video
)
Parent’s talk is that less is more.

52/42
SYSTEM ARCHITECTURE

§ Motivate Parents
Timer
(short-term goal)
Statistical graph
(long-term goal)
2
3
2
3
53/42
TALKLIME SYSTEM DESIGN
Interaction session

1. Select interaction time
2. Put the smartphone between the parent and the child
3. Interact with the child
TALKLIME USAGE DIRECTIONS
54/42

USER STUDY
Feedback
Study
Data
Collection
Parent Study
System Design
Pre-test
Intervention
Post-test
Phase1Phase2Phase3
§ Participants : 8 children with language
delays and their parents
• Children
(1) could communicate through vocalization
(2) no problems with perception and listening
• Parents
(1) primary caregiver for the child
(2) never experienced parent-training before
()
C9
8
AA7 5=67
2= A 5A =
45A
C9
8
AA7 5=67
2= A 5A =
45A
0 =A
9 3.(
( ( % # ) #(% # ) %(# )
1 7 7=A
9
3.(
( ( # ) % #) #) % #) #
55/42We received Ewha-womans university student’s help.

56/42
Day6weeks
-
Pre-test, post-test: 30 minutes parent-child interaction to collect # of utterance
USER STUDY
Experiment group
(N = 4)
Control group
(N = 4)

57/42
Day
1 1
1 1
6weeks
1
1
USER STUDY
1 A 45 1-
A 45 1-
5 1 211 - 6 - - 1 5 1 - 5
-61 611 41 5 1 - 5 3 5 3
- 3 -31 41 5 1 - 5
4- 5 3 6 - 5
Experiment group
(N = 4)
Control group
(N = 4)
1 2 3 4 5 6
- 1 -5 5 3
Parent-training was conducted every weekend by a language therapist.

58/42
Day6weeks
USER STUDY
Experiment group
(N = 4)
Control group
(N = 4)
• Control group : wrote the interaction diary
• Experiment group : used the TalkLIME

§ Qualitative results
– After post-tests, the post-surveys with the parents in the
experiment group were conducted.
: 33 2 3 3 25 15 : 5 :1 5 : :
: :14 3 3
: 3: 3
5: : : 1 31 :1 5 5:15 3
59/42
EXPERIMENT RESULTS
strongly disagree (1) strongly agree (5)

§ Screen-based conversation feedback
(+) “When I saw that my child’s ball was too small, I tried to give
my child a chance to speak” (P2)
(+) “After I knew the conversation was excessively led by me, I
tried to decrease my utterances and induce the child’s
utterances by establishing the conversation strategies that I
learned from the parent-training” (P4)
(-) “When I knew the number of utterances of my child was very
low, I tired to induce him to speak, but if the ball was still small
despite my efforts, I felt exhausted” (P3)
60/42
QUALITATIVE RESULTS
5 1 1
1 45 1 5 5

§ Timer
(+)“I, sometimes, want to finish or skip the interaction with my
child, because I am already too tired to talk due to house
chores or other thing to do. However, I cheered up by looking
the remaining time in the timer when I felt it” (P4)
(-) Most parents felt the interaction is a long time, one parent
didn’t feel it was too hard, so she didn’t think the timer was helpful
(P1)
61/42
QUALITATIVE RESULTS

§ Result graph
(+) “I especially liked the dotted line showing the average
number of utterances of the same age group. The difference
between the dotted line and my child’s line alerted me,
motivating to practice more.” (P2)
62/42
QUALITATIVE RESULTS

§ User experiences
QUALITATIVE RESULTS
(-) “This system was helpful in initially inducing my child’s utterance,
however, once I felt like I was interacting correctly, I did not felt like I
needed the system any longer” (P1)
(-) “Difficult to carry around the TalkLIME system and execute the
application when interacting with my child” (P3)
63/42
1 1 1 1

Parents
64/42
QUANTITATIVE RESULTS
%
(
)
1# 1 # 1
2101
5 3 1 461 3
)
%
%)
)
1# 1 # 1-4444
5 3 1 461 3
§ Parents
There was a non-significant main effect between the pre-test and the post-
test for both groups (!(#,%) = 1.071, , > .05).

From the result of the initiation ratio from the children,
There was a significant improvement in the experiment group than the
control group (!(#,%) = 7.754, , < .05).
65/42
QUANTITATIVE RESULTS
%
%
3-
( 3 1 5 ) 5 02 3 5
%
%
300030
( 3 1 5 ) 5 02 3 5
§ Children

MOBILE-SYSTEM ACCURACY
!""#$%"& =
# )* #++,$%-",. %-%/&.,0 1& +ℎ, ")34#+,$
# )* #++,$%-",. %-%/.,0 3%-#%//&
The children’s utterances were especially short and quiet.
66/42

DISCUSSION
§ Recruiting Participants
• The number of participants was relatively low.
• It is difficult to recruit special-needs.
• 6-week length for the user-study made it difficult.
• We focused on qualitative results.
67/42

DISCUSSION
§ Speaker Identification Accuracy
• Short Utterances : some utterances are very short (1-2s)
• Non-utterance Sounds : “hmm~” and “oh~”
• Surrounding Noises : overlapped sounds have different vocal
features and make the speaker recognition more difficult.
• User Behavior and Voice Change: emotional states,
pronunciation speeds
68/42

CONCLUSION
69/42
• We proposed TalkLIME mobile-system intervention system to
improve parent-child interaction for children with language
delays.
• TalkLIME provides realtime less-disruptive feedback to the
parent by using screen-based feedback.
• Simple timer and statistical graph provided encouragement
to the parents in their daily parent-child interaction.
• 6-week user study showed that mobile-system intervention
showed a significant improvement in the initiation ratio from
the children.

RESEARCH INTERN
§ NTT COMMUNICATIONS (KYOTO)
2018.12 ~ 2019.02, mentor : Dr. Naomi Yamashita
70/40

MOTIVATION
FH
F
HD A F
% )
D H FH B F DC
, DAD : A :HDF
• 0DFBDC : C
• 0 F H .1
• ,D DC H DC
: C
5D: A :HDF
• F CH C 5HF C :A C C : F C
• 5 H :H DC D 0 C DF B FF DF
• 5D: A DFH B A F C
2F C A 4 DFH 5H D F :H C D H FH B . F DC C H 4 :D F
:HDF FDB F FH B . F DC '%%(
) F
C CH
F CH C 5HF
H F DFH F A H HD H B DFH CH F :D F :HDF D F DC

영유아 발달 증진을 위한 모바일 시스템 디자인

Recommended

Recommended

More Related Content

Similar to 영유아 발달 증진을 위한 모바일 시스템 디자인

Similar to 영유아 발달 증진을 위한 모바일 시스템 디자인 (20)

More from NAVER Engineering

More from NAVER Engineering (20)

Recently uploaded

Recently uploaded (20)

영유아 발달 증진을 위한 모바일 시스템 디자인