The 2024 16th IEEE/SICE International Symposium on System Integration(SII2024)

1. Smartphone-Based Teaching System for Neonate
Soothing Motions
Hiyo ri Tsuji 1 ) , Takumi Yamamo to 1 ), So ra Yamaji 2 ) , Maiko
K o bayashi 3 ), K yo shiro Sasaki 4 ) ,No riko Aso 2 ) , Yuta Sugiura 1 )
1 ) K e i o U n i v e r s i t y , 2 ) K a n a g a w a U n i v e r s i t y , 3 ) W a s e d a
U n i v e r s i t y , 4 ) K a n s a i U n i v e r s i t y
The 2024 16th IEEE/SICE International Symposium on System Integration

2. 2
[1]X. Yao, T. Pl¨otz, M. Johnson, and K. de Barbaro, “Automated detection of infant holding using wearable sensing: Implications for developmental science and intervention,” Proc. ACM Interact. Mob. Wearable Ubiquitous Technol, vol.
3, no. 2, Article 64, Jun. 2019, doi: 10.1145/3328935.
Background: Neonate soothing motion
• Downsides of inappropriate soothing motion[1]
• Negatively impact the physical and socio-emotional health of infants
• Challenging for first-time caregivers
• Lack the knowledge for neonatal nursing
• Limited opportunities and time to learn skill

3. • Risky to learn soothing motions with real neonates
• Help to provide support for skill acquisition
• Attachment of sensors to neonates △
3
Background: Acquisition of neonatal nursing skill

4. • Effective support learning/training expert soothing motion
• Less invasive methods
4
Purpose

5. Smartphone-Based Teaching System for Neonate Soothing Motions
• A smartphone + a stuffed toy
• Easy training at home without specialized equipment
• Capture the posture angles and the magnitude of acceleration
• Real time feedback on the screen and through audio
5
Proposed Method
How to use the proposed system

6. Smartphone-Based Teaching System for Neonate Soothing Motions
6
Proposed Method

7. 7
[1] L. Zhang, L. Liu, W. Ying, M. Huang, C. Yao, and F. Ying, “ModHera: A modular kit for parents to take care babies,” in: Proceedings of the 20th Annual ACM Interaction Design and Children Conference (IDC ’21), Athens, Greece,
2021, pp. 547–551, doi: 10.1145/3459990.3465209.
[2] X. Yao, T. Pl¨otz, M. Johnson, and K. de Barbaro, “Automated detection of infant holding using wearable sensing: Implications for developmental science and intervention,” Proc. ACM Interact. Mob. Wearable Ubiquitous Technol,
vol. 3, no. 2, Article 64, Jun. 2019, doi: 10.1145/3328935.
[3] Jianhui Liu, Yijun Zhao, Jiadi Wang, Chenyi Xie, Fangyuan Cheng, Jintao Nie, Ge Yan, Cheng Yao, Leijing Zhou, Preben Hansen, Fangtian Ying, and Guanyun Wang. 2022. Pababy: An Interactive System for First-time Parents to Learn
Neonatal Nursing. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA '22). Association for Computing Machinery, New York, NY, USA, Article 320, 1–7.
Related Work: Support for neonatal nursing care
Pababy[3]
ModHera[1] Yao et al.[2]
Noninvasive
Specialized equipment
Invasive

8. Smartphone-Based Teaching System for Neonate Soothing Motions
• Target user: first-time caregivers
• Target posture: cradle hold during the neonatal period
• The basic posture to hold neonates before his/her neck control is established
• Experts: midwives
8
Proposed Method
Cradle hold

9. • Soothing motion by a midwife for 1min
• Cradle hold
• A baby doll with a Xsens Dot sensor[1][2]
9
Definition of true values
Average
Adduction Angle 12.8°
Inclination Angle 51.6°
Acceleration 0.97m/s2
Angle definitions
Average of a midwife’s soothing motion
[1] Kyotokagaku, Mokuyokuningyou jyuu shiri-zu “Shintaro” A gata(in Japanese). http://www.kyotokagaku.com/jp/products_data/m5963_a/ (12 July 2023)
[2]Zero C Seven, XSENS DOT. https://www.0c7.co.jp/products/sensing/xsens/product-lineup/xsens-dot.html (12 July 2023)

10. 10
Implementation: Hardware and data measurement
• Smartphone: HUAWEI P30 lite
• Gyroscope: posture angles while a user is holding a stuffed toy
• Accelerometer: magnitude of acceleration while performing the motions
A smartphone and a stuffed toy

11. 11
Implementation: Feedback
• Feedback
• Screen: 3D model, text prompt, arrow
indicating correct direction
• Audio: crying sound when posture is
inappropriate
• Threshold setting for data
• Posture angle: (true value)±5°
• Acceleration: (true value)＋0.5m/s2

12. Compare the effectiveness of learning
• Group V: only video
• Group V+PS: video + proposed system
12
Experiments
Participants
Group V Group V+PS
Gender Male:6
Female:2
Male:5
Female:3
Age Ave: 21.9
SD: 0.8
Ave: 22.5
SD: 1.5
[1][2]
[1] Kyotokagaku, Mokuyokuningyou jyuu shiri-zu “Shintaro” A gata(in Japanese). http://www.kyotokagaku.com/jp/products_data/m5963_a/ (12 July 2023)
[2] Zero C Seven, XSENS DOT. https://www.0c7.co.jp/products/sensing/xsens/product-lineup/xsens-dot.html (12 July 2023)
Experimental flow
V+PS

13. • Significant difference：Inclination angle(p=0.004, d=0.660)
• Useful for ensuring the safety of neonates
• Neck angle is directly linked to the ease of breathing
13
Result
Absolute errors between post-training data and true values
Angle definitions
V+PS

14. • Adduction angle: SDV+PS ＜ SDV
• Acceleration: Group V+PS deviated further from the true value
• More feedback elements on posture angle than acceleration
14
Result
Absolute errors between post-training data and true values
Angle definitions
V+PS
V+PS

15. • Smartphone-based approach
• Angle offset when attaching a smartphone / when calibrating
• Limited information that can be measured or provided
• Credibility of the true values
• Individual differences between midwives and neonates
• Change according to babies’ growth
• Experimental design
• Effects for long-term training
• Types of stuffed toys
15
Limitations and Future Work

16. 16
Summary
Background
The importance of learning soothing motion
Risky to learn soothing motions with real neonates
Related Work
Support for neonatal nursing care
Skill training
Proposed Method Smartphone-Based Teaching System for Neonate Soothing Motions
Implementation A Smartphone acquires angle and acceleration with built-in sensor and provides feedback
Result Effective system for teaching regarding inclination angles
Future Work Credibility of the true values, experimental design

17. Appendix

18. 20
Result: Group V
Adduction angle Inclination angle Acceleration
Before learning 5.56° 12.89° 1.18m/s2
After learning 16.02° 25.76° 0.87m/s2
True value 12.81° 51.57° 0.97m/s2
Distribution of data before and after training in Group V
Data in Group V

19. 21
Result: Group V+S
Adduction angle Inclination angle Acceleration
Before learning 16.12° 15.50° 0.83m/s2
After learning 15.99° 39.75° 0.65m/s2
True vale 12.81° 51.57° 0.97m/s2
Data in Group V+S
Distribution of data before and after training in Group V+S