Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)

Technologies, like augmented reality (AR), have the ability to catch children’s imagination and to promote their attention.

Children with ASD usually have difficulty to recognize facial expressions and to understand associated emotions.

We propose to design and develop an innovative GameBook to assist children with ASD to recognize and acquire emotions by engaging their attention and motivation.

The GameBook will contain a story that describe some scenarios and real world situations which will conduct the children to become involved on fictional contents associated with emotions.

An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)

  1. 1. An  Augmented  Reality   GameBook  for  Children   with  Au8sm  Spectrum   Disorder  (ASD)   Jorge  Brandão   University  of  Minho/Algoritmi  R&D  Centre,  Guimarães,  Portugal   Pedro  Cunha   University  of  Minho/Algoritmi  R&D  Centre,  Guimarães,  Portugal   José  Vasconcelos   University  of  Minho/Algoritmi  R&D  Centre,  Guimarães,  Portugal   Vítor  Carvalho   University  of  Minho/Algoritmi  R&D  Centre,  Guimarães,  Portugal   IPCA-­‐EST,  Barcelos,  Portugal   Filomena  Soares   University  of  Minho/Algoritmi  R&D  Centre,  Guimarães,  Portugal     InternaKonal  conference  focused  on  e-­‐learning  in  the  workplace   June  10-­‐12,  2015,  at  Columbia  University  in  New  York.  
  3. 3. •  Technologies,  like  augmented  reality  (AR),  have  the  ability  to  catch   children’s  imaginaKon  and  to  promote  their  aYenKon.     •  Children  with  ASD  usually  have  difficulty  to  recognize  facial   expressions  and  to  understand  associated  emoKons.     •  We  propose  to  design  and  develop  an  innovaKve  GameBook  to   assist  children  with  ASD  to  recognize  and  acquire  emoKons  by   engaging  their  aYenKon  and  moKvaKon.       •  The  GameBook  will  contain  a  story  that  describe  some  scenarios  and   real  world  situaKons  which  will  conduct  the  children  to  become   involved  on  ficKonal  contents  associated  with  emoKons.   ABSTRACT  
  4. 4. ABSTRACT   Play  with   AR  Avatar   Play  Quiz   Games   Play  Memory   Games   The  child  will  have  to  interact  with   these  scenarios,  by  playing  with   one  3D  AR  avatar  with  different   facial  expressions     In  this  GameBook  the  child  will   have  to  idenKfy  the  correct   emoKonal  face  to  the  situaKon  in   a  Quiz  Game.   This  GameBook  the  child  will  have   to  idenKfy  the  correct  emoKonal   face  to  the  situaKon  in  a  Quiz   Game.   x   x  
  5. 5. AUTISM  SPECTRUM  DISORDER  IN  USA   Source:  Center  for  disease  Control  and  PrevenKon    -­‐  hYp://   “The   AuKsm   Society   esKmates   that   the   United   States   is   facing   almost   $90   billion   annually   in   costs  for  au8sm.     (This   figure   includes   research,   insurance   costs   and   non-­‐covered   expenses,   Medicaid   waivers   for   auKsm,   educaKonal   spending,   h o u s i n g ,   t r a n s p o r t a K o n ,   employment,   related   therapeuKc   services  and  caregiver  costs.)”   hYp://www.auKsm-­‐­‐auKsm.    
  6. 6. hWp://­‐disease-­‐control   HIGH   FUNCTIONING   LOW   FUNCTIONING  
  7. 7. 1  out  of  every  54  boys  in  the  United  States  has  an  au8sm   spectrum  disorder   hYp://­‐disease-­‐control  
  8. 8. 1  out  of  every  252  girls  in  the  United  States  has  an  au8sm   spectrum  disorder   hYp://­‐disease-­‐control  
  9. 9. hYp://­‐disease-­‐control  
  10. 10. INTRODUTION   •  Technology  is  ubiquitous  and  can  be  a  safe  and  good  way  to  moKvate  and   engage  children  in  interacKve  learning  acKviKes  in  order  to  promote  their   cogniKve  and  social  skills   •  It  is  being  used  increasingly  on  a  variety  of  pedagogical  contexts,  both  as   assisKve  technologies  and  as  tools  for  helping  us  to  understand  user’s  moKvaKon   •  It  has  the  potenKal  to  provide  individually  tailored  intervenKons  that  are   suitable  for  a  wide  variety  of  abiliKes     •   Technological  intervenKons  allow  its  use  at  different  speeds  and   locaKons,  and  never  lose  paKence  with  the  frequent  repeKKon  that   many  people  with  ASD  has  
  11. 11. INTRODUCTION   •  ASD  children  struggle  with  significant  relaKonships  and  behavioural   challenges  that  in  most  cases  have  serious  implicaKons  for  inclusion  social   adulthood     •  One  of  the  most  promising  applicaKons  in  the  intervenKon  process  of  ASD   children  came  from  developing  tools  for  promoKng  children’s  social  and   communicaKon  skills     •  This  offers  the  opportunity  to  reduce  some  of  the  crippling  personnel  costs   associated  with  auKsm  care    
  12. 12. INTRODUTION   •  Children  with  ASD  usually  have  difficulty  to  recognize  facial  expressions   and  to  understand  associated  emoKons,  to  imitate  or  use  emoKonal   expressions,  to  understand  and  control  their  own  emoKons,  or  to   interpret  emoKons  or  empathy  with  others   •  Several  studies  show  that  the  majority  of  people  with  ASD  exhibit  a   natural  affinity  with  technology  and  a  posiKve  ahtude  towards   computer-­‐based  training      
  13. 13. INTRODUTION   •  Technologies,  like  augmented  reality  (AR),  have  the  ability  to   catch  the  children’s  imagina8on  and  to  promote  their  aWen8on,   because  they  can  experiment  ar8ficial,  safe  and  fascina8ng   environments.   •  AR  is  an  opKmal  interface  technology  and  a  helpful  tool  to  support   auKsKc  children  capabiliKes.  With  AR  it  is  possible  to  create  more   aYracKve  and  interacKve  interfaces  that  can  be  manipulated  by   hand,  without  using  convenKonal  peripherals  such  as  the   keyboard  and  the  mouse.     •  This  characterisKc  offers  greater  interacKon  between  the  child   and  the  object,  raising  their  interest  and  their  curiosity,  providing   greater  adherence  to  the  task/acKvity.    
  14. 14. THE  GAMEBOOK   Given  the  potenKal  benefits  of  AR  we  aim  to   design  a  tool  based  on  AR  in  order  to  improve   capabiliKes  of  ASD  children       Our  goal  is  to  create  an  innovaKve  serious  game   based  on  a  book  style  with  interacKon  inputs  to   assist  children  with  social  interacKon  and   communicaKon  skills.       We  aim  to  promote  children  with  ASD  recognizing   and  acquiring  emoKons  by  engaging  their  aYenKon   and  moKvaKon,  increasing  their  competence  on   this  handicap.  
  15. 15. THE  GAMEBOOK   The   GameBook   presents   the   story   of   Tobias’s   adventure   during   a   visit   to   a   zoo   park.       The  player  can  read  the  story  and  interact  with   it.  The  story  will  describe  scenarios  and  Tobias’   interacKons  with  animals  and  real  world   situaKons  which  will  conduct  the  children  to   become  involved  on  ficKonal  contents  associated   with  emoKons.  
  16. 16. THE  GAMEBOOK   We  believe  that  the  GameBook  will  put  the  ability  of  the  child  to  interact   with  the  story  and  to  promote  his/her  imagina8on  and  engaging.   hYp://   The  GameBook  can  be  played  on  any  mobile   technology,  such  as  a  tablet,  smartphone  or   laptop,  with  either  an  external  web  camera  or  an   inbuilt  camera.   The  GameBook  is  available  on:  
  17. 17. GAMEBOOK  CHARACTERS   •  The  character  Tobias  was  designed  to  capKvate  the   children  aYenKon  and  to  empathize  with  them.     •  On  his  design  was  pretended  to  promote  a  face  with   characterisKcs  that  allows  the  players  to  feel  some   aYracKon  with  him.     •  The  expressions  are  designed  by  special  details  in  the   avatar,  essenKally,  with  his  lips  and  eyes.     •  It  was  given  a  cartoonish  look  to  Tobias,  which  was   deemed  by  the  group  members  as  potenKally  more   familiar  and  more  fun  to  the  children  than  a  photo-­‐realisKc  
  18. 18. GAMEBOOK  SCENARIOS   The  game  includes  a  total  of  5  different  scenarios,  designed  to  involve  the  child   on  real  life  situa8ons.   The  scenarios,  all  in  the  zoo,  will  promote  the  child’s  contact  with  different  environments  and   will  create  some  emoKonal  reacKon  on  Tobias.     ANIVERSARY   GO  TO  THE  ZOO   WITH  ELEPHANT   WITH  LION   WITH  BIRD  
  19. 19. IMPLEMENTATION  OF  THE  GAMEBOOK   To  create  this  serious  game,  the  following  technologies  were  used:       The   3D   flip   book   uses   the   HTML5/CSS3   and   JavaScript/Jquery  web  languages.       To  develop  the  AR  feature  it  was  selected  the   ArUco®  (Augmented  Reality  library  from  the   University  of  Cordoba).  This  opKon  was  held   on  the  easiness  of  use,  reliability  and  cross-­‐ plakorm  compaKbility.    
  20. 20. CONCLUSIONS  AND  FUTURE  WORK   •  The  aim  of  this  project  was  to  make  available  a  cost  effec8ve  tool  implemented  in  a  plagorm   usually  pleasant  to  ASD  children.     •  In  order  to  test  the  GameBook,  an  exploratory  study  in  a  classroom  context  with  ASD  children   will  be  performed.       •  In  the  near  future,  a  system  that  will  allow  monitoring  the  child’s  results  and  these  outcomes   will  be  available  on  the  website  of  the  game.     •  Furthermore,  new  stories,  as  for  example,  Tobias  adventures  in  the  Sea  or  in  the  Space,  with   new  characters,  scenarios  and  situa8ons  can  be  easily  included.     •  It  is  the  authors’  belief  that  the  GameBook  will  increase  ASD  children  engagement  in  the   learning  ac8vi8es  promo8ng  their  memory,  cogni8ve  and  social  skills  and  allowing  them  to   reinforce  facial  expression  recogni8on.  
  22. 22. QUESTIONS?   Jorge  Brandão   University  of  Minho/Algoritmi  R&D  Centre,  Guimarães,  Portugal   E-­‐mail:     Pedro  Cunha   University  of  Minho/Algoritmi  R&D  Centre,  Guimarães,  Portugal   E-­‐mail:     José  Vasconcelos   University  of  Minho/Algoritmi  R&D  Centre,  Guimarães,  Portugal   E-­‐mail:     Vítor  Carvalho   University  of  Minho/Algoritmi  R&D  Centre,  Guimarães,  Portugal   IPCA-­‐EST,  Barcelos,  Portugal   E-­‐mail:         Filomena  Soares   University  of  Minho/Algoritmi  R&D  Centre,  Guimarães,  Portugal  E-­‐mail: