• Save
Nthu cs seminar_may21
Upcoming SlideShare
Loading in...5

Nthu cs seminar_may21






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds


Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Nthu cs seminar_may21 Nthu cs seminar_may21 Presentation Transcript

  •   On Grounding Human Communication with  Human-Computer Interaction Designs Hao-­‐Chuan  Wang  .  王浩全     Department  of  Computer  Science   Ins3tute  of  Informa3on  Systems  and  Applica3ons   Na3onal  Tsing  Hua  University   h-p://www.cs.nthu.edu.tw/~haochuan       May  21,  2014  @  NTHU-­‐CS  Seminar    
  •     Wang A Quick Overview of Human-Computer Interaction (HCI) 2  
  •   The  two  “senses”  of  Human-­‐Computer  Interac7on:  From  interface  …   “Interac<on”  in  the  sense  of  computers  listening  and  responding  to  people’s  input
  •   …  to  problem  solving  and  value  crea7on  in  the  real  world “Interac<on”  in  the  sense  of  designing  technologies  based  on  user  needs,  goals,   constraints,  and  characteris<cs.  UCD:  User-­‐Centered  Design. Iden7fying  &  fixing     usability  problems Technology     supported   educa7on Persuasive  (behavioral  change)     compu7ng
  •     Wang HCI: Studying Existing and Possible Relationships between Computers and People 5   ACM  SIGCHI  Curricula  1996
  •     Wang 30 Years of the HCI Community 6   ACM  SIGCHI:     9  Turing  Award  Winners  /  188  ACM  Fellows http://dl.acm.org/sig.cfm?id=SP923
  •     Wang What’s Changing in HCI Today? Big  picture  is  s<ll  there,  but:   •  More  emphasis  is  on  use  contexts  and   applica<ons.   •  Computers  are  of  many  forms,  doing  all   sort  of  things.   •  Compu<ng  is  not  necessarily  done     by  silicon  chips  computers.       -­‐  Input  and  output  are  versa<le.  Not   necessarily  “keyboard  and  mouse”,     “text,  speech  or  graphics”   -­‐  Collabora<on  and  social.  Not  necessarily   “one  human,  one  computer”.     7  
  •     Wang Computer-Mediated Communication (CMC) 8  
  •     Wang       What’s  the  longest  distance  in  the  world?     世界上最遠的距離是什麼? 9  
  •     Wang 10  
  •     Wang Supporting Human Communication Communica<on  in  the  sense  of  data  transmission  across   physical  distance  is  not  that  hard  today   •  Wired  and  wireless  computer  networking,  internet  etc.     Communica<on,  in  the  sense  of  understanding  each  other,  or   crossing  the  “psychological  distance”  between  people   remains  hard   •  Difficul<es  in  expressing  or  understanding  thoughts   •  Barriers  between  genera<ons,  genders,  professions,   languages,  and  cultures.       Suppor<ng  human  communica<on  con<nues  to  be  a   challenging  yet  worth-­‐of-­‐pursuing  topic  in  HCI.     11  
  •     Wang Supporting Human Communication Communica<on  in  the  sense  of  data  transmission  across   physical  distance  is  not  that  hard  today   •  Wired  and  wireless  computer  networking,  internet  etc.     Communica<on,  in  the  sense  of  understanding  each  other,  or   crossing  the  “psychological  distance”  between  people   remains  hard   •  Difficul<es  in  expressing  or  understanding  thoughts   •  Barriers  between  genera<ons,  genders,  professions,   languages,  and  cultures.       Suppor<ng  human  communica<on  con<nues  to  be  a   challenging  yet  worth-­‐of-­‐pursuing  topic  in  HCI.     12  
  •     Wang Ultimate Goal? Mind-Connecting! 13  
  •     Wang Lost in Technologies However,  technology  development  does  not  always  approach   the  goal  effec<vely.  For  example:     Video  conferencing   •  Bandwidth-­‐demanding.  Video  lagging     that  disrupts  conversa<on   •  Adop<on  is  not  guaranteed  .     Privacy  and  other  social  concerns   Machine  transla<on   •  Quality  concern   •  Influent  second  language  can  beat    MT  (cf.  Yamashita  &  Ishida,  2006).     14  
  •     Wang Observation Designs  of  CMC  can  work  be-er  when  features  and   constraints  of  human  communica<on  are  inves<gated   and  considered.     Ex.  Awareness  indicator  that  makes     “typing”  visible  in  instant  messaging.     Basic  research  stays  relevant!   What  are  the  features  of  successful  and  unsuccessful   communica<on?     What’s  the  nature  of  “understanding”? 15  
  •     Wang Grounding Communication 16  
  •     Wang Language as a Cognitive Inquiry 17   Noam  Chomsky
  •     Wang Language as a Social Inquiry 18   Pragma<cs  (linguis<cs),  language  use  (psychology,   communica<on)
  •     Wang How Would You Describe… Where  you  live  in  Hsinchu?     Where  you  lived  when  you  were  in  U.S.?     19  
  •     Wang My Answer Where  you  live  in  Hsinchu?     Near  清大後門.     Where  you  lived  when  you  were  in  U.S.?     In  Ithaca,  a  college  town  in  the  middle  of  New  York  state   if  you  know  where  it  is.  It’s  where  Cornell  University  is   located.     20  
  •     Wang My Answer Where  you  live  in  Hsinchu?     Near  清大後門.     Where  you  lived  when  you  were  in  U.S.?     In  Ithaca,  a  college  town  in  the  middle  of  New  York  state   if  you  know  where  it  is.  It’s  where  Cornell  University  is   located.     Do  you  see  the  general  difference?  Why?     21  
  •     Wang My Answer Where  you  live  in  Hsinchu?     Near  清大後門.     Where  you  lived  when  you  were  in  U.S.?     In  Ithaca,  a  college  town  in  the  middle  of  New  York  state   if  you  know  where  it  is.  It’s  where  Cornell  University  is   located.     Do  you  see  the  general  difference?  Why?  The  amount  of   knowledge  that  we  shared.     22  
  •     Wang Common Ground 23   Knowledge,  beliefs,  aitudes  we  share,   and  know  that  we  share,     and  know  that  we  know  that  we  share,     influence  how  we  use  language  to  communicate.     Grounding:  Interac<ve  process   by  which  communicators  exchange     evidence  of  their  understanding  to     arrive  at  the  state  of  common  ground.   Herbert  Clark
  •     Wang Evidence of Common Ground Physical  co-­‐presence  (being  co-­‐located)   •  “close  that  door”   Shared  community  membership   •  “Let’s  meet  at  小七”   Linguis<c  co-­‐presence  (can  access  same  u-erances)   24  
  •     Wang Evidence of Common Ground Physical  co-­‐presence  (being  co-­‐located)   •  “close  that  door”   Shared  community  membership   •  “Let’s  meet  at  小七”   Linguis<c  co-­‐presence  (can  access  same  u-erances)   25   “What’s  this?”  
  •     Wang Grounding is a Collaborative Process 26  
  •     Wang The Role of Media: Affordances An  influen<al  HCI-­‐rooted  concept,  which  roughly  means   “ac<on-­‐permiing  proper<es”  of  objects  that  people  see   •  Chair  affords  siing   •  Door-­‐knob  affords  door-­‐opening   •  Virtual  keyboard  affords  typing     (but  is  this  trivial?)   27   Don  Norman
  •     Wang Affordances of Communication Media 28  
  •     Wang Technology Changes Grounding Affordances  of  media  constrain  how  people  may  interact   with  one  another   •  E.g.,  if  no  visibility,  impossible  to  use  head-­‐nodding  as  a   technique  for  grounding     People  may  learn  to  adapt  their  grounding  behaviors   (this  happens.  E.g.,  emo<cons  in  IM)   or   Design  new  CMC  tools  with  useful  proper7es  to  support   grounding  and  communica7on.   29  
  •   30
  •   使用體感裝置探討在電腦中介傳播下之手勢使用行為 使用電腦作為訊息傳遞媒介進行人與人間的溝通已經是一個普遍的現象,我們亟需瞭解以電腦為中介 之溝通模式與面對面溝通的模式之間到底有那些差異,對於人際溝通的影響為何。過去這方面的研究 多著重在媒介的性質對於信任及語言使用的影響。對於非語言的溝通行為,例如溝通手勢的使用則探 討有限,其中一個原因在於缺少可快速有效量測細微手勢的方法。本論文提出一個應用技術,利用體 感裝置Microsoft Kinect來捕捉人與人溝通時肢體動作細微的變化。透過對肢體移動速度的分析以及多 重特徵值的截取,我們得以實驗比較面對面溝通(Face-to-Face)、視訊通訊(Video)與音訊通訊(Audio)三 種不同媒介對於溝通手勢行為所產生的影響,包括了手勢使用的程度以及兩個溝通者間行為的相似度。 此運用體感裝置作為行為科學量測工具的方法可用於快速評估新設計之線上溝通介面對於溝通行為的 影響,亦可用於傳播理論研究之發展與探討。在設計上,所提出之資料收集與分析方法亦可能作為未 來電腦中介傳播工具設計的基礎。 Microsoft Research Asia UR Project: FY13-RES-OPP-027 Wang, H-C., & Lai, C-T. (accepted). Kinect-taped Communication: Using Motion Sensing to Study Gesture Use and Similarity in Face-to-Face and Computer-Mediated Brainstorming. ACM Conference on Human Factors in Computing Systems (CHI) 2014. Full paper. [Acceptance rate: 22.8%]
  • Kinect-taped Communication:  Using Motion Sensing to Study Gesture Use  and Similarity in Face-to-Face and  Computer-Mediated Brainstorming Hao-Chuan Wang, Chien-Tung Lai National Tsing Hua University, Taiwan
  • [cf.  Bos  et  al.,  2002;  Setlock  et  al.,  2004;  Scissors  et  al.,  2008,  Wang  et  al.,  2009] Computer-mediated communication (CMC) tools are prevalent, but are they all equal? •  Ex. Video vs. Audio Media properties influence aspects of communication differently •  Task performance, grounding, styles, similarity of language patterns, social processes and outcomes etc. How media influence communication?
  • Communication could be more than speaking. Both verbal and non-verbal channels are active during conversations. Facial  expression Gesture [cf.  Goldin-­‐Meadow,  1999;  Giles  &  Coupland,  1991  ] The (missing) non-verbal aspect in CMC research
  • Studying gesture use in communication Current methods: •  Videotaping with manual coding. •  Giving specific instructions to participants (e.g., to gesture or not). •  Using confederates etc. Problems to solve: •  High cost. Labor-intensiveness. •  Resolution of manual analysis- Hard to recognize and reliably label small movements. •  Scalability- Hard to study arbitrary communication in the wild.
  • “Kinect-taping”method Like videotaping, we use motion sensing devices, such as Microsoft Kinect, to record hand and body movements during conversations. •  Detailed, easier-to-process representations. •  Behavioral science instrument (“microscope”) to study non-verbal communication in ad hoc groups. •  Low cost if automatic measures are satisfactory.
  • Re-appropriating motion sensors in HCI: Sensing-aided user research for  future designs From sensors as design elements to sensors as research instruments to help future designs. ! (a)!Face(to(face!(F2F)!communication! [cf.  Mark  et  al.,  2014]
  • A media comparison study Investigate how people use gestures during face- to-face and computer-mediated brainstorming Compare three communication media •  Face-to-Face •  Video •  Audio ! (a)!Face(to(face!(F2F)!communication! ! (b)!Video(mediated!communication! Figure'1.'A'sample'study'setting'that'compares'(a)'F2F'to'(b)'video<mediated'communication' by'using'Kinect'as'a'behavioral'science'instrument.' !
  • Hypotheses H1. Visibility increases gesture use Proportion of gesture Face-to-Face > Video > Audio H2. Visibility increases accommodation Similarity between group members’ gestures Face-to-Face > Video > Audio Also explore how gesture use, level of understanding, and ideation productivity correlate. [cf.  Clark  &  Brennan,  1991] [cf.  Giles  &  Coupland,  1991]
  • Experimental design 36 individuals, 18 two-person groups Kinect-taped group brainstorming sessions Face-to-Face Video Audio Three  trials  (15  min  each)     in  counterbalanced  order   Data analysis Amount and similarity of gestures, Level of understanding, Productivity
  • How to quantify gestures? How many gestures are there in a 15 min talk?
  • moving not moving
  • Two unit motions with speed threshold 0
  • Three unit motions with speed threshold 2
  • Choose the thresholds (m/s)
  • Choose the thresholds Too  few  signals Almost  everything Data  points  of  interest (m/s)
  • How to measure similarity between unit motions?
  • Feature extraction and representation Unit motions are represented as feature vectors •  Time length, path length, displacement, velocity, speed, angular movement etc. •  Features extracted for both hands and both elbows. 73 features extracted for each unit motion. Similarity between unit motions: Cosine value between the two vectors.
  • Validating the similarity metric 1 2 3 Machine Ranking Human Ranking 1 2 3 Randomly select motion queries Retrieve similar and dissimilar motions Kinect-taped motion database
  • Count Human Rank R1 R2 R3 Machine Rank R1 29 2 5 R2 7 27 2 R3 0 7 29 x2=107.97,  p<.001 Validating the similarity metric Contingency analysis
  • H1: Amount of gesture use H2: Similarity between group members Associations •  Amount of gesture and understanding •  Amount of gesture and ideation productivity •  Gesture similarity and ideation productivity Key Results
  • Visibility on proportion of gesture use 0 2 4 6 8 10 12 14 16 Face-to-face Video Audio ProportionofGestureUse(%) H1 not supported. Media did not influence percentage of gesture. People gesture as much in Audio as in F2F and Video.
  • Association between self-gesture and level of understanding Model&Predicted,UnderstandingModel&Predicted,Numb Propor9on,of,Individual’s,Own,Gesture,Use,(%) Audio F2F Video Individual’s Own Gesture Use (%) Non-communicative function of gesture. Understanding correlates with self-gesture but not partner-gesture Stronger correlation with reduced or no visibility.
  • Similarity between group members 0.46 0.47 0.48 0.49 0.5 0.51 0.52 0.53 0.54 0.55 Face-to-face Video Audio Between-participantGestural Similarity H2 supported. Similarity F2F > Video > Audio. People gesture more similarly when they can see each other.
  • Summary and implications   Media   Comparison     Study Kinect- taping Method
  • Motion sensing for studying non-verbal behaviors in CMC. Summary and implications   Media   Comparison     Study Kinect- taping Method Visibility influences similarity but not amount of gesture. Only self-gesture correlates with understanding. Gesture doesn’t seem to convey much meaning to the partner. Seeing the partner is not crucial to understanding.
  • Study communication of ad hoc groups in the wild. Distributed deployment study of CMC tools. Cross-lingual and cross-cultural communication. Summary and implications (cont.)   Media   Comparison     Study Kinect- taping Method The value of video may be relatively limited to the social and collaborative aspect (similarity etc.). Feedback that promotes self- gesturing may help understanding.
  •     Wang Key Messages   Suppor<ng  human  communica<on  con<nues  to  be  an  important   topic  in  HCI,  both  to  research  and  design  prac<ce.   •  Focusing  on  how  to  shorten  the  “psychological  distance”   between  people.  “Mind-­‐connec<ng”!   Basic  and  applied  behavioral,  cogni<ve  and  social  sciences  helps   to  understand  the  features  of  successful  and  unsuccessful   communica<on   •  Insight  that  we  should  focus  on  CMC  affordances  as  much  as   technicality.   Interdisciplinary  work  can  benefit  both  sides:  Social  and   behavioral  sciences  help  technology  design,  and  vice  versa.   59  
  •     Wang Ultimate Goal? Mind-Connecting! 60  
  •     Wang 61   國立清華大學人機合作與社群運算實驗室 NTHU  Collabora<ve  and  Social  Compu<ng  Lab  (CSC  Lab)   Acknowledgement  for  Support  from     Na<onal  Science  Council,  Taiwan   Google  Inc.   Microsou  Research  Asia  微軟亞洲研究院 Industrial  Technology  Research  Ins<tute  (ITRI)  工業技術研究院 Delta  Corp  台達電子公司 Na<onal  Science  Founda<on,  USA