Your SlideShare is downloading. ×
0
where	
  user	
  experience	
  and
               software	
  engineering	
  meet
                            Andrew	
  J....
in	
  method,	
  I’m	
  an	
  HCI	
  researcher


                                          I	
  study	
  
               ...
in	
  method,	
  I’m	
  an	
  HCI	
  researcher


                                          I	
  study	
  
               ...
in	
  topic,	
  I’m	
  an	
  SE	
  researcher


                     I	
  want	
  to	
  make	
  it	
  easy	
  to	
  
     ...
human-­‐computer	
       software	
  
      interaction           engineering


                            me




       ...
human-­‐computer	
                         software	
  
      interaction                             engineering


    ho...
user-­‐centered	
  
                            software	
  evolution



                            how	
  do	
  we	
  ge...
talk	
  outline

                                   why	
  is	
  software	
  evolution	
  difficult?
                       ...
talk	
  outline

                              why	
  is	
  software	
  evolution	
  difficult?
                            ...
information	
  needs	
  at	
  Microsoft
      with	
  the	
  Human	
  Interactions	
  in	
  Programming	
  group	
  at	
  ...
17	
  developers	
  
  hard	
  at	
  work	
  
  across	
  25	
  hours
   Visual	
  Studio
   Windows	
  Vista
   service	
...
8	
  activities
  writing	
  code
  submitting	
  code
  triaging	
  bugs
  reproducing	
  a	
  failure
  understanding	
 ...
8	
  activities
  writing	
  code
  submitting	
  code
  triaging	
  bugs
  reproducing	
  a	
  failure
  understanding	
 ...
9	
  reasons	
  for	
  
  switching	
  tasks
 face	
  to	
  face	
  conversation
 phone	
  call
 instant	
  message
 e-­‐m...
9	
  reasons	
  for	
  
  switching	
  tasks
 face	
  to	
  face	
  conversation
 phone	
  call
 instant	
  message
 e-­‐m...
software	
  
                             interrupted	
  every	
  
  development	
                ~5-­‐10	
  minutes
  wor...
next,	
  we	
  looked	
  
 for	
  information	
  
 that	
  developers	
  
 sought	
  to	
  get	
  
 their	
  work	
  
 don...
next,	
  we	
  looked	
  
 for	
  information	
  
 that	
  developers	
  
 sought	
  to	
  get	
  
 their	
  work	
  
 don...
21	
  information	
  needs	
  observed	
  (by	
  frequency)
      what	
  have	
  my	
  coworkers	
  been	
  doing?	
  
  ...
5	
  information	
  needs	
  least	
  often	
  satisfied
      what	
  have	
  my	
  coworkers	
  been	
  doing?	
         ...
3	
  were	
  debugging	
  related	
  ...
      what	
  have	
  my	
  coworkers	
  been	
  doing?	
  
      what	
  code	
 ...
2	
  were	
  design	
  related
      what	
  have	
  my	
  coworkers	
  been	
  doing?	
  
      what	
  code	
  could	
  ...
software	
  development	
  is	
  tacit
                            plans	
  and	
  specifications	
  are	
  unwritten




 ...
software	
  quality	
  depends	
  highly	
  on	
  the	
  quality	
  of	
  
         human	
  communication	
  and	
  cogni...
talk	
  outline

                              why	
  is	
  software	
  evolution	
  difficult?
                            ...
talk	
  outline

                              why	
  is	
  software	
  evolution	
  difficult?
                            ...
why	
  is	
  debugging	
  such	
  a	
  challenge?
                                                 wrong!


              ...
Whyline
                       a	
  Workspace	
  for	
  Helping	
  You	
  Link	
  Instructions,	
  Numbers,	
  and	
  Even...
an	
  example	
  ...

                            why	
  was	
  the	
  line	
  black?




                                ...
an	
  example	
  ...

                            why	
  was	
  the	
  line	
  black?




                                ...
record	
  the	
  problem
                                                       25
Monday, November 30, 2009
record	
  the	
  problem
                                                       25
Monday, November 30, 2009
load	
  the	
  recording
                                                       26
Monday, November 30, 2009
why	
  was	
  the	
  line	
  color	
  black?   27
Monday, November 30, 2009
why	
  was	
  the	
  line	
  color	
  black?   27
Monday, November 30, 2009
why	
  was	
  the	
  line	
  color	
  black?   28
Monday, November 30, 2009
code




                                         executions	
  of	
  code
                                         (execu...
why	
  was	
  the	
  line	
  color	
  black?   29
Monday, November 30, 2009
followup	
  
                            questions	
  about	
  
                            selected	
  event




        ...
followup	
  
                                   questions	
  about	
  
                                   selected	
  even...
selected	
  
                             dependency	
  
                             highlighted	
  
                    ...
why	
  was	
  the	
  line	
  color	
  black?   30
Monday, November 30, 2009
because	
  gSlider	
  
                                   was	
  used	
  twice,	
  
                                   ign...
a	
  comparison	
  study


                                                  vs

         Whyline	
                       ...
subject	
  program
       ArgoUML,	
  an	
  open	
  source	
  
       software	
  design	
  tool
       ~150,000	
  lines	...
bug	
  1	
  results
                                        whyline           control

                            #	
  su...
bug	
  2	
  results
                                      whyline                control

                            #	
 ...
unsolicited	
  quotes	
  from	
  users
             “This	
  is	
  great,	
  when	
  can	
  I	
  get	
  this	
  for	
  C?”...
talk	
  outline

                              why	
  is	
  software	
  evolution	
  difficult?
                            ...
talk	
  outline

                                   why	
  is	
  software	
  evolution	
  difficult?
                       ...
does	
  open	
  bug	
  reporting	
  work?
   with	
  my	
  PhD	
  student,	
  Parmit	
  Chilana



          comprehensive...
four	
  types	
  of	
  contributors
           1%    CORE	
  devs	
  drivers,	
  super	
  reviewers,	
  module	
  owners,	...
#	
  of	
  comment	
  contributors	
  over	
  time
          (#$!!!"
                                               !"#$%&...
#	
  of	
  comment	
  contributors	
  over	
  time
          (#$!!!"
                                               !"#$%&...
resolution	
  by	
  reporter	
  type
          $!!"#
           ,!"#
           +!"#                                      ...
resolution	
  by	
  reporter	
  type
          $!!"#
           ,!"#
           +!"#                                      ...
%	
  reports	
  FIXED	
  by	
  each	
  type

           (!!"#
                                                            ...
%	
  reports	
  FIXED	
  by	
  each	
  type

           (!!"#
                                                            ...
why	
  are	
  REPORTERs	
  ineffective?
        sampled	
  and	
  categorized	
  100	
  reports	
  of	
  each	
  
        r...
is	
  open	
  bug	
  reporting	
  useful?
        yes,	
  but	
  ...
              ■ significant	
  overhead	
  to	
  proce...
ongoing	
  work

        enabling	
  every	
  user	
  to	
  submit
                                            precise
   ...
three	
  takeaway	
  points
              software	
  quality	
  depends	
  highly	
  on	
  the	
  quality	
  
           ...
Upcoming SlideShare
Loading in...5
×

Where User Experience And Software Engineering Meet

442

Published on

My talk at CHOOSE 2009 in Bern, Switzerland.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
442
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Where User Experience And Software Engineering Meet"

  1. 1. where  user  experience  and software  engineering  meet Andrew  J.  Ko Monday, November 30, 2009
  2. 2. in  method,  I’m  an  HCI  researcher I  study   problems I  evaluate   I  design   solutions solutions 2 Monday, November 30, 2009
  3. 3. in  method,  I’m  an  HCI  researcher I  study   problems I  evaluate   I  design   solutions solutions 2 Monday, November 30, 2009
  4. 4. in  topic,  I’m  an  SE  researcher I  want  to  make  it  easy  to   create  useful,  usable  software that  empowers   and  enriches   users’  lives 3 Monday, November 30, 2009
  5. 5. human-­‐computer   software   interaction engineering me 4 Monday, November 30, 2009
  6. 6. human-­‐computer   software   interaction engineering how  do  we  get  the   how  do  get  the   right  design? design  right ? (the  lives  of  users) (the  lives  of  developers) 5 Monday, November 30, 2009
  7. 7. user-­‐centered   software  evolution how  do  we  get  the right  design right? design   6 Monday, November 30, 2009
  8. 8. talk  outline why  is  software  evolution  difficult? a  study  of  information  needs  at  Microsoft how  can  tools  help  software  evolution? (#$!!!" (!$!!!" debugging  with  the  Whyline !"#$%&'(, '$!!!" 9/829?/9B" how  can  users  help  software  evolution? &$!!!" !"#$%&'(+ !"#$%&'(- 5B/9B" %$!!!" !"#$%&'()*+ *>C3/" #$!!!" !" >29/" recent  work  in  leveraging  the  crowd )*+",-'" )*+",!(" ./0",--" ./0",!#" 489",!%" 489",!A" =>?",!<" =>?",!'" :5@",!&" 123",--" 123",!#" :*;",!<" :*;",!'" 456",!!" 456",!7" 7 Monday, November 30, 2009
  9. 9. talk  outline why  is  software  evolution  difficult? a  study  of  information  needs  at  Microsoft how  can  tools  help  software  evolution? debugging  with  the  Whyline !"#$%&'()*+ !"#$% how  can  users  help  software  evolution? recent  work  in  leveraging  the  crowd )*+",!(" ./0",!#" 489",!%" 123",!#" 456",!!" 456",!7" 8 Monday, November 30, 2009
  10. 10. information  needs  at  Microsoft with  the  Human  Interactions  in  Programming  group  at  Microsoft  Research observed  25  hours  of  coding  and  bug   fixing,  in  the  role  of  “new  hires” 357  pages  of  handwritten  notes 4,231  events  in   an  spreadsheet 9 Monday, November 30, 2009
  11. 11. 17  developers   hard  at  work   across  25  hours Visual  Studio Windows  Vista service  packs mobile  devices discussion  boards educational  tools SQL  server MS  Office Encryption DRM ... 10 Monday, November 30, 2009
  12. 12. 8  activities writing  code submitting  code triaging  bugs reproducing  a  failure understanding  behavior reasoning  about  design maintaining  awareness non-­‐work  activity 11 Monday, November 30, 2009
  13. 13. 8  activities writing  code submitting  code triaging  bugs reproducing  a  failure understanding  behavior reasoning  about  design maintaining  awareness non-­‐work  activity 11 Monday, November 30, 2009
  14. 14. 9  reasons  for   switching  tasks face  to  face  conversation phone  call instant  message e-­‐mail  alerts bug  report  change  alerts task  avoidance getting  blocked meetings task  completion 12 Monday, November 30, 2009
  15. 15. 9  reasons  for   switching  tasks face  to  face  conversation phone  call instant  message e-­‐mail  alerts bug  report  change  alerts task  avoidance getting  blocked meetings task  completion 12 Monday, November 30, 2009
  16. 16. software   interrupted  every   development   ~5-­‐10  minutes work  is  highly   fragmented blocked  every   ~10  minutes 13 Monday, November 30, 2009
  17. 17. next,  we  looked   for  information   that  developers   sought  to  get   their  work   done  ... 14 Monday, November 30, 2009
  18. 18. next,  we  looked   for  information   that  developers   sought  to  get   their  work   done  ... 14 Monday, November 30, 2009
  19. 19. 21  information  needs  observed  (by  frequency) what  have  my  coworkers  been  doing?   what  code  could  have  caused  this  behavior?   have  resources  I  depend  on  changed?   what  code  caused  this  program  state? how  do  I  use  this  data  structure  or  function? did  I  make  any  mistakes? what  is  the  program  supposed  to  do? in  what  situations  does  this  failure  occur? is  this  problem  worth  fixing? why  was  this  code  implemented  this  way? what’s  statically  related  to  this  code? what  are  the  implications  of  this  change? 15 Monday, Novemberdoes  the  failure  look  like? what   30, 2009
  20. 20. 5  information  needs  least  often  satisfied what  have  my  coworkers  been  doing?   %  unsatisfied what  code  could  have  caused  this  behavior?   36% have  resources  I  depend  on  changed?   what  code  caused  this  program  state? 61% how  do  I  use  this  data  structure  or  function? did  I  make  any  mistakes? what  is  the  program  supposed  to  do? 15% in  what  situations  does  this  failure  occur? 41% is  this  problem  worth  fixing? why  was  this  code  implemented  this  way? 44% what’s  statically  related  to  this  code? what  are  the  implications  of  this  change? 16 Monday, Novemberdoes  the  failure  look  like? what   30, 2009
  21. 21. 3  were  debugging  related  ... what  have  my  coworkers  been  doing?   what  code  could  have  caused  this  behavior? reproducing,   have  resources  I  depend  on  changed?   diagnosing,   what  code  caused  this  program  state? and  scoping   how  do  I  use  this  data  structure  or  function? failures  were   did  I  make  any  mistakes? the  most  time-­‐ what  is  the  program  supposed  to  do? consuming   activities in  what  situations  does  this  failure  occur? is  this  problem  worth  fixing? why  was  this  code  implemented  this  way? what’s  statically  related  to  this  code? what  are  the  implications  of  this  change? 17 Monday, Novemberdoes  the  failure  look  like? what   30, 2009
  22. 22. 2  were  design  related what  have  my  coworkers  been  doing?   what  code  could  have  caused  this  behavior? have  resources  I  depend  on  changed?   what  code  caused  this  program  state?   how  do  I  use  this  data  structure  or  function? did  I  make  any  mistakes? what  is  the  program  supposed  to  do? knowing  what   in  what  situations  does  this  failure  occur? software   is  this  problem  worth  fixing? should  do  was   rarely  known why  was  this  code  implemented  this  way? what’s  statically  related  to  this  code? what  are  the  implications  of  this  change? 18 Monday, Novemberdoes  the  failure  look  like? what   30, 2009
  23. 23. software  development  is  tacit plans  and  specifications  are  unwritten developers  have  to   communicate  to  make  progress 19 Monday, November 30, 2009
  24. 24. software  quality  depends  highly  on  the  quality  of   human  communication  and  cognition these  human  activities  are  faulty  and  unreliable 20 Monday, November 30, 2009
  25. 25. talk  outline why  is  software  evolution  difficult? a  study  of  information  needs  at  Microsoft how  can  tools  help  software  evolution? debugging  with  the  Whyline !"#$%&'()*+ !"#$% how  can  users  help  software  evolution? recent  work  in  leveraging  the  crowd )*+",!(" ./0",!#" 489",!%" 123",!#" 456",!!" 456",!7" 21 Monday, November 30, 2009
  26. 26. talk  outline why  is  software  evolution  difficult? a  study  of  information  needs  at  Microsoft how  can  tools  help  software  evolution? debugging  with  the  Whyline !"#$%&'()*+ !"#$% how  can  users  help  software  evolution? recent  work  in  leveraging  the  crowd )*+",!(" ./0",!#" 489",!%" 123",!#" 456",!!" 456",!7" 21 Monday, November 30, 2009
  27. 27. why  is  debugging  such  a  challenge? wrong! observe   guess  the   check  the   solved! failure cause guess 22 Monday, November 30, 2009
  28. 28. Whyline a  Workspace  for  Helping  You  Link  Instructions,  Numbers,  and  Events observe   click  the   follow   solved! failure faulty  output dependencies 23 Monday, November 30, 2009
  29. 29. an  example  ... why  was  the  line  black? 24 Monday, November 30, 2009
  30. 30. an  example  ... why  was  the  line  black? 24 Monday, November 30, 2009
  31. 31. record  the  problem 25 Monday, November 30, 2009
  32. 32. record  the  problem 25 Monday, November 30, 2009
  33. 33. load  the  recording 26 Monday, November 30, 2009
  34. 34. why  was  the  line  color  black? 27 Monday, November 30, 2009
  35. 35. why  was  the  line  color  black? 27 Monday, November 30, 2009
  36. 36. why  was  the  line  color  black? 28 Monday, November 30, 2009
  37. 37. code executions  of  code (execution  events) why  was  the  line  color  black? 28 Monday, November 30, 2009
  38. 38. why  was  the  line  color  black? 29 Monday, November 30, 2009
  39. 39. followup   questions  about   selected  event why  was  the  line  color  black? 29 Monday, November 30, 2009
  40. 40. followup   questions  about   selected  event why  was  the  line  color  black? 29 Monday, November 30, 2009
  41. 41. selected   dependency   highlighted   in  source why  was  the  line  color  black? 29 Monday, November 30, 2009
  42. 42. why  was  the  line  color  black? 30 Monday, November 30, 2009
  43. 43. because  gSlider   was  used  twice,   ignoring  bSlider why  was  the  line  color  black? 30 Monday, November 30, 2009
  44. 44. a  comparison  study vs Whyline   control group   group both  groups  had  modern  IDE  features    show  declaration,  show  callers,  show  references,  etc. 31 Monday, November 30, 2009
  45. 45. subject  program ArgoUML,  an  open  source   software  design  tool ~150,000  lines  of  code 22  external  libraries chose  two  bug  reports   from  version  18.1 ■ one  w/  simple  fix ■ one  w/  complex  fix 32 Monday, November 30, 2009
  46. 46. bug  1  results whyline control #  successful minutes 10 30 8 24 6 18 4 12 2 6 0 0 p<.05 p<.05 more  successful  in  half  the  time 33 Monday, November 30, 2009
  47. 47. bug  2  results whyline control #  successful minutes 10 10 8 4  of  10 8 6 gave  up 6 4 4 2 2 0 0 p<.05 more  successful  in  the  same  time 34 Monday, November 30, 2009
  48. 48. unsolicited  quotes  from  users “This  is  great,  when  can  I  get  this  for  C?” “My  god,  this  is  so  cool.” “It's  so  nice  and  straight  and  simple...” 35 Monday, November 30, 2009
  49. 49. talk  outline why  is  software  evolution  difficult? a  study  of  information  needs  at  Microsoft how  can  tools  help  software  evolution? debugging  with  the  Whyline !"#$%&'()*+ !"#$% how  can  users  help  software  evolution? recent  work  in  leveraging  the  crowd )*+",!(" ./0",!#" 489",!%" 123",!#" 456",!!" 456",!7" 36 Monday, November 30, 2009
  50. 50. talk  outline why  is  software  evolution  difficult? a  study  of  information  needs  at  Microsoft how  can  tools  help  software  evolution? (#$!!!" (!$!!!" debugging  with  the  Whyline !"#$%&'(, '$!!!" 9/829?/9B" how  can  users  help  software  evolution? &$!!!" !"#$%&'(+ !"#$%&'(- 5B/9B" %$!!!" !"#$%&'()*+ *>C3/" #$!!!" !" >29/" recent  work  in  leveraging  the  crowd )*+",-'" )*+",!(" ./0",--" ./0",!#" 489",!%" 489",!A" =>?",!<" =>?",!'" :5@",!&" 123",--" 123",!#" :*;",!<" :*;",!'" 456",!!" 456",!7" 36 Monday, November 30, 2009
  51. 51. does  open  bug  reporting  work? with  my  PhD  student,  Parmit  Chilana comprehensive  analysis  of  ~500,000   reports  from  the  Mozilla  community ■ quantitative  analysis  of  to  characterize  bug   report  resolution  trends ■ qualitative  analysis  to  explain  bug  report   resolution  trends 37 Monday, November 30, 2009
  52. 52. four  types  of  contributors 1% CORE  devs  drivers,  super  reviewers,  module  owners,  peers 1% ACTIVE  devs  assigned  bug  reports 80% REPORTERS 18% USERS reported  and   only  commented   commented  on   on  bug  reports bug  reports 38 Monday, November 30, 2009
  53. 53. #  of  comment  contributors  over  time (#$!!!" !"#$%&'(, (!$!!!" '$!!!" 9/829?/9B" &$!!!" !"#$%&'(+ !"#$%&'(- 5B/9B" %$!!!" !"#$%&'()*+ *>C3/" #$!!!" !" >29/" )*+",-'" )*+",!(" ./0",--" ./0",!#" 489",!%" 489",!A" =>?",!<" =>?",!'" :5@",!&" 123",--" 123",!#" :*;",!<" :*;",!'" 456",!!" 456",!7" REPORTERs  are  the  most  active  commenters 39 Monday, November 30, 2009
  54. 54. #  of  comment  contributors  over  time (#$!!!" !"#$%&'(, (!$!!!" '$!!!" 9/829?/9B" &$!!!" !"#$%&'(+ !"#$%&'(- 5B/9B" %$!!!" !"#$%&'()*+ *>C3/" #$!!!" !" >29/" )*+",-'" )*+",!(" ./0",--" ./0",!#" 489",!%" 489",!A" =>?",!<" =>?",!'" :5@",!&" 123",--" 123",!#" :*;",!<" :*;",!'" 456",!!" 456",!7" REPORTERs  are  the  most  active  commenters 39 Monday, November 30, 2009
  55. 55. resolution  by  reporter  type $!!"# ,!"# +!"# 67/89321.# *!"# )!"# 9:30;/8.1.# (!"# <0:=9># '!"# <0-?@A0-;.# &!"# %!"# 9:52896# $!"# B>.6# !"# -./0-1.-# 2345.# 30-.# most  REPORTER  reports  are  not  FIXED 40 Monday, November 30, 2009
  56. 56. resolution  by  reporter  type $!!"# ,!"# +!"# 67/89321.# *!"# )!"# 9:30;/8.1.# (!"# <0:=9># '!"# <0-?@A0-;.# &!"# %!"# 9:52896# $!"# B>.6# !"# -./0-1.-# 2345.# 30-.# most  REPORTER  reports  are  not  FIXED 40 Monday, November 30, 2009
  57. 57. %  reports  FIXED  by  each  type (!!"# !"#$%&'()*+ !"#$%&'(, !"#$%&'(- '!"# &!"# !"#$%&'(+ 07+/# %!"# $!"# *0A8/# !"# +/27+B/+# )*+#,-'# ./0#,-'# =/>#,!%# =/>#,!@# 1/2#,--# )*+#,!(# 94:#,!?# 94:#,!'# 678#,!(# 678#,!@# 678#,!%# 94:#,!$# 345#,!!# )*;#,!<# REPORTERs  have  dropped  in  effectiveness )*;#,!&# 41 Monday, November 30, 2009
  58. 58. %  reports  FIXED  by  each  type (!!"# !"#$%&'()*+ !"#$%&'(, !"#$%&'(- '!"# &!"# !"#$%&'(+ 07+/# %!"# $!"# *0A8/# !"# +/27+B/+# )*+#,-'# ./0#,-'# =/>#,!%# =/>#,!@# 1/2#,--# )*+#,!(# 94:#,!?# 94:#,!'# 678#,!(# 678#,!@# 678#,!%# 94:#,!$# 345#,!!# )*;#,!<# REPORTERs  have  dropped  in  effectiveness )*;#,!&# 41 Monday, November 30, 2009
  59. 59. why  are  REPORTERs  ineffective? sampled  and  categorized  100  reports  of  each   resolution  type... most  REPORTER  reports  =  technical  support  for   power  users’  tinkering  and  using  old  builds rarely  provided  static  and  dynamic  context   adequate  to  reproduce  problems reported  problems,  resolved  shortly  after   42 Monday, November 30, 2009
  60. 60. is  open  bug  reporting  useful? yes,  but  ... ■ significant  overhead  to  process  bad  reports ■ only  a  skewed  subset  of  users  report  bugs ■ users  who  report  bugs  are  bad  at  providing   the  static  and  dynamic  context  of  problems ■ text  is  a  terribly  imprecise  medium  for   expressing  this  context 43 Monday, November 30, 2009
  61. 61. ongoing  work enabling  every  user  to  submit precise structured aggregatable bug  reports with  zero  training? 44 Monday, November 30, 2009
  62. 62. three  takeaway  points software  quality  depends  highly  on  the  quality   of  human  communication  and  cognition human  communication  and  cognition  are   faulty  and  unreliable carefully  designed  interactive  tools  can   compensate  for  these  limitations questions? 45 Monday, November 30, 2009
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×