User	
  Involvement	
  in	
  
Software	
  Evolution	
  
Practice:	
  	
  
A	
  Case	
  Study	
  
Dennis	
  Pagano	
  (@den...
Foundations	
  
•  User	
  Involvement	
  
•  Aims	
  at	
  maximizing	
  system	
  usefulness	
  and	
  usability	
  by	
...
D.	
  Edsger	
  Dijkstra	
  D.	
  Edsger	
  Dijkstra	
  
Why	
  User	
  Involvement?	
  
«The	
  no7on	
  of	
  ’user’	
  ...
In?luential	
  Changes	
  
Dennis	
  Pagano	
  -­‐	
  TU	
  Munich	
  -­‐	
  May	
  2013	
  	
  
4	
  
•  From	
  programm...
Study	
  Goal	
  and	
  Questions	
  
Dennis	
  Pagano	
  -­‐	
  TU	
  Munich	
  -­‐	
  May	
  2013	
  	
  
5	
  
Understa...
Research	
  Method	
  and	
  Data	
  
Dennis	
  Pagano	
  -­‐	
  TU	
  Munich	
  -­‐	
  May	
  2013	
  	
  
6	
  
•  Semi-...
Dennis	
  Pagano	
  -­‐	
  TU	
  Munich	
  -­‐	
  May	
  2013	
  	
  
7	
  
1	
  
User	
  Involvement	
  Sefng	
  
Results...
User	
  Involvement	
  Setting	
  
•  Four	
  main	
  user	
  feedback	
  arYfacts:	
  Error	
  reports,	
  feature	
  
re...
2	
  
Results	
  
Dennis	
  Pagano	
  -­‐	
  TU	
  Munich	
  -­‐	
  May	
  2013	
  	
  
9	
  
User	
  Involvement	
  Workfl...
Bene?its	
  of	
  User	
  Feedback	
  
1.  Allows	
  for	
  conYnuous	
  assessment	
  of	
  product	
  acceptance	
  
(re...
User	
  Feedback	
  Analysis	
  
•  Developers	
  analyze	
  user	
  feedback	
  to	
  create	
  priori7zed	
  tasks	
  
fi...
1.  Natural	
  language	
  content,	
  low	
  feedback	
  quality,	
  
contradictory	
  feedback	
  
2.  Communica7on	
  g...
3	
  
Dennis	
  Pagano	
  -­‐	
  TU	
  Munich	
  -­‐	
  May	
  2013	
  	
  
13	
  
Tool	
  Requirements	
  
Results	
  
Tool	
  Requirements	
  
•  Consolidate,	
  structure,	
  analyze,	
  and	
  track	
  user	
  feedback	
  
•  Group	
  dup...
Implications	
  of	
  Findings	
  
Dennis	
  Pagano	
  -­‐	
  TU	
  Munich	
  -­‐	
  May	
  2013	
  	
  
15	
  
Researcher...
Feel	
  free	
  to	
  contact	
  us	
  for	
  feedback…	
  
16	
  
Dennis	
  Pagano	
  
TU	
  Munich	
  
pagano@cs.tum.edu...
Upcoming SlideShare
Loading in …5
×

User Involvement in Software Evolution Practice: A Case Study

1,008 views

Published on

User involvement in software engineering has been researched over the last three decades. However, existing studies concentrate mainly on early phases of user-centered design projects, while little is known about how professionals work with post-deployment end-user feedback. In this paper we report on an empirical case study that explores the current practice of user involvement during software evolution.
We found that user feedback contains important information for developers, helps to improve software quality and to identify missing features. In order to assess its relevance and potential impact, developers need to analyze the gathered feedback, which is mostly accomplished manually and consequently requires high effort. Overall, our results show the need for tool support to consolidate, structure, analyze, and track user feedback, particularly when feedback volume is high. Our findings call for a hypothesis-driven analysis of user feedback to establish the foundations for future user feedback tools.

Published in: Technology, Spiritual
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,008
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

User Involvement in Software Evolution Practice: A Case Study

  1. 1. User  Involvement  in   Software  Evolution   Practice:     A  Case  Study   Dennis  Pagano  (@dennispagano),  Bernd  Bruegge   Technische  Universität  München  
  2. 2. Foundations   •  User  Involvement   •  Aims  at  maximizing  system  usefulness  and  usability  by   understanding  users’  needs  and  expecta0ons     •  So3ware  Evolu7on   •  So<ware  has  been  delivered  (this  is  a  simplified  view)   Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013     2  
  3. 3. D.  Edsger  Dijkstra  D.  Edsger  Dijkstra   Why  User  Involvement?   «The  no7on  of  ’user’  cannot  be  precisely  defined,   and  therefore  has  no  place  in  computer  science  or   so>ware  engineering.»       A.  Alexander  G.  Bell   B.  Michael  Jackson   C.  Christopher  Columbus   That  was  in  1979.  Of  course,  the  rest  is  history.   3   Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013    
  4. 4. In?luential  Changes   Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013     4   •  From  programmers  to   arbitrary  persons   •  From  few  vicYms  to  many   paramount  stakeholders   •  More  demanding   Users   •  From  expensive  scienYfic   programs  to  cheap  apps   •  Running  on  new  devices   •  Delivered  via  applicaYon   distribuYon  pla[orms   So3ware   •  Users  and  use  context  unknown  upfront   •  Increasing  distance  between  users  and  developers   ➡  User  involvement  changed:  from  user-­‐centered  design  to               con7nuous  interpreta7on  of  post-­‐deployment  user  feedback  
  5. 5. Study  Goal  and  Questions   Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013     5   Understand  current  prac7ce  of     user  involvement  during  so3ware  evolu7on   Workflow:   Why  and  how  do   developers  work   with  user   feedback,  and   which  problems   arise?   2       Requirements:   What  do   developers   expect  from  tool   support  for  user   involvement?   3       SeIng:   How,  when,  and   where  may  users   provide  feedback?       1  
  6. 6. Research  Method  and  Data   Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013     6   •  Semi-­‐structured,  open  ended  interviews   •  Audio  recording  and  manual  transcripYon   •  Coding  and  building  hypotheses   •  Five  developers  from  small  and  medium  sized   companies  (SMEs)   •  Focus:  Large  number  of  applicaYon  users  (>  10,000)   •  Different  audiences  (consumers  and  professionals)   •  Different  plaOorms  (desktop  and  mobile)   Method  Data  
  7. 7. Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013     7   1   User  Involvement  Sefng   Results  
  8. 8. User  Involvement  Setting   •  Four  main  user  feedback  arYfacts:  Error  reports,  feature   requests,  feedback  on  exis7ng  features,  ra7ngs   •  User  feedback  scaQered:  email,  applicaYon  distribuYon   pla[orms,  integrated  feedback  mechanisms  etc.   •  Users  inten7onally  select  feedback  channel:  The  more  cri7cal   the  feedback,  the  more  public  the  channel   •  Users  not  systema7cally  involved:  No  agreed  pracYce  how  to   provide  nor  how  to  gather  user  feedback   Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013     8   1  
  9. 9. 2   Results   Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013     9   User  Involvement  Workflow  
  10. 10. Bene?its  of  User  Feedback   1.  Allows  for  conYnuous  assessment  of  product  acceptance   (real-­‐world  usage  data)   2.  Helps  to  improve  so3ware  quality   3.  PosiYve  raYngs  and  user  experience  convey  trust:   adver7sement  and  marke7ng   4.  Helps  idenYfying  missing  features,  BUT  developers  need  to   assess  how  many  users  will  benefit  from  new  feature   Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013     10   2  
  11. 11. User  Feedback  Analysis   •  Developers  analyze  user  feedback  to  create  priori7zed  tasks   fifng  into  current  product  roadmap   •  Feedback  importance  depends  on  frequency  of  its  occurrence   !"#$"%&'(#)"&% *+,#-),%&./$0"%1""23-)4&% 5&&"&&%./2.6.2(-0%7#.'#.,8% *&9:-,"%.:7-),% ;0-&&.18%1""23-)4% !"#$%&'()*+,&' !"#$%&',-*#.(' /(&+'0&&123,.' !"#$%"%&'()!*%+#) 4+")+"56' 7893,5' ,"-%.#)!*%+#) :++)+'+&9)+5''#'0&35*+&'+&;*&(5' </&"#,%./%)'/6"/9'/-0% ,''0&%-/2%='#4>'=&% Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013     11   2  
  12. 12. 1.  Natural  language  content,  low  feedback  quality,   contradictory  feedback   2.  Communica7on  gaps  in  user  feedback  channels  disconnect   users  and  developers   Problems  and  Challenges   Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013     12   2   ➡  Main  problem:  manual  analysis  process   •  Developers  read  feedback  mulYple  Ymes   •  Manual  esYmaYon  of  how  many  users  are  affected   ➡  Feedback  quan7ty  amplifies  these  problems  
  13. 13. 3   Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013     13   Tool  Requirements   Results  
  14. 14. Tool  Requirements   •  Consolidate,  structure,  analyze,  and  track  user  feedback   •  Group  duplicate  or  similar  feedback,  capture  feedback  type,  and   associated  feature   •  Measure  frequency  of  user  feedback   Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013     14   3   •  Collect  feedback  and  analyze   later   •  Less  feedback  and   professional  end-­‐users   Re-­‐ac7ve  approach   •  RecommendaYon  of  exisYng   feedback   •  High  volume  of  non-­‐ confidenYal  feedback   Pro-­‐ac7ve  approach  
  15. 15. Implications  of  Findings   Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013     15   Researchers   1.  Extract  informa7on  from  natural  language  user  feedback   (microtext  understanding,  island  parsing)   2.  Elicit  context  informa7on  for  analysis  process   Companies   1.  Consolidate,  structure,  analyze,  and  track  user  feedback   2.  IdenYfy  similar  reports,  feedback  type,  and  affected  feature   3.  Assess  feedback  impact  (frequency)  and  reporYng  user   Tool  Developers   1.  Offer  fewer,  suitable,  bidirec7onal  feedback  channels   2.  Educate  users  how  to  provide  helpful  feedback  
  16. 16. Feel  free  to  contact  us  for  feedback…   16   Dennis  Pagano   TU  Munich   pagano@cs.tum.edu   @dennispagano   Bernd  Bruegge   TU  Munich   bruegge@cs.tum.edu   Dennis  Pagano  -­‐  TU  Munich  -­‐  May  2013    

×