User	  Studies	  Are	  a	  Waste	  of	  Money! 	  	  …if	  you	  know	  what	  you	  are	  doing	  J	  Risto	  Sarvas	  F...
›  Head	  of	  UX	  &	  Service	  Design	     at	  Futurice.	  ›  Docentship	  at	  Aalto	  University	     (Media	  Techn...
Futurice	  in	  brief	                                                        TAMPERE	                                    ...
We	  offer	  tailored	  solutions	           done	  with	  	         agile	  methods.	  
›  How	  to	  spend	  the	  UX	                    budget?	  Overview	     ›  What	  to	  focus	  on	  if	  not	  on	     ...
20	  000	  €	  …to	  make	  a	  newsreader	  for	  iOS.	     How	  should	  we	  spend	  it?	  
›  Development	                        ›  Estimated	  12k€	      Budget	     ›  Management	  breakdown	           ›  Estim...
Typical	  choices	  for	  user	  research.	  »  Focus	  groups	                   »  1	  FG,	  2	  desgnrs,	  4	  days	  o...
›  Which	  is	  more	  critical	  a	  “Let’s	  cut	  the	        working	  software	  or	  UX?	                           ...
›  Sure…	  but	  does	  that	  really	                                 save	  money	  here?	  “Let’s	  scope	  it	     ›  ...
›  Yes	  the	  UX	  budget	  is	  too	                                           small.	          “The	  UX	  budget	     ...
›  Is	  this	  the	  first	  newsreader	  app	  in	  the	                                        world?	  	                ...
World	  of	  a	  difference:	                                  ~5k€	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	 ...
Our	  25	  latest	  mobile	  cosnumer	  apps	  are	  rated	  4.2	  in	  average.	  Amount	  of	  user	  studies	  varies,	...
How	  is	  it	  possible	  to	  make	  successful	  software	  without	  emphasizing	  user	                          stud...
•         Observation	       •         Literature	       •         Existing	  studies	       •         Focus	  groups	    ...
Bechmark	  competition	  and	                                       reverse	  engineer	  user	  reqs.	                    ...
New	  ideas	     Familiar	     Consumers	     Plenty	  of	      Customer	              Complex	         Large	        Stra...
New	  ideas	        Familiar	     Consumers	     Plenty	  of	      Customer	              Complex	         Large	        S...
New	  ideas	     Familiar	     Consumers	     Plenty	  of	      Customer	              Complex	         Large	        Stra...
›  Is	  the	  designer	  also	  a	  user?	  ›  Has	  the	  designer	  done	  newsreaders?	  ›  Has	  the	  designer	  work...
›  User	  studies	  are	  expensive.	                         They	  are	  not	  always	  worth	  the	                    ...
User	  Studies	  are	  a	  Waste	  of	  Money…	    …under	  specific	  circumstances.	  
Discussion	  But	  what	  next?	  
Discussion	  point	  1:	  We	  need	  more	  understanding	  on	  what	    shapes	  the	  design	  of	  technology	  	   a...
The	  Multi-­‐Dimensional	  Space	                                                                                 Experie...
Theory	                                                                 Practice	               •    Social	  construction...
Discussion	  point	  2:	    Real	  user	  knowledge	  is	  	     created	  after	  launch.	  From	  waterfall	  to	  agile...
User	  studies	                                                   as	  guesswork	                 Business	      Research	...
Thank	  you!	                           @rsarvas	                 risto.sarvas@futurice.com	                   www.hiit.fi/...
Upcoming SlideShare
Loading in …5
×

INUSE Seminar Risto Sarvas

1,555 views

Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

INUSE Seminar Risto Sarvas

  1. 1. User  Studies  Are  a  Waste  of  Money!    …if  you  know  what  you  are  doing  J  Risto  Sarvas  Futurice  &  Aalto  University    6/12/12  INUSE  seminar  
  2. 2. ›  Head  of  UX  &  Service  Design   at  Futurice.  ›  Docentship  at  Aalto  University   (Media  Technology).  ›  A  decade  at  HIIT.  
  3. 3. Futurice  in  brief   TAMPERE   HELSINKI   LONDON   BERLIN   DÜSSELDORF   1   Founded  in   150   2000  
  4. 4. We  offer  tailored  solutions   done  with     agile  methods.  
  5. 5. ›  How  to  spend  the  UX   budget?  Overview   ›  What  to  focus  on  if  not  on   user  studies?   ›  Discussion  points  
  6. 6. 20  000  €  …to  make  a  newsreader  for  iOS.   How  should  we  spend  it?  
  7. 7. ›  Development   ›  Estimated  12k€   Budget   ›  Management  breakdown   ›  Estimated  3k€   ›  UX  design  &  research   ›  What  is  left  is…  5k€.  
  8. 8. Typical  choices  for  user  research.  »  Focus  groups   »  1  FG,  2  desgnrs,  4  days  of  work  =  6400€  »  Observations   »  5  usrs,  1  designer,  5  days  of  work  =  4000€  »  User  interviews   »  5  usrs,  1  designer,  5days  of  work  =  4000€  »  PD  workshops   »  1  workshop,  2  designers,  3  days  =  4800€  »  Stakeholder  workshops   »  1  workshop,  2  designers,  3  days  =  4800€  »  etc.   »  Our  UX  budget  is  5000€  and  above  we   have  allocated  no  design  yet!   One  person  day  =  800€  in  budget  
  9. 9. ›  Which  is  more  critical  a  “Let’s  cut  the   working  software  or  UX?   Really.   developmet   budget?”   ›  Interviewing  5  people  would   be  33%  of  the  development   budget.    
  10. 10. ›  Sure…  but  does  that  really   save  money  here?  “Let’s  scope  it   ›  User  studies  do  not  scale   down:  Running  focus  groups  down  to  make   or  workshops  takes  few  days   it  possible”   each  anyway.     ›  We  have  to  scope  this  down   anyway  to  do  it  only  with  20k €.  
  11. 11. ›  Yes  the  UX  budget  is  too   small.   “The  UX  budget   ›  Nevertheless,  that  is  what   is  too  small!”   the  customer  was  willing  to     pay.  This  is  their  budget.   “We  should  not   ›  Should  we  take  the  project,   then?   accept  this   ›  Can  we  make  the   project!”   newsreader  with  a  5k€  UX   budget?  With  no  user   research!  Futurice  6/12/12   11  
  12. 12. ›  Is  this  the  first  newsreader  app  in  the   world?     ›  Have  we  done  newsreader  apps   before?   Should  we  do   ›  Does  the  customer  have  this  already  the  UX  for  5k€?     done  on  some  other  platform?     ›  Are  we  users  ourselves?   In  ~6  days?   ›  Is  this  a  familiar  customer?   ›  What  features  does  the  customer   want?  How  simple  is  it?   ›  Are  we  familiar  with  the  platform  UI   guidelines?  
  13. 13. World  of  a  difference:   ~5k€                                                                                                      ~50k€   Porting  an  existing  app.                                                                                                                            Creating  from  scratch.  “You  guys  did  a  great  job  with  the  Android  newsreader  for  our  company.     “We  want  something  previously  unseen   in  the  world  of  newsreaders.  We  do  not  Could  you  do  the  same  reader  for  iOS   know  what  it  is.  It  will  be  a  tool  for  stock  as  well?  It  will  be  for  public  use  to   market  analysts  and  investors.  This  accompany  our  newspaper.”   would  be  a  good  first  project  between   our  companies.”  
  14. 14. Our  25  latest  mobile  cosnumer  apps  are  rated  4.2  in  average.  Amount  of  user  studies  varies,  but  it  is  never  what  schoolbooks  would  expect.    
  15. 15. How  is  it  possible  to  make  successful  software  without  emphasizing  user   studies?  
  16. 16. •  Observation   •  Literature   •  Existing  studies   •  Focus  groups   Eliciting   •  Design  for  myself   user  reqs   •  Common  sense   User   •  Social  distance   participa Designer   as  user  •  Lead  users   tion   What  shapes  •  Participatory  design   the  product,   •  Professional  •  User  testing   and  its  UX   experience   design?   Designer   Existing   •  Existing  trends   as   •  Shared  visions  among   cultural   •  UI  genres   professio designers   •  Metaphores  and   genres   Tech  &   nal   associations   •  Use  practices   compone nts   •  Existing  UI  components   •  Existing  platform  APIs   •  Existing  content     Adapted  from  Hyysalo  2011.  
  17. 17. Bechmark  competition  and   reverse  engineer  user  reqs.   Ask  the  customer  to  tell   about  the  users.   Eliciting   Do  we  have  specific  user  domain   user  reqs   knowledge?   How  much  does  it  cost?   User   Does  the  customer  know  their   Is  it  worth  it?   participa Designer   customers?  User  testing  is  often  worth  it.   as  user   Are  we  gathering  user  data   tion   What  shapes   after  launch?.   the  product,   and  its  UX   How  experienced  is  our  designer?   design?   Designer   Existing   Have  we  done  these  kinds  of   as   products/services?   Is  this  rocket  science?   cultural   professio Are  we  learning  enough?  What  is  the  “metaphor”  of  the   genres   design?   Tech  &   nal   Are  we  using  well-­‐known   compone conventions?   nts   What  is  the  platform?   What  is  the  back-­‐end?   What  are  the  UI  guidelines?   What  are  the  customer’s  brand  Adapted  from  Hyysalo  2011.   guidelines?  
  18. 18. New  ideas   Familiar   Consumers   Plenty  of   Customer   Complex   Large   Strategy   domain   user  data   new  to  agile   service   organization   documents  5   5  4   4  3   3  2   2  1   1  0   0   1   1   2   2   3   3   4   4   5   5   Concept   Unknown   Specific   No  user   Customer   Single   Small   Gfx  for   ready   domain   users   data   knows  agile   functionality   organization   coding   18  
  19. 19. New  ideas   Familiar   Consumers   Plenty  of   Customer   Complex   Large   Strategy   domain   user  data   new  to  agile   service   organization   documents  5   p”   5  4   “Mobile  ap 4  3   3  2   2  1   1  0   0   1   1   2   2   “ERP   3   app”   3   4   4   5   5   Concept   Unknown   Specific   No  user   Customer   Single   Small   Gfx  for   ready   domain   users   data   knows  agile   functionality   organization   coding   19  
  20. 20. New  ideas   Familiar   Consumers   Plenty  of   Customer   Complex   Large   Strategy   domain   user  data   new  to  agile   service   organization   documents  5   5  4   4  3   3  2   2  1   1  0   0   1   1   2   2   3   3   4   4   5   5   Concept   Unknown   Specific   No  user   Customer   Single   Small   Gfx  for   ready   domain   users   data   knows  agile   functionality   organization   coding  The  20k€  newsreader   20  
  21. 21. ›  Is  the  designer  also  a  user?  ›  Has  the  designer  done  newsreaders?  ›  Has  the  designer  worked  with  this  customer?  ›  Does  iOS  have  clear  guidelines?    ›  Does  the  customer  have  clear  visual  &  brand  guidelines?  ›  Is  a  newsreader  app  a  common  product?  ›  Can  we  re-­‐use  the  design  from  existing  products?  ›  Can  the  customer  give  us  good  feedback  on  behalf  of  the  end-­‐users?  ›                                                             Can  we  do  the  UX  design  for  5000€?  ›                                                                                                           Sure  we  can!  
  22. 22. ›  User  studies  are  expensive.   They  are  not  always  worth  the   cost.   ›  Take  advantage  of  other  factors   shaping  the  UX:   ›  The  designer  as  a  user.  Key  points   ›  The  designer  as  a  professional   with  experience.   ›  Existing  UI  patterns,  guidelines  &   components.   ›  Obvious  use  patterns  and   practices.   ›  Consulting  other  people  who   know  the  users  (e.g.,  product   owner)  
  23. 23. User  Studies  are  a  Waste  of  Money…   …under  specific  circumstances.  
  24. 24. Discussion  But  what  next?  
  25. 25. Discussion  point  1:  We  need  more  understanding  on  what   shapes  the  design  of  technology     and  how  to  apply  that  into  design.  
  26. 26. The  Multi-­‐Dimensional  Space   Experience  of  designer   Enterprise  system  for  XY   ERP  for  X   Mobile  app  for  ZZ   Online  service  for  WZ   Web  page  for  G   ERP  for  XXY   #  of  featur Enterprise  system  for  XX   es   Domain  familiarity   Mobile  app  for  XY   Online  service  for  X  ERP  for  XXY   Mobile  app  for  ZZ   ERP  for  Z  
  27. 27. Theory   Practice   •  Social  construction  of  technology   •  Ethnography,  ethnomethods   •  Qualitative  &  quantitative  approaches   •  Use  practices,  domestication  of  tech   •  Design  research   This  should  be     a  rapid  cycle!   •  Empirical  data  on  projects   •  Case  studies   •  Real  world  context   •  Views  of  professionals   •  Reality  check  
  28. 28. Discussion  point  2:   Real  user  knowledge  is     created  after  launch.  From  waterfall  to  agile/lean.  
  29. 29. User  studies   as  guesswork   Business   Research   Develop Mainten objectives   &  Design   ment   ance   x  months   Several  user  studies  6/12/12   based  on  actual  use   29  
  30. 30. Thank  you!   @rsarvas   risto.sarvas@futurice.com   www.hiit.fi/risto.sarvas/  Futurice  6/12/12   30  

×