Last update: 2014.3.18
Zhao Wenbin Chen Cheng Wu Tianyi Xu Yihua Zhu Ruoyu Zuo Xingyuan
ROOMIE
Usability	
  Test	
  Report...
Content
1.
Introduction
About
Roomie
Target
population of
Roomie
Goal of
usability
testing
2.
Methodology
Overview
Partici...
 
	
   About	
  Roomie	
  
	
  	
  	
  Roomie	
  is	
  an	
  online	
  pla,orm	
  that	
  aims	
  at	
  helping	
  univers...
Goal	
  of	
  usability	
  tes4ng
	
  	
  	
  Our	
  goal	
  for	
  the	
  usability	
  tesCng	
  was	
  to	
  collect	
  ...
Overview	
  
Usability	
  tesCng	
  occurred	
  during	
  the	
  one-­‐week	
  span	
  of	
  March	
  
9-­‐16.	
  We	
  di...
Prototype Building
We used Axure to make the prototype of Roomie online
platform. Axure is a business drawing software for...
Par4cipant	
  Selec4on	
  &	
  Recruitment
	
  	
  	
  Since	
  our	
  goal	
  was	
  to	
  find	
  the	
  strengths	
  and...
Test	
  Protocol	
  
	
  	
  	
  Each	
  test	
  session	
  last	
  for	
  one	
  hour,	
  including	
  10	
  minutes	
  f...
Test	
  Protocol
	
  	
  The	
  two	
  tasks	
  we	
  gave	
  them	
  were	
  organized	
  by	
  the	
  sequenCal	
  
orde...
Test	
  Environment	
  and	
  Equipment
	
  	
  	
  The	
  tests	
  were	
  conducted	
  in	
  Learning	
  Commons	
  of	
...
Score	
  analysis:Test	
  1*
3.3
3.6 3.6
2.5
3 3.1
2.8
0
0.5
1
1.5
2
2.5
3
3.5
4
登
陆
注
册
页
邮
箱
注
册
填
写
信
息
个
人
主
页
首
页他
人
...
Score	
  analysis: Test	
  2
2.7
2.8
2.9
3
3.1
3.2
3.3
3.4
3.5
登
陆
注
册
页
邮
箱
注
册
填
信
息
页
个
人
主
页
首
页
搜
索
结
果
页
他
人
的
个
人
主...
Score	
  analysis:	
  Test	
  1	
  &	
  Test	
  2
0
0.5
1
1.5
2
2.5
3
3.5
4
登
陆
注
册
页
邮
箱
注
册
填
写
信
息
个
人
主
页
首
页
他
人
的
个
...
Time	
  spent	
  on	
  task:	
  Test	
  1	
  &	
  Test	
  2
T ime spent on task
0
5
10
15
20
25
1 2 3 4 5
Minutes
Test1
Te...
Findings
In	
  the	
  first-­‐round	
  test,	
  we	
  found	
  5	
  broken	
  experiences	
  and	
  
figured	
  out	
  soluC...
 
3	
  person	
  points	
  out	
  that	
  the	
  picture	
  on	
  the	
  first	
  page	
  is	
  not	
  
appropriate.	
  	
 ...
SoluCon	
  1	
  	
  
Changed	
  to	
  a	
  more	
  clear	
  and	
  high	
  quality	
  picture,	
  cannot	
  find	
  a	
  
s...
Broken	
  experience	
  2	
  
5	
  users	
  menConed	
  that	
  the	
  design	
  of	
  user’s	
  own	
  informaCon	
  page...
SoluCon	
  2	
  	
  
Change	
  table	
  to	
  listed	
  informaCon,	
  add	
  more	
  sucCons	
  in	
  user’s	
  
own	
  i...
3	
  person	
  points	
  out	
  that	
  	
  
“The	
  advanced	
  search	
  is	
  confusing”,	
  “	
  I	
  do	
  not	
  und...
SoluCon	
  3	
  	
  
Change	
  boxes	
  to	
  listed	
  opCons	
  to	
  make	
  it	
  easier	
  to	
  understand,	
  
and	...
4	
  users	
  menConed	
  
that	
  more	
  personal	
  
informaCon	
  such	
  as	
  
lifestyle	
  is	
  necessary.	
  
	
 ...
SoluCon	
  4	
  
Provide	
  choices	
  related	
  to	
  lifestyle	
  and	
  apartment	
  requirements	
  .	
  
23
3.Result...
3	
  person	
  points	
  out	
  that	
  the	
  whole	
  process	
  of	
  registraCon	
  and	
  
search	
  is	
  not	
  ver...
SoluCon	
  5	
  	
  
Adding	
  navigaCon	
  at	
  the	
  top	
  of	
  each	
  page,	
  and	
  change	
  “submit”	
  
to	
 ...
After we fixed problems from Test 1, we updated our
prototype from V1.0 to V2.0 and conducted Test 2.
26
Prototype
V1.0
Pr...
In	
  the	
  “fill-­‐in	
  personal	
  informaCon”	
  page,	
  we	
  want	
  to	
  collect	
  
more	
  informaCon	
  about	...
We	
  add	
  “*”	
  in	
  front	
  of	
  those	
  mandatory	
  fields	
  to	
  disCnguish	
  
from	
  opConal	
  ones,	
  s...
 2	
  users	
  pointed	
  out	
  that	
  they	
  don’t	
  want	
  to	
  type	
  in	
  the	
  
descripCons	
  of	
  their	
...
Besides	
  words	
  to	
  describe	
  the	
  characterisCc,we	
  first	
  provide	
  
menu	
  for	
  users	
  to	
  choose	...
31
2	
  person	
  points	
  out	
  that	
  they	
  can	
  not	
  	
  understand	
  the	
  word	
  
‘洋楼’and‘唐楼’,	
  and	
  ...
To	
  be	
  more	
  clear,	
  we	
  change	
  “唐楼”	
  and	
  “洋楼”	
  to	
  “无电梯
楼宇”	
  and	
  “有电梯楼宇”.	
  We	
  also	
  ad...
ALer	
  registraCon	
  process,	
  user	
  will	
  landing	
  to	
  personal	
  homepage,	
  
but	
  there	
  is	
  no	
  ...
SoluCon	
  4	
  
Besides	
  words	
  to	
  describe	
  the	
  characterisCc,we	
  first	
  provide	
  
menu	
  for	
  users...
RecommendaCon	
  funcCon	
  on	
  the	
  homepage	
  is	
  overlooked	
  by	
  
users.	
  
“See	
  I	
  can	
  search	
  t...
SoluCon	
  5	
  
We	
  put	
  recommendaCon	
  part	
  on	
  the	
  top	
  of	
  homepage	
  to	
  
reinforce	
  its	
  fu...
37
2 person	
  points	
  out	
  that	
  when	
  searching,	
  they	
  put	
  gender	
  in	
  
priority.	
  	
  	
  
“Where...
We	
  rearrange	
  the	
  choices	
  and	
  add	
  “同一性别”	
  so	
  as	
  it	
  can	
  
be	
  seen	
  more	
  easily.
38
3....
In	
  the	
  process	
  of	
  finding	
  search	
  funcCon,
user	
  click	
  “people	
  I	
  interested	
  in”	
  below	
  ...
We	
  build	
  list	
  page	
  of	
  users	
  who	
  I	
  am	
  interested	
  in	
  aLer	
  click	
  
“people	
  I	
  inte...
41
2 person	
  points	
  out	
  that	
  they	
  
can’t	
  understand	
  what	
  
grouping	
  means	
  when	
  I	
  want	
 ...
We	
  revise	
  Grouping	
  facCon,	
  add	
  a	
  drop	
  list	
  for	
  users	
  to	
  select	
  
and	
  add	
  group	
 ...
Further	
  demand	
  of	
  renCng	
  a	
  house	
  can	
  not	
  be	
  fulfilled	
  in	
  our	
  
website	
  
“Besides	
  fi...
 	
  	
  	
  The	
  tasks	
  are	
  about	
  the	
  basic	
  funcCons	
  of	
  Roomie.	
  Most	
  users	
  
successfully	
...
•  Roomie usability test video
•  Prototype (in the folder “V3.0/ index.html”)
45
5. Appendix
Thank you
46
Upcoming SlideShare
Loading in …5
×

Usability test report roomie

631 views

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
631
On SlideShare
0
From Embeds
0
Number of Embeds
30
Actions
Shares
0
Downloads
12
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Usability test report roomie

  1. 1. Last update: 2014.3.18 Zhao Wenbin Chen Cheng Wu Tianyi Xu Yihua Zhu Ruoyu Zuo Xingyuan ROOMIE Usability  Test  Report 1
  2. 2. Content 1. Introduction About Roomie Target population of Roomie Goal of usability testing 2. Methodology Overview Participant Selection & Recruitment Test Protocol Test Environment and Equipment 3. Results Score analysis Test 1 Findings Test 2 Findings 4. Conclusion Conclusion 5. Appendix Axure Propotype Usability test viedo 2
  3. 3.     About  Roomie        Roomie  is  an  online  pla,orm  that  aims  at  helping  university   students  find  suitable  roommates  in  Hong  Kong  according  to   their  personality,  life  style  and  other  daily  issues. 3 1.Introduction   Target  popula4on  of  Roomie        While  the  primary  users  of  Roomie  are  students  from  Mainland   China,  as  most  of  them  cannot  apply  for  the  university   dormitories,  we  also  prefer  to  target  graduates  who  determine   to  work  in  Hong  Kong  but  lack  roommates  to  share  an   apartment,  as  well  as  local  students  who  prefer  to  live   separately  from  parents.
  4. 4. Goal  of  usability  tes4ng      Our  goal  for  the  usability  tesCng  was  to  collect  both   quanCtaCve  and  qualitaCve  data  of  new  users  learning  how  to   use  Roomie  to  search  for  potenCal  roommates  for  the  first   Cme.  We  hope  that  the  results  of  the  usability  tesCng  can  be   used  by  us  to  understand  the  strengths  and  weaknesses  of  this   project  and  serve  as  a  benchmark  for  future  usability  tesCng. 4 1.Introduction
  5. 5. Overview   Usability  tesCng  occurred  during  the  one-­‐week  span  of  March   9-­‐16.  We  divided  our  tesCng  into  two  rounds,  the  first  round   on  March  9  and  the  second  round  on  March  16.  ALer  the  first   round  tesCng,  we  opCmized  our  prototype  according  to   parCcipants’  suggesCons,  then  held  the  second  round  tesCng   one  week  aLer.  Ten  parCcipants  were  recruited  who  had  never   heard  of  our  project  before  but  yet  had  a  need  to  look  for   roommates,  or  had  related  roommate-­‐finding  experiences.   During  the  half-­‐an-­‐hour  tesCng  sessions,  each  parCcipant  was   asked  to  do  the  task  of  searching  for  suitable  roommates  on   Roomie.  Time  spent  on  the  task  and  qualitaCve  comments   were  collected  for  each  parCcipant.  Each  session  was  recorded   using  QuickTime  to  allow  for  later  analysis. 5 2.  Methodology
  6. 6. Prototype Building We used Axure to make the prototype of Roomie online platform. Axure is a business drawing software for creating mockups and wireframes of application user interfaces. This easy to use software generates HTML prototypes and functional specifications. 6 2.  Methodology
  7. 7. Par4cipant  Selec4on  &  Recruitment      Since  our  goal  was  to  find  the  strengths  and  weaknesses  of  this   project,  we  recruited  users  who  had  never  heard  Roomie   before  but  had  a  need  to  look  for  roommates,  or  had  related   roommate-­‐finding  experiences.  They  should  be  future   students,  present  students  or  graduates  of  a  certain  university   in  Hong  Kong.  We  recruited  our  parCcipants  through  our  social   networks,  and  offered  them  a  small  giL  as  an  incenCve  for   parCcipaCon. 7 2.  Methodology
  8. 8. Test  Protocol        Each  test  session  last  for  one  hour,  including  10  minutes  for  a   pre-­‐test  background  interview  and  post-­‐test  debriefing   quesCons,  and  20  minutes  for  the  tasks.          We  began  each  test  session  by  asking  them  some  general   quesCons  about  how  they  got  along  with  their  former   roommates,  whether  they  had  some  good  or  bad  experiences,   and  what  they  cared  the  most  when  looking  for  a  roommate.   Then  we  gave  them  two  tasks,  and  asked  them  to  complete   each  one  while  talking  aloud  about  their  thought  process  and   any  confusion  they  encountered. 8 2.  Methodology
  9. 9. Test  Protocol    The  two  tasks  we  gave  them  were  organized  by  the  sequenCal   order.  The  summary  of  tasks  is  listed  below.   Task Number   Task Summary 1   Register for Roomie 2   Search for roommates on Roomie 9 2.  Methodology      ALer  the  parCcipant  completed  the  tasks,  we  conducted  a   post-­‐test  brief,  asking  them  about  their  interesCng  or  confused   points  about  Roomie.  We  also  asked  them  to  score  each  page   according  to  their  saCsfacCon  (4-­‐point  Likert  scale,  from  1   =dissaCsfied  to  4=saCsfied)  and  asked  if  they  thought  they   would  be  likely  to  use  Roomie,  based  on  their  iniCal   impression.
  10. 10. Test  Environment  and  Equipment      The  tests  were  conducted  in  Learning  Commons  of  Wu  Ho  Man   Yuen  Building.  We  used  QuickTime  to  record  our  test  screens.   We  each  took  turns  moderaCng  one  session  and  observing  one   session.  We  set  up  QuickTime  on  a  Mac,  and  give  this  Mac  to   parCcipant  to  use  during  the  test. 10 2.  Methodology Data  Collec4on        We  collected  both  qualitaCve  and  quanCtaCve  data  about  the   parCcipants’  behavior.  Our  quanCtaCve  data  consisted  of  the   performance  data  of  Cme  spent  on  tasks  for  each  parCcipant,   and  their  saCsfacCon  (4-­‐point  Likert  scale)  towards  Roomie.   The  qualitaCve  data  we  collected  were  quotes,  comments  and   observaCons  noted  down  by  the  observer  during  the  test.  
  11. 11. Score  analysis:Test  1* 3.3 3.6 3.6 2.5 3 3.1 2.8 0 0.5 1 1.5 2 2.5 3 3.5 4 登 陆 注 册 页 邮 箱 注 册 填 写 信 息 个 人 主 页 首 页他 人 的 个 人 主 页 加 好 友 Tes t 1均分 Test1均分 11 3.Results *Test 1 stands for the first-round test. After the first round, we optimized our prototype and conducted Test 2, which is the second–round test.
  12. 12. Score  analysis: Test  2 2.7 2.8 2.9 3 3.1 3.2 3.3 3.4 3.5 登 陆 注 册 页 邮 箱 注 册 填 信 息 页 个 人 主 页 首 页 搜 索 结 果 页 他 人 的 个 人 主 页 感 兴 趣 留 言 页 Tes t 2均分 Test2均分 12 3.Results
  13. 13. Score  analysis:  Test  1  &  Test  2 0 0.5 1 1.5 2 2.5 3 3.5 4 登 陆 注 册 页 邮 箱 注 册 填 写 信 息 个 人 主 页 首 页 他 人 的 个 人 主 页 test1均分 test2均分 13 3.Results
  14. 14. Time  spent  on  task:  Test  1  &  Test  2 T ime spent on task 0 5 10 15 20 25 1 2 3 4 5 Minutes Test1 Test2 14 3.Results From  this  chart  we  can  see  clearly  that  aLer  opCmizing  the  prototype,   parCcipants  spent  less  Cme  on  compleCng  tasks  than  the  first  Cme.  It   means  that  now  our  prototype  is  easier  to  understand,  and  the   searching  process  is  clearer  than  before.
  15. 15. Findings In  the  first-­‐round  test,  we  found  5  broken  experiences  and   figured  out  soluCons  for  each  broken  experience.  In  the  second-­‐ round  test,  we  found  8  problems  that  needed  to  be  solved. 15 3.Results Prototype V1.0 Prototype V2.0 Prototype V3.0 Fixed 5 problems from Test 1 Fixed 8 problems from Test 2
  16. 16.   3  person  points  out  that  the  picture  on  the  first  page  is  not   appropriate.     “The  picture  in  registraCon  page  cannot  clearly  represent  the   funcCon  of  Roomie.”  “The  picture  make  the  website  look  like  a   social  media.” 16 3.Results Test 1 Broken  experience  1  
  17. 17. SoluCon  1     Changed  to  a  more  clear  and  high  quality  picture,  cannot  find  a   suitable  picture  with  Asian  faces.   17 3.Results Test 1
  18. 18. Broken  experience  2   5  users  menConed  that  the  design  of  user’s  own  informaCon  page   is  not  clean  and  clear.   “I  find  the  table  very  stressful  and  the  classificaCon  of  informaCon   is  not  clear.  ”   18 3.Results Test 1
  19. 19. SoluCon  2     Change  table  to  listed  informaCon,  add  more  sucCons  in  user’s   own  informaCon  page.   19 3.Results Test 1
  20. 20. 3  person  points  out  that     “The  advanced  search  is  confusing”,  “  I  do  not  understand  the   relaConship  between  apartment  and  roommate.”   20 3.Results Test 1 Broken  experience  3  
  21. 21. SoluCon  3     Change  boxes  to  listed  opCons  to  make  it  easier  to  understand,   and  also  add  missed  bufons.     21 3.Results Test 1
  22. 22. 4  users  menConed   that  more  personal   informaCon  such  as   lifestyle  is  necessary.     “I  hope  to  see  more   personalized   informaCon  to  know   what  kind  of  person   the  potenCal   roommate  is.”     22 3.Results Test 1 Broken  experience  4  
  23. 23. SoluCon  4   Provide  choices  related  to  lifestyle  and  apartment  requirements  .   23 3.Results Test 1
  24. 24. 3  person  points  out  that  the  whole  process  of  registraCon  and   search  is  not  very  smooth,  some  bufons  are  missing.     “I  think  there  should  be  a  next  or  finish  bufon.”   “  I  can’t  go  back  to  the  homepage.”     24 3.Results Test 1 Broken  experience  5  
  25. 25. SoluCon  5     Adding  navigaCon  at  the  top  of  each  page,  and  change  “submit”   to  “finish”.   25 3.Results Test 1
  26. 26. After we fixed problems from Test 1, we updated our prototype from V1.0 to V2.0 and conducted Test 2. 26 Prototype V1.0 Prototype V2.0 Fixed 5 problems from Test 1
  27. 27. In  the  “fill-­‐in  personal  informaCon”  page,  we  want  to  collect   more  informaCon  about  the  user.  But  3  users  complained  that   they  don’t  want  to  fill  in  all  the  informaCon.     “Some  informaCon  I  don’t  want  to  share  to  others.  It’s  all   required?  ” 27 3.Results Test 2 Broken  experience  1  
  28. 28. We  add  “*”  in  front  of  those  mandatory  fields  to  disCnguish   from  opConal  ones,  so  as  users  can  clearly  figure  out  which   part  they  must  fill  in. 28 3.Results Test 2 SoluCon  1  
  29. 29.  2  users  pointed  out  that  they  don’t  want  to  type  in  the   descripCons  of  their  characters,  or  it  took  some  Cme  to  think   about  how  to  describe  themselves.  So  there  is  a  high   possibility  that  they  will  leave  in  this  stage.   “I  don’t  want  to  type,  I  want  to  choose.”   29 3.Results Test 2 Broken  experience  2  
  30. 30. Besides  words  to  describe  the  characterisCc,we  first  provide   menu  for  users  to  choose  their  character  type.   30 3.Results Test 2 SoluCon  2  
  31. 31. 31 2  person  points  out  that  they  can  not    understand  the  word   ‘洋楼’and‘唐楼’,  and  actually  the  type  of  the  building  is  not   that  important.  Moreover,  the  type  of  the  apartment  is   something  that  they  concern  more.     “I  don’t  know  how  to  choose, I  have  no  idea  what  it  is.” 3.Results Test 2 Broken  experience  3  
  32. 32. To  be  more  clear,  we  change  “唐楼”  and  “洋楼”  to  “无电梯 楼宇”  and  “有电梯楼宇”.  We  also  add  Type  of  Apartment  to   provide  more  details. 32 3.Results Test 2 SoluCon  3  
  33. 33. ALer  registraCon  process,  user  will  landing  to  personal  homepage,   but  there  is  no  instrucCons  for  what  to  do  next.     “OK,  I’ve  registered.  So  what  I  goanna  do  next?  ” 33 3.Results Test 2 Broken  experience  4  
  34. 34. SoluCon  4   Besides  words  to  describe  the  characterisCc,we  first  provide   menu  for  users  to  choose  their  character  type.   34 3.Results Test 2
  35. 35. RecommendaCon  funcCon  on  the  homepage  is  overlooked  by   users.   “See  I  can  search  the  website.  If  there  are  recommended   roommates  for  me,  that  would  be  great!”   35 3.Results Test 2 Broken  experience  5  
  36. 36. SoluCon  5   We  put  recommendaCon  part  on  the  top  of  homepage  to   reinforce  its  funcCon,  and  put  search  bar  on  the  right  side.  If  users   want  to  search  more  informaCon,  they  can  click  Advanced  Search   and  go  to  search  page.   36 3.Results Test 2
  37. 37. 37 2 person  points  out  that  when  searching,  they  put  gender  in   priority.       “Where  is  the  gender  in  the  search  box?  I  would  definitely   want  that  opCon  before  all  the  other  choices.    ” NO GENDER! 3.Results Test 2 Broken  experience  6  
  38. 38. We  rearrange  the  choices  and  add  “同一性别”  so  as  it  can   be  seen  more  easily. 38 3.Results Test 2 SoluCon  6  
  39. 39. In  the  process  of  finding  search  funcCon, user  click  “people  I  interested  in”  below   head  portrait  instead  of  “search”  bufon  on   the  header.  However, “people  I  interested   in”  is  not  the  right  search  bufon.  There  is   some  misunderstanding  of  “people  I   interested  in”.   “I’ll  click  this(‘people  I  interested  in’)  to  see   who  is  my  suitable  roommate. 39 3.Results Test 2 Broken  experience  7  
  40. 40. We  build  list  page  of  users  who  I  am  interested  in  aLer  click   “people  I  interested  in  ”    bufon.  Users  can  check  and   manage  all  the  list  here. 40 3.Results Test 2 SoluCon  7  
  41. 41. 41 2 person  points  out  that  they   can’t  understand  what   grouping  means  when  I  want   to  leave  a  note  to  other  users.   “I  don’t  know  what  the   grouping  here  means,  will  it   appear  every  Cme  I  leave   message  for  someone?  Will   other  people  see  my  grouping   informaCon?  Would  it  be   more  appropriate  if  it  shows   when  I  liked  someone?” 3.Results Test 2 Broken  experience  8  
  42. 42. We  revise  Grouping  facCon,  add  a  drop  list  for  users  to  select   and  add  group  category. 42 3.Results Test 2 SoluCon  8  
  43. 43. Further  demand  of  renCng  a  house  can  not  be  fulfilled  in  our   website   “Besides  finding  a  roommate,  I  want  to  find  a  flat.  If  there  is  only   informaCon  of  roommates,  I  think  that’s  not  enough.”     SoluCon   Unfortunately,we  cannot  solve  this  problem  so  far.     43 3.Results Test 2 Broken experience 9
  44. 44.        The  tasks  are  about  the  basic  funcCons  of  Roomie.  Most  users   successfully  finished  all  the  tasks.  All  of  them  experienced   some  difficulCes  but  all  successfully  figured  out  a  way  to   overcome  them.            From  the  first-­‐round  test  we  found  many  weaknesses  of   Roomie  and  opCmized  it  according  to  parCcipants’  suggesCons.   And  in  the  second-­‐round  test,  though  Roomie  had  been   opCmized,  parCcipants  sCll  figured  out  some  broken   experiences  that  need  to  be  solved.  It  means  that  to  saCsfied   users’  real  needs,  we  sCll  have  a  lot  to  do. 44 4. Conclusion
  45. 45. •  Roomie usability test video •  Prototype (in the folder “V3.0/ index.html”) 45 5. Appendix
  46. 46. Thank you 46

×