Dissemination Users In Focus


Published on

Published in: Travel, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Dissemination Users In Focus

  1. 1. Users in Focus Statistics Denmark Copenhagen 1-3 October 2008 Annegrete Wulff [email_address] © Statistics Denmark
  2. 2. User surveys <ul><li>User surveys or user satisfaction surveys </li></ul><ul><li>Method and timing </li></ul><ul><li>Benefits contra inconveniences </li></ul><ul><li>Some results </li></ul><ul><li>Alternatives </li></ul>
  3. 3. Who visits the web site? <ul><li>260,000 visitors monthly </li></ul><ul><li>50 % adult students </li></ul><ul><li>25 % first time users </li></ul>
  4. 4. Can we spy on the users? <ul><li>Reading the log files </li></ul>
  5. 5. Why are we doing a survey? <ul><li>We want to improve something </li></ul><ul><li>The results as such are not interesting </li></ul><ul><li>Do we plan to take action? </li></ul><ul><li>Questions should be related to the actions </li></ul><ul><li>Do we follow up? </li></ul>
  6. 6. Collecting information on usability Direct, simultaneous Indirect, reflective Highly structured Unstructured Participant observation Formal interviews Spontaneous observations Planned observations, Usability, think aloud Eye tracking Surveys, open questions Surveys closed questions Hot line support Feed back from users Open interview
  7. 7. Users or users’ satisfaction <ul><li>Who are the users? </li></ul><ul><ul><li>who are the respondents? </li></ul></ul><ul><li>How do they use our product or service? </li></ul><ul><li>Are they satisfied? </li></ul><ul><ul><li>in general </li></ul></ul><ul><ul><li>looking at specific elements </li></ul></ul>
  8. 8. Web surveys for web products <ul><li>Questionnaire pop-up to </li></ul><ul><ul><li>everybody </li></ul></ul><ul><ul><li>to external users only </li></ul></ul><ul><ul><li>to every tenth </li></ul></ul><ul><ul><li>answer now or later </li></ul></ul>
  9. 9. Pop-up Dear User Hopefully you will assist our improving Statistics Denmark’s homepage by answering these questions. It will take you only a couple of minutes. Answers will be handled confidentially. Thank you for your help! Go to the questions Return to the home page
  10. 10. 15 questions (2001-2004)
  11. 11. Response rate Evaluation reports 8% 36% 2005 8% 7% 10% 9% - www.dst.dk 53% 14% 13% 11% 18% www.statbank.dk 2007 2004 2003 2002 2001
  12. 12. 2005 StatBank survey <ul><li>Change of method: </li></ul><ul><li>15 questions 4 questions </li></ul><ul><li>General Specific </li></ul><ul><li>Questions related to the task </li></ul>
  13. 13. Comments from users <ul><li>User comments with e-mail address get a personal answer </li></ul><ul><li>Results published on the web : </li></ul><ul><li>www.dst.dk/usersurvey </li></ul>
  14. 14. Some results <ul><li>1444 users cared to fill in the web survey that was active for three days. The response rate in this survey has been very high: 36 %. This is almost three times the response rate from the previous year. </li></ul><ul><li>1/3 visits the StatBank for the first time. 2/3 find what they are looking for. A major part of the users are students. </li></ul><ul><li>1/4 comments on the usefulness of the StatBank. </li></ul><ul><li>The action plan for 2006 includes an improved effort on easing the accessability for first time visitors. </li></ul>
  15. 15. 2005 pop-up/pop-in
  16. 16. The questions 2005 <ul><li>How often do you use the StatBank ? </li></ul><ul><li>For what purpose do you need the statistics you just found? </li></ul><ul><li>Did you find the data you needed </li></ul><ul><li>A retrieval from StatBank can be saved in Excel. You can get automatic update to this file with the latest data, without opening the StatBank again. Is this something you can use? (See a demo on www.dst.dk/datashooting ) </li></ul><ul><li>Do you have any comments regarding the StatBank? </li></ul>
  17. 17. Alternatives – to be combined <ul><li>Full user satisfaction survey </li></ul><ul><li>Specific satisfaction survey </li></ul><ul><li>Usability test </li></ul><ul><ul><li>functionality </li></ul></ul><ul><ul><li>contents </li></ul></ul><ul><ul><li>structure </li></ul></ul><ul><li>Eye tracking </li></ul><ul><li>Focus groups </li></ul><ul><li>Help desk registrations </li></ul>
  18. 18. Eye tracking <ul><li>Municipality reform </li></ul><ul><li>Population in Denmark </li></ul><ul><li>Links from search </li></ul><ul><li>Subjects </li></ul>
  19. 19. The Set up <ul><li>Test common actions on our web site </li></ul><ul><ul><li>Test persons among typical users </li></ul></ul><ul><ul><li>Design 10 tasks </li></ul></ul><ul><ul><li>Define criteria for success </li></ul></ul><ul><ul><li>Conclude on alternatives </li></ul></ul>
  20. 20. Equipment <ul><li>Equipment built into a monitor </li></ul><ul><li>Web camera and infra light sources </li></ul><ul><li>Measure eye movements and transforms them to what the users see </li></ul>
  21. 21. Method <ul><li>Test leader introduces the tasks </li></ul><ul><li>Task 1 </li></ul><ul><ul><li>Solving task 1 </li></ul></ul><ul><ul><li>Interview </li></ul></ul><ul><li>Task 2 </li></ul><ul><ul><li>Solving task 2 </li></ul></ul><ul><ul><li>Interview </li></ul></ul><ul><li>…… </li></ul>
  22. 22. Task 1 Find information on municipalities AA A B
  23. 23. Results 1 4:24 1:31 Task 10 2:59 1:07 Task 9 2:50 0:28 Task 8 1 3:30 1:22 Task 7 3:47 0:57 Task 6 3 4:28 0:29 Task 5 3:10 0:34 Task 4 4:03 0:29 Task 3 0:53 0:17 Task 2 1 1:35 0:32 Task 1 Gave up Maximum Minimum Time in minutes:seconds
  24. 24. Task 1 Municipality information Municipality info Site map Find your way Municipality info
  25. 25. Task 3 – Search results
  26. 26. Task 6 Swedish citizens in Denmark Population
  27. 27. Evaluation of the method <ul><li>Observation + think aloud </li></ul><ul><li>Eye tracking + follow up interview </li></ul><ul><ul><li>combined with think aloud </li></ul></ul><ul><ul><li>documentation </li></ul></ul><ul><ul><li>overview </li></ul></ul><ul><ul><li>sharing of results </li></ul></ul>
  28. 28. Lessons learned <ul><li>Left side column too compact </li></ul><ul><li>Consider structure by subject areas </li></ul><ul><li>Meaningful lables on all links </li></ul><ul><li>Right side column hardly noticed </li></ul><ul><li>Help information not noticed </li></ul><ul><li>Logo in upper left side not noticed </li></ul><ul><li>Text lines too long </li></ul><ul><li>Differences more important than similarities </li></ul>
  29. 29. Eye tracking - Demo <ul><li>Municipality reform </li></ul><ul><li>Population in Denmark </li></ul><ul><li>Links from search </li></ul><ul><li>Subjects </li></ul>