Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Conducting Longitudinal    Diary Studies      Amanda Nance           Sage       June 10, 2009
Early diary study
Goal of this presentation• How to run a better diary study  – Create effective template  – Understand the data  – Keep use...
Topics for this presentation• Reason for the project• About the diary study• Diary study tips• Unique contributions of the...
Reasons for the project•   Assess “getting started” experience of our    desktop accounting software•   Focus on tasks aft...
Used multiple methods to get truer pictureSite visits         Support calls                 Survey analysis         Resear...
About the diary study• 5 participants: small business owners,  bookkeepers• Diaries kept for up to 8 weeks beginning at  i...
About the diary study• Provided template in Word to record  – Daily Work: tools they used, other tasks  – Issues and Comme...
Tips
Recruit extra participants• Recruited 12• 5 completed• Reasons for drop outs were interesting  findings, too
Use a recruiter to save time• Pros and cons• They can recruit and more  – Email documents  – Remind of deadlines  – Collec...
Communicate deadlines• Communicate due date for first submission• Disqualify participants who don’t respond• Close the loop
Template tips
Make it easy for participants• Keep it short  – This study: 3 questions daily, 6-7 per incident• Separate questions from a...
Separate questions from answers
Show sample entriesWhat did you do for your business today?• I answered several customer phone calls.• Called automated ba...
Ask for screenshots“Looking at Aged Receivables…. They donot match. Why are they different?”
Tips for screenshots• If they are important, require them• Include instructions
Structure template for easy analysis• Separate daily entries from critical  incidents• Include participant name and week n...
Iterate on the template• Prior to launching• As you receive user feedback
Process tips
Require daily entries• Keeps participants engaged• Gives you a reason to pay them• Opportunity to learn about the broader ...
Talk to the participants yourself• Intro call to understand business and  product expectations• Follow up on unclear entri...
Research tips• Read entries as soon as possible• Leave time to research
Data analysis• No different from other methods• Evaluate data  – Excel to categorize theme, second theme, week #• Share an...
Unique contributions of diary study• Infrequent tasks• Help use• Users resolving issues• Tasks over time
Reasons to run a diary study• No travel required• Non-intrusive, in context• Cheaper than site visits
Engagement and meaningful data• Pay for more complete responses• Encourage useful responses through  feedback on recent en...
Key Points• Daily entries• Explain what you expect• Sample entries• Reasonable number of questions• Personal contact with ...
ContactMore tips in conference proceedingsanance@gmail.comTwitter: @amandaux
Tips for conducting long-term diary studies - UPA 2009
Upcoming SlideShare
Loading in …5
×

Tips for conducting long-term diary studies - UPA 2009

1,641 views

Published on

Published in: Technology, Education
  • Be the first to comment

Tips for conducting long-term diary studies - UPA 2009

  1. 1. Conducting Longitudinal Diary Studies Amanda Nance Sage June 10, 2009
  2. 2. Early diary study
  3. 3. Goal of this presentation• How to run a better diary study – Create effective template – Understand the data – Keep users engaged• See benefits of the method
  4. 4. Topics for this presentation• Reason for the project• About the diary study• Diary study tips• Unique contributions of the diary method
  5. 5. Reasons for the project• Assess “getting started” experience of our desktop accounting software• Focus on tasks after initial set up• Create requirements for next release
  6. 6. Used multiple methods to get truer pictureSite visits Support calls Survey analysis Researcher calls Diary study Diary from: http://www.flickr.com/photos/bdorfman/15846725/
  7. 7. About the diary study• 5 participants: small business owners, bookkeepers• Diaries kept for up to 8 weeks beginning at install• Diaries submitted weekly by email• Compensation – at least $40/week
  8. 8. About the diary study• Provided template in Word to record – Daily Work: tools they used, other tasks – Issues and Comments: problems with our product
  9. 9. Tips
  10. 10. Recruit extra participants• Recruited 12• 5 completed• Reasons for drop outs were interesting findings, too
  11. 11. Use a recruiter to save time• Pros and cons• They can recruit and more – Email documents – Remind of deadlines – Collect diaries
  12. 12. Communicate deadlines• Communicate due date for first submission• Disqualify participants who don’t respond• Close the loop
  13. 13. Template tips
  14. 14. Make it easy for participants• Keep it short – This study: 3 questions daily, 6-7 per incident• Separate questions from answers
  15. 15. Separate questions from answers
  16. 16. Show sample entriesWhat did you do for your business today?• I answered several customer phone calls.• Called automated bank system to make sure wehave enough money for payroll.• Called in payroll to ADP (we outsource payroll).• Ordered parts for one of our jobs by phone.
  17. 17. Ask for screenshots“Looking at Aged Receivables…. They donot match. Why are they different?”
  18. 18. Tips for screenshots• If they are important, require them• Include instructions
  19. 19. Structure template for easy analysis• Separate daily entries from critical incidents• Include participant name and week number in footer and file name
  20. 20. Iterate on the template• Prior to launching• As you receive user feedback
  21. 21. Process tips
  22. 22. Require daily entries• Keeps participants engaged• Gives you a reason to pay them• Opportunity to learn about the broader work context
  23. 23. Talk to the participants yourself• Intro call to understand business and product expectations• Follow up on unclear entries• Provide support
  24. 24. Research tips• Read entries as soon as possible• Leave time to research
  25. 25. Data analysis• No different from other methods• Evaluate data – Excel to categorize theme, second theme, week #• Share and act on data – Summarize key findings, provide recommendation for each – Use customer quotes – Share complete diaries
  26. 26. Unique contributions of diary study• Infrequent tasks• Help use• Users resolving issues• Tasks over time
  27. 27. Reasons to run a diary study• No travel required• Non-intrusive, in context• Cheaper than site visits
  28. 28. Engagement and meaningful data• Pay for more complete responses• Encourage useful responses through feedback on recent entries• Continue participants who are useful
  29. 29. Key Points• Daily entries• Explain what you expect• Sample entries• Reasonable number of questions• Personal contact with researcher• Questions?
  30. 30. ContactMore tips in conference proceedingsanance@gmail.comTwitter: @amandaux

×