Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
End-to-end validation and optimisation
via Diary Study
Katie Phillips
User Experience Researcher -
Australia Post
Agenda
2
1. Project brief
2. Methodologies
3. Recruitment & planning
4. Why Slack?
5. Key Insights & implementation
6. Can...
3
Project brief
• New application for consumers & businesses to pay for & print Australia Post
labels online
• Application...
Choosing a
research
methodology
4
Key factors
• What is your objective?
• Validating the end-to-end journey of a customer
...
Moderated user testing
Pros
• Great insights into usability
• Defects obvious
• Can be observed/live streamed
• Remote opt...
Cons
• Moderator required (can do self-moderated as well)
• Logistics
Time requirements
• Try for at least 5 participants
...
Diary study
Pros
• Great insight into end-to-end process
• Real world factors
• Lots of raw quotes and photos
• Runs in ba...
Cons
• Dependent on what customers
say they do
• Drop outs
• Technology
• Requires very clear instructions
• Prototype mus...
Timeline
Prepared
stakeholders
Plan & recruit
Analysis &
Report
Diary study &
in-flight
activities
Diary study
planning &
recruitment
• Recruited 12 small business
participants for a 3 week
study
Tips
• Be tight on recrui...
Why Slack?
• Free!
• Great desktop and mobile apps
• Private channels
• Easily invite others from your
team to observe
• K...
Setting up Slack
• Write clear instructions for participants BEFORE the start date
• Set up a private channel for each par...
Setting up Slack
Setting up Slack
Key insights & resolutions
16
“If you could let it accept .3kg and
auto add the zero that would be
great - adding the leading zero
stuffs me up each t...
17
Insights & resolutions
Resolutions
• Non-interface issues raised with
international product team
• Non-application issu...
Can you run
it?
18
Key factors
• Is it the right method for your product?
• $$ for incentives
• $150-$250 per participant ...
19
The real question:
Can you afford NOT to run something?
“If I wasn't participating in this
there is no way in heck I wou...
Nielsen Norman Group on choosing methodologies
ethn.io's Remote Research tools guide
Frog’s Design Research bookshelf
IDEO...
Thanks for
your time!
Katie Phillips
User Experience Researcher -
Australia Post
Upcoming SlideShare
Loading in …5
×

Learning from your customers - A diary study with Slack

2,226 views

Published on

Katie Phillips talk on using Diary Studies for customer research including how she used Slack for a study at Australia Post. From Product Anonymous March 2017 event.

Published in: Internet
  • Be the first to comment

  • Be the first to like this

Learning from your customers - A diary study with Slack

  1. 1. End-to-end validation and optimisation via Diary Study Katie Phillips User Experience Researcher - Australia Post
  2. 2. Agenda 2 1. Project brief 2. Methodologies 3. Recruitment & planning 4. Why Slack? 5. Key Insights & implementation 6. Can you run it? 7. Useful links and resources
  3. 3. 3 Project brief • New application for consumers & businesses to pay for & print Australia Post labels online • Application live since October • New features being deployed frequently • “International” feature soft-released November
  4. 4. Choosing a research methodology 4 Key factors • What is your objective? • Validating the end-to-end journey of a customer sending internationally through this application & using insights to enhance it • Where are you fidelity-wise? • In production • How much time can you commit? • Endless (aka 2.5 iterations) • What’s your budget? • Enough for professional recruitment, incentives, and a dedicated researcher Create order Get to MyPost Business page Review & confirm Pay Print labels Prepare package Lodge parcel
  5. 5. Moderated user testing Pros • Great insights into usability • Defects obvious • Can be observed/live streamed • Remote options • Can watch what customers are actually doing, not just what they say they do • Any fidelity
  6. 6. Cons • Moderator required (can do self-moderated as well) • Logistics Time requirements • Try for at least 5 participants • Run time: 45min-1hr each • Analysis time: 1-3hr per participant Moderated user testing
  7. 7. Diary study Pros • Great insight into end-to-end process • Real world factors • Lots of raw quotes and photos • Runs in background • Helps team understand value of product
  8. 8. Cons • Dependent on what customers say they do • Drop outs • Technology • Requires very clear instructions • Prototype must be functional Time requirements • Try for at least 8 participants • Run time: 1-2 weeks • Analysis time: 2-4 hours per participant Diary study
  9. 9. Timeline Prepared stakeholders Plan & recruit Analysis & Report Diary study & in-flight activities
  10. 10. Diary study planning & recruitment • Recruited 12 small business participants for a 3 week study Tips • Be tight on recruitment • Be clear on what they need to do to receive incentive • Incentives at the end (or segment) • Find an easy platform Recruiting specs • Demographics (age, gender, location) • Behavioural (online habits, social, travel) • Exclusions (colleagues, language restrictions)
  11. 11. Why Slack? • Free! • Great desktop and mobile apps • Private channels • Easily invite others from your team to observe • Keep the team open for as long as you need • Group channels (help, participant chat) • Bots (Anonymous feedback, Polly surveys, To Do, SlackBot) • Easy to moderate & set up notification preferences
  12. 12. Setting up Slack • Write clear instructions for participants BEFORE the start date • Set up a private channel for each participant • Set up a group help channel • Be prepared to moderate at all hours • Provide an alternative
  13. 13. Setting up Slack
  14. 14. Setting up Slack
  15. 15. Key insights & resolutions
  16. 16. 16 “If you could let it accept .3kg and auto add the zero that would be great - adding the leading zero stuffs me up each time.” Insights & resolutions Resolutions • Development team allocated 1/2 - 1 iteration to incorporate critical feedback during study • Optimisation backlog was defined and prioritised based on insights • Instructions page re-designed Insights • Confirmation of value • Form fixes needed (12 usability issues) • No one reads instructions • It’s easy to learn “All of my friends who run online businesses will be jumping for joy”
  17. 17. 17 Insights & resolutions Resolutions • Non-interface issues raised with international product team • Non-application issues raised with segment team • Conducted additional user testing around language • Input into website IA • Insights remained with our team for future features and projects Insights • Plastic pockets aren’t easy to get or use! • Trouble navigating to application • Awareness issues • Language confusion “Oh, there it is! It's like hidden! That's way too hidden”
  18. 18. Can you run it? 18 Key factors • Is it the right method for your product? • $$ for incentives • $150-$250 per participant for a 2 week diary-study • Plus $100-$200 per participant for recruitment services • your time if you choose to recruit on your own • Time • time requirements as mentioned previously • development time to incorporate feedback and changes
  19. 19. 19 The real question: Can you afford NOT to run something? “If I wasn't participating in this there is no way in heck I would even know this is available” “I'm just assuming I would take this into my local post office and have them scan and hopefully they don't tell me that I can't use them.” “ It does not like the price I have entered for the shoes to the US yet it is the price she paid for them. I have tried varying the amount but it still remains highlighted in red no matter what I do”
  20. 20. Nielsen Norman Group on choosing methodologies ethn.io's Remote Research tools guide Frog’s Design Research bookshelf IDEO’s Method Cards 20 Useful resources
  21. 21. Thanks for your time! Katie Phillips User Experience Researcher - Australia Post

×