Running Successful User Surveys         Surprises and Lessons Learned                                   April 6, 2005     ...
Today’s Discussion                                         Fast facts about MedeFinance                                   ...
Fast Facts About MedeFinance   Founded                                              By Jim Quist in 1994   Who We Are     ...
Service MedeFinance Offers                                                    MedeFinance Performance OnDemand    TM      ...
A Few MedeFinance Clients©2005 MedeFinance. Confidential & Proprietary   /   4
Today’s Discussion                                         Fast facts about MedeFinance                                   ...
Why survey?Week 1                        Week 2                    Week 3   Week 4   Week 5   Week 6     Week 7   Week 8  ...
What defines success?Week 1                        Week 2                    Week 3   Week 4   Week 5   Week 6   Week 7   ...
What’s important to know?Week 1                        Week 2                     Week 3        Week 4   Week 5    Week 6 ...
How should we survey?Week 1                        Week 2                     Week 3   Week 4    Week 5    Week 6   Week 7...
Is it easy to read and use?Week 1                        Week 2                     Week 3   Week 4   Week 5   Week 6   We...
Who should we survey?Week 1                        Week 2                        Week 3       Week 4    Week 5      Week 6...
Why is it important?Week 1                        Week 2                      Week 3    Week 4   Week 5     Week 6   Week ...
Did you forget?Week 1                        Week 2                Week 3    Week 4   Week 5      Week 6   Week 7   Week 8...
What does it all mean?Week 1                        Week 2                     Week 3   Week 4   Week 5   Week 6   Week 7 ...
What is important?Week 1                        Week 2                     Week 3   Week 4   Week 5   Week 6   Week 7   We...
How to publish the results?      Week 1                        Week 2                      Week 3             Week 4      ...
Get the value!Week 1                        Week 2                      Week 3      Week 4     Week 5   Week 6      Week 7...
Today’s Discussion                                         Fast facts about MedeFinance                                   ...
Satisfaction©2005 MedeFinance. Confidential & Proprietary   /   19
Benefits©2005 MedeFinance. Confidential & Proprietary   /   20
Credible Results©2005 MedeFinance. Confidential & Proprietary   /   21
Usability & Training©2005 MedeFinance. Confidential & Proprietary   /   22
MVQ - Most Valuable Questions©2005 MedeFinance. Confidential & Proprietary   /   23
Today’s Discussion                                         Fast facts about MedeFinance                                   ...
Surprises                        The survey was a success!                                    Validation and reality check...
Next time…                            Now that we have experience, next time                              we will…        ...
Go survey!                                   This was a helpful presentation.                                             ...
Upcoming SlideShare
Loading in...5
×

Running Successful User Surveys

128

Published on

Running Successful User Surveys: Surprises and Lessons Learned by Doug Stark at SVPMA Monthly Event April 2005

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
128
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Running Successful User Surveys

  1. 1. Running Successful User Surveys Surprises and Lessons Learned April 6, 2005 Doug Stark Sr. Director, Product Management dstark@medefinance.com 510.647.1300
  2. 2. Today’s Discussion Fast facts about MedeFinance Our survey process Key survey questions Surprises & Next time…©2005 MedeFinance. Confidential & Proprietary / 1
  3. 3. Fast Facts About MedeFinance Founded By Jim Quist in 1994 Who We Are Web analytic software and client services provider What We Do Help financial executives improve performance How We Do It Our Performance On Demand Service 1) Adaptive Web Analytics – personalized 2) Instant, on demand IT environment and 3) Client services – deep healthcare domain experience Headquartered Emeryville, CA Ownership Private, Profitable, 100% Growth Each of Last 3 Years Clients Over 400 hospitals…100% Healthcare Focused…©2005 MedeFinance. Confidential & Proprietary / 2
  4. 4. Service MedeFinance Offers MedeFinance Performance OnDemand TM Adaptive Web Client Analytics Services On Demand IT Environment©2005 MedeFinance. Confidential & Proprietary / 3
  5. 5. A Few MedeFinance Clients©2005 MedeFinance. Confidential & Proprietary / 4
  6. 6. Today’s Discussion Fast facts about MedeFinance Our survey process Key survey questions Surprises & Next time…©2005 MedeFinance. Confidential & Proprietary / 5
  7. 7. Why survey?Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 1) Determine the Objectives Measure end-user satisfaction with services & product Measure the company’s perception Gather input on future product enhancements Deepen knowledge about our user base Gain experience with direct user surveys©2005 MedeFinance. Confidential & Proprietary / 6
  8. 8. What defines success?Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 2) Establish Survey Goals 10 – 15 minutes to complete Response rate of 20 – 25% Complete in time for roadmap planning©2005 MedeFinance. Confidential & Proprietary / 7
  9. 9. What’s important to know?Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 3) Create Survey Questions 50 initial questions created for: 50 Qs User Profile Services Product Functionality Future Functionality Open Feedback & Optional Self Identification 35 Qs ~75% Quantitative vs. ~25% Qualitative Cross-functional team wrote questions Product Management / Client Services / Account Management / Executives©2005 MedeFinance. Confidential & Proprietary / 8
  10. 10. How should we survey?Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 4) Select Survey Engine There are many options on the market Key considerations: Advanced Analysis Email Distribution Anonymous Responses Export Functionality Branching Graphics Branding & Easy to Use Response Tracking Cost Unsubscribe/Remove©2005 MedeFinance. Confidential & Proprietary / 9
  11. 11. Is it easy to read and use?Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 The Survey:©2005 MedeFinance. Confidential & Proprietary / 10
  12. 12. Who should we survey?Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 5) Target Users Frequent to infrequent users Recent to past users CFOs, Directors, Managers, Consultants Gain customer stakeholders’ permission! Targeted users logged in: Number of times targeted users logged in: Quarterly 10% Monthly 15% Daily 50% Weekly 25%©2005 MedeFinance. Confidential & Proprietary / 11
  13. 13. Why is it important?Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 6) Conduct Survey - Invitation Personalize email invitation from CEO “As a valued friend and partner of MedeFinance, I trust you have experienced the commitment of our Company to excellence and our goal to make you successful… The survey starts at: http://www.surveymonkey.com/a... I look forward to your response by… Sincerely, Jim Quist, CEO©2005 MedeFinance. Confidential & Proprietary / 12
  14. 14. Did you forget?Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 7) Conduct Survey - Reminder After 7 days, we experienced a response rate of 25% Reminder invitation to those who had not responded boosted response rate to 35%! “Last week, Jim Quist, sent an online survey invitation…I want to follow up personally to ensure you receive this invitation… We truly are shaping the product based on our customers direct feedback… I look forward to your response by…©2005 MedeFinance. Confidential & Proprietary Sincerely, / 13 Doug Stark Sr. Director, Product Management
  15. 15. What does it all mean?Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 8) Analyze Results Determine validity Compare averages/means Observe distribution Correlate results with conjoint analysis Use common sense…©2005 MedeFinance. Confidential & Proprietary / 14
  16. 16. What is important?Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 9) Report Results Share key results early! Don’t make it a science project… Present results in actionable manner Publish by account, function, etc… Provide supporting detail for outliers Correlate key questions with supporting and/or opposing questions©2005 MedeFinance. Confidential & Proprietary / 15
  17. 17. How to publish the results? Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Example Results Graphs Response Rate = n% “We really like…” Extremely Dissatisfied Neutral Satisfied Extremely “We really like…” Dissatisfied (2) (3) (4) Satisfied 9. How well do you KNOW HOW TO USE...? (1) (5) Extremely Well Strongly Agree 27% n Enough Well Agree 58% Support Some Neutral 14% Not at Disagree 1% AllStrongly Disagree 0% StronglyStrongly Disagree Neutral Agree “It would be helpful Agree Disagree ©2005 MedeFinance. Confidential & Proprietary / 16 “It would be helpful if…” 10. I feel I was ADEQUATELY TRAINED on how to use... if…”
  18. 18. Get the value!Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 10) Take Action Respondents expect action! Assign functional leads to create action plans for: Immediate, Tactical, Strategic Product vs. Service By Functional Area Earn a strong future response rate by responding to respondents’ feedback!©2005 MedeFinance. Confidential & Proprietary / 17
  19. 19. Today’s Discussion Fast facts about MedeFinance Our survey process Key survey questions Surprises & Next time…©2005 MedeFinance. Confidential & Proprietary / 18
  20. 20. Satisfaction©2005 MedeFinance. Confidential & Proprietary / 19
  21. 21. Benefits©2005 MedeFinance. Confidential & Proprietary / 20
  22. 22. Credible Results©2005 MedeFinance. Confidential & Proprietary / 21
  23. 23. Usability & Training©2005 MedeFinance. Confidential & Proprietary / 22
  24. 24. MVQ - Most Valuable Questions©2005 MedeFinance. Confidential & Proprietary / 23
  25. 25. Today’s Discussion Fast facts about MedeFinance Our survey process Key survey questions Surprises & Next time…©2005 MedeFinance. Confidential & Proprietary / 24
  26. 26. Surprises The survey was a success! Validation and reality check - significant value can be achieved from a small group n<300 Response rate - reminders really do drive response rates Customer affinity counts How much work it really takes©2005 MedeFinance. Confidential & Proprietary / 25
  27. 27. Next time… Now that we have experience, next time we will… Segment the survey - ask the right questions to the right people Quality over quantity – perform more analysis with fewer questions. How will the answer be used? Increase survey group size Seriously consider outsourcing!©2005 MedeFinance. Confidential & Proprietary / 26
  28. 28. Go survey! This was a helpful presentation. Strongly Disagree Disagree Average Agree Strongly Agree©2005 MedeFinance. Confidential & Proprietary / 27
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×