The document discusses methods for evaluating product improvement, including surveys, interviews, face-to-face meetings, live prototyping, and gathering customer feedback. It emphasizes the importance of beta users, in-app feedback, and analytics in the process of validated learning and product development. Success stories illustrate the effectiveness of these methods in transforming user frustration into constructive insights.

1. Objective
Question: Are we improving the product?
The first steps to answering this question…

2. • Surveys
• Phone interviews
• Face-to-face meetings
• Live prototyping
Customer Feedback
Might be useful… but there are other ways

3. Validated Learning
How lean teams build great products
• Beta users – Customers as employees
• In-app feedback – Turning frustrated users into useful allies
• Analytics – Actions speak louder than words

4. Beta Users
Customers as employees
• From 2004 until July 7, 2009 Gmail was in beta
• Extending our testing team - A/B testing
• Privileged users as a sales strategy
• SUCCESS STORY - Clever

5. In-app Feedback
Turning frustrated users into useful allies
SUCCESS STORY – Mint.com

6. Analytics
Actions speak louder than words
• Do users find this report useful?
• How does she use the report we created?
• Is Report A better than Report B?
• How long does it take her to complete action A?
• To complete process B?

7. In-app Feedback
Turning frustrated users into useful allies

8. Implementation – Validated Learning
Feedback?
• Beta users – Customers as employees
• In-app feedback – Turning frustrated users into useful allies
• Analytics – Actions speak louder than words