Customer FeedbackThe missing piece of the Agile Puzzle June 22, 2012 Agile Roots 2012 Salt Lake City, Utah
Who am I?• This guy• Maciej (“Ski”) Skierkowski• @skierkowski
Why are we here?• We are learning and we are sharing• I learned a lot and I’ve got a lot more to learn
Why are we really here?Building products is easy,building products customers want is hard.• I will share a few… – basic rules – basic mechanisms
Rule #1: Your opinion is wrong• Problem: you think you know what the customer wants, but you are probably wrong.• Customers will use your product in a different way than you expected• They might not use the product at all
Rule #1: Your opinion is wrong• The litmus test – “I think customers need…” (ur doin it wrong) – “Customer *xyz+ needs…” (ur doing it right)
Rule #1: Your opinion is wrong• Solution: build a machine that validates (or rejects) your assumptions about customer needs
Rule #2: Don’t ask what people wantProblem: Customers ask for features. Your are inthe business of solving problems, not buildingfeatures; you need to identify the problems tosolve, not the features to build.
Rule #2: Don’t ask what people wantExample 1Consider a feature request such as “GitHubshould let me FTP up a documentation site formy project.” What this customer is really tryingto say is “I want a simple way to publish contentrelated to my project…” – Tom Preston-Wernerfrom “Ten lessons from GitHub’s first year”
Rule #2: Don’t ask what people wantExample 2… it’s not just about features.• Friend: “How much are you willing to pay for an iPhone”?• Me: “No more than $250”• (3 months later)• Me: “I just bought an iPhone for $450”
Rule #2: Don’t ask what people wantExample 3…Customers: “we need custom SSL”• That was a no-brainer, but we had more questions: – How much are people willing to pay for it? – How much demand is there? – Do they need to update certificates?
Rule #2: Don’t ask what people wantSolution: when someone asks for a feature, ask“why”, and do it at least 3 times
Rule #3: Fake it till you make it• Problem: If you build it first… – Will people use it? – Will they pay for it? – How much are they willing to pay for it?• Solution…• First build a façade to allow users to express interest• Once you have traction/evidence, then build the real thing
Rule #4: Make it cheap, throw it out• Problem: you invest in building a product but nobody wants it• Don’t get too attached• Take shortcuts• Build only what you need. Cutting is king.• Throw it out if people aren’t using it; it’s just dead code.
Rule #5: Scalability is not a problem, make it one• Problem: Engineers want to make it scalable… but nobody is using it• The real problem is getting people to use your product and to pay for it. Scalability is just premature optimization.• Scalability is a great problem to have.• (e.g. PHP Fog hitting 2^15-1 apps)• Keep an eye out for “quality” too.
Rule #6: Overcommit and deliver• Keep resources fixed, keep schedule fixed, and keep the functionality• Leverage creativity and 3rd party services to deliver without compromise. – Pusher, Loggly, Airbrake, Sendgrid, Mailchimp, Google Forms, GitHub, RedisToGo, …
Rule #7: Don’t bother with feature lists• If you can’t remember it, it’s probably not worth building• The things you need to build are the ones your customers won’t shut-up about.• If you have a big backlog/icebox… delete it. Nobody will miss it.
Rule #8: Not all users are customers• Problem: Listening to the wrong customer will lead you to solving the wrong problems.• Learn why customers use your product• Learn to identify users who are your target customers
Recap of the rules1. Your opinion is always wrong2. Don’t ask what people want3. Fake it till you make it4. Make it cheap, throw it out5. Scalability is not a problem, make it one6. Overcommit and deliver7. Don’t bother with feature lists8. Not all users are customers
Rules => Mechanisms• Rules give us structure• Product team delivers value to customers• Customers tell us what is valuable… but how?
Outstanding Support• Everyone provides support – Executive team (CEO, Director) – Developers – Support team• A business differentiator• Very easy to identify problems for existing customers
Net Promoter Score• The survey: – [0 to 10] Would you recommend this service? – Why?• Powerful free response• Score doesn’t tell you much, but lots of great candid feedback about product• Identifies with both existing and lost customers
Live Chat• “Hi, anyone there”• Makes the company personal• Immediate gratification and solutions for support issues
Interviews• Every week interview a customer for 15-30 minutes over the phone or over coffee• Common questions: Name, Job, story, their business, what they love/hate, etc.• Special question: Ask for advice• Share info with the company• Helps make the customer a promoters• Makes customers the advisors
Surveys• Use sparingly• “Data driven decision making” or “decision driven data making” (hint: it’s the latter)• Great for targeted data (e.g. marketing material)• Terrible for testing your value proposition
Usability study• Don’t make this formal• Make it cheap and easy• Hire a good designer/usability person• Everybody focuses on usability• KEEP IT SIMPLE!!
Discussion Forums• http://community.phpfog.com/• Surprised at engagement• Customers helping other customers• Great way to engage with groups of passionate people
Third party services• Customer information is everywhere• Services we monitor and engage – Quora – Blogs and comments – Stack overflow – Google Alerts – Hacker News
Workflow analytics• Tools – Google Analytics – KISS Metrics for flow measurement• Great for optimization• Bad for testing hypothesis
Retention Emails• “I noticed you haven’t logged in over the past 2 weeks”• You got the user over the biggest hurdle (getting their attention) but never got them to the finish line (converting them to a customer).
List of mechanisms• Outstanding support• Net Promoter Score (NPS)• Live Chat• Interviews• Surveys• Usability studies• Discussion forums• Third party services• Workflow analysis• Retention emails