Experiments never killed anybody - Rajiv Srivatsa, UrbanLadder & Thiagarajan, Intuit #PNCamp, Pune
•Download as PPTX, PDF•
0 likes•1,962 views
The document discusses the importance of experiments in validating ideas and making decisions. It provides examples from Urban Ladder of how they designed and conducted experiments to test assumptions and gather user feedback. Key aspects of experiment design discussed are identifying the problem, developing hypotheses to test, running targeted experiments to collect meaningful data, and using the results to inform next steps. Conducting many small, low-cost experiments is advocated over large projects to reduce risks and costs from failed assumptions.
Recommended
More Related Content
Viewers also liked
Viewers also liked (19)
Similar to Experiments never killed anybody - Rajiv Srivatsa, UrbanLadder & Thiagarajan, Intuit #PNCamp, Pune
Similar to Experiments never killed anybody - Rajiv Srivatsa, UrbanLadder & Thiagarajan, Intuit #PNCamp, Pune (20)
More from ProductNation/iSPIRT
More from ProductNation/iSPIRT (20)
Recently uploaded
Recently uploaded (20)
Experiments never killed anybody - Rajiv Srivatsa, UrbanLadder & Thiagarajan, Intuit #PNCamp, Pune
- 1. Experiments never killed anybody… Rajiv Srivatsa (Urban Ladder) / Rajan (Intuit) #PNCamp Dec 04 2013
- 2. 1. Vision, Target, Problem These are more important to crystallize than the solution or the idea itself…
- 3. 1. Vision, Target, Problem • The Urban Ladder Example • Vision: Beautiful homes for millions of Indians Tip: The vision is always user-facing! Bold. Specific. Solution-free. Customer-focused. Memorable. • Target: ‘Upper Middle-Class’ ‘Home-proud’ ‘Digitally-Savvy’ ‘Urban Indian’ ‘Earning > 1 lak family income’ ‘Well connected’ ’Travelled internationally’ Tip: The sharper it is defined, the better it is for engaging with a clear set of customers for each experiment • Problem: Getting good quality, well-designed, trust-worthy furniture at reasonable prices Tip: Balance going too wide or too narrow
- 4. 1. Exercise • Vision, Target Audience & Problem Statement • Pair with 1 other person • 2-min to write the following on a post-it • The vision • Target audience • Problem statement • Your partner to introduce you from the post-it you have prepared!
- 6. 2. Experiments – the pre-work Pick a broad decision / feature / initiative that you last did. Please answer the following questions (write in a post – it ) • • • • What was the decision? Was it a successful one? (Y/N) How long did it take to realize the learning about the decision? What were some implicit assumptions made in the decision?
- 8. 2. Why experiments observe idea analyze & present decision design build launch & marketing users
- 10. 2. Why experiments In the age of rapid changes... observe idea Design analyze Build & Test present From: Decisions by “Opinions and Powerpoint” users decision To: Decisions by “ Experimentation & Learning”
- 11. 3. Designing the experiments There are no failed experiments, only negated hypotheses… need-gap analysis product-market concept test [ Testing if the need-gap is a big enough need; understanding priorities ] [ A feasible brand and product concept that delivers on the vision: how much are users willing to pay for this ] leap of faith assumption [ Key behavioral assumption about our idea that’s keeping us up at night if this assumption is false, nothing else matters ] hypothesis [ Solutions/features that could support our leap of faith assumption . It can also be considered as a restatement of leap of faith in a numerical way] repeat tests experiment reflect [ Conditions created to measure behavioral response to learn about the hypothesis ] [ Pivot the idea or persevere ]
- 12. 3. Designing the experiments need-gap analysis product-market concept test leap of faith assumption • The Urban Ladder Examples • Confirm need-gap priority Discovery: Quality and design were more important than price Tools: SurveyMonkey, Customer Interviews Dataset: 100 Responses; Over 40 in-depth interviews • Is the brand promise exciting? Question: People should relate to the brand-name, logo and promise Tools: LaunchRock, FB, Customer Interviews Dataset: 350 sign-ups in 2.5 months; 25 FB shares; Over 100 conversations • Validate product-market concept fit Validation: People should like the product at a feasible price-point Tools: Polls, PPT, Email, A/B Tests Dataset: Over 40 responses; 25 in-depth interviews
- 13. 3. Designing the experiments need-gap analysis product-market fit leap of faith assumption • The Urban Ladder Example • Hypothesis: People get a sense of the furniture quality online and buy it • Dataset: Friends and family • MVP definition: Beta version of the site using outsourced technology, design integrated with Google Analytics; Calls to check source and feedback; Basic range in 2 categories; Merchandised products with story • Target Metrics: If 2% of the visitors buy items for at least Rs. 5k, then we can say that this experiment can be taken to the next level • Data Gathering: Spread over a 2 week period to get to the first 25 transactions, largely from family and friends • Results Analysis: Strong interest in buying product; ability to get a sense of colors and size from images; strong interest in other cities, categories Experiment Repeats: Neutralize the audience, check with friends of friends who probably don’t have direct affinity to people or brand; Test with service / without service; test repeat rates
- 14. 3. Designing the experiments • Write down on a post – it ( 5 min) • Problem • • • • Leap of Faith Assumption Numerical Hypothesis Experiment Metric • Pair up & review
- 15. 4. Also important are… • • • • • • A strong business plan with valid numbers and realistic targets The right questions to ask during the probe phase Messaging, Communication, Design, Brand Focus on doing few things well Clear milestones A smile
- 16. Q&A What would you do differently in your work?
- 17. Credits Thanks to www.dilbert.com and Scott Adams for the awesomeness that is Dilbert!
- 18. MVP Fidelity
Editor's Notes
- Examples of big bang launch is WebVan and Iridium. WebVan is a classic case from the dotcom time which tried to sell groceries online. The problem with the model is by the time launch happens the user needs even if model right changes.
- Add a recent example in the last 5 years , an expensive failure Think of an example which used the big funnel model Think of an example of experimentation