Stop Running Random Tests
•Download as PPTX, PDF•
0 likes•311 views
Validate experiments, improve results.
Recommended
Recommended
More Related Content
More from Theo van der Zee
More from Theo van der Zee (13)
Recently uploaded
Recently uploaded (20)
Stop Running Random Tests
- 1. Stop Running Random Tests Validate experiments, improve results Theo van der Zee - ConversionReview @ Conversion Hotel 2014
- 2. “Split testing is not about throwing all ideas at users to see which ones make sense.”
- 3. Helping visitors To achieve their goals
- 4. Experiments
- 10. Customer data
- 15. Expert review
- 18. Variations
- 19. Research
- 27. “Split testing doesn’t give you any excuses not to do your research first.”
- 28. Theo van der Zee ConversionReview.com
Editor's Notes
- Tim Ash, author of best-seller LPO, best picture I could find of him ;-) Important message. [HANDS] who has run test with less than 10 mins pre-research?
- Even in the light of testing quotas, performance-based work, getting paid by the hour, being on a payroll from 9-5, or just starting out with testing
- Is our proposed experiment fixing something that actually bothers visitors? Or do we just THINK it bothers them? Is there data (quantitative or qualitative) to back this up?
- “A lot of times, people don't know what they want until you show it to them.” — Steve Jobs
- “If I had asked people what they wanted, they would have said faster horses.” — Henry Ford
- People as numbers, data points, learnings derived via statistical modeling. Learn the ‘what’ is happening, no why given
- Find conversion bottlenecks. Landing pages, bounce rates, dashboards, demographics, interests, segments?
- Log events: is really nobody clicking button X? Which of the new dynamic landing pages aren’t converting? [QUESTION] Who uses it? how?
- From CRM, sales data, surveys, etc. Who is buying, what, why, from where, when? Patterns? Predictions? [EXAMPLES]
- PMA, backend of database. SalesForce / CRM? Use wisely and ethically. Which item (im)popular for which user? Useful for segmentation! [QUESTION] Who uses it? how?
- People as people, n=1 at a time, asking questions. Learn the ‘Why’ behind the ‘What’
- Remote or real life, usertesting.com. Scenarios, tasks, demographic selection. [QUESTION] who used user testing? results?
- 5 minutes (often more), English websites, 3 per month, valuable insights.
- Heuristic analysis of user scenarios: what do users want as a given time? How to give that to them optimally? (Check out ConversionReview.com)
- Contrast analysis tool, lightest setting, shows part of the text is unreadable
- Algorithmically generated heatmap shows menu gets zero attention (doesn’t have to be a problem though)
- Is there a real problem? Then look for proper solutions in the form of variations.
- What information can we find about (solving this) problem? Articles, blogs, journals
- Feedly, I’ll publish a blog with CRO resources I’m following. [QUESTION] Which sources do you recommend?
- Can the variations work for visitors? Do they understand what is going on? Choose from: 5 second, click, navigation or preference test.
- Some of the usability tests I’ve done. Important to focus on strengths of each method
- Do the variations work for real visitors (and why)? Currently in public beta, gonna disrupt the market if the play it right
- Let me know if you want a little demo
- No need for any credits for this quote right? ;-)
- Get a free review, check it out!