Slides Ladislav Bartos recently used in his discussion w/ mentees of The Product Mentor. The Product Mentor is a program designed to pair Product Mentors and Mentees from around the World, across all industries, from start-up to enterprise, guided by the fundamental goals…Better Decisions. Better Products. Better Product People. Throughout the program, each mentor leads a conversation in an area of their expertise that is live streamed and available to both mentee and the broader product community. http://TheProductMentor.com

1. Lorem Ipsum Dolor
User Testing by
example by Ladislav Bartos

2. Agenda
❖ About me!
❖ About User testing!
❖ The big picture, where User testing sits!
❖ The type of testing!
❖ Testing and implementing changes ( the circle )!
❖ What can you test!
❖ Example 1!
❖ Example 2!
❖ Example 3!
❖ Example 4!
❖ Conclusion

3. About me
❖ 8+ year product management experience!
❖ passionate about building marketplaces !
❖ 4 years marketplace which run in 10 countries in 6 different
languages!
❖ Agile, lean, Continuous improvements, development and
integration!
❖ User centric from day 1!
❖ Training centre manager, teaching users to use internet

4. User Testing - and other Terms
❖ What is user testing?!
❖ It’s technique used to evaluate a product, concept, … by
testing it on users.!
❖ Why is it important?!
❖ Help to create a successful business via understanding
the target customers.!
❖ Is User Testing key to success?!
❖ No! You need to consider other ….

5. Test Cycle
❖ Set a goal!
❖ Why? Set assumptions!
❖ Implement and Test solutions!
❖ Evaluate results

6. Setting Goals
❖ Goals can be:!
❖ Test new idea/concept!
❖ Conversion-rate optimisation, Landing-page optimisation!
❖ Marketing optimisation!
❖ Business process optimisation!
❖ Search engine optimisation, Video search engine optimisation!
❖ Website optimisation!
❖ Social media optimisation!

7. Setting Assumptions
❖ You might have assumptions!
❖ Assumption generation - Qualitative testing!
❖ Usability testing!
❖ Hallway testing!
❖ Expert review!
❖ Reviewing analytics report on usability!
❖ Consumer consulting board!
❖ Surveys, polls

8. Implementing and Testing Solutions
❖ Quantitative testing!
❖ Controlled A/B testing

9. Evaluating Results

10. Example 1
❖ Company ABC wanted to launch a new product however they were not sure how to market it!
❖ Phase 1:!
❖ Goal:!
❖ Test multiple slogans!
❖ Assumptions:!
❖ Hallway testing!
❖ If the message is easy to understand the more users will subscribe to the product!
❖ Testing:!
❖ Create 5 Identical landing pages only the key message different!
❖ Email capture, encouraging users to subscribe!
❖ 1 Adwords campaign, split test to 5 pages!
❖ Evaluation:!
❖ The most converting message is used in Phase 2

11. Example 1
❖ Company ABC wanted to make sure the chosen message is the best messaging they can use!
❖ Phase 2:!
❖ Goal:!
❖ Test multiple slogans compared to the winner slogan in phase 1!
❖ Assumptions:!
❖ Online tool!
❖ If the message is easy to understand the more users will subscribe to the product!
❖ Testing:!
❖ Create 5 Identical landing pages only the key message different!
❖ Email capture, encouraging users to subscribe!
❖ 1 Adwords campaign, split test to 5 pages!
❖ Evaluation:!
❖ The most converting message is used in Phase 2

12. Example 2
❖ Company ABC launched a new responsive website and wanted to make sure …!
❖ Phase 1:!
❖ Goal:!
❖ Test multiple slogans!
❖ Assumptions:!
❖ Hallway testing!
❖ If the message is easy to understand the more users will subscribe to the product!
❖ Testing:!
❖ Create 5 Identical landing pages only the key message different!
❖ Email capture, encouraging users to subscribe!
❖ 1 Adwords campaign, split test to 5 pages!
❖ Evaluation:!
❖ The most converting message is used in Phase 2

13. Example 3
❖ Company ABC wanted to find out why the competitor is better !
❖ Phase 1:!
❖ Goal:!
❖ Test multiple slogans!
❖ Assumptions:!
❖ Hallway testing!
❖ If the message is easy to understand the more users will subscribe to the product!
❖ Testing:!
❖ Create 5 Identical landing pages only the key message different!
❖ Email capture, encouraging users to subscribe!
❖ 1 Adwords campaign, split test to 5 pages!
❖ Evaluation:!
❖ The most converting message is used in Phase 2

14. Example 4
❖ Company ABC wanted to test concept!
❖ Goal:!
❖ Test multiple slogans!
❖ Assumptions:!
❖ Hallway testing!
❖ If the message is easy to understand the more users will subscribe to the product!
❖ Testing:!
❖ Create 5 Identical landing pages only the key message different!
❖ Email capture, encouraging users to subscribe!
❖ 1 Adwords campaign, split test to 5 pages!
❖ Evaluation:!
❖ The most converting message is used in Phase 2

15. ❖ Strategic Marketing Optimization!
❖ Test your value-proposition message!
❖ Dramatically lift your sales without spending!
❖ Generate more business!
!

16. ❖ Conversion-rate optimisation:!
❖ The science and art of getting more revenue-generating
actions from the same number of visitors!

17. ❖ I’ve never seen a website that couldn’t be improved. In
fact the best online companies in the world are
committed to continuous improvement on their
websites.!
❖ If your company is like most, the vast majority of
visitors leave without indicating the most basic level of
interest. Are you satisfied that the majority of your
expensive traffic is being wasted? Should you be
allocating the majority of your scarce marketing budget
to driving more people to this under performing
website?

18. ❖ If you claim that your product or service is easy, take a
look at how easy your website is to understand and use:!
❖ How many steps are in your signup process?!
❖ How many fields of information are required in your
purchase process?!
❖ How easy is to find your shipping information?!
❖ Is your product information understandable?!

19. ❖ Why “Best practices” aren’t Best practices?!
❖ What is often missing in “best practice”
recommendations is a consideration for your unique
business environment, goals, and target audience. Your
website should be a vehicle designed to uniquely
communicate the value proposition of your product or
service and then make it easy for visitors to take the
desired action.!

20. ❖ HiPPO - Highest-paid person’s opinion !
❖ The Black Turtleneck Method - Guru’s advising best
practise!
❖ The customer tested method - Using scientific method of
hypothesis and testing to find optimised solutions. You
formulate hypotheses about which page layouts will
encourage the highest conversion rate and use controlled
A/B split testing to find out with statistical confidence. In
this case, when your designer presents design alternatives,
you say, “Those look like interesting design options. We
should test that”

21. ❖ Website re-design!
❖ A dirty little secret of website redesign is that the new site often
reduces conversion rates and revenue:!
❖ Your most valuable repeat customers may be accustomed to the
current site layout and confused by a new site. Their confusion
will result in lost revenue for you!
❖ The new design may fix some conversion problems but
unintentionally introduce worse ones, with a net effect of
reduced conversion rates!
❖ Most important, your website design may not be a problem, and
the redesign could end up being a complete waste of time and
money with no improvements to show for it!

22. ❖ Use Evolutionary Site Redesign - where new design,
layout, and content elements are tested and your website
gradually evolves toward the best-performing ideal.!
❖ In ESR redesign you prioritise your most important pages
and page templates and test new layouts, designs, and
content with controlled testing. You will get all the benefits
of a new site design without the risks.!
❖ As you’ve probably guessed, ESR essentially uses
conversion-rate optimisation principles to redesign your
site. You end up with a new site “look and feel” and
conversion-rate lift at the same time.