Your optimization and testing strategy can potentially determine the success or failure of your product. A/B testing is the most common approach used to validate optimization hypothesis. This approach while powerful has many limitations and does not always result in the highest ROI. Personalization is an alternative approach which can maximize value for specific audiences but can be costly to implement at scale. Understanding the strengths and weaknesses of each approach will be critical in forming a winning optimization strategy.

1. What Are Product Optimization Tactics by
Bills.com Senior PM
www.productschool.com

2. FREE INVITE
Join 23,000+ Product Managers on

3. COURSES
Product Management
Learn the skills you need to land a product manager job

4. COURSES
Coding for Managers
Build a website and gain the technical knowledge to lead software engineers

5. COURSES
Data Analytics for Managers
Learn the skills to understand web analytics, SQL and machine learning concepts

6. COURSES
Blockchain and Cryptocurrencies
Learn how to trade cryptocurrencies and build products using the blockchain

7. Ali Daftarian
TONIGHT’S SPEAKER

8. Online Funnel Optimization
Tactics
Ali Daftarian - Product School - 05/02/2018

9. Introduction
● Currently PM in mortgage lead gen @ Bills.com (part of FFN)
● Prior PM roles at Western Union & Simplikate (acquired by
Phunware)
● Prior career as a QA Engineer
● Bachelors in CS from UBC & MBA from SMU (in Canada)
● Fun fact: certified yoga instructor (Yoga to the people)

10. Agenda
● Mortgage Lead Gen flow Overview
● Overview & use cases of below in online lead gen funnels:
○ A/B testing
○ Personalization
○ Predictive personalization
● Summary
● Q&A

11. Online Mortgage Lead Gen Overview
● Focus on single transaction (‘lead’)
● Sensitive to market conditions (Ex. Increase in interest rates)
● Impacted by government regulations (Ex. HARP extension)
● Marketing driven (Ex. changes in marketing costs)
● Highly competitive and dynamic
● Lead buyers exert great influence on the quality and price for leads they
accept

12. A/B Testing Overview
A way to compare two versions of something to see which one performs better
● Benefits:
○ Allows for distinction between causation and correlation
○ Streamlined and easy to use tools available (ex. optimizely)
○ Can yield significant improvements to KPI goals
● Cons:
○ Does not maximize value across audiences
○ Not easily adaptable to changes in user behavior or market conditions
○ Can be time consuming to get statistically significant results and
business value

13. A/B Testing Examples
Appropriate for testing fundamental changes in the online experience
Examples:
● Test difference number of steps and grouping of question for online forms
(Ex. LMB)
● Test fundamentally different designs or user experiences (Ex. LMB)
● Longer forms and moving deeper in the funnel (Ex. LT)

14. Personalization
Customizing a service or a product to meet the specific needs of a segment of
consumers.
● Benefits:
○ Maximizes the business value from specific consumers groups
○ Presents the best customer experience to specific consumers groups
● Cons:
○ Difficult to discover differences in audiences
○ Costly to implement personalized experiences at scale

15. Predictive Personalization
Use predictive technologies to determine the variations of the product or service
which perform best for specific consumer groups.
● Benefits:
○ Maximize the business value across all consumer groups
○ Presents the best customer experience across all consumer groups
● Cons:
○ Fewer commercial tools available
○ Costly or complex to implement technology in-house

16. Personalization Examples
Ideal approach for sustainable ongoing optimizations, capturing incremental value
and maximizing value.
Examples:
● Engaging headlines on landing pages
● Convenient ‘common’ answers
● Consistent CTA placements
● Setting expectations and progress indicators
● Effects of branding and site logos
● Helpful tips

17. Summary
● Use A/B testing to find the best base site experience
● Use personalization to optimize for differences in audiences
● Leverage predictive personalization if available to:
○ Maximize business value and consumer satisfaction
○ Adapt to changes in market and consumer behavior
○ Streamline ongoing optimizations

18. Part-time Product Management Courses in
San Francisco, Silicon Valley, Los Angeles, New
York, Austin, Boston, Seattle, Chicago, Denver,
London, Toronto
www.productschool.com