SlideShare a Scribd company logo
1 of 24
Download to read offline
1
A/B Testing with
Behavioral Insights
Building hypothesis and analysing experiments with behavioral insights
A Product Webinar by VWO
2
What I do day in, day out?
“Look at session recordings &
heatmaps of mobile apps of our
customers and suggest them UX
improvement ideas”
Little about me
12+ years of managing and marketing
websites and apps, for B2B & B2C.
Piyush Sharma
Product Marketing, VWO
3
Agenda
1
2
3
4
5
6
Mobile app experiment lifecycle
Collect data
Analyse test results
Popular use cases
Tour of session replays, heatmaps & funnels
Q & A
4
Mobile app experiment lifecycle
Define Objective
Activation, Engagement, Retention,
Conversions
Develop Hypothesis
Building hypothesis for multiple variations
Analyse Test Results
Check movement in metrics. “And why?”
Collect Data
Quantitative Analysis, Qualitative Analysis
Run Experiment
Running A/B tests, multivariate or split tests
Deploy or Record Learnings
1
2
3
5
4
6
Repeat
5
Collect data
5
6
Collect Data
Quantitative Data Qualitative Data
What Users Say
VWO Insights - Mobile App
Session recordings
Heatmaps
Surveys
User Interviews
Reviews
Emails
What Users Do
7
Collect Data - Quantitative Data
Limitations
Not easy to find the why behind the numbers.
60%
Completed app
onboarding
78%
Dropping off from the
checkout screen
9%
Users use the
new feature
8
Search what users are
saying around the issue.
Only 1 in 25 unhappy
customers complain
directly to you.
Source: Esteban Kolsky Research
App store reviews
Support emails
8
Collect Data - Qualitative Data
Moreover, lot of to and fro with the
user to understand issue.
9
Deal with conscious (biased)
responses.
App is used with subconscious
instincts
Gaps between what they say
and what they do
Difficult to deduce exact
actionables
9
“Onboarding form is long”
“Price is too high”
“Got confused between options”
Collect Data - Qualitative Data
Surveys
User interviews
10
10
Collect Data - Qualitative Data
Session recordings
Heatmaps
Understand why something
might be happening
See exactly what the users see
Get team’s confidence into
your next A/B test with a
glaring insight you observed
11
11
Quantitative Data
Collect Data
Example
Why is this user cohort
exiting the app?
Insight
The navigation of the app
is not clearly visible on
iOS so users don’t know
where to go.
Qualitative Data
Example
Unusual number of exits
from home screen.
Insight
Majority of iOS users
using the latest app
version are exiting the
home screen.
Power-packed Hypothesis
Example
Test with a new version of
navigation which is much more
visible.
12
Analyse Test Results
12
13
13
Analysing Test Results
14
Analysing Test Results
Quantitative Data
Qualitative Data
(Session Recordings & Heatmaps)
Statistical significance and winning variant
Compare across multiple KPIs
How different segments performed?
External and internal factors?
Why there was an improvement?
How was the behavior different in
both the variations?
Insights that quantitative data might
miss
15
Use cases
15
16
Quantitative Insight - Low use of the search
function within the app.
Behavioral Insight - “Users try the search feature
but quickly give up. They seem to prefer
browsing through categories.”
A/B Test Hypothesis - Improve search
functionality with auto-suggestions and filters to
make it more user-friendly.
Analyzing Results - Monitor if there's an
increase in the use of the search function without
users getting confused about what to search.
Use cases
16
Feature adoption
17
Quantitative Insight - Only 20% women
completed app onboarding.
Behavioral Insight - “Users want to skip the DOB
option but they can’t, hence dropping off”
A/B test Hypothesis - Make the DOB optional in a
new version.
Analysing Results - More women completing the
onboarding. Verify if the users are indeed skipping
the DOB and going past next screens.
Use cases
17
Onboarding completion
18
Use cases
18
Quantitative Insight - Product screen showing
unusual drop-offs
Behavioral Insight - “Users tapping on the image
to magnify it but failing. Rage taps on the image.”
A/B test Hypothesis - Provide option to see a
magnified view with multiple images, so that user
can see product in detail
Analyse Results - Check if enough users are
actually tapping and checking out the product the
way you expected.
Improved conversions
19
Use cases
More abandons in new app version How
users compare between new app
versions?
New users not sticking - How new vs
returning users compare?
UK users churning - How users from
different geographies behave?
Check it out →
19
20
More events?
Add to cart? Checkout?
Upgrade plan?
20
Use Cases
21
Use Cases
21
Which are important areas?
Which areas are ignored?
Test UX elements
CTA placements
Eliminate unnecessary elements
Identify new clickable areas
22
22
Let’s see
Session Recordings &
Heatmaps
in action
23
23
Open for any questions.
24
Want to understand how it could help your
app and explore features in detail?
Try VWO free
24

More Related Content

Similar to Fueling A_B experiments with behavioral insights (1).pdf

Optimizing Mobile App UX for better engagement & conversions.pdf
Optimizing Mobile App UX for better engagement & conversions.pdfOptimizing Mobile App UX for better engagement & conversions.pdf
Optimizing Mobile App UX for better engagement & conversions.pdfVWO
 
Understand user struggles. Optimize mobile app UX.
Understand user struggles. Optimize mobile app UX.Understand user struggles. Optimize mobile app UX.
Understand user struggles. Optimize mobile app UX.VWO
 
Seven Peaks Speaks: User testing to action
Seven Peaks Speaks: User testing to actionSeven Peaks Speaks: User testing to action
Seven Peaks Speaks: User testing to actionSeven Peaks Speaks
 
12 steps that will lead you to data driven mobile app development
12 steps that will lead you to data driven mobile app development12 steps that will lead you to data driven mobile app development
12 steps that will lead you to data driven mobile app developmentInnopplinc
 
Ultimate Guide to Moblie App A/B Testing
Ultimate Guide to Moblie App A/B TestingUltimate Guide to Moblie App A/B Testing
Ultimate Guide to Moblie App A/B TestingEvgeny Tsarkov
 
Mob4Hire What is mobile app and website usability?
Mob4Hire What is mobile app and website usability?Mob4Hire What is mobile app and website usability?
Mob4Hire What is mobile app and website usability?Stephen King
 
Elizabeth Hom's UX Portfolio
Elizabeth Hom's UX PortfolioElizabeth Hom's UX Portfolio
Elizabeth Hom's UX PortfolioElizabeth Hom
 
Appurify Performance Automation Whitepaper FINAL
Appurify Performance Automation Whitepaper FINALAppurify Performance Automation Whitepaper FINAL
Appurify Performance Automation Whitepaper FINALJennifer Shambroom
 
Personalization Strategy Workshop - Los Angeles
Personalization Strategy Workshop - Los AngelesPersonalization Strategy Workshop - Los Angeles
Personalization Strategy Workshop - Los AngelesOptimizely
 
3 Steps To Improving Your Online Performance
3 Steps To Improving Your Online Performance3 Steps To Improving Your Online Performance
3 Steps To Improving Your Online PerformanceBecome Customer-Centric
 
Are you a lean mobile startup? Applying lean startup principles to mobile app...
Are you a lean mobile startup? Applying lean startup principles to mobile app...Are you a lean mobile startup? Applying lean startup principles to mobile app...
Are you a lean mobile startup? Applying lean startup principles to mobile app...Aravind Krishnaswamy
 
Clover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive ExperimentationClover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive ExperimentationOptimizely
 
A_B Testing For Progressive Web Apps – Implement Effectively.pdf
A_B Testing For Progressive Web Apps – Implement Effectively.pdfA_B Testing For Progressive Web Apps – Implement Effectively.pdf
A_B Testing For Progressive Web Apps – Implement Effectively.pdfLucas Lagone
 
Modeling Online Hotel Choice: Conjoint analysis as a multivariate alternative...
Modeling Online Hotel Choice: Conjoint analysis as a multivariate alternative...Modeling Online Hotel Choice: Conjoint analysis as a multivariate alternative...
Modeling Online Hotel Choice: Conjoint analysis as a multivariate alternative...SKIM
 
Looksoft Mobile Transformation
Looksoft Mobile TransformationLooksoft Mobile Transformation
Looksoft Mobile TransformationLooksoft
 
Looksoft Mobile Transformation
Looksoft Mobile TransformationLooksoft Mobile Transformation
Looksoft Mobile TransformationLooksoft
 
Analytics in Action: How to Build Data-Informed Products
Analytics in Action: How to Build Data-Informed ProductsAnalytics in Action: How to Build Data-Informed Products
Analytics in Action: How to Build Data-Informed ProductsAggregage
 
Analytics in Action: How to Build Data-Informed Products
Analytics in Action: How to Build Data-Informed ProductsAnalytics in Action: How to Build Data-Informed Products
Analytics in Action: How to Build Data-Informed ProductsHannah Flynn
 
Improving traffic quality in a more private world
Improving traffic quality in a more private worldImproving traffic quality in a more private world
Improving traffic quality in a more private worldAmyStamper3
 

Similar to Fueling A_B experiments with behavioral insights (1).pdf (20)

Optimizing Mobile App UX for better engagement & conversions.pdf
Optimizing Mobile App UX for better engagement & conversions.pdfOptimizing Mobile App UX for better engagement & conversions.pdf
Optimizing Mobile App UX for better engagement & conversions.pdf
 
Understand user struggles. Optimize mobile app UX.
Understand user struggles. Optimize mobile app UX.Understand user struggles. Optimize mobile app UX.
Understand user struggles. Optimize mobile app UX.
 
Seven Peaks Speaks: User testing to action
Seven Peaks Speaks: User testing to actionSeven Peaks Speaks: User testing to action
Seven Peaks Speaks: User testing to action
 
12 steps that will lead you to data driven mobile app development
12 steps that will lead you to data driven mobile app development12 steps that will lead you to data driven mobile app development
12 steps that will lead you to data driven mobile app development
 
Ultimate Guide to Moblie App A/B Testing
Ultimate Guide to Moblie App A/B TestingUltimate Guide to Moblie App A/B Testing
Ultimate Guide to Moblie App A/B Testing
 
Mob4Hire What is mobile app and website usability?
Mob4Hire What is mobile app and website usability?Mob4Hire What is mobile app and website usability?
Mob4Hire What is mobile app and website usability?
 
Elizabeth Hom's UX Portfolio
Elizabeth Hom's UX PortfolioElizabeth Hom's UX Portfolio
Elizabeth Hom's UX Portfolio
 
Appurify Performance Automation Whitepaper FINAL
Appurify Performance Automation Whitepaper FINALAppurify Performance Automation Whitepaper FINAL
Appurify Performance Automation Whitepaper FINAL
 
Personalization Strategy Workshop - Los Angeles
Personalization Strategy Workshop - Los AngelesPersonalization Strategy Workshop - Los Angeles
Personalization Strategy Workshop - Los Angeles
 
3 Steps To Improving Your Online Performance
3 Steps To Improving Your Online Performance3 Steps To Improving Your Online Performance
3 Steps To Improving Your Online Performance
 
Are you a lean mobile startup? Applying lean startup principles to mobile app...
Are you a lean mobile startup? Applying lean startup principles to mobile app...Are you a lean mobile startup? Applying lean startup principles to mobile app...
Are you a lean mobile startup? Applying lean startup principles to mobile app...
 
Clover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive ExperimentationClover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive Experimentation
 
A_B Testing For Progressive Web Apps – Implement Effectively.pdf
A_B Testing For Progressive Web Apps – Implement Effectively.pdfA_B Testing For Progressive Web Apps – Implement Effectively.pdf
A_B Testing For Progressive Web Apps – Implement Effectively.pdf
 
Modeling Online Hotel Choice: Conjoint analysis as a multivariate alternative...
Modeling Online Hotel Choice: Conjoint analysis as a multivariate alternative...Modeling Online Hotel Choice: Conjoint analysis as a multivariate alternative...
Modeling Online Hotel Choice: Conjoint analysis as a multivariate alternative...
 
Looksoft Mobile Transformation
Looksoft Mobile TransformationLooksoft Mobile Transformation
Looksoft Mobile Transformation
 
Looksoft Mobile Transformation
Looksoft Mobile TransformationLooksoft Mobile Transformation
Looksoft Mobile Transformation
 
Lean User Testing Intro
Lean User Testing IntroLean User Testing Intro
Lean User Testing Intro
 
Analytics in Action: How to Build Data-Informed Products
Analytics in Action: How to Build Data-Informed ProductsAnalytics in Action: How to Build Data-Informed Products
Analytics in Action: How to Build Data-Informed Products
 
Analytics in Action: How to Build Data-Informed Products
Analytics in Action: How to Build Data-Informed ProductsAnalytics in Action: How to Build Data-Informed Products
Analytics in Action: How to Build Data-Informed Products
 
Improving traffic quality in a more private world
Improving traffic quality in a more private worldImproving traffic quality in a more private world
Improving traffic quality in a more private world
 

More from VWO

The Science of Landing Page Messaging.pdf
The Science of Landing Page Messaging.pdfThe Science of Landing Page Messaging.pdf
The Science of Landing Page Messaging.pdfVWO
 
Uncover Insightful User Journey Secrets Using GA4 Reports
Uncover Insightful User Journey Secrets Using GA4 ReportsUncover Insightful User Journey Secrets Using GA4 Reports
Uncover Insightful User Journey Secrets Using GA4 ReportsVWO
 
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdfResearch and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdfVWO
 
Life between variants by Tomas Voves.pdf
Life between variants by Tomas Voves.pdfLife between variants by Tomas Voves.pdf
Life between variants by Tomas Voves.pdfVWO
 
Message-Market Fit VWO Workshop by Daphne tideman
Message-Market Fit VWO Workshop by Daphne tidemanMessage-Market Fit VWO Workshop by Daphne tideman
Message-Market Fit VWO Workshop by Daphne tidemanVWO
 
A_B Testing Personalized Meditation Recommendations.pdf
A_B Testing Personalized Meditation Recommendations.pdfA_B Testing Personalized Meditation Recommendations.pdf
A_B Testing Personalized Meditation Recommendations.pdfVWO
 
Converting with Comedy: Research Parallels for CRO
Converting with Comedy: Research Parallels for CROConverting with Comedy: Research Parallels for CRO
Converting with Comedy: Research Parallels for CROVWO
 
Unifying feature management with experiments - Server Side Webinar (1).pdf
Unifying feature management with experiments - Server Side Webinar (1).pdfUnifying feature management with experiments - Server Side Webinar (1).pdf
Unifying feature management with experiments - Server Side Webinar (1).pdfVWO
 
Run more experiments with fewer resources
Run more experiments with fewer resourcesRun more experiments with fewer resources
Run more experiments with fewer resourcesVWO
 
Crafting High-Converting eCommerce Landing Pages
Crafting High-Converting eCommerce Landing PagesCrafting High-Converting eCommerce Landing Pages
Crafting High-Converting eCommerce Landing PagesVWO
 
Diseñando según la regulación actual de cookies para lograr más conversión
Diseñando según la regulación actual de cookies para lograr más conversiónDiseñando según la regulación actual de cookies para lograr más conversión
Diseñando según la regulación actual de cookies para lograr más conversiónVWO
 
VWO - Mark de Winter - Run more experiments with fewer resources.pdf
VWO - Mark de Winter - Run more experiments with fewer resources.pdfVWO - Mark de Winter - Run more experiments with fewer resources.pdf
VWO - Mark de Winter - Run more experiments with fewer resources.pdfVWO
 
Turn Website Visits into Conversions (2).pdf
Turn Website Visits into Conversions (2).pdfTurn Website Visits into Conversions (2).pdf
Turn Website Visits into Conversions (2).pdfVWO
 
Principales obstáculos de la experimentación y cómo abordarlos en StartUps...
Principales obstáculos de la experimentación y cómo abordarlos en StartUps...Principales obstáculos de la experimentación y cómo abordarlos en StartUps...
Principales obstáculos de la experimentación y cómo abordarlos en StartUps...VWO
 
Turn Website Visits into Conversions (1).pdf
Turn Website Visits into Conversions (1).pdfTurn Website Visits into Conversions (1).pdf
Turn Website Visits into Conversions (1).pdfVWO
 
Converting with Comedy: Research Parallels for CRO
Converting with Comedy: Research Parallels for CROConverting with Comedy: Research Parallels for CRO
Converting with Comedy: Research Parallels for CROVWO
 
7 puntos clave y prácticos en todo proyecto de CRO
7 puntos clave y prácticos en todo proyecto de CRO7 puntos clave y prácticos en todo proyecto de CRO
7 puntos clave y prácticos en todo proyecto de CROVWO
 
Tackling Conversion Problems with Personalization | VWO Webinars
Tackling Conversion Problems with Personalization | VWO WebinarsTackling Conversion Problems with Personalization | VWO Webinars
Tackling Conversion Problems with Personalization | VWO WebinarsVWO
 
Turning Business Challenges into Testable Ideas - 29 Nov '23.pdf
Turning Business Challenges into Testable Ideas - 29 Nov '23.pdfTurning Business Challenges into Testable Ideas - 29 Nov '23.pdf
Turning Business Challenges into Testable Ideas - 29 Nov '23.pdfVWO
 
First 30 days of Your CRO Program
First 30 days of Your CRO ProgramFirst 30 days of Your CRO Program
First 30 days of Your CRO ProgramVWO
 

More from VWO (20)

The Science of Landing Page Messaging.pdf
The Science of Landing Page Messaging.pdfThe Science of Landing Page Messaging.pdf
The Science of Landing Page Messaging.pdf
 
Uncover Insightful User Journey Secrets Using GA4 Reports
Uncover Insightful User Journey Secrets Using GA4 ReportsUncover Insightful User Journey Secrets Using GA4 Reports
Uncover Insightful User Journey Secrets Using GA4 Reports
 
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdfResearch and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
 
Life between variants by Tomas Voves.pdf
Life between variants by Tomas Voves.pdfLife between variants by Tomas Voves.pdf
Life between variants by Tomas Voves.pdf
 
Message-Market Fit VWO Workshop by Daphne tideman
Message-Market Fit VWO Workshop by Daphne tidemanMessage-Market Fit VWO Workshop by Daphne tideman
Message-Market Fit VWO Workshop by Daphne tideman
 
A_B Testing Personalized Meditation Recommendations.pdf
A_B Testing Personalized Meditation Recommendations.pdfA_B Testing Personalized Meditation Recommendations.pdf
A_B Testing Personalized Meditation Recommendations.pdf
 
Converting with Comedy: Research Parallels for CRO
Converting with Comedy: Research Parallels for CROConverting with Comedy: Research Parallels for CRO
Converting with Comedy: Research Parallels for CRO
 
Unifying feature management with experiments - Server Side Webinar (1).pdf
Unifying feature management with experiments - Server Side Webinar (1).pdfUnifying feature management with experiments - Server Side Webinar (1).pdf
Unifying feature management with experiments - Server Side Webinar (1).pdf
 
Run more experiments with fewer resources
Run more experiments with fewer resourcesRun more experiments with fewer resources
Run more experiments with fewer resources
 
Crafting High-Converting eCommerce Landing Pages
Crafting High-Converting eCommerce Landing PagesCrafting High-Converting eCommerce Landing Pages
Crafting High-Converting eCommerce Landing Pages
 
Diseñando según la regulación actual de cookies para lograr más conversión
Diseñando según la regulación actual de cookies para lograr más conversiónDiseñando según la regulación actual de cookies para lograr más conversión
Diseñando según la regulación actual de cookies para lograr más conversión
 
VWO - Mark de Winter - Run more experiments with fewer resources.pdf
VWO - Mark de Winter - Run more experiments with fewer resources.pdfVWO - Mark de Winter - Run more experiments with fewer resources.pdf
VWO - Mark de Winter - Run more experiments with fewer resources.pdf
 
Turn Website Visits into Conversions (2).pdf
Turn Website Visits into Conversions (2).pdfTurn Website Visits into Conversions (2).pdf
Turn Website Visits into Conversions (2).pdf
 
Principales obstáculos de la experimentación y cómo abordarlos en StartUps...
Principales obstáculos de la experimentación y cómo abordarlos en StartUps...Principales obstáculos de la experimentación y cómo abordarlos en StartUps...
Principales obstáculos de la experimentación y cómo abordarlos en StartUps...
 
Turn Website Visits into Conversions (1).pdf
Turn Website Visits into Conversions (1).pdfTurn Website Visits into Conversions (1).pdf
Turn Website Visits into Conversions (1).pdf
 
Converting with Comedy: Research Parallels for CRO
Converting with Comedy: Research Parallels for CROConverting with Comedy: Research Parallels for CRO
Converting with Comedy: Research Parallels for CRO
 
7 puntos clave y prácticos en todo proyecto de CRO
7 puntos clave y prácticos en todo proyecto de CRO7 puntos clave y prácticos en todo proyecto de CRO
7 puntos clave y prácticos en todo proyecto de CRO
 
Tackling Conversion Problems with Personalization | VWO Webinars
Tackling Conversion Problems with Personalization | VWO WebinarsTackling Conversion Problems with Personalization | VWO Webinars
Tackling Conversion Problems with Personalization | VWO Webinars
 
Turning Business Challenges into Testable Ideas - 29 Nov '23.pdf
Turning Business Challenges into Testable Ideas - 29 Nov '23.pdfTurning Business Challenges into Testable Ideas - 29 Nov '23.pdf
Turning Business Challenges into Testable Ideas - 29 Nov '23.pdf
 
First 30 days of Your CRO Program
First 30 days of Your CRO ProgramFirst 30 days of Your CRO Program
First 30 days of Your CRO Program
 

Recently uploaded

ATRIUM GAMING : SLOT GACOR MUDAH MENANG TERBARU
ATRIUM GAMING : SLOT GACOR MUDAH MENANG TERBARUATRIUM GAMING : SLOT GACOR MUDAH MENANG TERBARU
ATRIUM GAMING : SLOT GACOR MUDAH MENANG TERBARUsayangkamuu240203
 
Back Office Outsourcing Services Company
Back Office Outsourcing Services CompanyBack Office Outsourcing Services Company
Back Office Outsourcing Services Companybrandzus11
 
Marketing Automation Insights - Unlocking Success
Marketing Automation Insights - Unlocking SuccessMarketing Automation Insights - Unlocking Success
Marketing Automation Insights - Unlocking SuccessLean Summits
 
Klinik Jual Obat Aborsi Di Bandung wa 0851/7541/5434 Misoprostol 200mcg Pfize...
Klinik Jual Obat Aborsi Di Bandung wa 0851/7541/5434 Misoprostol 200mcg Pfize...Klinik Jual Obat Aborsi Di Bandung wa 0851/7541/5434 Misoprostol 200mcg Pfize...
Klinik Jual Obat Aborsi Di Bandung wa 0851/7541/5434 Misoprostol 200mcg Pfize...Spesialis Kandungan Resmi BPOM
 
WA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di Sampang
WA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di SampangWA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di Sampang
WA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di Sampanginfoobataborsi24
 
Niche Analysis for Client Outreach Outside Marketplace.pptx
Niche Analysis for Client Outreach Outside Marketplace.pptxNiche Analysis for Client Outreach Outside Marketplace.pptx
Niche Analysis for Client Outreach Outside Marketplace.pptxAhnaf Tahmid Haque
 
Admya Infotech is the Best Digital Marketing Company
Admya Infotech is the Best Digital Marketing CompanyAdmya Infotech is the Best Digital Marketing Company
Admya Infotech is the Best Digital Marketing CompanyAdmya Infotech Pvt. Ltd.
 
Top tips for effective SEO copywriting.pdf
Top tips for effective SEO copywriting.pdfTop tips for effective SEO copywriting.pdf
Top tips for effective SEO copywriting.pdfJuan Pineda
 
WA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di Pasuruan
WA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di PasuruanWA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di Pasuruan
WA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di Pasuruaninfoobataborsi24
 
The Future Normal - DIGGIT - Henry Coutinho-Mason.pdf
The Future Normal - DIGGIT - Henry Coutinho-Mason.pdfThe Future Normal - DIGGIT - Henry Coutinho-Mason.pdf
The Future Normal - DIGGIT - Henry Coutinho-Mason.pdfDIGGIT
 
Fungible Tokens on Bitcoin: Runes Protocol
Fungible Tokens on Bitcoin: Runes ProtocolFungible Tokens on Bitcoin: Runes Protocol
Fungible Tokens on Bitcoin: Runes Protocolnehapardhi711
 
The Vital Role of Keyword Density in Crafting SEO-Optimized Content
The Vital Role of Keyword Density in Crafting SEO-Optimized ContentThe Vital Role of Keyword Density in Crafting SEO-Optimized Content
The Vital Role of Keyword Density in Crafting SEO-Optimized ContentKhalid332898
 
Fantasy Cricket Apps: A New Viewpoint for Online Cricket Betting Apps
Fantasy Cricket Apps: A New Viewpoint for Online Cricket Betting AppsFantasy Cricket Apps: A New Viewpoint for Online Cricket Betting Apps
Fantasy Cricket Apps: A New Viewpoint for Online Cricket Betting AppsCricket Betting Online
 
Digital PR & Content Marketing Lecture for Advanced Digital & Social Media St...
Digital PR & Content Marketing Lecture for Advanced Digital & Social Media St...Digital PR & Content Marketing Lecture for Advanced Digital & Social Media St...
Digital PR & Content Marketing Lecture for Advanced Digital & Social Media St...Valters Lauzums
 
Why Digital Marketing Important for our Business.pdf
Why Digital Marketing Important for our Business.pdfWhy Digital Marketing Important for our Business.pdf
Why Digital Marketing Important for our Business.pdfInfyQ Seo Experts
 
Building Your Customer Base with MailPoet.pdf
Building Your Customer Base with MailPoet.pdfBuilding Your Customer Base with MailPoet.pdf
Building Your Customer Base with MailPoet.pdfChristopher Ross
 
Using GA 4 to to Prove Value - Greg Jarboe - Aug 8, 2023.pptx
Using GA 4 to to Prove Value - Greg Jarboe - Aug 8, 2023.pptxUsing GA 4 to to Prove Value - Greg Jarboe - Aug 8, 2023.pptx
Using GA 4 to to Prove Value - Greg Jarboe - Aug 8, 2023.pptxGreg Jarboe
 
Tea Gobec, Kako pluti po morju tehnoloških sprememb, Innovatif.pdf
Tea Gobec, Kako pluti po morju tehnoloških sprememb, Innovatif.pdfTea Gobec, Kako pluti po morju tehnoloških sprememb, Innovatif.pdf
Tea Gobec, Kako pluti po morju tehnoloških sprememb, Innovatif.pdfDIGGIT
 
Beyond the Basics: Enhanced Strategies for Next-Level Advertising
Beyond the Basics: Enhanced Strategies for Next-Level AdvertisingBeyond the Basics: Enhanced Strategies for Next-Level Advertising
Beyond the Basics: Enhanced Strategies for Next-Level AdvertisingJessieGoodrum1
 

Recently uploaded (20)

ATRIUM GAMING : SLOT GACOR MUDAH MENANG TERBARU
ATRIUM GAMING : SLOT GACOR MUDAH MENANG TERBARUATRIUM GAMING : SLOT GACOR MUDAH MENANG TERBARU
ATRIUM GAMING : SLOT GACOR MUDAH MENANG TERBARU
 
Back Office Outsourcing Services Company
Back Office Outsourcing Services CompanyBack Office Outsourcing Services Company
Back Office Outsourcing Services Company
 
Marketing Automation Insights - Unlocking Success
Marketing Automation Insights - Unlocking SuccessMarketing Automation Insights - Unlocking Success
Marketing Automation Insights - Unlocking Success
 
Klinik Jual Obat Aborsi Di Bandung wa 0851/7541/5434 Misoprostol 200mcg Pfize...
Klinik Jual Obat Aborsi Di Bandung wa 0851/7541/5434 Misoprostol 200mcg Pfize...Klinik Jual Obat Aborsi Di Bandung wa 0851/7541/5434 Misoprostol 200mcg Pfize...
Klinik Jual Obat Aborsi Di Bandung wa 0851/7541/5434 Misoprostol 200mcg Pfize...
 
WA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di Sampang
WA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di SampangWA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di Sampang
WA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di Sampang
 
Niche Analysis for Client Outreach Outside Marketplace.pptx
Niche Analysis for Client Outreach Outside Marketplace.pptxNiche Analysis for Client Outreach Outside Marketplace.pptx
Niche Analysis for Client Outreach Outside Marketplace.pptx
 
Admya Infotech is the Best Digital Marketing Company
Admya Infotech is the Best Digital Marketing CompanyAdmya Infotech is the Best Digital Marketing Company
Admya Infotech is the Best Digital Marketing Company
 
Top tips for effective SEO copywriting.pdf
Top tips for effective SEO copywriting.pdfTop tips for effective SEO copywriting.pdf
Top tips for effective SEO copywriting.pdf
 
WA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di Pasuruan
WA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di PasuruanWA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di Pasuruan
WA | 0821-8888-6412 | Apotik Jual Obat Aborsi Cytotec Asli Di Pasuruan
 
The Future Normal - DIGGIT - Henry Coutinho-Mason.pdf
The Future Normal - DIGGIT - Henry Coutinho-Mason.pdfThe Future Normal - DIGGIT - Henry Coutinho-Mason.pdf
The Future Normal - DIGGIT - Henry Coutinho-Mason.pdf
 
buy best digital marketing course in india
buy best digital marketing course in indiabuy best digital marketing course in india
buy best digital marketing course in india
 
Fungible Tokens on Bitcoin: Runes Protocol
Fungible Tokens on Bitcoin: Runes ProtocolFungible Tokens on Bitcoin: Runes Protocol
Fungible Tokens on Bitcoin: Runes Protocol
 
The Vital Role of Keyword Density in Crafting SEO-Optimized Content
The Vital Role of Keyword Density in Crafting SEO-Optimized ContentThe Vital Role of Keyword Density in Crafting SEO-Optimized Content
The Vital Role of Keyword Density in Crafting SEO-Optimized Content
 
Fantasy Cricket Apps: A New Viewpoint for Online Cricket Betting Apps
Fantasy Cricket Apps: A New Viewpoint for Online Cricket Betting AppsFantasy Cricket Apps: A New Viewpoint for Online Cricket Betting Apps
Fantasy Cricket Apps: A New Viewpoint for Online Cricket Betting Apps
 
Digital PR & Content Marketing Lecture for Advanced Digital & Social Media St...
Digital PR & Content Marketing Lecture for Advanced Digital & Social Media St...Digital PR & Content Marketing Lecture for Advanced Digital & Social Media St...
Digital PR & Content Marketing Lecture for Advanced Digital & Social Media St...
 
Why Digital Marketing Important for our Business.pdf
Why Digital Marketing Important for our Business.pdfWhy Digital Marketing Important for our Business.pdf
Why Digital Marketing Important for our Business.pdf
 
Building Your Customer Base with MailPoet.pdf
Building Your Customer Base with MailPoet.pdfBuilding Your Customer Base with MailPoet.pdf
Building Your Customer Base with MailPoet.pdf
 
Using GA 4 to to Prove Value - Greg Jarboe - Aug 8, 2023.pptx
Using GA 4 to to Prove Value - Greg Jarboe - Aug 8, 2023.pptxUsing GA 4 to to Prove Value - Greg Jarboe - Aug 8, 2023.pptx
Using GA 4 to to Prove Value - Greg Jarboe - Aug 8, 2023.pptx
 
Tea Gobec, Kako pluti po morju tehnoloških sprememb, Innovatif.pdf
Tea Gobec, Kako pluti po morju tehnoloških sprememb, Innovatif.pdfTea Gobec, Kako pluti po morju tehnoloških sprememb, Innovatif.pdf
Tea Gobec, Kako pluti po morju tehnoloških sprememb, Innovatif.pdf
 
Beyond the Basics: Enhanced Strategies for Next-Level Advertising
Beyond the Basics: Enhanced Strategies for Next-Level AdvertisingBeyond the Basics: Enhanced Strategies for Next-Level Advertising
Beyond the Basics: Enhanced Strategies for Next-Level Advertising
 

Fueling A_B experiments with behavioral insights (1).pdf

  • 1. 1 A/B Testing with Behavioral Insights Building hypothesis and analysing experiments with behavioral insights A Product Webinar by VWO
  • 2. 2 What I do day in, day out? “Look at session recordings & heatmaps of mobile apps of our customers and suggest them UX improvement ideas” Little about me 12+ years of managing and marketing websites and apps, for B2B & B2C. Piyush Sharma Product Marketing, VWO
  • 3. 3 Agenda 1 2 3 4 5 6 Mobile app experiment lifecycle Collect data Analyse test results Popular use cases Tour of session replays, heatmaps & funnels Q & A
  • 4. 4 Mobile app experiment lifecycle Define Objective Activation, Engagement, Retention, Conversions Develop Hypothesis Building hypothesis for multiple variations Analyse Test Results Check movement in metrics. “And why?” Collect Data Quantitative Analysis, Qualitative Analysis Run Experiment Running A/B tests, multivariate or split tests Deploy or Record Learnings 1 2 3 5 4 6 Repeat
  • 6. 6 Collect Data Quantitative Data Qualitative Data What Users Say VWO Insights - Mobile App Session recordings Heatmaps Surveys User Interviews Reviews Emails What Users Do
  • 7. 7 Collect Data - Quantitative Data Limitations Not easy to find the why behind the numbers. 60% Completed app onboarding 78% Dropping off from the checkout screen 9% Users use the new feature
  • 8. 8 Search what users are saying around the issue. Only 1 in 25 unhappy customers complain directly to you. Source: Esteban Kolsky Research App store reviews Support emails 8 Collect Data - Qualitative Data Moreover, lot of to and fro with the user to understand issue.
  • 9. 9 Deal with conscious (biased) responses. App is used with subconscious instincts Gaps between what they say and what they do Difficult to deduce exact actionables 9 “Onboarding form is long” “Price is too high” “Got confused between options” Collect Data - Qualitative Data Surveys User interviews
  • 10. 10 10 Collect Data - Qualitative Data Session recordings Heatmaps Understand why something might be happening See exactly what the users see Get team’s confidence into your next A/B test with a glaring insight you observed
  • 11. 11 11 Quantitative Data Collect Data Example Why is this user cohort exiting the app? Insight The navigation of the app is not clearly visible on iOS so users don’t know where to go. Qualitative Data Example Unusual number of exits from home screen. Insight Majority of iOS users using the latest app version are exiting the home screen. Power-packed Hypothesis Example Test with a new version of navigation which is much more visible.
  • 14. 14 Analysing Test Results Quantitative Data Qualitative Data (Session Recordings & Heatmaps) Statistical significance and winning variant Compare across multiple KPIs How different segments performed? External and internal factors? Why there was an improvement? How was the behavior different in both the variations? Insights that quantitative data might miss
  • 16. 16 Quantitative Insight - Low use of the search function within the app. Behavioral Insight - “Users try the search feature but quickly give up. They seem to prefer browsing through categories.” A/B Test Hypothesis - Improve search functionality with auto-suggestions and filters to make it more user-friendly. Analyzing Results - Monitor if there's an increase in the use of the search function without users getting confused about what to search. Use cases 16 Feature adoption
  • 17. 17 Quantitative Insight - Only 20% women completed app onboarding. Behavioral Insight - “Users want to skip the DOB option but they can’t, hence dropping off” A/B test Hypothesis - Make the DOB optional in a new version. Analysing Results - More women completing the onboarding. Verify if the users are indeed skipping the DOB and going past next screens. Use cases 17 Onboarding completion
  • 18. 18 Use cases 18 Quantitative Insight - Product screen showing unusual drop-offs Behavioral Insight - “Users tapping on the image to magnify it but failing. Rage taps on the image.” A/B test Hypothesis - Provide option to see a magnified view with multiple images, so that user can see product in detail Analyse Results - Check if enough users are actually tapping and checking out the product the way you expected. Improved conversions
  • 19. 19 Use cases More abandons in new app version How users compare between new app versions? New users not sticking - How new vs returning users compare? UK users churning - How users from different geographies behave? Check it out → 19
  • 20. 20 More events? Add to cart? Checkout? Upgrade plan? 20 Use Cases
  • 21. 21 Use Cases 21 Which are important areas? Which areas are ignored? Test UX elements CTA placements Eliminate unnecessary elements Identify new clickable areas
  • 22. 22 22 Let’s see Session Recordings & Heatmaps in action
  • 23. 23 23 Open for any questions.
  • 24. 24 Want to understand how it could help your app and explore features in detail? Try VWO free 24