Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
NCR Confidential
MILLION DOLLAR
IMPACT THROUGH
METRICS, ANALYTICS,
AND A/B TESTING
AZHAR BANDE-ALI
SEPTEMBER 17, 2016
NCR Confidential
What will we talk about?
• Who?
• What is NCR?
• The Product
• How we use Google Analytics
• Opportunity
...
NCR Confidential
What will we talk about?
AZHAR
PM, NCR
NCR Confidential
Azhar Bande-Ali
NCR Confidential
NCR Confidential
NCR POWERS
WORLDWIDE,
EVERY SECOND
DOLLARS IN CASH
GOES THROUGH NCR
ATMs EACH YEAR
NCR SERVICES ARE HERE
...
NCR Confidential
Product
NCR Confidential
NCR Confidential
Product
NCR Confidential
Scorecard Metrics
Sessions that end on
page/Unique page visits
Monitor each stage in
ordering funnel
Conv...
NCR Confidential
NCR Confidential
Google Analytics
• Ecommerce
• Funnel Visualization
• Events
• Custom events not tracked by page views
• ...
NCR Confidential
Google Analytics
Ecommerce & Goals Events
NCR Confidential
Opportunity
NCR Confidential
Ordering Funnel – Conversion Rate
Select Location: 83%
Select Pick Up Time: 85%
Build Bag: 56%
NCR Confidential
Events
• Track every action on menu
page that could result in
abandonment
• Events can be a part of a
Cat...
NCR Confidential
AIDA Framework
Exits
%
Interest: are users exploring? Submenu Events 5.26%
Desire: are they interested in...
NCR Confidential
ROI
ROI
Improve by # transactions $ $ yearly
Interest: are
users exploring?
Submenu Events 176.1 110 $3,3...
NCR Confidential
$1,085,136.00
How to get buy in from your boss and customers
NCR Confidential
A/B Test
NCR Confidential
Google Tag Manager
• Inject “tags” to websites
without needing a
deployment
• Add backdoor to your
website
NCR Confidential
Google Analytics Experiments
• Create experiment
• Define variants
• Use experiment ID to inject
code via...
NCR Confidential
Current State
Hypothesis: Making “Continue”
button more prominent will improve
the checkout conversion ra...
NCR Confidential
Variant 1
Remove “Cancel” button
NCR Confidential
Variant 2
Remove “Cancel” and change
button text from “Continue” to
“Checkout”
NCR Confidential
Results
bit.ly/pcncr
NCR Confidential
Questions?
NCR Confidential
Upcoming SlideShare
Loading in …5
×

Million Dollar Impact Through Metrics, Analytics, and A/B Testing

291 views

Published on

This talk was given by Azhar Bande-Ali at ProductCamp ATL 2016 on September 17, 2016. It explains the methodology and frameworks used by NCR to discover opportunities for improvement and tools used to conduct A/B testing to prototype solutions.

https://www.linkedin.com/in/azharb

  • Be the first to comment

  • Be the first to like this

Million Dollar Impact Through Metrics, Analytics, and A/B Testing

  1. 1. NCR Confidential MILLION DOLLAR IMPACT THROUGH METRICS, ANALYTICS, AND A/B TESTING AZHAR BANDE-ALI SEPTEMBER 17, 2016
  2. 2. NCR Confidential What will we talk about? • Who? • What is NCR? • The Product • How we use Google Analytics • Opportunity • Discovery • Buy-in • A/B test Questions
  3. 3. NCR Confidential What will we talk about? AZHAR PM, NCR
  4. 4. NCR Confidential Azhar Bande-Ali
  5. 5. NCR Confidential
  6. 6. NCR Confidential NCR POWERS WORLDWIDE, EVERY SECOND DOLLARS IN CASH GOES THROUGH NCR ATMs EACH YEAR NCR SERVICES ARE HERE TO SUPPORT OUR CUSTOMERS SPEAKING AND SPANNING FROM: THE WEST COAST OF THE US… …TO THE FAR EAST OF RUSSIA CONSUMERS USING FASTLANE SELFSERV CHECKOUTS SCAN EVERY YEAR – OVER 6 ITEMS FOR EVERY MAN, WOMAN AND CHILD ON EARTH. ON AVERAGE, AN AUTOMOBILE HAS NCR ACTIVELY MANAGES IN 2014 VISITED OUR EXECUTIVE BRIEFING CENTERS IN DULUTH, DUNDEE AND NYC TO LEARN HOW NCR IS SOLUTIONS NCR CONNECTED PAYMENTS CURRENTLY PROTECTS MORE THAN AND MORE THAN ANNUALLY (SPECIALIZES IN CLOUD, MOBILE eWALLET PAYMENTS) NCR SERVICES CENTERS RESPOND TO CUSTOMER INCIDENTS EACH YEAR ATM’S + APTRA SOFTWARE SERVE OVER MORE THAN TWITTER AND IPHONE COMBINED NCR FACILITATES EVERY SECOND
  7. 7. NCR Confidential Product
  8. 8. NCR Confidential
  9. 9. NCR Confidential Product
  10. 10. NCR Confidential Scorecard Metrics Sessions that end on page/Unique page visits Monitor each stage in ordering funnel Conversion Rate Drop Off Rate Return Rate Food industry standard: 10% Highlights areas of friction and confusion for users Most annoying metric of all Sessions with a transaction/Total Sessions Monitor weekly change Cohort: new transacting users between 5 and 4 weeks ago Repeat visits from cohort in last 4 weeks/ Cohort size
  11. 11. NCR Confidential
  12. 12. NCR Confidential Google Analytics • Ecommerce • Funnel Visualization • Events • Custom events not tracked by page views • Metrics • Conversion Rate • Drop-off Rate
  13. 13. NCR Confidential Google Analytics Ecommerce & Goals Events
  14. 14. NCR Confidential Opportunity
  15. 15. NCR Confidential Ordering Funnel – Conversion Rate Select Location: 83% Select Pick Up Time: 85% Build Bag: 56%
  16. 16. NCR Confidential Events • Track every action on menu page that could result in abandonment • Events can be a part of a Category, have Labels, and an Event Value • Careful! Can lead to analysis paralysis
  17. 17. NCR Confidential AIDA Framework Exits % Interest: are users exploring? Submenu Events 5.26% Desire: are they interested in shopping? Add Item Events 18.22% Revenue: are users checking out? Checkout Events 23.84%
  18. 18. NCR Confidential ROI ROI Improve by # transactions $ $ yearly Interest: are users exploring? Submenu Events 176.1 110 $3,303.73 $171,793.86 Desire: are they interested in shopping? Add Item Events 720.99 557 $16,713.02 $869,077.24 Revenue: are users checking out? Checkout Events 695.6 696 $20,868.00 $1,085,136.00
  19. 19. NCR Confidential $1,085,136.00 How to get buy in from your boss and customers
  20. 20. NCR Confidential A/B Test
  21. 21. NCR Confidential Google Tag Manager • Inject “tags” to websites without needing a deployment • Add backdoor to your website
  22. 22. NCR Confidential Google Analytics Experiments • Create experiment • Define variants • Use experiment ID to inject code via GTM
  23. 23. NCR Confidential Current State Hypothesis: Making “Continue” button more prominent will improve the checkout conversion rate.
  24. 24. NCR Confidential Variant 1 Remove “Cancel” button
  25. 25. NCR Confidential Variant 2 Remove “Cancel” and change button text from “Continue” to “Checkout”
  26. 26. NCR Confidential Results bit.ly/pcncr
  27. 27. NCR Confidential Questions?
  28. 28. NCR Confidential

×