Measurements and Metrics




MOBILE ANALYTICS
VALIDATED LEARNING LOOP

                                • Analytics and Metrics Process =
                                  Build + Measure + Learn
                                • Best summarized by Eric Ries’s
                                  Validated Learning Loop in Lean
                                  Startup methodology




                                        Not just code. Also alignment of
                                        measurement and business
                                        strategy



              REFERENCE: Lean Startup by Eric Ries
OUTCOMES

• Analytics don’t end with
  measurement
• Must translate data into desirable
  outcomes




 Test, measure, and take action




                           REFERENCE: Lean Startup by Eric Ries
BUSINESS REQUIREMENTS
• FUNDAMENTAL QUESTION: WHAT DEFINES
  SUCCESS?
• All analysis start with a question
• Understand what metrics and data are needed to make
  better decisions and perform better
• Analyze mobile app architecture
   – Any constraints that may inhibit measurements
   – How to leverage technology
PRODUCT USAGE METRICS


                               AARRR!
• What do you need to do to
  build your product and
  learn about your users?
• Dave McClure’s AARRR
  model provides 5 useful
  metrics to learning about
  your product and how it is
  used



          Reference: dave McClure, 500 startups. Picture courtesy of walt Disney pictures
THE AARRR MODEL




            Reference: dave McClure, 500 startups
FUNNEL ANALYSIS
• A funnel of steps that a user go through before meeting a
  goal, for example
   –   Steps leading to contacting the company
   –   Steps leading to purchasing the product
   –   Steps leading to purchasing in-app modules/features
   –   Steps leading to purchasing merchandise or tokens (for games(
• Funnel analysis = understanding conversions
• A step in a funnel = a page view (web) = a screen or action
  (mobile app)
WEB VS MOBILE

                 WEB                                 MOBILE APP

 Session tracking done primarily thru       Session tracking done primarily thru
        cookies and Javascript                             UDID

Human user interface is keyboard and        Human user interface is gestural and
          mouse based                                 touch-based
Web measurement model is centered
                                             Measurement model is less about
around page views, referrals, search,
                                                  referrals and search
            and visits
                                                Unique visitors are measured
Unique visitors are tied to individual or
                                            differently because of gateway IPs of
          server IP addresses
                                                           carriers
SOLUTIONS
• Flurry – http://flurry.com/
• Localytics – http://localytics.com/
• Webtrends –
  http://webtrends.com/products/analytics/mobile/
• AppClix – http://www.appclix.com/
• Kontagent – http://kontagent.com/
• Bango – http://bango.com/
• Apsalar – http://apsalar.com/
• Claritics – http://claritics.com/
• Others that we may have missed…
MOBILE METRICS CATEGORIES

•   Application
•   Content
•   User Behavior
•   People/Location
•   Technical
COMMON MOBILE METRICS 1

• Applications                • Content
  – #Downloads                  –   Screens
  – Conversions                 –   Visits, unique visits
    (Monetization)              –   In-app
  – Engagement/loyalty          –   Ads
    (over time)                 –   Links
  – User acquisition            –   Other events
  – User retention
  – Cohort analysis
    (retention, engagement,
     monetization)
COMMON MOBILE METRICS 2

• User Behavior                 • People/Location
  – Screen flow (useful for       –   Users
    navigation and usability)     –   Social identity
  – Exits (how users are          –   Countries/Regions
    exiting an app)               –   Languages
  – Sessions                      –   Marketplaces
    (length, frequency, type
    of users)                     –   Carriers
                                  –   Age
COMMON MOBILE METRICS 3

• Technical
  – Errors
  – Devices
  – Operating systems
  – App Versions
  – Connections
REPORT TERMINOLOGY

 • Common report terminology
 • Example: Breakdown of OS versions used to run the app
                                              FILTER: Time Duration
DIMENSION: OS Version




                                                         METRIC: Session


                          Reference: Localytics report
STRATEGIES
• Define a few funnels to understand how user
  usage drives towards a goal like
  registration, purchase
• If possible apply some of the advanced tracking
  and reporting features in the analytics tool to
  provide deeper insights
   – Filters (at the log and report levels)
   – Funnel analysis
   – Profiling
• Keep refining how metrics are tracked
EVENT TAGGING

• Data collection
• Mapping: Actions » Events » Metrics
• Need to define events to tag so that we can
  measure the metrics
• Example:
  – Start Time = Time when a player starts playing a
    game
  – End Time = Time when a player ends a game
  – Defines the names for both events
Questions?




                        Picture: Sean Dreilinger -
http://www.flickr.com/photos/seandreilinger/2326448445/in/photostream/

Mobile Analytics

  • 1.
  • 2.
    VALIDATED LEARNING LOOP • Analytics and Metrics Process = Build + Measure + Learn • Best summarized by Eric Ries’s Validated Learning Loop in Lean Startup methodology Not just code. Also alignment of measurement and business strategy REFERENCE: Lean Startup by Eric Ries
  • 3.
    OUTCOMES • Analytics don’tend with measurement • Must translate data into desirable outcomes Test, measure, and take action REFERENCE: Lean Startup by Eric Ries
  • 4.
    BUSINESS REQUIREMENTS • FUNDAMENTALQUESTION: WHAT DEFINES SUCCESS? • All analysis start with a question • Understand what metrics and data are needed to make better decisions and perform better • Analyze mobile app architecture – Any constraints that may inhibit measurements – How to leverage technology
  • 5.
    PRODUCT USAGE METRICS AARRR! • What do you need to do to build your product and learn about your users? • Dave McClure’s AARRR model provides 5 useful metrics to learning about your product and how it is used Reference: dave McClure, 500 startups. Picture courtesy of walt Disney pictures
  • 6.
    THE AARRR MODEL Reference: dave McClure, 500 startups
  • 7.
    FUNNEL ANALYSIS • Afunnel of steps that a user go through before meeting a goal, for example – Steps leading to contacting the company – Steps leading to purchasing the product – Steps leading to purchasing in-app modules/features – Steps leading to purchasing merchandise or tokens (for games( • Funnel analysis = understanding conversions • A step in a funnel = a page view (web) = a screen or action (mobile app)
  • 8.
    WEB VS MOBILE WEB MOBILE APP Session tracking done primarily thru Session tracking done primarily thru cookies and Javascript UDID Human user interface is keyboard and Human user interface is gestural and mouse based touch-based Web measurement model is centered Measurement model is less about around page views, referrals, search, referrals and search and visits Unique visitors are measured Unique visitors are tied to individual or differently because of gateway IPs of server IP addresses carriers
  • 9.
    SOLUTIONS • Flurry –http://flurry.com/ • Localytics – http://localytics.com/ • Webtrends – http://webtrends.com/products/analytics/mobile/ • AppClix – http://www.appclix.com/ • Kontagent – http://kontagent.com/ • Bango – http://bango.com/ • Apsalar – http://apsalar.com/ • Claritics – http://claritics.com/ • Others that we may have missed…
  • 10.
    MOBILE METRICS CATEGORIES • Application • Content • User Behavior • People/Location • Technical
  • 11.
    COMMON MOBILE METRICS1 • Applications • Content – #Downloads – Screens – Conversions – Visits, unique visits (Monetization) – In-app – Engagement/loyalty – Ads (over time) – Links – User acquisition – Other events – User retention – Cohort analysis (retention, engagement, monetization)
  • 12.
    COMMON MOBILE METRICS2 • User Behavior • People/Location – Screen flow (useful for – Users navigation and usability) – Social identity – Exits (how users are – Countries/Regions exiting an app) – Languages – Sessions – Marketplaces (length, frequency, type of users) – Carriers – Age
  • 13.
    COMMON MOBILE METRICS3 • Technical – Errors – Devices – Operating systems – App Versions – Connections
  • 14.
    REPORT TERMINOLOGY •Common report terminology • Example: Breakdown of OS versions used to run the app FILTER: Time Duration DIMENSION: OS Version METRIC: Session Reference: Localytics report
  • 15.
    STRATEGIES • Define afew funnels to understand how user usage drives towards a goal like registration, purchase • If possible apply some of the advanced tracking and reporting features in the analytics tool to provide deeper insights – Filters (at the log and report levels) – Funnel analysis – Profiling • Keep refining how metrics are tracked
  • 16.
    EVENT TAGGING • Datacollection • Mapping: Actions » Events » Metrics • Need to define events to tag so that we can measure the metrics • Example: – Start Time = Time when a player starts playing a game – End Time = Time when a player ends a game – Defines the names for both events
  • 17.
    Questions? Picture: Sean Dreilinger - http://www.flickr.com/photos/seandreilinger/2326448445/in/photostream/