Analytics for Product Managers


Published on

video here:

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • * the role analytics plays in product management
    * how analytics can guide PMs
    * data points you may need as a PM
    * tools and techniques for getting that data
  • * my experience with the PM role at
    * OI elevator pitch
    * anecdote about how widely the role can vary
    * in all but one of these bullet points I’ve had the title product manager
    * 1st pm: organization, prioritization of competing projects, and process for product planning and development
    * one-woman product team
    * operational role: creating traffic and revenue projections and directing strategy for content acquisition
    * one of two specialized PMs reporting to new VP of Product
    * focus on content analysis, data integrity, search rankings, and our unreleased API
  • * probably goes without saying that there are a variety of viewpoints
    * no PM is at either extreme
    * since this talk is so data-focused want to point out that there are a variety of degrees to which data will play a role in different projects
    * data is a sounding board
    * in a situation where you feel like you don’t have data --> use your creativity to find some
    * use your instincts and understanding of your product to guide your interpretation of any data
  • * most important place for analytics in product management is in prioritization
    * PMs make tough decisions about how to spend our time and developers’ time
    * these debates are always going to be rife with politics and emotions
    * numbers or even the promise of numbers to the table can help you be a voice of clarity and direction
    * e.g.: OI built a simple A/B testing platform prior to our redesign and the promise of data-driven decisions helped to diffuse tension over how aggressively to change the site

    * tracking succes
    * e.g.: OI tracked change from text link to graphic button-->clicks went up--> clicks went down--> sometimes change makes people click

    * numbers don't prevent subjectivity, guarantee success, or help you win every argument
  • * hellish experience trying to reconcile Google Analytics and Quantcast --> choose a tool to be authoritative
    * prioritize --> make a decision based on the key value propositions of your product
    * if you can't control your stats, you may not be optimizing for the right metric for your product or the age of your product
    * e.g. tracking PVs/visit on OI is frustrating, because it currently has a very simple value proposition: to get users to content they care about. instead we track clicks to third-party content per visit, which is more central to our value proposition
  • * FW: (biz metrics, not just analytics) meetup’s “successful meetups” and etsy’s customer service tracking
    * DM: acquisition, activation, retention, referral, revenue
    * +1 to FW’s notion that metrics have to change and be specific to biz/product
    * DM calls out really specific terms that clarify the mystery behind some of the more general terms
    * I’m going to cite four buckets of analytics (in general terms) and clarify the spirit behind them
  • * how many people are interacting with your product
    * consumer website: unique visitors
    * desktop/mobile app or subscription service: downloads/signups
    * publishing tools: like Twitter or Tumblr or for Publishers
    * pub tools: DLs/signups + uniques on your users’ content
  • * how much people are interacting with your product
    * promise behind it is like that of the engagement ring: how much people love your product and how loyal they’ll be
    * conversion should measure what you hope you’re giving users
    * choose an engagement metric that exposes the main value proposition that your product makes for users
    * e.g. pageviews/visit versus clicks per visit for OI
    * typical: PVs/visit, time on site, bounce rate, return visitors, logged-in sessions
    * for pub tools: on your users content
  • * conversion should measure what you hope your users are giving you
    * generally: how frequently people do a specific thing that you want
    * account registration
    * upgrade
    * providing info
    * OI e.g.: clicks on ads as conversion
  • * revenue is a sad ghost sad ghost bcs many early-stage companies defer tracking revenue
    * other analytics should drive revenue
    * some companies have a mandate from investors to focus on building a product they love, but it's important to keep the spectre of revenue and how you'll track it in mind while you build your product
    * ads can be paid on CPM (flat rate per thousand impressions), cost per click, or cost per action
    * subscription or license feeds
    * one-time purchases
    * BizDev deals like foursquare’s deals with bravo and the history channel

  • * a few practical tips to help you tracking and understanding these types of numbers
    * most of these can be boiled down to: think like a hacker about where you can cram variables into your tracking tool and add as many as possible

  • * Google Analytics is limiting and often inaccuarate but startups can’t afford to invest in Omniture
    * suck it up and hack GA
    ==campaign trackers==
    * campaign trackers page is written for people tracking the success of ad campaigns
    * just a bunch of query strings that you can define any value for (utm_content, utm_campaign, utm_medium
    * in twitter’s emails
    * track clicks in the “campaigns” view under “traffic sources”
    ==detailed paths and title tags==
    * analytics “content” view lets you filter types of pages by info in the URL paths or title tags
    * make sure they have as much info as possible, but optimize your identifying strings for search queries because paths and titles are important to SEO
    * set up goals for engagement or conversion tracking
    * multi-step conversions create funnel visualizations
    * if you want to use funnels, make sure each step of your conversion process has a unique URL (variables ok, just use head match or regular expression)
    * see the results in the funnels section under goals
    * shows where users are coming from to enter your conversion process, how many proceed, and where they drop off
    * identify steps in conversion that need usability testing or distracting links along the way
  • * see the results under “goals :: funnel visualizations”
    * diagram that shows what pages users are coming from to reach the first step in your conversion process
    * for each step, how many proceed to the next step and how many drop off
    * shows how many drop offs go where
    * identify pain points for usability testing
    * identify distracting links

  • * a/b testing lets you fear commitment to new features without stifling progress and iteration
    * Google’s Website Optimizer can set up simple a/b tests for static landing pages
    * OI built a simple a/b testing tool that sets a number between 0-1 in the session and compares it to the % of visitors we want to show each template
    * make sure you can get every KPI for your product in both templates
    * a change you make for engagement could negatively affect monetization
    * have a token representing the template you’re testing in your title tag for analytics, in your ad tags for revenue monetization, in your campaign URL trackers for conversion or engagement
  • * pay attention to public reach-trackers because other people are* if the numbers you say in public don't match the numbers on Compete and Quantcast, you'll be laughed at or distrusted
    * sign up for a Quantcast account --> greater accuracy + privacy settings
    * labels let you add a string you define 3x in your tracking code
    * labelled traffic shows up in “audience segments” in your Quantcast profile (can be made private)
    * use labels to segment your traffic however makes sense
    * OI uses these to measure which users of our publisher dashboard users are generating the most reach and traffic
  • * have more than one window into your numbers so you can make sure they are generally aligned
    * don’t waste your time trying to reconcile them perfectly, but choose a package to be authoritative. this should be the numbers that pay you.
    * be on the lookout for dramatic differences as a heads-up that something is awry
    * OI found that Analytics was reporting 200% fewer visitors and pageviews than Quantcast for the widgets distributed by users of our self-service publishing product
    * ad server, Google Ad Manager (which doesn’t track reach) roughly agreed with Quantcast on the pageviews
    * no discrepancy for our main site
    * hack for tracking JS widgets on other domains --> wrote in iframe that called a source page on our domain w/tracking code for GA + Quantcast and ad tags for Ad Manager
    * Google Analytics could not track a large portion of visits made in IE via this hack but Quantcast and AdManager code didn't have that discrepancy
    * AdManager numbers were luckily the highest of the bunch
  • * engagement metrics should play to the key value proposition you want to give users
    * conversion metrics should play to the key value you want to GET from users
    * learn how your analytics package works and cram as many details into it as possible
  • * email
    * twitter
    * neglected blog
  • Analytics for Product Managers

    1. 1. Analytics for Product Managers What numbers should I pay attention to and how?
    2. 2. about me Being a Product Manager at has meant: • first PM hired • one-person product team • business manager • one of two specialized PMs
    3. 3. extreme views let numbers make every decision only listen to your gut and creative vision
    4. 4. you <3 numbers Analytics help: • guide debates • evaluate a new opportunity • track performance
    5. 5. you <3 numbers Analytics hurt: • different numbers in different tools • not sure which stats to prioritize • numbers you canʼt control
    6. 6. KPI theories • Fred Wilson: 4-6 metrics that change as the business grows • Dave McClure: Startup Metrics for Pirates (AARRR!)
    7. 7. reach
    8. 8. engagement
    9. 9. conversion
    10. 10. revenue
    11. 11. tips & details
    12. 12. make the most of Google Analytics • campaign trackers: • detailed URL paths + title tags: brooklyn-ny/tags/crime <title>Boston Bars and Clubs | local places near Boston, MA | alt4</title> • goals --> funnels
    13. 13. funnels and drop users come off here from here
    14. 14. A/B test • fear commitment • donʼt fear change • track all KPIs
    15. 15. own your external numbers • keep an eye on Compete • get Quantified + use labels <script type="text/ javascript">_qoptions={qacct:"YourAccountNumber", labels: "SegmentName"};</script> <script type="text/javascript" src="http://"></script> <noscript><a href=" YourAccountNumber"; target="_blank"><img src=" YourAccountNumber.gif?labels=Segment_Name"; style="display: none;" border="0" height="1" width="1" alt="Segment_Name"/></a></noscript>
    16. 16. donʼt reconcile • sanity check metrics across tools • your numbers wonʼt reconcile perfectly • trust the numbers that pay you
    17. 17. key points • engagement metrics that track the core value of your product • cram details into your analytics package
    18. 18. talk to me @kenspeckle