SlideShare a Scribd company logo
1 of 38
Download to read offline
Using data to guide product development
Mat Clayton
Co-Founder
What is Mixcloud?
The Plan
The Plan
What does failure look like?
turntable.fm - MAU
Text
turntable.fm - DAU
Competitor analysis
• Sitemaps are an amazing resource
• Google - “site:competitor.com“
• OpenSiteExplorer, can be used to find links and any growth
tactics being used
• graph.facebook.com/<app_id> can give you monthly Facebook
actives for FB connected apps
• API’s
Key Performance Indicators
(KPI’s)
Pick a single KPI
• Easily understood
• Provides focus and align the team
• Represent the core of the product, and usually isn’t uniques
Mixcloud - Listener minutes / 28 days
AirBnb - Nights stayed / 28 days
Eventbrite - Tickets Booked / 28 days
SendGrid - Mail Sent / 28 days
Dashboards
• Distribute information amongst the team
• Make trends observable
• Give the team a solid understanding in
top level metrics
• We’re big fans of graphite and grafana
• Alert on changes
User Accounting
User accounting
Site A Site B
Active Users 1000 1000
Born +800 +100
Die -100 0
Sleep -300 -50
Awaken +100 +450
Next Period’s Active Users 1500 1500
Retention Curves
Identifying key
product features for retention
The Brainstorm
What correlates with retention?
First week Listens
Key User Metrics - Twitter
Twitter’s data
Correlation does not imply Causation
Cause or effect?
How do you choose
what to build?
What facts do we know about our users?
• What information did they give us? name, email, Facebook?
• Do they use mobile web/desktop/iOS/Android?
• Do they have email notifications enabled?
• Which core features have they used and to what degree?
• Which features did they miss?
• What was their last interaction with the service?
• Where did they come from? (Referrer)
• Do they use competitor products?
Evaluating potential product features
• Do we have the resources to build it?
• How many users will the feature be relevant to?
• How frequently will they interact with it?
• What is the conversion rate against our primary goal?
AB Testing
(Always Be Testing)
AB Testing Examples
Big changes == Big results
How big is big?
AB testing
• Continually run AATests
• Requirements
- Fast to implement, a single line of code
- Negligible cost to product performance, including SEO
- Results need to be available to the entire team
• Tools
- Django Experiments (we wrote it)
- A/Bingo (Ruby on Rails)
- Optimizely
- Google Website Optimizer
Failure is acceptable
“A good plan, violently executed now, is better than
a perfect plan next week.” - George S. Patton
Thanks for listening!
Any Questions?
@matclayton
mat@mixcloud.com

More Related Content

Viewers also liked

MozCon Seattle 2011 - Social Design
MozCon Seattle 2011 - Social DesignMozCon Seattle 2011 - Social Design
MozCon Seattle 2011 - Social Design
Mat Clayton
 

Viewers also liked (20)

Social Design - ProSEO
Social Design - ProSEOSocial Design - ProSEO
Social Design - ProSEO
 
Nodejs - A quick tour (v5)
Nodejs - A quick tour (v5)Nodejs - A quick tour (v5)
Nodejs - A quick tour (v5)
 
Cloud Computing
Cloud ComputingCloud Computing
Cloud Computing
 
MozCon Seattle 2011 - Social Design
MozCon Seattle 2011 - Social DesignMozCon Seattle 2011 - Social Design
MozCon Seattle 2011 - Social Design
 
Nodejs - A quick tour (v4)
Nodejs - A quick tour (v4)Nodejs - A quick tour (v4)
Nodejs - A quick tour (v4)
 
Marketing with Linked Data (MIT)
Marketing with Linked Data (MIT)Marketing with Linked Data (MIT)
Marketing with Linked Data (MIT)
 
Using Lean Principles to Manage the Data Value Chain
Using Lean Principles to Manage the Data Value ChainUsing Lean Principles to Manage the Data Value Chain
Using Lean Principles to Manage the Data Value Chain
 
Agile Data Strategy and Lean Execution
Agile Data Strategy and Lean ExecutionAgile Data Strategy and Lean Execution
Agile Data Strategy and Lean Execution
 
Data Driven Product Development
Data Driven Product Development Data Driven Product Development
Data Driven Product Development
 
BIG DATA AS A STRATEGY : CUSTOMER EXPERIENCE AND ANALYTICS
BIG DATA AS A STRATEGY : CUSTOMER EXPERIENCE AND ANALYTICSBIG DATA AS A STRATEGY : CUSTOMER EXPERIENCE AND ANALYTICS
BIG DATA AS A STRATEGY : CUSTOMER EXPERIENCE AND ANALYTICS
 
Data-Driven Product Development
Data-Driven Product DevelopmentData-Driven Product Development
Data-Driven Product Development
 
Data: Strategy and Storytelling
Data: Strategy and StorytellingData: Strategy and Storytelling
Data: Strategy and Storytelling
 
LOD for Entrepreneurs 20111115
LOD for Entrepreneurs 20111115LOD for Entrepreneurs 20111115
LOD for Entrepreneurs 20111115
 
Big Data: Strategy
Big Data: StrategyBig Data: Strategy
Big Data: Strategy
 
Agile and data driven product development oleh Dhiku VP Product KMK Online
Agile and data driven product development oleh Dhiku VP Product KMK OnlineAgile and data driven product development oleh Dhiku VP Product KMK Online
Agile and data driven product development oleh Dhiku VP Product KMK Online
 
Modernizing Architecture for a Complete Data Strategy
Modernizing Architecture for a Complete Data StrategyModernizing Architecture for a Complete Data Strategy
Modernizing Architecture for a Complete Data Strategy
 
Developing a Data Strategy -- A Guide For Business Leaders
Developing a Data Strategy -- A Guide For Business LeadersDeveloping a Data Strategy -- A Guide For Business Leaders
Developing a Data Strategy -- A Guide For Business Leaders
 
Data strategy in a Big Data world
Data strategy in a Big Data worldData strategy in a Big Data world
Data strategy in a Big Data world
 
Becoming a Data-Driven Organization - Aligning Business & Data Strategy
Becoming a Data-Driven Organization - Aligning Business & Data StrategyBecoming a Data-Driven Organization - Aligning Business & Data Strategy
Becoming a Data-Driven Organization - Aligning Business & Data Strategy
 
Lessons in Data Modeling: Why a Data Model is an Important Part of Your Data ...
Lessons in Data Modeling: Why a Data Model is an Important Part of Your Data ...Lessons in Data Modeling: Why a Data Model is an Important Part of Your Data ...
Lessons in Data Modeling: Why a Data Model is an Important Part of Your Data ...
 

Similar to Using data to guide product development

Similar to Using data to guide product development (20)

Mozilla Foundation Metrics - presentation to engineers
Mozilla Foundation Metrics - presentation to engineersMozilla Foundation Metrics - presentation to engineers
Mozilla Foundation Metrics - presentation to engineers
 
NACDEP 2015 - Are we entrepreneurs?
NACDEP 2015  - Are we entrepreneurs?NACDEP 2015  - Are we entrepreneurs?
NACDEP 2015 - Are we entrepreneurs?
 
Anatomy of Relevance - From Data to Action: Presented by Saïd Radhouani, Yell...
Anatomy of Relevance - From Data to Action: Presented by Saïd Radhouani, Yell...Anatomy of Relevance - From Data to Action: Presented by Saïd Radhouani, Yell...
Anatomy of Relevance - From Data to Action: Presented by Saïd Radhouani, Yell...
 
Anatomy of Search Relevance: From Data To Action
Anatomy of Search Relevance: From Data To ActionAnatomy of Search Relevance: From Data To Action
Anatomy of Search Relevance: From Data To Action
 
Harshal cv-UI-developer
Harshal cv-UI-developerHarshal cv-UI-developer
Harshal cv-UI-developer
 
Data-Driven Software Testing: The New, Lean Approach to Quality
Data-Driven Software Testing: The New, Lean Approach to QualityData-Driven Software Testing: The New, Lean Approach to Quality
Data-Driven Software Testing: The New, Lean Approach to Quality
 
Product management with analytics
Product management with analyticsProduct management with analytics
Product management with analytics
 
Using online tools to help us assess our public legal education work
Using online tools to help us assess our public legal education work Using online tools to help us assess our public legal education work
Using online tools to help us assess our public legal education work
 
Excel with Enterprise SEO
Excel with Enterprise SEOExcel with Enterprise SEO
Excel with Enterprise SEO
 
UX Webinar: Always Be Testing
UX Webinar: Always Be TestingUX Webinar: Always Be Testing
UX Webinar: Always Be Testing
 
Intro to Data Analytics with Oscar's Director of Product
 Intro to Data Analytics with Oscar's Director of Product Intro to Data Analytics with Oscar's Director of Product
Intro to Data Analytics with Oscar's Director of Product
 
Data Driven Design: Using Web Analytics to Improve Information Architectures
Data Driven Design: Using Web Analytics to Improve Information ArchitecturesData Driven Design: Using Web Analytics to Improve Information Architectures
Data Driven Design: Using Web Analytics to Improve Information Architectures
 
Tuli eServices_Mobile portfolio
 Tuli eServices_Mobile portfolio Tuli eServices_Mobile portfolio
Tuli eServices_Mobile portfolio
 
User testing methodology
User testing methodologyUser testing methodology
User testing methodology
 
Data Science for Online Services: Problems & Frontiers (Changbal Conference 2...
Data Science for Online Services: Problems & Frontiers (Changbal Conference 2...Data Science for Online Services: Problems & Frontiers (Changbal Conference 2...
Data Science for Online Services: Problems & Frontiers (Changbal Conference 2...
 
How To Structure Your Search Team for Success
How To Structure Your Search Team for SuccessHow To Structure Your Search Team for Success
How To Structure Your Search Team for Success
 
SEO - What is it?
SEO - What is it?SEO - What is it?
SEO - What is it?
 
Intuit - How to Scale Your Experimentation Program
Intuit - How to Scale Your Experimentation ProgramIntuit - How to Scale Your Experimentation Program
Intuit - How to Scale Your Experimentation Program
 
Tips for Choosing A New College Planning Technology
Tips for Choosing A New College Planning TechnologyTips for Choosing A New College Planning Technology
Tips for Choosing A New College Planning Technology
 
Webinar - SEO for Beginners: Simple Steps for Nonprofits and Libraries - 2016...
Webinar - SEO for Beginners: Simple Steps for Nonprofits and Libraries - 2016...Webinar - SEO for Beginners: Simple Steps for Nonprofits and Libraries - 2016...
Webinar - SEO for Beginners: Simple Steps for Nonprofits and Libraries - 2016...
 

Recently uploaded

Microsoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdfMicrosoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdf
Overkill Security
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
Wonjun Hwang
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
FIDO Alliance
 

Recently uploaded (20)

Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
Microsoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdfMicrosoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdf
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cf
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch Tuesday
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDM
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptx
 
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfFrisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 

Using data to guide product development