Data That Sparks Joy by Heap Analytics Sr Product Manager

Product Management presentation given during #ProductCon Online in July 2020 by Heap Analytics Sr Product Manager.

CERTIFICATES
Your Product Management Certiﬁcate Path
Product Leadership
Certiﬁcate™
Full Stack Product
Management Certiﬁcate™
Product Management
Certiﬁcate™
20 HOURS40 HOURS40 HOURS

Corporate
Training
Level up your team’s Product
Management skills

Free Product Management Resources
BOOKS
EVENTS
JOB PORTAL
COMMUNITIES
bit.ly/product_resources
COURSES

Data That Sparks Joy ✨
July 9th, 2020 | ProductCon Online

Agenda
Introduction
● Step 1: Inventory of your (data) space
● Step 2: What to keep and what to discard
Conclusion

👋 About Me
Data Analyst Data Engineer Product Manager
(Experimentation
Platform)
Product Manager
(Product Analytics
Software)
There is definitely a conducting line here.

👩‍💻 Our framework today
Ease of Access
Accuracy Understandability
CompletenessReliability

As a PM data can be anywhere
Surveys
Interactions/behavioral data
A/B tests
Interviews
Transactional/financial data
Performance data

Survey data
Ease of Access
Accuracy Understandability
CompletenessReliability

Behavioral data
Ease of Access
Accuracy Understandability
CompletenessReliability

A/B testing data
Ease of Access
Accuracy Understandability
CompletenessReliability

Interviews
Ease of Access
Accuracy Understandability
CompletenessReliability

Transactional data
Ease of Access
Accuracy Understandability
CompletenessReliability

Performance data
Ease of Access
Accuracy Understandability
CompletenessReliability

3 tips for data hygiene
🗑 Toss the metrics that don’t spark joy
🗂 Get an organization system in place
🏋 Build a good routine

🗑 Toss the metrics that don’t spark joy
Toss: Metrics that don’t spark joy
Vanity Metrics
# of users
# of impressions
Etc.
Objective-oriented
metrics
Feature engagement
Conversion rates

🗑 Toss the metrics that don’t spark joy
Toss: Outliers
Outliers

🗑 Toss the metrics that don’t spark joy
Keep: Medians and Modes

🗑 Toss the metrics that don’t spark joy
Toss: Metrics that don’t spark joy

Get an organization system!
● Data Dictionary
● Internal wiki
● Alerting

Build a good routine
When looking at your dataset:
- Sample size unbiased and large enough?
- Are you cherry picking?
Credit: Geckoboard

Learn how to ask questions
- Open ended vs. close ended questions
- Neutral vs. leading questions
- Answers need to be balanced (not “bad”, “good”,
“very good”, “excellent”)
- Ask one question at the time

3 things to remember:
🗑 Toss the metrics that don’t spark joy
🗂 Get an organization system in place
🏋 Build a good routine

www.productschool.com
Part-time Product Management Training Courses
and
Corporate Training

From maximizing the crave-ability of food additives to notification addiction, Product Managers have a profound impact on society. In the not too distant future, a number of those Product decisions will be delivered by artificial intelligence. In this talk, we discussed ethical lessons from the history of Product Management and how we can learn from them to build ethical AI. Former Director of Product at Yammer also talked about how to understand data quality, biases, and potential impacts, and learn what your self-driving car will do when it encounters The Trolley Problem.

Data Science: The Product Manager's Primer

Building a Culture of Product Experimentation by Optimizely PM

Data-Driven Decision Making by Expedia Sr PM

Intro to Data Analytics with Oscar's Director of Product

Kevin Yien, a Product Manager at Square, talked about how to listen for learnings to influence any decisions you need to make, including: building, marketing and selling a product. Main points from his talk: FIND people to interview who will be able to provide valuable feedback PREPARE scripts for your interviews CONDUCT discovery interviews to in/validate hypotheses WRITE actionable notes from interviews (whether in-person or over the phone) ANALYZE your notes and extract trends/insights

Test for Success: A Guide to A/B Testing on Emails & Landing Pages

Optimizely

Email marketing is a key component to any successful marketing strategy — and it's constantly evolving! That's why testing and optimizing your communications is just as important as the strategy itself. Sometimes knowing how, what, and when to test can seem overwhelming but don't worry, we've got your back. Join this informative webinar with Jessica Langensand of Marketo and Allison Sparrow of Optimizely to discover: How to design an effective email A/B test What to test in your emails and landing pages Testing ideas you'll want to share with your team!

[CXL Live 16] Opening Keynote by Peep Laja

Kissmetrics on SlideShare

Analytics for Startups

I’m not going to spend any time on Google Analytics. How healthy is this business? It’d be great to track metrics like these: 1 MRR, Churn, LTV, acquisition cost 2 Virality, DAU, MAU 3 Average order value, repurchase rate 4 Funnels and conversions But you don’t have any data yet Your data is in a constant rate of decay Your data is messy Use metrics that measure your biggest problem. Ignore the rest. Gateway Metrics When picking metrics, always ask yourself: What’s my biggest constraint right now and which metric will tell me if I’m making progress? You need to do the right things in the right order. Gateway #1: Is your idea any good? Your main constraint: Getting anyone to care about your idea. Your main metric: Get someone to pay or use your product regularly. Bad metrics for this gateway: 1 Asking people if they’ll pay 2 AdWords clicks 3 Beta or waiting list signups 4 Traffic Gateway #2: Is your product good enough? Your main constraint: Having a product that’s good enough to build a business on. Your main metric: Ask 500 users the Product/Market Fit Question What is the P/M Fit Question? Your goal for the P/M Fit Question: At least 40% of users should say “Very disappointed.” *Sean Ellis and Hiten Shah get credit for this one. How do you get to the first 500 users/customers? Hustle. The P/M Fit Question isn’t perfect, verify with a retention metric. Gateway #3: Can you grow? Your main constraint: Acquiring customers consistently from at least one channel. You have plenty of options to choose from: 1 Inbound (Google, Content, Social) 2 Paid (PPC, Affiliates) 3 Virality (Invites, Referrals) Pick just one to start Work on your channel for at least 3 months. Assume it’ll work and get the resources needed to execute. Your main metrics: Your main business metric and acquisition funnel. Main business metrics: 1 SaaS: Monthly Recurring Revenue 2 Ecommerce: Monthly Revenue 3 Consumer Tech: Monthly Active Users Why not cost per acquisition or lifetime value? You have no idea how much it costs to acquire customers or how much they’ll spend (yet). Gateway #4: Do you have a stable model? Your main constraint: In order to keep scaling, you need a stable model for your business. Your main metrics: Depends entirely on what business model you have. The SaaS Model The Ecommerce The Consumer Tech Model Find someone in your industry that knows the key benchmarks. Finally, get serious with data. If you have a sales team, pile data into your CRM. If consumer tech, do everything in-house. Google Analytics plus an internal database will take you far. Start with constraints, hack together what you need to measure them. How to get data you really need: 1 One team owns data quality. 2 Hire a data engineer. 3 Clean up and integrate your data. 4 Use customer analytics. 5 Build a Growth Team.

Agile Marketing Meetup: Moving Beyond the Marketing Plan So You Remain Relevant

Annual marketing plans can't keep up with the rapidly changing landscape of digital marketing acquisition channels and tactics. To remain relevant, CMOs and marketing teams need to completely rethink their marketing/growth approach. This presentation highlights the agile process that today's fastest growing companies are adopting - from building the right team to executing an agile marketing process.

[CXL Live 16] SaaS Optimization - Effective Metrics, Process and Hacks by Ste...

Stephen will be talking on SaaS optimization strategy, including: - The data, insight and metrics you need to track to identify opportunity - Flow optimization: from landing page to trial, usage, purchase and retention - Problems and opportunities: e.g. how do you test a landing page when the sale happens 14 or 30 days later? How do you manage testing across multiple KPIs simultaneously? How do you understand and segment your key product offering?

Three Rules Every Mobile Product Needs to Be Successful With The Author Of M...

How Marketplace Products Build Customer Trust by eBay Sr PM

Main takeaways: - Trust is as elementary a problem for marketplace companies as is balancing Supply and Demand. - There are two types of Trust that companies need to solve for - Intrinsic (trust in the marketplace itself) and Extrinsic (trust in the players on the marketplace). - Companies such as Airbnb, eBay, Etsy, Uber have solved for the above two through various products and features that form the backbone of their product experiences.

Ask Me Anything with LinkedIn's Senior Product Manager

Aligning Quantitative Analytics for Better Product Decisions

3 Ways of Working with Support Leads to Success by Shopify Sr PM

Design Sprints

Jeremy Horn

Slides Nis Frome recently used in his discussion w/ mentees of The Product Mentor. Synopsis: The design sprint is a concrete vehicle for deploying necessary user research techniques to go from problem to solution. You can use it to prioritize opportunities, clarify a roadmap, or iterate on the nitty gritty of a user experience. The Product Mentor is a program designed to pair Product Mentors and Mentees from around the World, across all industries, from start-up to enterprise, guided by the fundamental goals…Better Decisions. Better Products. Better Product People. Throughout the program, each mentor leads a conversation in an area of their expertise that is live streamed and available to both mentee and the broader product community. http://TheProductMentor.com

Measure Your Way to Success by Sephora's former Dir. of Product

Product management is about creating change. Metrics are the guideposts that help us ensure that the changes we make are leading to the results we want. They let us forecast, provide early warning signals, and create incentives for action. In this talk Meghan Cochran talked about designing good metrics, gaining alignment among a broad range of stakeholders, and communicating progress effectively. She discussed the trade-offs of looking at rates & ratios vs absolute numbers, and talked about funnels, cohorts, and other fascinating and exciting measures of success.

VWO Webinar: Scaling Your Testing Program

VWO

Better Living Through Analytics - Strategies for Data Decisions

Data is king! Get ready to understand how a successful analytics team can empower managers from product, marketing, and other areas to make effective, data-driven decisions. Louis Cialdella, a data scientist at ZipRecruiter, shared some case studies and successful strategies that he has used at ZipRecruiter as well as previous experiences. The purpose of this data talk was to enlighten people on how to make sure that analysts can successfully partner with other departments and get them the information they need to do great things.

The Biggest Growth Opportunity is Right Under Your Nose

Hkwaw event 20100622Hong Kong Web Analytics

Product management with analytics

Daryl Hemeon

[CXL Live 16] Growth Hacking BS: Fixing Marketing One Truth at a Time by Morg...

Growth hacking has exploded in popularity, with companies scrambling to find magical 'hackers' who create massive user bases and revenue out of thin air. Unfortunately, a closer look shows us that much of the hype is just that. To find growth, companies must stop searching for unicorns and do something much less sexy: get back to work. In this talk I'll cover: - how companies actually grow online based on studying dozens of fast-growing startups - why growth is based on companies finding winning tests faster - how we did that at growth hackers - how other companies do it (HubSpot, Twitter, more) - how to start hacking and start growing - things to avoid, etc.

Product Management and Metrics by Amazon Sr PM

Measuring What Matters by Google PM

What's hot

Measuring Success of Data Platforms by Booking.com Product Leader

How to Interview Customers with Square's PM

Test for Success: A Guide to A/B Testing on Emails & Landing Pages

Optimizely

[CXL Live 16] Opening Keynote by Peep Laja

Kissmetrics on SlideShare

Analytics for Startups

Agile Marketing Meetup: Moving Beyond the Marketing Plan So You Remain Relevant

[CXL Live 16] SaaS Optimization - Effective Metrics, Process and Hacks by Ste...

Three Rules Every Mobile Product Needs to Be Successful With The Author Of M...

How Marketplace Products Build Customer Trust by eBay Sr PM

Ask Me Anything with LinkedIn's Senior Product Manager

Aligning Quantitative Analytics for Better Product Decisions

3 Ways of Working with Support Leads to Success by Shopify Sr PM

Design Sprints

Jeremy Horn

Measure Your Way to Success by Sephora's former Dir. of Product

VWO Webinar: Scaling Your Testing Program

VWO

Better Living Through Analytics - Strategies for Data Decisions

The Biggest Growth Opportunity is Right Under Your Nose

Hkwaw event 20100622Hong Kong Web Analytics

Product management with analytics

Daryl Hemeon

[CXL Live 16] Growth Hacking BS: Fixing Marketing One Truth at a Time by Morg...

What's hot (20)

Measuring Success of Data Platforms by Booking.com Product Leader

How to Interview Customers with Square's PM

Test for Success: A Guide to A/B Testing on Emails & Landing Pages

[CXL Live 16] Opening Keynote by Peep Laja

Analytics for Startups

Agile Marketing Meetup: Moving Beyond the Marketing Plan So You Remain Relevant

[CXL Live 16] SaaS Optimization - Effective Metrics, Process and Hacks by Ste...

Three Rules Every Mobile Product Needs to Be Successful With The Author Of M...

How Marketplace Products Build Customer Trust by eBay Sr PM

Ask Me Anything with LinkedIn's Senior Product Manager

Aligning Quantitative Analytics for Better Product Decisions

3 Ways of Working with Support Leads to Success by Shopify Sr PM

Design Sprints

Measure Your Way to Success by Sephora's former Dir. of Product

VWO Webinar: Scaling Your Testing Program

Better Living Through Analytics - Strategies for Data Decisions

The Biggest Growth Opportunity is Right Under Your Nose

Hkwaw event 20100622

Product management with analytics

[CXL Live 16] Growth Hacking BS: Fixing Marketing One Truth at a Time by Morg...

Similar to Data That Sparks Joy by Heap Analytics Sr Product Manager

Product Management and Metrics by Amazon Sr PM

Measuring What Matters by Google PM

Data Informed Product Management by Eventbrite Sr PM

Webinar: How to be Data Driven with Product by Carbon Five Sr PM

How to Effectively Experiment in PM by LendingTree Sr PM

Leverage The Power of Small Data

Karyn Zuidinga

How to be an AI Product Manager by Amazon Product Leader

MEASURING THE BUSINESS IMPACT OF LEARNING: WHAT WE’VE LEARNED

Human Capital Media

The ‘Measuring the Business Impact of Learning’ benchmarking survey, conducted by LEO Learning and Watershed (on behalf of Learning Technologies Group) is entering its fourth year. With the survey launching on November 1st and closing on December 13th, LEO Learning and Watershed are holding a webinar to reflect on the results so far, plus discuss how organizations they’re working with have overcome the barriers in measurement planning and implementation. The insights are drawn from their group experience working with a range of clients in this field and should be valuable for anyone who wants to get going in learning analytics and sustainable business impact assessment. Join your hosts as they cover the state of the world of measurement, and you’ll receive: An understanding of how well-known organizations have overcome the barriers in measurement planning and implementation. Real-world examples of how to get management buy-in, designing for data, building data ecosystems, implementing a learning analytics strategy and more. The opportunity to take this years ‘Measuring the Business Impact of Learning’ survey, and see the results coming in live!

Focusing on Metrics that Matter as a Prod Mgr by T-Mobile Sr PM

Data - How to Use it & When by Square and Call Rail Product Leader

Main Takeaways: -It’s important to define success metrics before you start building your product or feature -The goal is to validate a hypothesis--what you think users want--and the data might invalidate your hypothesis. This is a good thing! Keep iterating in that direction to identify and test the next hypothesis. -Data is important to point you in the right direction, but it won’t answer “what is the perfect product?” To figure this out, you need to experiment, fail and repeat as fast as possible to find success.

How to Make Your Product a Data-Driven Product by Pluralsight PM

An Experimentation Framework: How to Position for Triple Digit Growth

Optimizely

You’ve done the button color A/B test, you’ve optimized your landing pages for better conversion. What next? At B2B organizations large and small, there is still tremendous potential for experimentation to drive innovation and growth. Learn how Brion’s growth team enables rapid iteration across a variety of different domains, teams, and organizations within Cisco. With an organization of 70,000 employees and many distributed divisions, enabling experimentation can be a complex initiative. Learn the framework for upleveling from random testing to explicit strategy to position your org for triple digit growth.

Dev's Guide to Feedback Driven Development

Marty Haught

How to Use Data to Build Better Products by fmr NY Times PM

Organizational development measures

Eric Bruggeman ET

Finding Meaning in the Numbers

TechSoup Canada

Startup Marketing: A Systems Approach

April Dunford

The Top 5 Challenges Product Faces with Jira

Cprime

Data-Driven Performance Feedback Helps Teams Make Better Customer Outcomes

Aggregage

Planning your analytics journey - webinar slides

Sprout Labs

In this webinar you’ll learn how to plan and implement sophisticated learning analytics in your learning programs. As our workplaces are becoming more data-driven, Learning & Development has new opportunities for measuring learning and the impact of your learning programs. You don’t need to become a data scientist to take advantage of these opportunities. You’ll learn: - What are data-driven approaches to learning - Useful statistics and visualisation techniques for L&D - The link between learning analytics and xAPI - Test-driven approaches to learning analytics - Emerging approaches using artificial intelligence and machine learning - How to plan a successful learning analytics project

Similar to Data That Sparks Joy by Heap Analytics Sr Product Manager (20)

Product Management and Metrics by Amazon Sr PM

Measuring What Matters by Google PM

Data Informed Product Management by Eventbrite Sr PM

Webinar: How to be Data Driven with Product by Carbon Five Sr PM

How to Effectively Experiment in PM by LendingTree Sr PM

Leverage The Power of Small Data

How to be an AI Product Manager by Amazon Product Leader

MEASURING THE BUSINESS IMPACT OF LEARNING: WHAT WE’VE LEARNED

Focusing on Metrics that Matter as a Prod Mgr by T-Mobile Sr PM

Data - How to Use it & When by Square and Call Rail Product Leader

How to Make Your Product a Data-Driven Product by Pluralsight PM

An Experimentation Framework: How to Position for Triple Digit Growth

Dev's Guide to Feedback Driven Development

How to Use Data to Build Better Products by fmr NY Times PM

Organizational development measures

Finding Meaning in the Numbers

Startup Marketing: A Systems Approach

The Top 5 Challenges Product Faces with Jira

Data-Driven Performance Feedback Helps Teams Make Better Customer Outcomes

Planning your analytics journey - webinar slides

More from Product School

De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...

How world-class product teams are winning in the AI era by CEO and Founder, P...

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...

Designing Great Products: The Power of Design and Leadership by Chief Designe...

From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...

From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...

Mission to Decommission: Importance of Decommissioning Products to Increase E...

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech

Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...

Relationship Counselling: From Disjointed Features to Product-First Thinking ...

Launching New Products In Companies Where It Matters Most by Product Director...

Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...

Revolutionizing The Banking Industry: The Monzo Way by CPO, Monzo

Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...

Act Like an Owner, Challenge Like a VC by former CPO, Tripadvisor

The Future of Product, by Founder & CEO, Product School

Webinar How PMs Use AI to 10X Their Productivity by Product School EiR.pdf

Explore AI tools hands-on and smoothly integrate them into your work routine. This practical experience is here to empower you, offering insights into the mindset of successful Product Managers. Learn the skills to become a more effective Product Manager. Main Takeaways: Hands-On AI Integration: Learn practical strategies for integrating AI tools into your workflow effectively. Mindset Insights for Success: Gain valuable insights into the mindset of successful Product Managers, unlocking the secrets to their achievements. Skill Empowerment for Growth: Acquire essential skills that empower your evolution toward becoming a more effective and impactful Product Manager.

Webinar: Using GenAI for Increasing Productivity in PM by Amazon PM Leader

Unlocking High-Performance Product Teams by former Meta Global PMM