Recommended
Recommended
More Related Content
Similar to Customer Obsession at Scale
Similar to Customer Obsession at Scale (20)
Recently uploaded
Recently uploaded (20)
Customer Obsession at Scale
- 1. Sam Nordstrom November 28, 2017 Customer Obsession At Scale
- 2. I’m Sam! Product Manager QuickBooks Payments 1. Invoicing 2. Data & Analytics Hi!
- 3. 1. What I work on - invoicing 2. Intuit’s approach to building products 3. How we use product analytics to stay customer obsessed at scale What I’m Going To Talk About Today Empowering teams to be active (not passive) consumers of data
- 5. What if Small Businesses got paid on all of their invoices at once?
- 8. Reduce US Unemployment By 27%
- 9. And make an incremental $30,000 above their current pay
- 11. Customer Obsession How Intuit Builds Products
- 12. 12Intuit Confidential and ProprietaryIntuit Confidential and ProprietaryIntuit Confidential and Proprietary Where we startedWhere we started: Intuit’s journey began over 35 years ago when our founder Scott Cook sat at his kitchen table and watched his wife as she balanced their checkbook and thought there must be a better way.
- 13. All companies claim they want to get close to their customers, but few do it as obsessively as Intuit… The underlying reality is that you can’t believe what customers tell you. Customer behavior is the truth. – Fortune Magazine #8 On Future 50 Top Companies
- 14. 14Intuit Confidential and ProprietaryIntuit Confidential and ProprietaryIntuit Confidential and Proprietary From: Quicken ½ of customers used the product in the office To: QuickBooks The #1 small business software in 2 months at double the price of the competition
- 15. 15Intuit Confidential and ProprietaryIntuit Confidential and ProprietaryIntuit Confidential and Proprietary From: Mint Customers used the app to manage income & expenses for Uber & Lyft To: QuickBooks Self-Employed The fastest growing & stickiest product in our 35 year history
- 16. Savoring The Surprise Digging into the insight or result that makes no sense. Focusing on the outlier, the information that doesn’t fit, to know the customer better than they know themselves. Active Customer Obsession Is...
- 17. We take the same approach with data We don’t stop with what the numbers tell us…
- 18. 18Intuit Confidential and ProprietaryIntuit Confidential and ProprietaryIntuit Confidential and Proprietary Using Product Analytics To Stay Customer Obsessed Metrics That Matter Data Driven Culture Infrastructure That Moves Fast Metrics Consistent Consumable Confident Culture Active > Passive Evangelize Encourage Infrastructure Time-to-insight Time-to-test Barriers to access 2 3 1
- 20. 1. Removing The Bottleneck 2. Single Source Of Truth 3. Multiple Versions Of Truth Enabling Speed
- 22. 1) product teams move at the speed of curiosity 2) data teams get time back to focus on the work that matters Team of Detectives > Single Sherlock
- 23. 23Intuit Confidential and ProprietaryIntuit Confidential and ProprietaryIntuit Confidential and Proprietary Single Source of Truth & Multiple Versions of Truth
- 25. • How you track – chickpeas aren’t garbanzo beans • Taxonomy – don’t rely on humans to govern & enforce a standard • Metric Definition – align, re-align, and talk about them constantly • Data Accuracy – get everyone on the same page Consistent & Consumable Metrics
- 26. 26Intuit Confidential and ProprietaryIntuit Confidential and ProprietaryIntuit Confidential and Proprietary Confidence Is Binary Product Data Doesn’t Have To Be Just Directional
- 27. 27Intuit Confidential and ProprietaryIntuit Confidential and ProprietaryIntuit Confidential and Proprietary Engagement Breadth, Depth & Height • Breadth = actions that matter • Height = usage frequency • Depth coefficient = correlation with desired outcome Building The Right Metric Teams engage with data when they fell 100% ownership over their metrics KEY TAKEAWAY: The right metrics aren’t always readily captured in a single event. Don’t use proxies for customer getting value; invest to the measure the whole story. TIP: Look in the speaker notes of this slide for instructions on selecting and using media types. And don’t forget to remove this box!
- 28. Infrastructure That Moves Fast Data-Driven Culture Infrastructure That Moves Fast Metrics That Matter Data Driven Culture Infrastructure That Moves Fast 3
- 29. Data democratization helps us move fast… A data-driven culture helps us disrupt our own products and change before it is needed
- 30. 1. Educate Up Front 2. Bake Data Into Your Day 3. Encourage Active Consumption Building A Data-Driven Culture
- 31. 31Intuit Confidential and ProprietaryIntuit Confidential and ProprietaryIntuit Confidential and Proprietary Great For Reporting… Not Insight Reading = Passive Consumption • Focus is on the metric that’s already being measured • Reactive data consumption instead of proactive data analysis • Discourages active thinking & doesn’t build the analytical mindset • Everyone should run the queries they are interested in Encouraging Active Consumption Reading Charts An excellent jump-off point, but it can’t stop here Saving ChartsMy least favorite feature…
- 32. Savoring The Surprise With Data
- 34. The Problem Customers are creating, saving & downloading invoices in QuickBooks, but not emailing them The Result Fewer invoices paid on time or at all, friction on both the QuickBooks user & their customer
- 35. Let’s fix the email creation workflow in QuickBooks!... Right?
- 36. Digging into the insight that makes no sense
- 37. 37Intuit Confidential and ProprietaryIntuit Confidential and ProprietaryIntuit Confidential and Proprietary Digging In Optimizing on the right platform • If we just read the invoicing funnel we would have optimized the email workflow in QuickBooks • It took the entire team digging into this workflow on a daily basis to figure out what these customers were doing. • Follow-Me-Homes confirmed suspected customer behavior Who are the merchants not emailing? And Why? We can’t see where the users are going after the leave the product, but we can see what traits they have in common KEY Insights: ½ of customers use Gmail for their small business Shared invoice links & downloaded attachments are viewed from chrome & Gmail TIP: Look in the speaker notes of this slide for instructions on selecting and using media types. And don’t forget to remove this box!
- 39. 1. Invest in a infrastructure that enables speed through access & accuracy 2. Focus on metrics that are consistent and 100% actionable 3. Encourage a data culture that embraces active > passive consumption Takeaways Disrupt Your Own Products To Change Before It Is Needed
- 40. Thanks!
Editor's Notes
- Hi Everyone – I am super excited to be here and thanks again for coming out – it’s great to see so many people taking an interest in product analytics. This is an aspect of great product building I am really passionate about, and looking forward to sharing some of the things our team has done with product data to improve how we approach product development
- I'm sam - I'm a PM at Intuit on our payments teams - focusing on invoicing & data and analytics our primary product is QuickBooks - but we also handle money movement for our entire ecosystem of payments products - from turbo tax to mint
- I want to touch on 3 different areas today To start with a little bit about what I work on - invoicing touch on how intuit approaches product building before talking about we use product analytics to drive this same approach of customer-driven development at scale all by empowering teams to be active, not passive, consumers data
- For those of you who aren’t familiar with Intuit - we make a wide range of products and services for small business and consumers that help them manage and control the most intimidating aspects of their financial lives - from taxes and accounting, to financing and payments Our mission is powering prosperity around the world because we allow our customers to focus on what they are truly passionate about - which coincidentally is almost never taxes and finance - while also delivering more time and money back. My team focuses on invoicing - the job of creating a statement of work, billing a client for the goods & services you’ve provided, tracking and getting paid for that work. Pretty simple right? So how exactly do we power prosperity through invoicing?
- Well although invoicing may seem straightforward, its actually one of the most painful parts of transacting as a small business or self-employed entity. Does anyone know the total dollar value of unpaid invoices at this moment for small businesses US?
- 825 billion… This gap has a massive impact that extends beyond just the businesses who aren’t getting paid for the work they have done If all unpaid invoices were suddenly paid tomorrow
- SBOs would hire an additional 2.1 million employees
- Reducing unemployment by 27%
- And increasing their own individual pay by more than $30,000
- This impact is huge and we build products that help at every stage in the invoicing process - from creating and sending a beautiful and customized invoice, to tracking and receive payments for your hard work. This is a little about the problem my team focuses on. In addition to invoicing I also drive data and analytics for the payments ecosystem. Since starting in this role one of my main areas of focus has been improving the use data in decision making. in order to explain how we think about this topic I want to walk through how Intuit approaches re-inventing their products.
- For us - it all boils down to customer obsession I’m sure everyone has heard about customer backed thinking, but at intuit we think about customer obsession a little differently going all the way back to our founding.
- intuit was founded over 35 years ago when Scott Cook watched his wife struggle to balance the family checkbook at their kitchen table and thought to himself there has to be a better way. Scott obsessed over this problem - not just the trouble of managing one’s money - but the underlying emotion of anxiety & confusion he witnessed his wife experience while trying to take control of their finances he realized the problem wasn’t accounting, or rather just accounting - but the underlying emotions of anxiety and confusion that accompany dealing with money - and built Intuits first personal finance product - quicken
- This idea - of knowing the customer so deeply - you are able to get to the underlying emotion of their problem has been at the core of nearly every large product transformation since that moment at the kitchen table We believe you can’t stop at what the customer explicitly tells you, you need to go deeper, experience their problems first hand in order to uncover ways of working & behaviors customer aren’t explicitly aware of themselves. We call this type of real-world observation a follow me home and we conduct over 10,000 hours of these sessions annually This idea – of knowing the customer so deeply, you get to the underlying emotion of their problem has been with nearly every large step the company has taken since that moment at the kitchen table. When quicken first launched intuit employees would follow customers back home to watch them unwrap the software and use it so they could uncover behaviors and ways of working the customers weren’t explicitly aware of themselves Watching our customers use our products in the real world is something we call a Follow-Me-Home and to this day we conduct over 10,000 hours of these sessions So why do we invest so much in seeing how our customers use our products?
- Engaged, real world observation has served as the catalyst for some of our biggest product transformations. With quicken, scott & team surveyed customers about how they used the product and found over half of customer said they used it in the office rather than at home. While the rest of the team started responded to usability requests scott couldn’t let this one out-of-place answer go, as it didn’t make sense to use a personal finance product in the workplace He dug in and found users were using quicken to run their small business. As a result be built quickbooks which became the #1 small business software in under 2 months at double the price of the competition.
- The same pattern was at the center of a more recent transformation with Mint. PMs digging into mint usage patterns discovered a group of customers were using mint to track and manage income and expenses related to uber & lyft Instead of focusing on how they could improve expense management for the consumer, they built QBSE for the gig economy worker, which became the fastest growing product in our 35 year history
- Our founder has a saying that encompasses this drive to know the customer better than they know themselves – Savoring the surprise. It means digging into the insight the makes no sense, and getting to the why behind what’s not expected. This brand of customer obsession is how we’ve been able to contiguously re-invent our business and products over 35 years
- We take the same approach with data – we don’t just stop at what a report tells us, we have built a culture the consistently digs into the data, the outliers that may not fit with the overall trend, and tries to understand why.
- Using data to stay customer obsessed as we’ve grown has helped us consistently maintain this pattern of re-invention For me – it comes down to 3 core capabilities and areas of focus – that enable our product analytics to to drive this type of customer obsession an infrastructure that enables speed, metrics that are thoughtfully crafted and 100% actionable And a culture that actively engages with data rather than passively consuming data
- Let’s start with the building blocks – the infrastructure that enable these metrics to be tracked and culture to thrive
- For us we’ve focused on 3 capabilities when building out our infrastructure to enable – removing the bottleneck, investing in a single source of truth, and branching multiple versions of truth off of it
- Widespread access to data is absolutely critical. As a big company we need every member of our organization to engage with data as they see fit. The idea of data democratization is well circulated these days – but I can’t stress how important this is, especially for a company as it begins to scale
- True data democratization does 2 equally important things – it enables product teams to move at the speed of curiosity, but it also give time back to data teams to work on the important stuff The chance of an aha! Moment – like the one that lead to quicken, or qb from quicken, or qbse from mint – can’t be a one off. By removing the barrier to access, every team member can dig into the data to find game changing insights Additionally – if everyone can self-serve on the metrics that matter to them, they connect more deeply with their metrics, and begin the foundation for a culture that embraces data in decision making
- The way we remove this bottleneck is through an architecture where all data is aggregated in a single immutable & accurate source of truth – off of which many different ways of working with that data are enabled We believe speed is a product of both confidence & accessibility. the ability to maintain accuracy with a single source of truth, and flexibility to move fast with multiple ways of working is at the core of data program
- The second important tenant of our data strategy is focusing on metrics that truly matter
- For us there are 4 keys to great metrics - from how they are defined, to how they are captured and read Starting with how you track - tracking the same things in the same ways is critical. if one team is tracking a subscription by the user initiated click, another by the viewing of the success screen, and another by the server call back, data consumability quickly breaks apart at scale - putting the time to ensure consistency here it critical Taxonomy - consistency and readability are key to start, but moving beyond a single data governance team to enforce this standard is necessary for any team looking to scale. We’ve begun investing in programmatic ways to construct and test instrumentation & analytics that allow both the engineers to work faster and frees up the data team who is no-longer the bottleneck. Metric definition - spend time here, get really serious about what you’re tracking, how your tracking it We use 4 principles in constructing every KPI actionable, customer-centric, comparative, and drives our business goals. Of these the actionable piece is this most important – it must have the ability to change how the team behaves and reflect a customer outcome the team has complete ownership over data accuracy - align on what each data source should be used for - if product data is directional and only for decision making, that’s fine but make sure every consumer knows it. we’ve taken the stance the confidence in data is binary - once it’s called into question as directional, folks loose faith in it to inform decision making, and if teams have to go to multiple sources to get different answers you loose speed.
- To address this accuracy concern we didn’t just focus on data collection integrity, we’ve built in a couple different ways of measuring and testing how accurate our data was: We track multiple events that should line up 1:1 and build dashboards solely around the parity of these metrics Additionally we pull data from multiple sources – webhooks from our third party partners & processors, user initiated events from the product and server initiated callbacks and aggregate them into a single metric that show how performant our clickstream data is.
- the outcome of these efforts should be a metric that is 100% actionable and reflects the customer outcomes a team is driving for. The more aligned a team is with their metric in feeling like they own the outcome, the more engaged they will be with data on the whole. We recently experience this shift in mindset with one of our teams focused on invoicing engagement – by changing the metric they were held accountable for to accurately reflect their areas of ownership, not only were we able to better capture customer outcomes, but the team felt more connected with the metric they were driving as it reflected all 3 dimensions of engagement.
- The culmination of these efforts is building a culture that embraces engaging with data in an active way
- While data access & the right metrics helps us move quickly, the truly game changing insights come from a culture the embraces active consumption and questioning of data
- A few areas of focus that have helped us foster this culture. It starts with education - the best way to get your whole team embracing data in decisions is to educate up front. we host analytics days, lunch n learns and bootcamps to get the whole team proficient in how to work with our data & tools Working data into your day – checking you metrics with every op-mec, expecting data in every decision. When it comes to data-backed thinking, repetition doesn’t ruin the payer, and the more ways you can weave data into team habits – the better Finally encouraging active data consumption – with the same type of engagement that doesn’t stop at what a customer tells you, but digs into the why behind the trend apart from encouraging all data consumers to do this through education, there is a really simple tactic we’ve adopted that helps drive this type of thinking – moving away from static reports to interactive queries
- we do this be having every consumer create the queries they are interested in in real time rather than relying on shared reports for them to check against It’s critical to dig into the details of a report – not just take the conversion metric or outcome at face value – and really understand why that % of users is converting, who they are and what else they are doing Interactive charts and queries that allow users to work freely with the data is crucial to allowing the whole team to dig deep into insights and get folks thinking in an analytical way
- So what does this culture look like in action… I want to end with a story of how we used active data consumption to build one of our most recent products from the ground up
- As apart of the invoicing team – our main area of focus is helping merchants get paid on their invoices. One of the main ways we’ve been able to help is by pay enabling invoices over email so clients can pay easily We know clients prefer to pay electronically online, and merchants have a better chance of getting paid through these means as well, so we’ve built our invoicing product around pay enabling invoices through email
- However – we recently noticed a pattern of customers creating, saving and downloading invoices in QB without emailing them This group of users experiences a lower paid rate, more delayed payments and fewer dollars in their bank account
- Upon identifying this group – the immediate reaction was to fix the email workflow for these users so they wouldn’t drop-off before sending the Pay enabled invoice. We could see the drop-off at this step, we knew their clients wanted emailed invoices, pretty straightforward - right
- However this drop-off wasn’t the whole story & we were able to uncover an critical trend while digging into the our product usage data
- While we could see the step our users were having difficulty – it didn't tell us why they were dropping off, where they were going instead, and what about these users they all had in common So before we focused on fixing this step we dug into to answer these questions and found a few key insights all by looking at the shared traits and behavior for the cohort that wasn’t converting 1 – ½ of these merchants who were dropping off used gmail as their business email address 2 – the downloaded invoice were being accessed through gmail and the chrome browser So what was happening? Though these shared user trait of didn’t seem pertinent at first it actual revealed a big opportunity. The story we started to see was that customers were downloading their invoices then attaching them to existing email threads in gmail with their clients – but because of this friction of migrating invoices over to another conversation out of context, there was a hit in the pay rate When we asked customers about this behavior they didn’t see the transition of bringing invoices over to a gmail conversation as particularly painful – however the data showed this break in the workflow negatively impacted how much they were getting paid. We identified the problem, & also identified a common behavior - using gmail for business conversation.
- So rather than trying to improve our email workflow, we built invoicing right into gmail so merchants can invoice and get paid all from the same conversations they were already having with their clients. As a result we are seeing higher paid rates and fewer clicks to get paid Digging into the data – the insight that made no sense - allowed us to solve the problem of emailing invoices in the right way by identifying comfortable behavior and moving our product this habit, rather than trying to induce behavior change
- In conclusion – there are 3 key aspects of our approach to data & product that help us re-invent our products time and time again 1. invest in an infrastructure that enables speed through access and accuracy 2. Focus on metric that are consistent and 100% actionable 3. And encourage a data culture that embraces active > passive consumption by focusing on these areas we are able to disrupt our own products and change before it is needed
- Thanks for listening & I think now we have time for a few questions