Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

John Cutler


Published on

What I learned from talking to 100+ product managers about data

Published in: Business
  • Be the first to comment

  • Be the first to like this

John Cutler

  1. 1. I talked to 100 teams about data. This is what I learned … @johncutlefish @johncutlefish
  2. 2. Why this talk?
  3. 3. Why Trust Learn Ritual Access Action Safety
  4. 4. WhyIf Google is doing it … then we should be doing it right? The VP got us together one day and told us that we needed to become more data-driven. Our features are going to be measured now. It was pretty stressful! I have no idea whether the work we are doing is actually having an impact. Whether it really makes it difference. I’m getting discouraged, actually. People say it does, but I don’t see it. We are just throwing spaghetti at the wall right now! We just have more information at our fingertips. We build better products. It is just another skillset and tool Obstacles The Jones “Alignment” Tiebreaker Catalysts Customer outcomes Sense of impact Probe and listen
  5. 5. TrustI’m just not sure I trust the team to act on the data appropriately. What about nuance? What about my experience? Just to be blunt, I don’t trust her. We talk about being a team, but it is always about her ideas and her intuition. My intuition matters too! Right? I know we all have the best intentions. Of course I get nervous, and sometimes out of my comfort zone, but I know we’re together on this Obstacles The unknown Cherry picking Intuition sucks! Catalysts Just try! Data quality Full team activity
  6. 6. LearnJust between you and me, there aren’t really any open questions at the moment. We have to execute, and we have to move quickly. Yes … I’m giving them a solution Do you think the code is a thing? That is has value, really? I don’t think so. It is what this founding team has learned over the last four, hard years. You can’t quantify it. We could tear the stack down and start over All I hear all day is “experiment” and “hypothesis” and “assumption”. I think we’re bastardizing the words. This isn’t science! This is shipping software Obstacles Relearn / rework Zombie features Perfect plan Catalysts Curiosity & stories Great questions We learned ...
  7. 7. RitualIt is just ad hoc queries for right now. When someone needs something, they have run the reports. I’m always asking engineers about SQL We share our findings on Thursdays. Like a big show and tell. It was rocky at first because some people were new to doing this type of analysis. But we learned quickly. It makes it more routine I have no idea where to start to be honest. Each time feels like a restart. New questions. New metrics. I get the idea, but it just doesn’t seem feasible. I have a day job! Obstacles Scrumfall “Listen for …” Reporting Catalysts Show and tell Cadence Radiators
  8. 8. AccessSo we had to downsize a bit. Not cool. But because of that a small group of us got together and started learning more. The basics, really. We had relied on someone else previously The product team doesn’t want us to have a login. It is like they want to own MixPanel. Only they can run reports I’m just an intern! I’m doing an MBA, and I’m not a data scientist. But for some reason they gave me the keys to all of this data. I used some tools from [school] and figured out some key predictors of churn and success. I feel kind of bad for the people who never get that freedom Obstacles Tool festishism Data hoarding The “Data Scientist” Catalysts Study groups Explore/exploit Democratize data
  9. 9. ActionWe had so much learning … so many interviews and stuff. And then some good data. We made the effort as a whole team. But suddenly we stopped. Everyone was kind of pissed. Why didn’t we act on it? I feel like we are looking for a needle in a haystack. And meanwhile we have everyday questions that need answers. Why does it need to be this big deal? Teams know that you’ll never get it right the first time. We just accept that. The drive is to keep moving the needle, and we mostly let people do that until it stops working, or never works Obstacles Feature factory “Owner” Get ahead of it Catalysts Missions/outcomes Explore/exploit Design a dashboard
  10. 10. SafetyThere is a certain vulnerability to all this. That’s a good word for it. It was kind of easier before when we’d build what they asked for, and then they took responsibility for it. Now it is on our watch! What if it fucks up? The problem with experiments is that our experiments would never fail. Never. Experiment was another word for “do what I say!” Having the new [title] here made so much difference. We’re not pushing hard enough unless we mess some things up I learned some stats. And I totally botched up a test. Completely. But it was OK. The world didn’t end Obstacles Success theater Penalty! The inmates! Catalysts Celebrate learning One team / all data By example
  11. 11. Discuss