Downloaded 37 times
Uploaded byYael Garten
How to use your data science team: Becoming a data-driven organization
The document discusses strategies for organizations to become data-driven by fostering a culture that treats data as a primary resource, mapping data usage throughout product lifecycles, and investing in tools to enhance data quality and accessibility. It emphasizes the role of data scientists as partners in decision-making, reinforcing the importance of measurable success metrics and actionable insights to drive product innovation. The talk outlines steps for implementing a data-driven framework, including the identification of relevant metrics, ongoing monitoring, and iterative improvements based on data analysis.
More Related Content
Similar to How to use your data science team: Becoming a data-driven organization
How to use your data science team: Becoming a data-driven organization
- 1. Data 101: How touse your data science team: Becoming a data-driven organization March 29, 2016 Yael Garten Director of Data Science, LinkedIn http://www.linkedin.com/in/yaelgarten, @yaelgarten
- 2. Data is thepresent: every industry, organization, product, decision data driven, or at least data informed.
- 3. Good vs Great?Data is Expected, Discussed, & Accessible Expected For decision making e.g. Determining strategy, goal setting, impact estimates of initiatives Discussed Talked about e.g. product strategy reviews, design discussions, in the hallway Accessible When questions come up, easy to get hands on data? e.g. democratized data, self-serve tools Culture Process Tools
- 4. How do weget started? (whether from scratch or to transform existing organization) Invest in the data Hire the right “data scientists” Culture Process Tools
- 5. Data Scientist –which kind? human machine Find & communicate data insights Producing for: Deliverables: Skills: Machine learning models
- 6. This talk willbe a success if we: 1. Review the steps of data driven product innovation 2. Understand what is needed to best enable fostering of (or transformation into) a data-driven organization: culture, process, and tools.
- 7. 1. Culture: Createa culture which expects decisions are informed by data & which treats data as a first class citizen 2. Process: Consciously map how you use data (in each phase of the product lifecycle, in making exec decisions) 3. Tools: invest in your data ecosystem (data quality, pipeline, and access tools) To create a data driven organization
- 8. Data driven productinnovation Product Lifecycle Ideation Design and Specs Development Test & Iterate Release Data driven product innovation framework: Use data to measure, understand, and improve the product: Build Measure Track Ship Tweak If data scientist is not involved until this stage, it may be too late.
- 9. Well-connected. Get relevance right. Fewconnections. Give them inventory. 1. Opportunity sizing: how big or important is the problem? 2. Use data to predict successful product initiatives: • Show news articles • Suggest new connections • Suggest following active content creators Build Measure Track Ship Tweak
- 10. Hypothesis: Following activesources leads to improved user experience with LinkedIn Feed Success Metric – progression of definition: • Total clicks on Follow • # clicks / #impressions of Follow suggestions • % Feed Inventory created by new followees • Downstream sustained engagement with items created by these followees • What is engaging? # of clicks? Time spent? # Shares? Combo? Actionable insights lead to product feature Success Metric Definition Tracking Instrumentation spec Experimentation Setup and Analysis Post-Launch Analysis Build Measure Track Ship Tweak is creating content we think you’ll find interesting Invest in developing the right success metric.
- 11. ~Metric = Downstreamengagement with items created by these followees Must enable attributing future clicks on feed items to that campaign as a source for the Follow. FeedActivityClick { memberID = 77777 actor = 55555 } FollowSources { followCampaign666 memberID = 77777 followeeID = 55555 } Need accurate reliable standardized data logging to enable metric computation. Actionable insights lead to product feature Success Metric Definition Tracking Instrumentation spec Experimentation Setup and Analysis Post-Launch Analysis Build Measure Track Ship Tweak
- 12. Actionable insights lead toproduct feature Success Metric Definition Tracking Instrumentation spec Experimentation Setup and Analysis Post-Launch Analysis Design: How long to run experiment, on whom? Implement: properly set up & randomize to ensure no bias Analyze: Go or no-go? monitor success metric, ideally automated on company-wide platform for holistic view of impacts Build Measure Track Ship Tweak is creating content we think you’ll find interesting Rigorously set up, then identify whether the feature increased the success metric.
- 13. Reporting, monitoring, adhoc analysis Long term measures of engagement/success Analysis to inform revision of design Actionable insights lead to product feature Success Metric Definition Tracking Instrumentation spec Experimentation Setup and Analysis Post-Launch Analysis Build Measure Track Ship Tweak is creating content we think you’ll find interesting Iterate. How can we revise? How can we tweak to optimize?
- 14. Invest in foundations.Culture Process Tools
- 15. 1. Culture: Createa culture which expects decisions are informed by data & which treats data as a first class citizen 2. Process: Consciously map how you use data (in each phase of the product lifecycle, in making exec decisions) 3. Tools: invest in your data ecosystem (data quality, pipeline, and access tools) To create a data driven organization
- 16. Data scientist asa partner, not a service – give context and ownership Strong bias to actionable impactful insights, speed of iteration & feedback • Data Foundations: governed datasets, consistent shared datasets and metrics • Data Democratization: self serve data exploration platform • Enable innovation: environment supports speedy ad-hoc analysis Culture Process Tools Actionable insights lead to product feature Success Metric Definition Tracking Instrumentation spec Experimentation Setup and Analysis Post-Launch Analysis Build Measure Track Ship Tweak
- 17. Actionable insights lead toproduct feature Success Metric Definition Tracking Instrumentation spec Experimentation Setup and Analysis Post-Launch Analysis Build Measure Track Ship Tweak Democratize data – self served data exploration platform Enable people in your organization (execs, product managers, designers) to have data at their fingertips – to ask and answer questions
- 18. s • Invest increating the right metrics • user centric mindset; optimize for user value not team success all stakeholders agree upon success metrics prior to launching the feature test • Platform of shared tiered metrics visible to entire company • a metrics pipeline that enables easy implementation of metrics (and not manual one-offs) Actionable insights lead to product feature Success Metric Definition Tracking Instrumentation spec Experimentation Setup and Analysis Post-Launch Analysis Build Measure Track Ship Tweak Culture Process Tools
- 19. sActionable insights lead toproduct feature Success Metric Definition Tracking Instrumentation spec Experimentation Setup and Analysis Post-Launch Analysis Build Measure Track Ship Tweak
- 20. • Data asa first class citizen • Data tracking bugs as a launch blocker • Proactive joint definition of data requirements and contract (schemas) by producers and consumers • Data Model Review Committee • Data tracking spec tool for source of truth Automated testing of data tracking • Data Quality monitoring tool to ensure data contract is met Actionable insights lead to product feature Success Metric Definition Tracking Instrumentation spec Experimentation Setup and Analysis Post-Launch Analysis Build Measure Track Ship Tweak Culture Process Tools
- 21. s • Belief inproactive controlled experiment as decision making tool • Efficient and principled lifecycle, from inception to decision • Before: Review experiment design & implementation • After: stakeholders discuss impacts and implications of tradeoffs • Company-wide experimentation platform, with tiered key metrics • Automated metric reporting & analysis capability Actionable insights lead to product feature Success Metric Definition Tracking Instrumentation spec Experimentation Setup and Analysis Post-Launch Analysis Build Measure Track Ship Tweak Culture Process Tools
- 22. sActionable insights lead toproduct feature Success Metric Definition Tracking Instrumentation spec Experimentation Setup and Analysis Post-Launch Analysis Build Measure Track Ship Tweak
- 23. s • Iteration andinnovation • Metrics meeting: weekly to understand performance and product value • Combine qualitative user feedback with quantitative results • Long term hold-out groups for deep understanding of impacts Actionable insights lead to product feature Success Metric Definition Tracking Instrumentation spec Experimentation Setup and Analysis Post-Launch Analysis Build Measure Track Ship Tweak Culture Process Tools
- 24. This talk willbe a success if we: 1. Review the steps of data driven product innovation 2. Understand what is needed to best enable fostering of (or transformation into) a data-driven organization: culture, process, and tools.
- 25. 1. Culture: Createa culture which expects decisions are informed by data & which treats data as a first class citizen 2. Process: Consciously map how you use data (in each phase of the product lifecycle, in making exec decisions) 3. Tools: invest in your data ecosystem (data quality, pipeline, and access tools) To create a data driven organization
- 26.
Editor's Notes
- #6 To have a mental framework of a definition: Splits roughly into two types: Is your data scientist producing analytics for human or computer? Who is the consumer or decision maker? We’ll focus on analytics. So, what is the culture process and tools needed? such as: Product recommendations Opportunity identification Analyses Metrics such as: Recommendation systems Classifiers Predictive models
- #12 Lets say we go with some measure like the last. So we know what we want to measure. Are we logging the data in a way that we can compute this metric?
- #19 Invest in created more sophisticated measure of value created for the user or customer. If Profile team increases Profile edits (better matching), but it decreases connections made (which might impact growth, new user acquisition) what does that mean? Example - Total signups vs quality signups At LinkedIn we have true norths. And we have tiers. Total Follows vs follow that drives to more liquidity as seen by long term engagement, click through of content generated by those followees