How Pinterest Scaled To Build The World's Catalog of 100+ Billion Ideas
•
2 likes•475 views
When a company goes through a hyper-growth phase, things move faster than you can control or even sometimes comprehend. This pushes teams to take shortcuts to facilitate faster data-driven decisions. If not prudently managed, this can result in a high “data debt,” which inhibits future growth opportunities. During Pinterest’s hypergrowth journey over the past six years, the data team made conscious decisions about technology, infrastructure, and processes to ensure that the data infrastructure scaled with the user growth, teams had the right tools to derive insights from the data, decision-making velocity was not impacted, and data debt was kept under control. Romit Jadhwani (BI Engineering Manager) deep dives into the different phases of Pinterest’s journey, focusing on questions asked, analytics and data solutions designed, and value added during each phase, offering a high-level blueprint for data teams to consider for their own hypergrowth journeys. This presentation was delivered at Strata + Hadoop World Conference in San Jose, CA on March 16, 2017.
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to How Pinterest Scaled To Build The World's Catalog of 100+ Billion Ideas
Similar to How Pinterest Scaled To Build The World's Catalog of 100+ Billion Ideas (20)
Recently uploaded
Recently uploaded (20)
How Pinterest Scaled To Build The World's Catalog of 100+ Billion Ideas
- 1. HowPinterest scaledtobuild theworld’s catalogof100+ billionideas March 16, 2017Strata + Hadoop WorldRomit Jadhwani
- 2. Business Intelligence Information Management Data Analytics Romit Jadhwani
- 13. DataDebt Yes,it’sarealthing …isaconsequenceof shortcutswe,datastewards, taketofacilitatefastdecision making • Inaccuratemetrics • Inconsistentdefinitions • Unavailableordelayeddata • Misalignedortoomanydashboards • Resourceintensive
- 19. Actors ? I am the ASKER. I ask questions to help make decisions.
- 20. Actors ? I am the ASKER. I ask questions to help make decisions. I am the ANALYST. I analyze the data to identify insights and trends.
- 21. Actors ? I am the ASKER. I ask questions to help make decisions. I am the ANALYST. I analyze the data to identify insights and trends. I am the ENGINEER. I build systems and tools.
- 23. Bigbang1 ? ASKER What is causing the spike in revenue?
- 24. Bigbang ? ASKER ANALYST 1 What is causing the spike in revenue? It’s the feature launch we did yesterday … I think
- 25. Bigbang ? ASKER ANALYST ENGINEER 1 What is causing the spike in revenue? Let’s check our dataIt’s the feature launch we did yesterday … I think
- 26. Bigbang What is happening? What or “Awareness” Why or “Understanding” Constant need for data availability Heroic effort by engineer/analyst … high value, high debt 1
- 27. Bigbang What is happening? What will help? What or “Awareness” Why or “Understanding” Constant need for data availability Heroic effort by engineer/analyst … high value, high debt KPIs Hire data engineer/s Be scrappy—launch and iterate 1
- 28. Bigbang What is happening? What will help? What or “Awareness” Why or “Understanding” Constant need for data availability Heroic effort by engineer/analyst … high value, high debt Diminishing returns! KPIs Hire data engineer/s Be scrappy—launch and iterate 1
- 34. ? Pedaltothemetal2 ASKER What is our most profitable customer segment?
- 35. We don’t have a good idea. How can we find out, Engineer? Pedaltothemetal2 ? ASKER ANALYST What is our most profitable customer segment?
- 36. We need to more data signals Pedaltothemetal2 ? ASKER ANALYST ENGINEER What is our most profitable customer segment? We don’t have a good idea. How can we find out, Engineer?
- 37. Pedaltothemetal What is happening? What or “Awareness” Why or “Understanding” How or “Process” Constant need for data breadth and depth Different “slices” of the company … still high value, still high debt 2
- 38. Pedaltothemetal What is happening? What will help? What or “Awareness” Why or “Understanding” How or “Process” Constant need for data breadth and depth Different “slices” of the company … still high value, still high debt Standards Data automation & infrastructure Wide vs deep focus 2
- 39. Pedaltothemetal What is happening? What will help? What or “Awareness” Why or “Understanding” How or “Process” Constant need for data breadth and depth Different “slices” of the company … still high value, still high debt Silos! Standards Data automation & infrastructure Wide vs deep focus 2
- 43. ? Payingitdown3 ASKER Why doesn’t my revenue match with what CFO presented?
- 44. The dashboards are owned by different teams ? Why doesn’t my revenue match with what CFO presented? Payingitdown3 ASKER ANALYST
- 45. #Facepalm Let’s fix this The dashboards are owned by different teams ? Payingitdown3 ASKER ANALYST ENGINEER Why doesn’t my revenue match with what CFO presented?
- 46. Payingitdown What is happening? What or “Awareness” Why or “Understanding” How or “Process” How well or “Efficiency” Constant need for data coverage “Slices” give way to the full pie 3
- 47. Payingitdown What is happening? What will help? What or “Awareness” Why or “Understanding” How or “Process” How well or “Efficiency” Constant need for data coverage “Slices” give way to the full pie Metric governance Golden data democracy Self service and up-skilling 3
- 48. Payingitdown What is happening? What will help? What or “Awareness” Why or “Understanding” How or “Process” How well or “Efficiency” Constant need for data coverage “Slices” give way to the full pie Entropy! Metric governance Golden data democracy Self service and up-skilling 3
- 50. Payingitdown3 Persistent Storage Real Time Consumers Events Logging Endpoint Persistent Storage CRM
- 51. Payingitdown3 Persistent Storage Real Time Consumers Events Logging Endpoint Persistent Storage CRM
- 52. Payingitdown3 Persistent Storage Real Time Consumers Events Logging Endpoint Persistent Storage CRM
- 53. Payingitdown Week over Week (WoW Change Breakdown) Component 1 Component 2 Component 3 Metric 1 Component 4 Metric 2 Component 5 KPI p % of Total Trending High Time Low Historical Trending High Time Low 3
- 55. KeyTakeaways
- 58. DataDebtisGood.
- 64. © Copyright, All Rights Reserved, Pinterest Inc. 2017