Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building a Mature Analytics Workflow: The Analyst Collective Viewpoint


Published on

The past few years have seen massive changes in the analytics ecosystem. Problems we've been struggling with for years are essentially solved. But we're still not good at analytics. The workflow of most analysts today needs to adapt, and we're building the thinking and the tooling to let that happen.

Published in: Data & Analytics
  • Be the first to comment

Building a Mature Analytics Workflow: The Analyst Collective Viewpoint

  1. 1. Building a Mature Analytics Workflow Tristan Handy, RJMetrics
  2. 2. − What is the state of analytics today? − Our vision for the future − Bringing this vision to life − Conversation What are we going to talk about?
  3. 3. − Open source project building: ○ Analytic workflow tools ○ Reusable analysis − Focused on solving the problems we’re going to discuss today − Community-run, sponsored by RJMetrics, in progress for 3 months − Currently 5 active contributors But first...what is Analyst Collective?
  4. 4. The State of Analytics
  5. 5. So much good stuff happening!
  6. 6. Interactive query response time is low and requires less thought.1
  7. 7. You have many choices for ETL. Data warehouses no longer need to be expensive. 2
  8. 8. BI is converging around SQL.3
  9. 9. Visualization options abound. Many are free.4
  10. 10. There has never been a better time to be an analyst... but we still have huge problems to solve.
  11. 11. Most businesses don’t have a single source of truth.5
  12. 12. Most analytic code is produced with limited (if any) quality control.6
  13. 13. Analysts typically specialize in narrow domains leading to low bus factors.7
  14. 14. It still takes a long time to get an answer.8
  15. 15. We’ve solved some core technical challenges. We need to learn how to build analytical processes and teams on this tech.
  16. 16. Our Vision for Analytics
  17. 17. “ Analytics doesn’t have to be this way. In fact, the playbook for solving these problems already exists — on our software engineering teams.
  18. 18. “ In order to be managed like software, analytics must be done in code.
  19. 19. Core Belief #1 Analytics is collaborative.
  20. 20. Analyst Collective Viewpoint #1 Version Control All analytic code should be version controlled.
  21. 21. Analyst Collective Viewpoint #2 Quality Control Any code that generates data or analysis should be reviewed and tested.
  22. 22. Analyst Collective Viewpoint #3 Documentation Your code should come packaged with a basic description of how it should be interpreted.
  23. 23. Analyst Collective Viewpoint #4 Modularity Create tables, views, or other data sets that expose a consistent schema and can be modified if business logic changes.
  24. 24. Exploring modularity with Stripe Invoices Plans Subscriptions Mapping layer Raw data layer Invoices cleaned Invoices Plans Subscriptions Invoices transformed Query 1 Query 2 Query 3
  25. 25. Core Belief #2 Analytics code is an asset.
  26. 26. Analyst Collective Viewpoint #5 Maintainability Analytic code should be written to minimize long-term maintenance burden.
  27. 27. Analyst Collective Viewpoint #6 Environments Analytics requires multiple environments.
  28. 28. Analyst Collective Viewpoint #6 SLAs Production analytics should be consistently available and accurate.
  29. 29. Core Belief #3 Analytics requires automation.
  30. 30. Bringing This to Life
  31. 31. Thank you! Thoughts or questions? More information at: