Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

1,002 views

Published on

Several People Are Tuning: Data and Machine Learning at Slack: Slack has only been publicly available for a little over two years, but during that time we have been able to create one of the most interesting data sets in the world about how teams form, grow, and collaborate in order to get things done. Our next great challenge is to find ways to leverage what we’ve learned in order to help people find the information they need faster and to help large organizations work together more effectively.

Although our mission is broad, we are still at the very start of our journey; our machine learning team was only created six months ago, and our data team just six months before that. I would like to talk for a bit about what that journey has been like: how we hired our team, how we chose our tools and developed our initial data infrastructure and machine learning models, and how we’ve framed the product problems that we’re trying to solve.

Published in: Technology
  • Be the first to comment

Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

  1. 1. Several People Are Tuning: Data and ML at Slack
  2. 2. About Me ● Slack’s Head of Data Engineering ● Used to work at Cloudera, Google, other places ● Wrote popular tweet that I’m sort of tired of talking about ● Only owns one hat
  3. 3. Our Mission: To make people’s working lives simpler, more pleasant, and more productive.
  4. 4. Investing in Data (when you’re not an ad/fintech company)
  5. 5. Data Is A Portfolio Business
  6. 6. The Vision Thing
  7. 7. Org Stuff: Analytics, Data, SLI
  8. 8. In The Beginning...
  9. 9. Enter Data Engineering Prod MySQL (sharded), Solr, Redis Third Party Data Warehouse S3, Hive, Spark, Presto, MySQL Webapp (Apache, PHP) Sqooper Logs (via Kafka)
  10. 10. The Ghost City
  11. 11. Staffing Up
  12. 12. Sprucing Up The Place
  13. 13. The Nature of the Work
  14. 14. The Product Problem
  15. 15. “We Don’t Sell Saddles”
  16. 16. Building A Knowledge Base
  17. 17. Weird, Wonderful ML
  18. 18. Minimum Viable Data Products
  19. 19. Running Experiments
  20. 20. Channel Recommendations
  21. 21. Data Models for Serving
  22. 22. Giant Shoulders
  23. 23. Thanks! We’re hiring!

×