Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Ewa Dominowska, Engineering Manager, Facebook at MLconf SEA - 5/01/15


Published on

Managing Machine Learning Projects in Industry: As the use of machine learning techniques to analyze and find value in ‘big data’ is being adopted more broadly by industry, we see an increasing need to build teams that can execute on large and complex projects. It is not possible for a single machine learning expert to cover problems of the scope and magnitude that are encountered. The scale of these projects requires teams of researchers and engineers to coordinate and collaborate to deliver impact. In this talk I will touch on some learnings and considerations when building or expanding such a team. I will cover building a group, framing the problem, finding a solution, and evaluating the results. I will illustrate the points with examples drawn from my experience in large companies and startups. I hope to provoke consideration and discussion for the challenges in this area, as well as to illustrate some of the complexities.

Published in: Technology
  • Be the first to comment

Ewa Dominowska, Engineering Manager, Facebook at MLconf SEA - 5/01/15

  1. 1. Managing Machine Learning Projects in Industry Ewa Dominowska Facebook, Engineering Manager
  2. 2. Agenda • Building a Team • Selecting and Framing a Problem • Problem Solving Approach • Evaluating Solutions • Delivering Impact
  3. 3. Building a Team • Engineer + ML Expert + Statistician + IR Expert • Domain expertise • Academic vs. industry experience • Research + engineering + experimentation = applied research • Investing (domain) vs. outsourcing (science)
  4. 4. Selecting a Lead ML Expert EngineerManager 1 8 8 8
  5. 5. Organizational Structure • Centralized Research Team • Centralized Applied Research Team • Embedded Researchers • Team Members • Academia • Conferences, competitions, data releases, benchmarks MSR, Facebook AI Lab LiveLabs, FB Applied ML Source: Bonkers World
  6. 6. Motivating • Intellectual challenge • Creative work • Autonomy • Purpose • Mastery • Recognition • Publishing • Conferences Source: Motivationhacker
  7. 7. Source: Oreilly Selecting and Framing a Problem
  8. 8. Selecting and Framing a Problem Start with a business problem Break down the problem Understand the impact Find the right data Select an objective function Build Models Measure and Evaluate Experiment Productionalize / Scale
  9. 9. Problem Solving Approach • Establish a baseline • Check your assumptions • Select a modellearning technique • Select features • Measure and evaluation • Experiment • Stability, scalability and robustness Source: Sheldoncomics
  10. 10. Evaluating Solutions • Defining the right metrics • Offline evaluation • A|B testing • Meaningful vs. representative • Representativeness and stability of results • Offline vs. online metrics
  11. 11. • How to split traffic • user, request, budget effects • How long to run a test • statistical significance, power, seasonality, novelty • Calibration • Model interactions • Residue effects from previous experiments Experimentation – Practical Lessons
  12. 12. Delivering Impact • Plan for valuable failure • Measure long term/steady state effects • Engineering improvements • Re-use of components, tools, models and frameworks • Durability and robustness • Data, context changes • Measurement, monitoring, experimentation
  13. 13. Thank you! We are hiring at Facebook!