Sergey Karayev, profile picture
Uploaded bySergey Karayev
PDF, PPTX16,411 views

Lecture 13: ML Teams (Full Stack Deep Learning - Spring 2021)

The document discusses machine learning (ML) teams and roles. It provides an overview of common ML roles including ML product manager, DevOps engineer, data engineer, ML engineer, ML researcher/scientist, and data scientist. It describes the typical job functions, work products, and tools for each role. It also discusses how ML skills vary across roles and provides a visualization. Finally, it discusses different organizational structures for ML teams, including nascent/ad-hoc, ML R&D-focused, and embedded ML product structures.

Technology◦
Related topics:
Deep Learning•

Embed presentation

Download as PDF, PPTX
Full Stack Deep Learning - UC Berkeley Spring 2021 Week 8 Machine Learning Teams
Full Stack Deep Learning - UC Berkeley Spring 2021 Running ML teams is hard 2 ML Teams - overview Running any technical team is hard… • ML talent is expensive and scarce • ML teams have a diverse set of roles • Projects have unclear timelines and high uncertainty • The field is moving fast and ML is the “high- interest credit card of technical debt” • Leadership often doesn’t understand AI • Hiring great people • Managing and developing those people • Managing your team’s output and making sure your vectors are aligned • Making good long-term technical choices & managing technical debt • Managing expectations from leadership … And ML adds complexity
Full Stack Deep Learning - UC Berkeley Spring 2021 Goal of this module • Give you some insight into how to think about building and managing ML teams • Help you get a job in ML 3 ML Teams - overview
Full Stack Deep Learning - UC Berkeley Spring 2021 Managing Module overview 4 Hiring • How to manage a ML team • How to hire ML engineers. How to get hired. ML Teams - overview Orgs • How ML teams are organized and how they fit into the broader organization Roles • ML-related roles and the skills they require
Full Stack Deep Learning - UC Berkeley Spring 2021 Managing Module overview 5 Hiring • How to manage a ML team • How to hire ML engineers. How to get hired. Orgs • How ML teams are organized and how they fit into the broader organization Roles • ML-related roles and the skills they require ML Teams - roles
Full Stack Deep Learning - UC Berkeley Spring 2021 Most common ML roles • ML product manager • DevOps • Data engineer • ML engineer • ML researcher / ML scientist • Data scientist 6 ML Teams - roles
Full Stack Deep Learning - UC Berkeley Spring 2021 Most common ML roles • ML product manager • DevOps • Data engineer • ML engineer • ML researcher / ML scientist • Data scientist 7 What’s the difference? ML Teams - roles
Full Stack Deep Learning - UC Berkeley Spring 2021 Breakdown of job function by role 8 Role Job Function Work product Commonly used tools ML product manager Work with ML team, business, users, data owners to prioritize & execute projects Design docs, wireframes, work plans Jira, etc ML Teams - roles
Full Stack Deep Learning - UC Berkeley Spring 2021 Breakdown of job function by role 9 Role Job Function Work product Commonly used tools ML product manager Work with ML team, business, users, data owners to prioritize & execute projects Design docs, wireframes, work plans Jira, etc DevOps engineer Deploy & monitor production systems Deployed product AWS, etc. ML Teams - roles
Full Stack Deep Learning - UC Berkeley Spring 2021 Breakdown of job function by role 10 Role Job Function Work product Commonly used tools ML product manager Work with ML team, business, users, data owners to prioritize & execute projects Design docs, wireframes, work plans Jira, etc DevOps engineer Deploy & monitor production systems Deployed product AWS, etc. Data engineer Build data pipelines, aggregation, storage, monitoring Distributed system Hadoop, Kafka, Airflow ML Teams - roles
Full Stack Deep Learning - UC Berkeley Spring 2021 Breakdown of job function by role 11 Role Job Function Work product Commonly used tools ML product manager Work with ML team, business, users, data owners to prioritize & execute projects Design docs, wireframes, work plans Jira, etc DevOps engineer Deploy & monitor production systems Deployed product AWS, etc. Data engineer Build data pipelines, aggregation, storage, monitoring Distributed system Hadoop, Kafka, Airflow ML engineer Train & deploy prediction models Prediction system running on real data (often in production) Tensorflow, Docker ML Teams - roles
Full Stack Deep Learning - UC Berkeley Spring 2021 Breakdown of job function by role 12 Role Job Function Work product Commonly used tools ML product manager Work with ML team, business, users, data owners to prioritize & execute projects Design docs, wireframes, work plans Jira, etc DevOps engineer Deploy & monitor production systems Deployed product AWS, etc. Data engineer Build data pipelines, aggregation, storage, monitoring Distributed system Hadoop, Kafka, Airflow ML engineer Train & deploy prediction models Prediction system running on real data (often in production) Tensorflow, Docker ML researcher Train prediction models (often forward looking or not production-critical) Prediction model & report describing it Tensorflow, pytorch, Jupyter ML Teams - roles
Full Stack Deep Learning - UC Berkeley Spring 2021 Breakdown of job function by role 13 Role Job Function Work product Commonly used tools ML product manager Work with ML team, business, users, data owners to prioritize & execute projects Design docs, wireframes, work plans Jira, etc DevOps engineer Deploy & monitor production systems Deployed product AWS, etc. Data engineer Build data pipelines, aggregation, storage, monitoring Distributed system Hadoop, Kafka, Airflow ML engineer Train & deploy prediction models Prediction system running on real data (often in production) Tensorflow, Docker ML researcher Train prediction models (often forward looking or not production-critical) Prediction model & report describing it Tensorflow, pytorch, Jupyter Data scientist Blanket term used to describe all of the above. In some orgs, means answering business questions using analytics Prediction model or report SQL, Excel, Jupyter, Pandas, SKLearn, Tensorflow ML Teams - roles
Full Stack Deep Learning - UC Berkeley Spring 2021 What skills are needed for the roles? 14 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM Software engineering
Full Stack Deep Learning - UC Berkeley Spring 2021 What skills are needed for the roles? 15 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM Primarily a software engineering role. Often from standard SWE pipeline. Software engineering
Full Stack Deep Learning - UC Berkeley Spring 2021 What skills are needed for the roles? 16 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM SWE with ML team as an active customer Software engineering
Full Stack Deep Learning - UC Berkeley Spring 2021 What skills are needed for the roles? 17 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM Rare mix of ML skills and SWE skills. Often SWEs with significant self- teaching or science / engineering PhDs who worked as traditional SWEs after grad school Software engineering
Full Stack Deep Learning - UC Berkeley Spring 2021 What skills are needed for the roles? 18 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM ML experts. Usually have MS/PhD in CS or Stats or did an industrial fellowship program Software engineering
Full Stack Deep Learning - UC Berkeley Spring 2021 What skills are needed for the roles? 19 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM Wide range of backgrounds from undergrad-only to science PhD Software engineering
Full Stack Deep Learning - UC Berkeley Spring 2021 What skills are needed for the roles? 20 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM Traditional PMs, but with a deep understanding of the ML development process & mindset Software engineering
Full Stack Deep Learning - UC Berkeley Spring 2021 Questions? 21 ML Teams - roles
Full Stack Deep Learning - UC Berkeley Spring 2021 Managing Module overview 22 Hiring • How to manage a ML team • How to hire ML engineers. How to get hired. Orgs • How ML teams are organized and how they fit into the broader organization Roles • ML-related roles and the skills they require ML Teams - orgs
Full Stack Deep Learning - UC Berkeley Spring 2021 ML org structures - lessons learned • No consensus yet on the right way to structure a ML team • This lecture: taxonomy of best practices for different organizational maturity levels 23 ML Teams - orgs
Full Stack Deep Learning - UC Berkeley Spring 2021 ML organization archetypes 24 The ML Organization Mountain ML Teams - orgs
Full Stack Deep Learning - UC Berkeley Spring 2021 ML organization archetypes 25 The ML Organization Mountain Nascent / Ad-Hoc ML What it looks like • No one is doing ML, or ML is done on an ad-hoc basis • Little ML expertise in-house Example organizations • Most small-medium businesses • Less technology-forward large companies (education, logistics, etc) Advantages • Often low-hanging fruit for ML Dis- advantages • Little support for ML projects, difficult to hire and retain good talent ML Teams - orgs
Full Stack Deep Learning - UC Berkeley Spring 2021 ML R&D ML organization archetypes 26 What it looks like • ML efforts are centered in the R&D arm of the organization • Often hire researchers / PhDs & write papers Example organizations • Larger Oil & gas, manufacturing, telecom companies Advantages • Often can hire experienced researchers • Can work on long-term business priorities & big wins Dis- advantages • Difficult to get data • Rarely translates into actual business value, so usually the amount of investment remains small The ML Organization Mountain ML Teams - orgs
Full Stack Deep Learning - UC Berkeley Spring 2021 ML embedded into business / product teams ML organization archetypes 27 What it looks like • Certain product teams or business units have ML expertise along-side their software or analytics talent • ML reports up to the team’s engineering lead or tech lead Example organizations • Software / technology companies • Financial services companies Advantages • ML improvements are likely to lead to business value • Tight feedback cycle between idea and product improvement Dis- advantages • Hard to hire and develop top talent • Access to resources (data / compute) can lag • ML project cycles conflict with engineering mgmt • Long-term projects can be hard to justify The ML Organization Mountain ML Teams - orgs
Full Stack Deep Learning - UC Berkeley Spring 2021 Independent ML Function ML organization archetypes 28 What it looks like • ML division reporting to senior leadership (often CEO) • ML PMs work with MLRs, MLEs, and customers to build ML into products • Teams sometimes publish long-term research Example organizations • Large financial services companies Advantages • Talent density allows to hire & train top practitioners • Senior leaders can marshal data / compute resources • Can invest in tooling, practices, and culture around ML development Dis- advantages • Model handoffs to lines of business can be challenging - users need to buy-in and be educated on model use • Feedback cycles can be slow The ML Organization Mountain ML Teams - orgs
Full Stack Deep Learning - UC Berkeley Spring 2021 ML-First Organizations ML organization archetypes 29 What it looks like • CEO buy-in • ML division working on challenging, long-term projects • ML expertise in every line of business focusing on quick wins and working with central ML division Example organizations • Large tech companies • ML-focused startups Advantages • Best data access: data thinking permeates the org • Recruiting: ML team works on hardest problems • Easiest deployment: product teams understand ML Dis- advantages • Hard to implement • Challenging & expensive to recruit enough talent • Culturally difficult to embed ML thinking everywhere ML Teams - orgs
Full Stack Deep Learning - UC Berkeley Spring 2021 ML team structures - design choices 30 Software engineering vs research Data ownership Model ownership Key questions • To what extent is the ML team responsible for building or integrating with software? • How important are SWE skills on the team? • How much control does the ML team have over data collection, warehousing, labeling, and pipelining? • Is the ML team responsible for deploying models into production? • Who maintains deployed models? ML Teams - orgs
Full Stack Deep Learning - UC Berkeley Spring 2021 ML team structures - design choices 31 Software engineering vs research Data ownership Model ownership • Research prioritized over SWE skills • Researcher-SWE collaboration lacking ML R&D • ML team has no control over data • ML team typically will not have data engineering component • Models are rarely deployed into production ML Teams - orgs
Full Stack Deep Learning - UC Berkeley Spring 2021 ML team structures - design choices 32 Software engineering vs research Data ownership Model ownership Embedded ML • SWE skills prioritized over research skills • Often, all researchers need strong SWE as everyone expected to deploy • ML team generally does not own data production / mgmt • Work with data engineers to build pipelines • ML engineers own the models that they deploy into production ML Teams - orgs • Research prioritized over SWE skills • Researcher-SWE collaboration lacking ML R&D • ML team has no control over data • ML team typically will not have data engineering component • Models are rarely deployed into production
Full Stack Deep Learning - UC Berkeley Spring 2021 ML team structures - design choices 33 Software engineering vs research Data ownership Model ownership ML Function • Each team has a strong mix of SWE and research skills • SWE and researchers work closely together within team • ML team has a voice in data governance discussions • ML team has strong internal data engineering function • ML team hands off models to user, but is responsible for maintaining them ML Teams - orgs Embedded ML • Research prioritized over SWE skills • Researcher-SWE collaboration lacking ML R&D • ML team has no control over data • ML team typically will not have data engineering component • Models are rarely deployed into production • SWE skills prioritized over research skills • Often, all researchers need strong SWE as everyone expected to deploy • ML team generally does not own data production / mgmt • Work with data engineers to build pipelines • ML engineers own the models that they deploy into production
Full Stack Deep Learning - UC Berkeley Spring 2021 ML team structures - design choices 34 Software engineering vs research Data ownership Model ownership Embedded ML • Research prioritized over SWE skills • Researcher-SWE collaboration lacking ML R&D ML Function ML First • ML team has no control over data • ML team typically will not have data engineering component • Models are rarely deployed into production • SWE skills prioritized over research skills • Often, all researchers need strong SWE as everyone expected to deploy • Each team has a strong mix of SWE and research skills • SWE and researchers work closely together within team • Different teams are more or less research oriented • Research teams collaborate closely with SWE teams • ML team generally does not own data production / mgmt • Work with data engineers to build pipelines • ML engineers own the models that they deploy into production • ML team has a voice in data governance discussions • ML team has strong internal data engineering function • ML team hands off models to user, but is responsible for maintaining them • ML team often owns company-wide data infrastructure • ML team hands off models to user, who operates and maintains them ML Teams - orgs
Full Stack Deep Learning - UC Berkeley Spring 2021 ML team structures - design choices 35 Software engineering vs research Data ownership Model ownership Embedded ML • Research prioritized over SWE skills • Researcher-SWE collaboration lacking ML R&D ML Function ML First • ML team has no control over data • ML team typically will not have data engineering component • Models are rarely deployed into production • SWE skills prioritized over research skills • Often, all researchers need strong SWE as everyone expected to deploy • Each team has a strong mix of SWE and research skills • SWE and researchers work closely together within team • Different teams are more or less research oriented • Research teams collaborate closely with SWE teams • ML team generally does not own data production / mgmt • Work with data engineers to build pipelines • ML engineers own the models that they deploy into production • ML team has a voice in data governance discussions • ML team has strong internal data engineering function • ML team hands off models to user, but is responsible for maintaining them • ML team often owns company-wide data infrastructure • ML team hands off models to user, who operates and maintains them ML Teams - orgs
Full Stack Deep Learning - UC Berkeley Spring 2021 Questions? 36 ML Teams - orgs
Full Stack Deep Learning - UC Berkeley Spring 2021 Managing Module overview 37 Hiring • How to manage a ML team • How to hire ML engineers. How to get hired. Orgs • How ML teams are organized and how they fit into the broader organization Roles • ML-related roles and the skills they require ML Teams - managing
Full Stack Deep Learning - UC Berkeley Spring 2021 Managing ML teams is challenging • It’s hard to tell in advance how hard or easy something is 38 ML Teams - managing
Full Stack Deep Learning - UC Berkeley Spring 2021 Managing ML teams is challenging 39 https://medium.com/@l2k/why-are-machine-learning-projects-so-hard-to-manage-8e9b9cf49641 It’s hard to tell in advance how easy or hard something is ML Teams - managing
Full Stack Deep Learning - UC Berkeley Spring 2021 Managing ML teams is challenging • It’s hard to tell in advance how hard or easy something is • ML progress is nonlinear • Very common for projects to stall for weeks or longer • In early stages, difficult to plan project because unclear what will work • As a result, estimating project timelines is extremely difficult • I.e., production ML is still somewhere between “research” and “engineering” 40 ML Teams - managing
Full Stack Deep Learning - UC Berkeley Spring 2021 Managing ML teams is challenging • It’s hard to tell in advance how hard or easy something is • ML progress is nonlinear • There are cultural gaps between research and engineering • Different values, backgrounds, goals, norms • In toxic cultures, the two sides often don’t value one another 41 ML Teams - managing
Full Stack Deep Learning - UC Berkeley Spring 2021 Managing ML teams is challenging • It’s hard to tell in advance how hard or easy something is • ML progress is nonlinear • There are cultural gaps between research and engineering • Leaders often don’t understand it 42 ML Teams - managing
Full Stack Deep Learning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML Project planning probabilistically 43 ML Teams - managing
Full Stack Deep Learning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML project planning probabilistically • From:
 
 
 
 
 • To:
 
 44 ML Teams - managing Task A Task C Task D Task E Task F Task G Week 1 Week 2 Week 3 Week 4
Full Stack Deep Learning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML project planning probabilistically • From:
 
 
 
 
 • To:
 
 45 ML Teams - managing Task A Task C Task D Task E Task F Task G Week 1 Week 2 Week 3 Week 4 Task A (50%) Task B (25%) Task C (50%) Week 1 Week 2 Week 3 Week 4 Task D (75%)
Full Stack Deep Learning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML project planning probabilistically • From:
 
 
 
 
 • To:
 
 46 ML Teams - managing Task A Task C Task D Task E Task F Task G Week 1 Week 2 Week 3 Week 4 Task A (50%) Task B (25%) Task C (50%) Week 1 Week 2 Week 3 Week 4 Task D (75%)
Full Stack Deep Learning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML project planning probabilistically • From:
 
 
 
 
 • To:
 
 47 ML Teams - managing Task A Task C Task D Task E Task F Task G Week 1 Week 2 Week 3 Week 4 Task A Task B Task C Week 1 Week 2 Week 3 Week 4 Task D
Full Stack Deep Learning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML project planning probabilistically • From:
 
 
 
 
 • To:
 
 48 ML Teams - managing Task A Task C Task D Task E Task F Task G Week 1 Week 2 Week 3 Week 4 Task A Task B Task C Week 1 Week 2 Week 3 Week 4 Task E (50%) Task F (50%) Task G (10%) Task D (75%)
Full Stack Deep Learning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML Project planning probabilistically • Attempt a portfolio of approaches • Measure progress based on inputs, not results • Have researchers and engineers work together • Get end-to-end pipelines together quickly to demonstrate quick wins • Educate leadership on ML timeline uncertainty 49 ML Teams - managing
Full Stack Deep Learning - UC Berkeley Spring 2021 Resources for educating execs • https://a16z.com/2016/06/10/ai-deep-learning-machines/ • Pieter’s upcoming AI Strategy class:
 https://emeritus-executive.berkeley.edu/artificial-intelligence/ 50 ML Teams - managing
Full Stack Deep Learning - UC Berkeley Spring 2021 Questions? 51 ML Teams - managing
Full Stack Deep Learning - UC Berkeley Spring 2021 Managing Module overview 52 Hiring • How to manage a ML team • How to hire ML engineers. How to get hired. Orgs • How ML teams are organized and how they fit into the broader organization Roles • ML-related roles and the skills they require ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 Hiring for ML - outline • The AI Talent Gap • Sourcing • Interviewing • Finding a job 53 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 Hiring for ML - outline • The AI Talent Gap • Sourcing • Interviewing • Finding a job 54 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 The AI Talent Gap 55 How many people know how to build AI systems? 5,000 (actively publishing research [Element AI]) 10,000 (estimated num people with the right skillset [Element AI]) 22,000 (PhD-educated AI researchers [Bloomberg]) 90,000 (upper bound on number of people [Element AI]) 200,000 - 300,000 (Number of AI researcher / practitioners [Tencent]) 3.6M (Number of software developers in the US) 18.2M (Number of software developers in the world) Sources: The AI Talent Shortage (Nikolai Yakovenko) https://medium.com/@Moscow25/the-ai-talent-shortage-704d8cf0c4cc Just How Shallow is the Artificial Intelligence Talent Pool (Jeremy Kahn) 
 https://www.bloomberg.com/news/articles/2018-02-07/just-how-shallow-is-the-artificial-intelligence-talent-pool ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 56 The AI talent gap Fierce competition for AI talent “Everyone agrees that the competition to hire people who know how to build artificial intelligence systems is intense. It’s  turned once- staid academic conferences into frenzied meet markets for corporate recruiters and driven the salaries of the top researchers to seven- figures.” 
 (Bloomberg) Sources: The AI Talent Shortage (Nikolai Yakovenko) https://medium.com/@Moscow25/the-ai-talent-shortage-704d8cf0c4cc Just How Shallow is the Artificial Intelligence Talent Pool (Jeremy Kahn) 
 https://www.bloomberg.com/news/articles/2018-02-07/just-how-shallow-is-the-artificial-intelligence-talent-pool ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 57 The AI talent gap Fierce competition for AI talent “Hiring is crazy right now. ML is a young field that got popular very quickly. There’s a ton of demand and not a lot of supply.” 
 (Computer Vision Engineer at Series C startup) ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 58 The AI talent gap Fierce competition for AI talent “Hiring for ML is really challenging and takes way more time and effort than we expected. We have someone working on it full-time and we’re still only able to get a few people per quarter” (Startup Founder) ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 Hiring for ML - outline • The AI Talent Gap • Sourcing • Interviewing • Finding a job 59 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 Most common ML roles • ML product manager • DevOps • Data engineer • ML engineer • ML researcher / ML scientist • Data scientist 60 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 Most common ML roles • ML product manager • DevOps • Data engineer • ML engineer • ML researcher / ML scientist • Data scientist 61 Slightly different mindset required. 
 Helpful to look for demonstrated interest in AI - courses, conferences, re-implementations, etc ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 Most common ML roles • ML product manager • DevOps • Data engineer • ML engineer • ML researcher / ML scientist • Data scientist 62 Our focus ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 How to hire MLEs - the wrong way • Job Description (Unicorn Machine Learning Engineer) • Duties • Keep up with the state of the art • Implement models from scratch • Deep understanding of mathematics & ability to come up with new models • Build tooling & infrastructure for the ML team • Build data pipelines for the ML team • Deploy & monitor models into production • Requirements • PhD • At least 4 years tensorflow experience • At least 4 years as a software engineer • Publications in top ML conference • Experience building large-scale distributed systems 63 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 How to hire MLEs - the right way • Hire for software engineering skills, interest in ML, and desire to learn. Train to do ML. • Go more junior. Most undergrad computer science students graduate with ML experience. • Be more specific about what you need. Not every ML engineer needs to do DevOps. 64 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 How to hire MLRs • Look for quality of publications, not quantity (e.g., originality of ideas, quality of execution) • Look for researchers with an eye for working on important problems (many researchers focus on trendy problems without considering why they matter) • Look for researchers with experience outside of academia • Consider hiring talented people from adjacent fields (physics, statistics, math) • Consider hiring people without PhDs (e.g., talented undergraduate / masters students, graduates of Google/Facebook/OpenAI fellowship programs, dedicated self-studiers) 65 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 How to find MLE/MLR candidates • Standard sources: LinkedIn, recruiters, on-campus recruiting, etc • Monitor arXiv and top conferences and flag first authors of papers you like • Look for good reimplementations of papers you like • Attend ML research conferences (NeurIPS, ICLR, ICML) 66 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 How to attract MLR / MLE candidates 67 What do machine learning practitioners want? How to make your company stand out? • Work with cutting edge tools & techniques • Build skills / knowledge in an exciting field • Work with excellent people • Work on interesting datasets • Do work that matters • Work on research-oriented projects. Publicize them. Invest in tooling for your team & empower employees to try new tools. • Build team culture around learning (reading groups, learning days, professional development budget, conference budget) • Hire high-profile people. Help your best people build their profile through publishing blogs & papers. • Sell the uniqueness of your dataset in recruiting materials. • Sell the mission of your company and potential impact of machine learning on that mission. Work on projects that have a tangible impact today. ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 Hiring for ML - outline • The AI Talent Gap • Sourcing • Interviewing • Finding a job 68 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 What to test in an ML interview? • Hire for strengths • Meet a minimum bar for everything else 69 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 What to test in an ML interview? • Validate your hypotheses of candidate’s strengths • Researchers: make sure they can think creatively about new ML problems, probe how thoughtful they were about previous projects • Engineers: make sure they are great generalist SWEs • Make sure candidates meet a minimum bar on weaker areas • Researchers: test SWE knowledge and ability to write good code • SWEs: test ML knowledge 70 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 What happens in a ML interview? • Much less well-defined than software engineering interviews • Common types of assessments: • Background & culture fit • Whiteboard coding (similar to SWE interviews) • Pair coding (similar to SWE interviews) • Pair debugging (often ML-specific code) • Math puzzles (e.g., involving linear algebra) • Take-home ML project • Applied ML (e.g., explain how you’d solve this problem with ML) • Previous ML projects (e.g., probing on what you tried, why things did / didn’t work) • ML theory (e.g., bias-variance tradeoff, overfitting, underfitting, understanding of specific algorithms) 71 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 Hiring for ML - outline • The AI Talent Gap • Sourcing • Interviewing • Finding a job 72 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 Where to look for a ML job? • Standard sources: LinkedIn, recruiters, on-campus recruiting, etc • ML research conferences (NeurIPS, ICLR, ICML) • Apply directly (remember, there’s a talent gap!) • This course • Those who pass the exam will get access to our recruiting database 73 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 How to stand out for ML roles? • Build software engineering skills (e.g., work at a well-known software company) • Exhibit interest in ML (e.g., conference attendance, online courses taken) • Show you have broad knowledge of ML (e.g., write blog posts synthesizing a research area) • Demonstrate ability to get ML projects done (e.g., create side projects, re- implement papers) • Prove you can think creatively in ML (e.g., win Kaggle competitions, publish papers) 74 More impressive ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 How to prepare for the interview? • Prepare for a general SWE interview (e.g., “Cracking the Coding Interview”) • Prepare to talk in detail about your past ML projects (remember details, prepare to talk about tradeoffs and decisions you made) • Review how basic ML algorithms work (linear / logistic regression, nearest neighbor, decision trees, k-means, MLPs, ConvNets, recurrent nets, etc) • Review ML theory • Think about the problems the company you’re interviewing with may face and what ML techniques may apply to them 75 ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 Conclusion 76 Orgs • ML teams are becoming more standalone, hence more interdisciplinary Roles • Lots of different skills involved in production ML, so there’s an opportunity for many to contribute Hiring • Talent is scarce, so be specific about what is must-have. It can be hard to break in as an outsider - use projects to build awareness. Managing • Managing ML teams is hard. There’s no silver bullet, but shifting toward probabilistic planning can help ML Teams - hiring
Full Stack Deep Learning - UC Berkeley Spring 2021 Thank you! 77

More Related Content

PDF
Data Management - Full Stack Deep Learning
bySergey Karayev
 
PDF
Lecture 9: AI Ethics (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
PDF
Lecture 5: ML Projects (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
PDF
Machine Learning Teams - Full Stack Deep Learning
bySergey Karayev
 
PDF
Troubleshooting Deep Neural Networks - Full Stack Deep Learning
bySergey Karayev
 
PDF
Testing and Deployment - Full Stack Deep Learning
bySergey Karayev
 
PDF
Lecture 10: ML Testing & Explainability (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
PDF
Lecture 7: Troubleshooting Deep Neural Networks (Full Stack Deep Learning - S...
bySergey Karayev
 
Data Management - Full Stack Deep Learning
bySergey Karayev
 
Lecture 9: AI Ethics (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
Lecture 5: ML Projects (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
Machine Learning Teams - Full Stack Deep Learning
bySergey Karayev
 
Troubleshooting Deep Neural Networks - Full Stack Deep Learning
bySergey Karayev
 
Testing and Deployment - Full Stack Deep Learning
bySergey Karayev
 
Lecture 10: ML Testing & Explainability (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
Lecture 7: Troubleshooting Deep Neural Networks (Full Stack Deep Learning - S...
bySergey Karayev
 

What's hot

PDF
Infrastructure and Tooling - Full Stack Deep Learning
bySergey Karayev
 
PDF
Lecture 8: Data Management (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
PDF
Setting up Machine Learning Projects - Full Stack Deep Learning
bySergey Karayev
 
PDF
Lecture 1: Deep Learning Fundamentals - Full Stack Deep Learning - Spring 2021
bySergey Karayev
 
PDF
Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
PDF
On Impact in Software Engineering Research (HU Berlin 2021)
byCISPA Helmholtz Center for Information Security
 
PDF
On impact in Software Engineering Research (ICSE 2018 New Faculty Symposium)
byCISPA Helmholtz Center for Information Security
 
PDF
Lecture 3: RNNs - Full Stack Deep Learning - Spring 2021
bySergey Karayev
 
PDF
On Impact in Software Engineering Research (Dagstuhl 2020)
byCISPA Helmholtz Center for Information Security
 
PDF
On Impact in Software Engineering Research
byCISPA Helmholtz Center for Information Security
 
PPTX
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...
byACM Chicago
 
PDF
ISEC'18 Keynote: Intelligent Software Engineering: Synergy between AI and Sof...
byTao Xie
 
PDF
Data Workflows for Machine Learning - Seattle DAML
byPaco Nathan
 
PDF
MSRA 2018: Intelligent Software Engineering: Synergy between AI and Software ...
byTao Xie
 
PDF
AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...
byBill Liu
 
PPTX
GALE: Geometric active learning for Search-Based Software Engineering
byCS, NcState
 
PPTX
Intelligent Software Engineering: Synergy between AI and Software Engineering
byTao Xie
 
PPTX
Ai use cases
bySparsh Agarwal
 
PDF
Data ops: Machine Learning in production
byStepan Pushkarev
 
PDF
Software Mining and Software Datasets
byTao Xie
 
Infrastructure and Tooling - Full Stack Deep Learning
bySergey Karayev
 
Lecture 8: Data Management (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
Setting up Machine Learning Projects - Full Stack Deep Learning
bySergey Karayev
 
Lecture 1: Deep Learning Fundamentals - Full Stack Deep Learning - Spring 2021
bySergey Karayev
 
Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
On Impact in Software Engineering Research (HU Berlin 2021)
byCISPA Helmholtz Center for Information Security
 
On impact in Software Engineering Research (ICSE 2018 New Faculty Symposium)
byCISPA Helmholtz Center for Information Security
 
Lecture 3: RNNs - Full Stack Deep Learning - Spring 2021
bySergey Karayev
 
On Impact in Software Engineering Research (Dagstuhl 2020)
byCISPA Helmholtz Center for Information Security
 
On Impact in Software Engineering Research
byCISPA Helmholtz Center for Information Security
 
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...
byACM Chicago
 
ISEC'18 Keynote: Intelligent Software Engineering: Synergy between AI and Sof...
byTao Xie
 
Data Workflows for Machine Learning - Seattle DAML
byPaco Nathan
 
MSRA 2018: Intelligent Software Engineering: Synergy between AI and Software ...
byTao Xie
 
AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...
byBill Liu
 
GALE: Geometric active learning for Search-Based Software Engineering
byCS, NcState
 
Intelligent Software Engineering: Synergy between AI and Software Engineering
byTao Xie
 
Ai use cases
bySparsh Agarwal
 
Data ops: Machine Learning in production
byStepan Pushkarev
 
Software Mining and Software Datasets
byTao Xie
 

Similar to Lecture 13: ML Teams (Full Stack Deep Learning - Spring 2021)

PPTX
From Notebook to Production: What Most ML Tutorials Don’t Teach
byvivekbharti311
 
PDF
Introducing MLOps.pdf
byDr. Anish Cheriyan (PhD)
 
PPTX
MLOps Training Online | MLOps Course in Hyderabad
byranjithvisualpath44
 
PDF
Integrating MLOps into Your Data Science Workflow.pdf
bysakethv1308
 
PDF
Lecture 11: ML Deployment & Monitoring (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
PDF
CI/CD for Machine Learning
byC4Media
 
PDF
DevOps Days Rockies MLOps
byMatthew Reynolds
 
PDF
Real World End to End machine Learning Pipeline
bySrivatsan Srinivasan
 
PDF
AIML Learning Path Cheat Sheet Essential Tools & Frameworks to Crack Your Int...
byTapp AI
 
PDF
Key MLOps (Mahine Learning Operational) Features
byAgus Kurniawan
 
PPTX
Expert Machine Learning Solutions for Smarter Business
byDigital Hub Solution
 
PDF
Mykola Mykytenko: MLOps: your way from nonsense to valuable effect (approache...
byLviv Startup Club
 
PDF
MLSEV Virtual. ML Platformization and AutoML in the Enterprise
byBigML, Inc
 
PDF
Using MLOps to Bring ML to Production/The Promise of MLOps
byWeaveworks
 
PDF
MLOps – Applying DevOps to Competitive Advantage
byDATAVERSITY
 
PPTX
Expert ML Development Services | AI Solutions for Your Business
byDigital Hub Solution
 
PDF
DutchMLSchool 2022 - Automation
byBigML, Inc
 
PPTX
AI_ML_Career_Orientation_Presentation.pptx
byBipradipChakraborty
 
PDF
Machine Learning Platformization & AutoML: Adopting ML at Scale in the Enterp...
byEd Fernandez
 
PDF
Building Your Dream Machine Learning Team with Python Expertise
byriyak40
 
From Notebook to Production: What Most ML Tutorials Don’t Teach
byvivekbharti311
 
Introducing MLOps.pdf
byDr. Anish Cheriyan (PhD)
 
MLOps Training Online | MLOps Course in Hyderabad
byranjithvisualpath44
 
Integrating MLOps into Your Data Science Workflow.pdf
bysakethv1308
 
Lecture 11: ML Deployment & Monitoring (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
CI/CD for Machine Learning
byC4Media
 
DevOps Days Rockies MLOps
byMatthew Reynolds
 
Real World End to End machine Learning Pipeline
bySrivatsan Srinivasan
 
AIML Learning Path Cheat Sheet Essential Tools & Frameworks to Crack Your Int...
byTapp AI
 
Key MLOps (Mahine Learning Operational) Features
byAgus Kurniawan
 
Expert Machine Learning Solutions for Smarter Business
byDigital Hub Solution
 
Mykola Mykytenko: MLOps: your way from nonsense to valuable effect (approache...
byLviv Startup Club
 
MLSEV Virtual. ML Platformization and AutoML in the Enterprise
byBigML, Inc
 
Using MLOps to Bring ML to Production/The Promise of MLOps
byWeaveworks
 
MLOps – Applying DevOps to Competitive Advantage
byDATAVERSITY
 
Expert ML Development Services | AI Solutions for Your Business
byDigital Hub Solution
 
DutchMLSchool 2022 - Automation
byBigML, Inc
 
AI_ML_Career_Orientation_Presentation.pptx
byBipradipChakraborty
 
Machine Learning Platformization & AutoML: Adopting ML at Scale in the Enterp...
byEd Fernandez
 
Building Your Dream Machine Learning Team with Python Expertise
byriyak40
 

More from Sergey Karayev

PDF
Lecture 12: Research Directions (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
PDF
Lecture 6: Infrastructure & Tooling (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
PDF
Lecture 2.B: Computer Vision Applications - Full Stack Deep Learning - Spring...
bySergey Karayev
 
PDF
Lecture 2.A: Convolutional Networks - Full Stack Deep Learning - Spring 2021
bySergey Karayev
 
PDF
Lab 1: Intro and Setup - Full Stack Deep Learning - Spring 2021
bySergey Karayev
 
PDF
Research Directions - Full Stack Deep Learning
bySergey Karayev
 
PDF
AI Masterclass at ASU GSV 2019
bySergey Karayev
 
KEY
Attentional Object Detection - introductory slides.
bySergey Karayev
 
Lecture 12: Research Directions (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
Lecture 6: Infrastructure & Tooling (Full Stack Deep Learning - Spring 2021)
bySergey Karayev
 
Lecture 2.B: Computer Vision Applications - Full Stack Deep Learning - Spring...
bySergey Karayev
 
Lecture 2.A: Convolutional Networks - Full Stack Deep Learning - Spring 2021
bySergey Karayev
 
Lab 1: Intro and Setup - Full Stack Deep Learning - Spring 2021
bySergey Karayev
 
Research Directions - Full Stack Deep Learning
bySergey Karayev
 
AI Masterclass at ASU GSV 2019
bySergey Karayev
 
Attentional Object Detection - introductory slides.
bySergey Karayev
 

Recently uploaded

PDF
Building Voice Agents with Google Agent Development Kit
bySuresh Peiris
 
PPTX
Becoming Frontier for Dynamics 365 Finance PPT
byjonbravetti
 
PDF
Why Enterprises Are Moving to Dedicated Servers in the Netherlands
byAtal Networks
 
PDF
Chapter 2 Computer Threat .pdf
byGetnet Tigabie Askale -(GM)
 
PDF
Implementing A Data Lakehouse Using Iceberg & Spark
byAnass Nabil
 
PPTX
UNIT 1- Introduction to Basic of Computer Fundamentals
bypawarbhaktiit
 
PDF
Adding Copilot+ PCs with Snapdragon to your HP fleet won’t require IT deploym...
byPrincipled Technologies
 
PDF
Profiting from Technological Innovation: Managing Intellectual Property to Bu...
byDavid Teece
 
PPT
connectives-linking words in English.ppt
byAmrMohammed82
 
PPTX
Fortify Your Digital Defenses with ServiceNow Security Operations by Suma Soft
byGavin Ellis
 
PDF
Hart, Inc. M&A Data Transition Checklist
byHart, Inc.
 
PPTX
Microsoft Azure News - January 2026 - BAUG
byDaniel Toomey
 
PDF
TopMate ES11 3-Wheel Electric Scooter: Comfort, Stability, and Independence f...
byTopmate
 
PDF
Are AI Hallucinations Getting Better or Worse? We Analyzed the Data.
byScott M. Graffius
 
PDF
UiPath Coded Agents - A Developer Driven Agentic Automation Era
byUiPathCommunity
 
PDF
How Automation Powers Data Quality and Governance at Scale
bySafe Software
 
PPTX
Design Flaws and Known Attacks in Agentic AI - ITI Business Session
byInformation Technology Institute
 
PPTX
Webinar: EVerest for Production: Accelerating EV Charger Development w/ Indus...
byDanBrown980551
 
PDF
One String, Many Prompts: AI Code Generation
byGeoffrey Goetz
 
PDF
Mercury Computer Systems & The Cell Broadband Engine
bySlide_N
 
Building Voice Agents with Google Agent Development Kit
bySuresh Peiris
 
Becoming Frontier for Dynamics 365 Finance PPT
byjonbravetti
 
Why Enterprises Are Moving to Dedicated Servers in the Netherlands
byAtal Networks
 
Chapter 2 Computer Threat .pdf
byGetnet Tigabie Askale -(GM)
 
Implementing A Data Lakehouse Using Iceberg & Spark
byAnass Nabil
 
UNIT 1- Introduction to Basic of Computer Fundamentals
bypawarbhaktiit
 
Adding Copilot+ PCs with Snapdragon to your HP fleet won’t require IT deploym...
byPrincipled Technologies
 
Profiting from Technological Innovation: Managing Intellectual Property to Bu...
byDavid Teece
 
connectives-linking words in English.ppt
byAmrMohammed82
 
Fortify Your Digital Defenses with ServiceNow Security Operations by Suma Soft
byGavin Ellis
 
Hart, Inc. M&A Data Transition Checklist
byHart, Inc.
 
Microsoft Azure News - January 2026 - BAUG
byDaniel Toomey
 
TopMate ES11 3-Wheel Electric Scooter: Comfort, Stability, and Independence f...
byTopmate
 
Are AI Hallucinations Getting Better or Worse? We Analyzed the Data.
byScott M. Graffius
 
UiPath Coded Agents - A Developer Driven Agentic Automation Era
byUiPathCommunity
 
How Automation Powers Data Quality and Governance at Scale
bySafe Software
 
Design Flaws and Known Attacks in Agentic AI - ITI Business Session
byInformation Technology Institute
 
Webinar: EVerest for Production: Accelerating EV Charger Development w/ Indus...
byDanBrown980551
 
One String, Many Prompts: AI Code Generation
byGeoffrey Goetz
 
Mercury Computer Systems & The Cell Broadband Engine
bySlide_N
 

Lecture 13: ML Teams (Full Stack Deep Learning - Spring 2021)

  • 1.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Week 8 Machine Learning Teams
  • 2.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Running ML teams is hard 2 ML Teams - overview Running any technical team is hard… • ML talent is expensive and scarce • ML teams have a diverse set of roles • Projects have unclear timelines and high uncertainty • The field is moving fast and ML is the “high- interest credit card of technical debt” • Leadership often doesn’t understand AI • Hiring great people • Managing and developing those people • Managing your team’s output and making sure your vectors are aligned • Making good long-term technical choices & managing technical debt • Managing expectations from leadership … And ML adds complexity
  • 3.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Goal of this module • Give you some insight into how to think about building and managing ML teams • Help you get a job in ML 3 ML Teams - overview
  • 4.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Managing Module overview 4 Hiring • How to manage a ML team • How to hire ML engineers. How to get hired. ML Teams - overview Orgs • How ML teams are organized and how they fit into the broader organization Roles • ML-related roles and the skills they require
  • 5.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Managing Module overview 5 Hiring • How to manage a ML team • How to hire ML engineers. How to get hired. Orgs • How ML teams are organized and how they fit into the broader organization Roles • ML-related roles and the skills they require ML Teams - roles
  • 6.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Most common ML roles • ML product manager • DevOps • Data engineer • ML engineer • ML researcher / ML scientist • Data scientist 6 ML Teams - roles
  • 7.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Most common ML roles • ML product manager • DevOps • Data engineer • ML engineer • ML researcher / ML scientist • Data scientist 7 What’s the difference? ML Teams - roles
  • 8.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Breakdown of job function by role 8 Role Job Function Work product Commonly used tools ML product manager Work with ML team, business, users, data owners to prioritize & execute projects Design docs, wireframes, work plans Jira, etc ML Teams - roles
  • 9.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Breakdown of job function by role 9 Role Job Function Work product Commonly used tools ML product manager Work with ML team, business, users, data owners to prioritize & execute projects Design docs, wireframes, work plans Jira, etc DevOps engineer Deploy & monitor production systems Deployed product AWS, etc. ML Teams - roles
  • 10.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Breakdown of job function by role 10 Role Job Function Work product Commonly used tools ML product manager Work with ML team, business, users, data owners to prioritize & execute projects Design docs, wireframes, work plans Jira, etc DevOps engineer Deploy & monitor production systems Deployed product AWS, etc. Data engineer Build data pipelines, aggregation, storage, monitoring Distributed system Hadoop, Kafka, Airflow ML Teams - roles
  • 11.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Breakdown of job function by role 11 Role Job Function Work product Commonly used tools ML product manager Work with ML team, business, users, data owners to prioritize & execute projects Design docs, wireframes, work plans Jira, etc DevOps engineer Deploy & monitor production systems Deployed product AWS, etc. Data engineer Build data pipelines, aggregation, storage, monitoring Distributed system Hadoop, Kafka, Airflow ML engineer Train & deploy prediction models Prediction system running on real data (often in production) Tensorflow, Docker ML Teams - roles
  • 12.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Breakdown of job function by role 12 Role Job Function Work product Commonly used tools ML product manager Work with ML team, business, users, data owners to prioritize & execute projects Design docs, wireframes, work plans Jira, etc DevOps engineer Deploy & monitor production systems Deployed product AWS, etc. Data engineer Build data pipelines, aggregation, storage, monitoring Distributed system Hadoop, Kafka, Airflow ML engineer Train & deploy prediction models Prediction system running on real data (often in production) Tensorflow, Docker ML researcher Train prediction models (often forward looking or not production-critical) Prediction model & report describing it Tensorflow, pytorch, Jupyter ML Teams - roles
  • 13.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Breakdown of job function by role 13 Role Job Function Work product Commonly used tools ML product manager Work with ML team, business, users, data owners to prioritize & execute projects Design docs, wireframes, work plans Jira, etc DevOps engineer Deploy & monitor production systems Deployed product AWS, etc. Data engineer Build data pipelines, aggregation, storage, monitoring Distributed system Hadoop, Kafka, Airflow ML engineer Train & deploy prediction models Prediction system running on real data (often in production) Tensorflow, Docker ML researcher Train prediction models (often forward looking or not production-critical) Prediction model & report describing it Tensorflow, pytorch, Jupyter Data scientist Blanket term used to describe all of the above. In some orgs, means answering business questions using analytics Prediction model or report SQL, Excel, Jupyter, Pandas, SKLearn, Tensorflow ML Teams - roles
  • 14.
    Full Stack DeepLearning - UC Berkeley Spring 2021 What skills are needed for the roles? 14 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM Software engineering
  • 15.
    Full Stack DeepLearning - UC Berkeley Spring 2021 What skills are needed for the roles? 15 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM Primarily a software engineering role. Often from standard SWE pipeline. Software engineering
  • 16.
    Full Stack DeepLearning - UC Berkeley Spring 2021 What skills are needed for the roles? 16 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM SWE with ML team as an active customer Software engineering
  • 17.
    Full Stack DeepLearning - UC Berkeley Spring 2021 What skills are needed for the roles? 17 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM Rare mix of ML skills and SWE skills. Often SWEs with significant self- teaching or science / engineering PhDs who worked as traditional SWEs after grad school Software engineering
  • 18.
    Full Stack DeepLearning - UC Berkeley Spring 2021 What skills are needed for the roles? 18 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM ML experts. Usually have MS/PhD in CS or Stats or did an industrial fellowship program Software engineering
  • 19.
    Full Stack DeepLearning - UC Berkeley Spring 2021 What skills are needed for the roles? 19 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM Wide range of backgrounds from undergrad-only to science PhD Software engineering
  • 20.
    Full Stack DeepLearning - UC Berkeley Spring 2021 What skills are needed for the roles? 20 Machine learning Low High Low High Size of bubble = communication / technical writing ML Researcher ML Engineer Data scientist Data engineer ML DevOps ML Teams - roles ML PM Traditional PMs, but with a deep understanding of the ML development process & mindset Software engineering
  • 21.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Questions? 21 ML Teams - roles
  • 22.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Managing Module overview 22 Hiring • How to manage a ML team • How to hire ML engineers. How to get hired. Orgs • How ML teams are organized and how they fit into the broader organization Roles • ML-related roles and the skills they require ML Teams - orgs
  • 23.
    Full Stack DeepLearning - UC Berkeley Spring 2021 ML org structures - lessons learned • No consensus yet on the right way to structure a ML team • This lecture: taxonomy of best practices for different organizational maturity levels 23 ML Teams - orgs
  • 24.
    Full Stack DeepLearning - UC Berkeley Spring 2021 ML organization archetypes 24 The ML Organization Mountain ML Teams - orgs
  • 25.
    Full Stack DeepLearning - UC Berkeley Spring 2021 ML organization archetypes 25 The ML Organization Mountain Nascent / Ad-Hoc ML What it looks like • No one is doing ML, or ML is done on an ad-hoc basis • Little ML expertise in-house Example organizations • Most small-medium businesses • Less technology-forward large companies (education, logistics, etc) Advantages • Often low-hanging fruit for ML Dis- advantages • Little support for ML projects, difficult to hire and retain good talent ML Teams - orgs
  • 26.
    Full Stack DeepLearning - UC Berkeley Spring 2021 ML R&D ML organization archetypes 26 What it looks like • ML efforts are centered in the R&D arm of the organization • Often hire researchers / PhDs & write papers Example organizations • Larger Oil & gas, manufacturing, telecom companies Advantages • Often can hire experienced researchers • Can work on long-term business priorities & big wins Dis- advantages • Difficult to get data • Rarely translates into actual business value, so usually the amount of investment remains small The ML Organization Mountain ML Teams - orgs
  • 27.
    Full Stack DeepLearning - UC Berkeley Spring 2021 ML embedded into business / product teams ML organization archetypes 27 What it looks like • Certain product teams or business units have ML expertise along-side their software or analytics talent • ML reports up to the team’s engineering lead or tech lead Example organizations • Software / technology companies • Financial services companies Advantages • ML improvements are likely to lead to business value • Tight feedback cycle between idea and product improvement Dis- advantages • Hard to hire and develop top talent • Access to resources (data / compute) can lag • ML project cycles conflict with engineering mgmt • Long-term projects can be hard to justify The ML Organization Mountain ML Teams - orgs
  • 28.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Independent ML Function ML organization archetypes 28 What it looks like • ML division reporting to senior leadership (often CEO) • ML PMs work with MLRs, MLEs, and customers to build ML into products • Teams sometimes publish long-term research Example organizations • Large financial services companies Advantages • Talent density allows to hire & train top practitioners • Senior leaders can marshal data / compute resources • Can invest in tooling, practices, and culture around ML development Dis- advantages • Model handoffs to lines of business can be challenging - users need to buy-in and be educated on model use • Feedback cycles can be slow The ML Organization Mountain ML Teams - orgs
  • 29.
    Full Stack DeepLearning - UC Berkeley Spring 2021 ML-First Organizations ML organization archetypes 29 What it looks like • CEO buy-in • ML division working on challenging, long-term projects • ML expertise in every line of business focusing on quick wins and working with central ML division Example organizations • Large tech companies • ML-focused startups Advantages • Best data access: data thinking permeates the org • Recruiting: ML team works on hardest problems • Easiest deployment: product teams understand ML Dis- advantages • Hard to implement • Challenging & expensive to recruit enough talent • Culturally difficult to embed ML thinking everywhere ML Teams - orgs
  • 30.
    Full Stack DeepLearning - UC Berkeley Spring 2021 ML team structures - design choices 30 Software engineering vs research Data ownership Model ownership Key questions • To what extent is the ML team responsible for building or integrating with software? • How important are SWE skills on the team? • How much control does the ML team have over data collection, warehousing, labeling, and pipelining? • Is the ML team responsible for deploying models into production? • Who maintains deployed models? ML Teams - orgs
  • 31.
    Full Stack DeepLearning - UC Berkeley Spring 2021 ML team structures - design choices 31 Software engineering vs research Data ownership Model ownership • Research prioritized over SWE skills • Researcher-SWE collaboration lacking ML R&D • ML team has no control over data • ML team typically will not have data engineering component • Models are rarely deployed into production ML Teams - orgs
  • 32.
    Full Stack DeepLearning - UC Berkeley Spring 2021 ML team structures - design choices 32 Software engineering vs research Data ownership Model ownership Embedded ML • SWE skills prioritized over research skills • Often, all researchers need strong SWE as everyone expected to deploy • ML team generally does not own data production / mgmt • Work with data engineers to build pipelines • ML engineers own the models that they deploy into production ML Teams - orgs • Research prioritized over SWE skills • Researcher-SWE collaboration lacking ML R&D • ML team has no control over data • ML team typically will not have data engineering component • Models are rarely deployed into production
  • 33.
    Full Stack DeepLearning - UC Berkeley Spring 2021 ML team structures - design choices 33 Software engineering vs research Data ownership Model ownership ML Function • Each team has a strong mix of SWE and research skills • SWE and researchers work closely together within team • ML team has a voice in data governance discussions • ML team has strong internal data engineering function • ML team hands off models to user, but is responsible for maintaining them ML Teams - orgs Embedded ML • Research prioritized over SWE skills • Researcher-SWE collaboration lacking ML R&D • ML team has no control over data • ML team typically will not have data engineering component • Models are rarely deployed into production • SWE skills prioritized over research skills • Often, all researchers need strong SWE as everyone expected to deploy • ML team generally does not own data production / mgmt • Work with data engineers to build pipelines • ML engineers own the models that they deploy into production
  • 34.
    Full Stack DeepLearning - UC Berkeley Spring 2021 ML team structures - design choices 34 Software engineering vs research Data ownership Model ownership Embedded ML • Research prioritized over SWE skills • Researcher-SWE collaboration lacking ML R&D ML Function ML First • ML team has no control over data • ML team typically will not have data engineering component • Models are rarely deployed into production • SWE skills prioritized over research skills • Often, all researchers need strong SWE as everyone expected to deploy • Each team has a strong mix of SWE and research skills • SWE and researchers work closely together within team • Different teams are more or less research oriented • Research teams collaborate closely with SWE teams • ML team generally does not own data production / mgmt • Work with data engineers to build pipelines • ML engineers own the models that they deploy into production • ML team has a voice in data governance discussions • ML team has strong internal data engineering function • ML team hands off models to user, but is responsible for maintaining them • ML team often owns company-wide data infrastructure • ML team hands off models to user, who operates and maintains them ML Teams - orgs
  • 35.
    Full Stack DeepLearning - UC Berkeley Spring 2021 ML team structures - design choices 35 Software engineering vs research Data ownership Model ownership Embedded ML • Research prioritized over SWE skills • Researcher-SWE collaboration lacking ML R&D ML Function ML First • ML team has no control over data • ML team typically will not have data engineering component • Models are rarely deployed into production • SWE skills prioritized over research skills • Often, all researchers need strong SWE as everyone expected to deploy • Each team has a strong mix of SWE and research skills • SWE and researchers work closely together within team • Different teams are more or less research oriented • Research teams collaborate closely with SWE teams • ML team generally does not own data production / mgmt • Work with data engineers to build pipelines • ML engineers own the models that they deploy into production • ML team has a voice in data governance discussions • ML team has strong internal data engineering function • ML team hands off models to user, but is responsible for maintaining them • ML team often owns company-wide data infrastructure • ML team hands off models to user, who operates and maintains them ML Teams - orgs
  • 36.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Questions? 36 ML Teams - orgs
  • 37.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Managing Module overview 37 Hiring • How to manage a ML team • How to hire ML engineers. How to get hired. Orgs • How ML teams are organized and how they fit into the broader organization Roles • ML-related roles and the skills they require ML Teams - managing
  • 38.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Managing ML teams is challenging • It’s hard to tell in advance how hard or easy something is 38 ML Teams - managing
  • 39.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Managing ML teams is challenging 39 https://medium.com/@l2k/why-are-machine-learning-projects-so-hard-to-manage-8e9b9cf49641 It’s hard to tell in advance how easy or hard something is ML Teams - managing
  • 40.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Managing ML teams is challenging • It’s hard to tell in advance how hard or easy something is • ML progress is nonlinear • Very common for projects to stall for weeks or longer • In early stages, difficult to plan project because unclear what will work • As a result, estimating project timelines is extremely difficult • I.e., production ML is still somewhere between “research” and “engineering” 40 ML Teams - managing
  • 41.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Managing ML teams is challenging • It’s hard to tell in advance how hard or easy something is • ML progress is nonlinear • There are cultural gaps between research and engineering • Different values, backgrounds, goals, norms • In toxic cultures, the two sides often don’t value one another 41 ML Teams - managing
  • 42.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Managing ML teams is challenging • It’s hard to tell in advance how hard or easy something is • ML progress is nonlinear • There are cultural gaps between research and engineering • Leaders often don’t understand it 42 ML Teams - managing
  • 43.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML Project planning probabilistically 43 ML Teams - managing
  • 44.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML project planning probabilistically • From:
 
 
 
 
 • To:
 
 44 ML Teams - managing Task A Task C Task D Task E Task F Task G Week 1 Week 2 Week 3 Week 4
  • 45.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML project planning probabilistically • From:
 
 
 
 
 • To:
 
 45 ML Teams - managing Task A Task C Task D Task E Task F Task G Week 1 Week 2 Week 3 Week 4 Task A (50%) Task B (25%) Task C (50%) Week 1 Week 2 Week 3 Week 4 Task D (75%)
  • 46.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML project planning probabilistically • From:
 
 
 
 
 • To:
 
 46 ML Teams - managing Task A Task C Task D Task E Task F Task G Week 1 Week 2 Week 3 Week 4 Task A (50%) Task B (25%) Task C (50%) Week 1 Week 2 Week 3 Week 4 Task D (75%)
  • 47.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML project planning probabilistically • From:
 
 
 
 
 • To:
 
 47 ML Teams - managing Task A Task C Task D Task E Task F Task G Week 1 Week 2 Week 3 Week 4 Task A Task B Task C Week 1 Week 2 Week 3 Week 4 Task D
  • 48.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML project planning probabilistically • From:
 
 
 
 
 • To:
 
 48 ML Teams - managing Task A Task C Task D Task E Task F Task G Week 1 Week 2 Week 3 Week 4 Task A Task B Task C Week 1 Week 2 Week 3 Week 4 Task E (50%) Task F (50%) Task G (10%) Task D (75%)
  • 49.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to manage ML teams better • Do ML Project planning probabilistically • Attempt a portfolio of approaches • Measure progress based on inputs, not results • Have researchers and engineers work together • Get end-to-end pipelines together quickly to demonstrate quick wins • Educate leadership on ML timeline uncertainty 49 ML Teams - managing
  • 50.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Resources for educating execs • https://a16z.com/2016/06/10/ai-deep-learning-machines/ • Pieter’s upcoming AI Strategy class:
 https://emeritus-executive.berkeley.edu/artificial-intelligence/ 50 ML Teams - managing
  • 51.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Questions? 51 ML Teams - managing
  • 52.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Managing Module overview 52 Hiring • How to manage a ML team • How to hire ML engineers. How to get hired. Orgs • How ML teams are organized and how they fit into the broader organization Roles • ML-related roles and the skills they require ML Teams - hiring
  • 53.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Hiring for ML - outline • The AI Talent Gap • Sourcing • Interviewing • Finding a job 53 ML Teams - hiring
  • 54.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Hiring for ML - outline • The AI Talent Gap • Sourcing • Interviewing • Finding a job 54 ML Teams - hiring
  • 55.
    Full Stack DeepLearning - UC Berkeley Spring 2021 The AI Talent Gap 55 How many people know how to build AI systems? 5,000 (actively publishing research [Element AI]) 10,000 (estimated num people with the right skillset [Element AI]) 22,000 (PhD-educated AI researchers [Bloomberg]) 90,000 (upper bound on number of people [Element AI]) 200,000 - 300,000 (Number of AI researcher / practitioners [Tencent]) 3.6M (Number of software developers in the US) 18.2M (Number of software developers in the world) Sources: The AI Talent Shortage (Nikolai Yakovenko) https://medium.com/@Moscow25/the-ai-talent-shortage-704d8cf0c4cc Just How Shallow is the Artificial Intelligence Talent Pool (Jeremy Kahn) 
 https://www.bloomberg.com/news/articles/2018-02-07/just-how-shallow-is-the-artificial-intelligence-talent-pool ML Teams - hiring
  • 56.
    Full Stack DeepLearning - UC Berkeley Spring 2021 56 The AI talent gap Fierce competition for AI talent “Everyone agrees that the competition to hire people who know how to build artificial intelligence systems is intense. It’s  turned once- staid academic conferences into frenzied meet markets for corporate recruiters and driven the salaries of the top researchers to seven- figures.” 
 (Bloomberg) Sources: The AI Talent Shortage (Nikolai Yakovenko) https://medium.com/@Moscow25/the-ai-talent-shortage-704d8cf0c4cc Just How Shallow is the Artificial Intelligence Talent Pool (Jeremy Kahn) 
 https://www.bloomberg.com/news/articles/2018-02-07/just-how-shallow-is-the-artificial-intelligence-talent-pool ML Teams - hiring
  • 57.
    Full Stack DeepLearning - UC Berkeley Spring 2021 57 The AI talent gap Fierce competition for AI talent “Hiring is crazy right now. ML is a young field that got popular very quickly. There’s a ton of demand and not a lot of supply.” 
 (Computer Vision Engineer at Series C startup) ML Teams - hiring
  • 58.
    Full Stack DeepLearning - UC Berkeley Spring 2021 58 The AI talent gap Fierce competition for AI talent “Hiring for ML is really challenging and takes way more time and effort than we expected. We have someone working on it full-time and we’re still only able to get a few people per quarter” (Startup Founder) ML Teams - hiring
  • 59.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Hiring for ML - outline • The AI Talent Gap • Sourcing • Interviewing • Finding a job 59 ML Teams - hiring
  • 60.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Most common ML roles • ML product manager • DevOps • Data engineer • ML engineer • ML researcher / ML scientist • Data scientist 60 ML Teams - hiring
  • 61.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Most common ML roles • ML product manager • DevOps • Data engineer • ML engineer • ML researcher / ML scientist • Data scientist 61 Slightly different mindset required. 
 Helpful to look for demonstrated interest in AI - courses, conferences, re-implementations, etc ML Teams - hiring
  • 62.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Most common ML roles • ML product manager • DevOps • Data engineer • ML engineer • ML researcher / ML scientist • Data scientist 62 Our focus ML Teams - hiring
  • 63.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to hire MLEs - the wrong way • Job Description (Unicorn Machine Learning Engineer) • Duties • Keep up with the state of the art • Implement models from scratch • Deep understanding of mathematics & ability to come up with new models • Build tooling & infrastructure for the ML team • Build data pipelines for the ML team • Deploy & monitor models into production • Requirements • PhD • At least 4 years tensorflow experience • At least 4 years as a software engineer • Publications in top ML conference • Experience building large-scale distributed systems 63 ML Teams - hiring
  • 64.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to hire MLEs - the right way • Hire for software engineering skills, interest in ML, and desire to learn. Train to do ML. • Go more junior. Most undergrad computer science students graduate with ML experience. • Be more specific about what you need. Not every ML engineer needs to do DevOps. 64 ML Teams - hiring
  • 65.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to hire MLRs • Look for quality of publications, not quantity (e.g., originality of ideas, quality of execution) • Look for researchers with an eye for working on important problems (many researchers focus on trendy problems without considering why they matter) • Look for researchers with experience outside of academia • Consider hiring talented people from adjacent fields (physics, statistics, math) • Consider hiring people without PhDs (e.g., talented undergraduate / masters students, graduates of Google/Facebook/OpenAI fellowship programs, dedicated self-studiers) 65 ML Teams - hiring
  • 66.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to find MLE/MLR candidates • Standard sources: LinkedIn, recruiters, on-campus recruiting, etc • Monitor arXiv and top conferences and flag first authors of papers you like • Look for good reimplementations of papers you like • Attend ML research conferences (NeurIPS, ICLR, ICML) 66 ML Teams - hiring
  • 67.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to attract MLR / MLE candidates 67 What do machine learning practitioners want? How to make your company stand out? • Work with cutting edge tools & techniques • Build skills / knowledge in an exciting field • Work with excellent people • Work on interesting datasets • Do work that matters • Work on research-oriented projects. Publicize them. Invest in tooling for your team & empower employees to try new tools. • Build team culture around learning (reading groups, learning days, professional development budget, conference budget) • Hire high-profile people. Help your best people build their profile through publishing blogs & papers. • Sell the uniqueness of your dataset in recruiting materials. • Sell the mission of your company and potential impact of machine learning on that mission. Work on projects that have a tangible impact today. ML Teams - hiring
  • 68.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Hiring for ML - outline • The AI Talent Gap • Sourcing • Interviewing • Finding a job 68 ML Teams - hiring
  • 69.
    Full Stack DeepLearning - UC Berkeley Spring 2021 What to test in an ML interview? • Hire for strengths • Meet a minimum bar for everything else 69 ML Teams - hiring
  • 70.
    Full Stack DeepLearning - UC Berkeley Spring 2021 What to test in an ML interview? • Validate your hypotheses of candidate’s strengths • Researchers: make sure they can think creatively about new ML problems, probe how thoughtful they were about previous projects • Engineers: make sure they are great generalist SWEs • Make sure candidates meet a minimum bar on weaker areas • Researchers: test SWE knowledge and ability to write good code • SWEs: test ML knowledge 70 ML Teams - hiring
  • 71.
    Full Stack DeepLearning - UC Berkeley Spring 2021 What happens in a ML interview? • Much less well-defined than software engineering interviews • Common types of assessments: • Background & culture fit • Whiteboard coding (similar to SWE interviews) • Pair coding (similar to SWE interviews) • Pair debugging (often ML-specific code) • Math puzzles (e.g., involving linear algebra) • Take-home ML project • Applied ML (e.g., explain how you’d solve this problem with ML) • Previous ML projects (e.g., probing on what you tried, why things did / didn’t work) • ML theory (e.g., bias-variance tradeoff, overfitting, underfitting, understanding of specific algorithms) 71 ML Teams - hiring
  • 72.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Hiring for ML - outline • The AI Talent Gap • Sourcing • Interviewing • Finding a job 72 ML Teams - hiring
  • 73.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Where to look for a ML job? • Standard sources: LinkedIn, recruiters, on-campus recruiting, etc • ML research conferences (NeurIPS, ICLR, ICML) • Apply directly (remember, there’s a talent gap!) • This course • Those who pass the exam will get access to our recruiting database 73 ML Teams - hiring
  • 74.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to stand out for ML roles? • Build software engineering skills (e.g., work at a well-known software company) • Exhibit interest in ML (e.g., conference attendance, online courses taken) • Show you have broad knowledge of ML (e.g., write blog posts synthesizing a research area) • Demonstrate ability to get ML projects done (e.g., create side projects, re- implement papers) • Prove you can think creatively in ML (e.g., win Kaggle competitions, publish papers) 74 More impressive ML Teams - hiring
  • 75.
    Full Stack DeepLearning - UC Berkeley Spring 2021 How to prepare for the interview? • Prepare for a general SWE interview (e.g., “Cracking the Coding Interview”) • Prepare to talk in detail about your past ML projects (remember details, prepare to talk about tradeoffs and decisions you made) • Review how basic ML algorithms work (linear / logistic regression, nearest neighbor, decision trees, k-means, MLPs, ConvNets, recurrent nets, etc) • Review ML theory • Think about the problems the company you’re interviewing with may face and what ML techniques may apply to them 75 ML Teams - hiring
  • 76.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Conclusion 76 Orgs • ML teams are becoming more standalone, hence more interdisciplinary Roles • Lots of different skills involved in production ML, so there’s an opportunity for many to contribute Hiring • Talent is scarce, so be specific about what is must-have. It can be hard to break in as an outsider - use projects to build awareness. Managing • Managing ML teams is hard. There’s no silver bullet, but shifting toward probabilistic planning can help ML Teams - hiring
  • 77.
    Full Stack DeepLearning - UC Berkeley Spring 2021 Thank you! 77