Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Presentatie Holger Hoos

2 views

Published on

Presentatie van het Datacongres ''data science voor maatschappelijke uitdagingen'' op 22 november 2018

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Presentatie Holger Hoos

  1. 1. Automated machine learning: Learning how to learn from data Holger H. Hoos LIACS Universiteit Leiden The Netherlands CS Department University of British Columbia Canada Data Science Congress The Hague (The Netherlands) 2018/11/22
  2. 2. Cai Huang et al., ScIentific Reports 8:16444 (2018)
  3. 3. “We report here on the application of our previously established open-source support vector machine (SVM)-based algorithm to predict the responses of 175 individual cancer patients to a variety of standard-of-care chemotherapeutic drugs from the gene-expression profiles (RNA-seq or microarray) of individual patient tumors.
  4. 4. “We report here on the application of our previously established open-source support vector machine (SVM)-based algorithm to predict the responses of 175 individual cancer patients to a variety of standard-of-care chemotherapeutic drugs from the gene-expression profiles (RNA-seq or microarray) of individual patient tumors. The models were found to predict patient responses with > 80% accuracy.”
  5. 5. The age of machines Holger Hoos: Automated machine learning: Learning how to learn from data 2
  6. 6. The age of machines “As soon as an Analytical Engine exists, it will necessarily guide the future course of the science. (Charles Babbage, 1864) Holger Hoos: Automated machine learning: Learning how to learn from data 2
  7. 7. The nature of computation Holger Hoos: Automated machine learning: Learning how to learn from data 3
  8. 8. The nature of computation Clear, precise instructions – flawlessly executed Holger Hoos: Automated machine learning: Learning how to learn from data 3
  9. 9. The nature of computation Clear, precise instructions – flawlessly executed algorithm Holger Hoos: Automated machine learning: Learning how to learn from data 3
  10. 10. Holger Hoos: Automated machine learning: Learning how to learn from data 4
  11. 11. The age of computation “The maths that computers use to de- cide stuff [is] infiltrating every aspect of our lives.” Holger Hoos: Automated machine learning: Learning how to learn from data 4
  12. 12. The age of computation “The maths that computers use to de- cide stuff [is] infiltrating every aspect of our lives.” financial markets social interactions Holger Hoos: Automated machine learning: Learning how to learn from data 4
  13. 13. The age of computation “The maths that computers use to de- cide stuff [is] infiltrating every aspect of our lives.” financial markets social interactions cultural preferences artistic production . . . Holger Hoos: Automated machine learning: Learning how to learn from data 4
  14. 14. Machine learning Holger Hoos: Automated machine learning: Learning how to learn from data 5
  15. 15. Machine learning is old ... Alan Turing (1950): Computing machinery and intelligence Holger Hoos: Automated machine learning: Learning how to learn from data 5
  16. 16. Machine learning is old ... Alan Turing (1950): Computing machinery and intelligence Farley and Clark (1954): Simulation of Self-Organizing Systems by Digital Computer Holger Hoos: Automated machine learning: Learning how to learn from data 5
  17. 17. Machine learning is old ... Alan Turing (1950): Computing machinery and intelligence Farley and Clark (1954): Simulation of Self-Organizing Systems by Digital Computer Arthur Samuel (1959): Some Studies in Machine Learning Using the Game of Checkers Holger Hoos: Automated machine learning: Learning how to learn from data 5
  18. 18. Machine learning is old ... Alan Turing (1950): Computing machinery and intelligence Farley and Clark (1954): Simulation of Self-Organizing Systems by Digital Computer Arthur Samuel (1959): Some Studies in Machine Learning Using the Game of Checkers Paul Werbos (1974): Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences . . . Holger Hoos: Automated machine learning: Learning how to learn from data 5
  19. 19. The machine learning revolution manually constructed algorithms automatic adaptation to given set / distribution of inputs Holger Hoos: Automated machine learning: Learning how to learn from data 6
  20. 20. The machine learning revolution manually constructed algorithms automatic adaptation to given set / distribution of inputs through optimisation of performance metric (loss minimisation) Holger Hoos: Automated machine learning: Learning how to learn from data 6
  21. 21. The machine learning revolution manually constructed algorithms automatic adaptation to given set / distribution of inputs through optimisation of performance metric (loss minimisation) machine learning procedures = meta-algorithms Holger Hoos: Automated machine learning: Learning how to learn from data 6
  22. 22. The machine learning revolution manually constructed algorithms automatic adaptation to given set / distribution of inputs through optimisation of performance metric (loss minimisation) machine learning procedures = meta-algorithms (procedures for optimising algorithm) Holger Hoos: Automated machine learning: Learning how to learn from data 6
  23. 23. Automating the automation of automation Holger Hoos: Automated machine learning: Learning how to learn from data 7
  24. 24. Automating the automation of automation Machine learning is powerful, but successful application is far from trivial. Holger Hoos: Automated machine learning: Learning how to learn from data 7
  25. 25. Automating the automation of automation Machine learning is powerful, but successful application is far from trivial. Fundamental problem: Which of many available algorithms (models) applicable to given machine learning problem to use, and with which hyper-parameter settings? Holger Hoos: Automated machine learning: Learning how to learn from data 7
  26. 26. Cai Huang et al., ScIentific Reports 8:16444 (2018)
  27. 27. “We report here on the application of our previously established open-source support vector machine (SVM)-based algorithm to predict the responses of 175 individual cancer patients to a variety of standard-of-care chemotherapeutic drugs from the gene-expression profiles (RNA-seq or microarray) of individual patient tumors. The models were found to predict patient responses with > 80% accuracy.”
  28. 28. Automating the automation of automation Machine learning is powerful, but successful application is far from trivial. Fundamental problem: Which of many available algorithms (models) applicable to given machine learning problem to use, and with which hyper-parameter settings? Example: WEKA contains 39 classification algorithms, Example: 3 × 8 feature selection methods Holger Hoos: Automated machine learning: Learning how to learn from data 7
  29. 29. Automating the automation of automation Machine learning is powerful, but successful application is far from trivial. Fundamental problem: Which of many available algorithms (models) applicable to given machine learning problem to use, and with which hyper-parameter settings? Solution: Automatically select ML methods and hyper-parameter settings Holger Hoos: Automated machine learning: Learning how to learn from data 7
  30. 30. Automating the automation of automation Machine learning is powerful, but successful application is far from trivial. Fundamental problem: Which of many available algorithms (models) applicable to given machine learning problem to use, and with which hyper-parameter settings? Solution: Automatically select ML methods and hyper-parameter settings Automated machine learning (AutoML) Holger Hoos: Automated machine learning: Learning how to learn from data 7
  31. 31. AutoML ... achieves substantial performance improvements over solutions hand-crafted by human experts Holger Hoos: Automated machine learning: Learning how to learn from data 8
  32. 32. AutoML ... achieves substantial performance improvements over solutions hand-crafted by human experts helps non-experts effectively apply ML techniques Holger Hoos: Automated machine learning: Learning how to learn from data 8
  33. 33. AutoML ... achieves substantial performance improvements over solutions hand-crafted by human experts helps non-experts effectively apply ML techniques intense international research focus in academia + industry: Auto-WEKA (Thornton, Hutter, HH, Leyton-Brown 2013; Kotthoff, Thornton Hutter, HH, Leyton-Brown 2017) Holger Hoos: Automated machine learning: Learning how to learn from data 8
  34. 34. AutoML ... achieves substantial performance improvements over solutions hand-crafted by human experts helps non-experts effectively apply ML techniques intense international research focus in academia + industry: Auto-WEKA (Thornton, Hutter, HH, Leyton-Brown 2013; Kotthoff, Thornton Hutter, HH, Leyton-Brown 2017) autosklearn (Feurer et al. 2015) Holger Hoos: Automated machine learning: Learning how to learn from data 8
  35. 35. AutoML ... achieves substantial performance improvements over solutions hand-crafted by human experts helps non-experts effectively apply ML techniques intense international research focus in academia + industry: Auto-WEKA (Thornton, Hutter, HH, Leyton-Brown 2013; Kotthoff, Thornton Hutter, HH, Leyton-Brown 2017) autosklearn (Feurer et al. 2015) neural architecture search (e.g., Google Cloud AutoML) Holger Hoos: Automated machine learning: Learning how to learn from data 8
  36. 36. AutoML ... achieves substantial performance improvements over solutions hand-crafted by human experts helps non-experts effectively apply ML techniques intense international research focus in academia + industry: Auto-WEKA (Thornton, Hutter, HH, Leyton-Brown 2013; Kotthoff, Thornton Hutter, HH, Leyton-Brown 2017) autosklearn (Feurer et al. 2015) neural architecture search (e.g., Google Cloud AutoML) demonstrated to outperform human ML experts Holger Hoos: Automated machine learning: Learning how to learn from data 8
  37. 37. AutoML ... achieves substantial performance improvements over solutions hand-crafted by human experts helps non-experts effectively apply ML techniques intense international research focus in academia + industry: Auto-WEKA (Thornton, Hutter, HH, Leyton-Brown 2013; Kotthoff, Thornton Hutter, HH, Leyton-Brown 2017) autosklearn (Feurer et al. 2015) neural architecture search (e.g., Google Cloud AutoML) demonstrated to outperform human ML experts, widely believed to be “next big thing” in ML Holger Hoos: Automated machine learning: Learning how to learn from data 8
  38. 38. AutoML: Under the hood Flexible ML framework with many algorithms, components, parameters Expose all choices as parameters Holger Hoos: Automated machine learning: Learning how to learn from data 9
  39. 39. AutoML: Under the hood Flexible ML framework with many algorithms, components, parameters Expose all choices as parameters Optimise parameter settings for specific use case using powerful machine learning and optimisation techniques Holger Hoos: Automated machine learning: Learning how to learn from data 9
  40. 40. AutoML: Under the hood Flexible ML framework with many algorithms, components, parameters Expose all choices as parameters Optimise parameter settings for specific use case using powerful machine learning and optimisation techniques “learn how to learn” Holger Hoos: Automated machine learning: Learning how to learn from data 9
  41. 41. AutoML: Under the hood Flexible ML framework with many algorithms, components, parameters Expose all choices as parameters Optimise parameter settings for specific use case using powerful machine learning and optimisation techniques “learn how to learn” Our approach: Use general-purpose automated algorithm configurator (Hutter, HH, Leyton-Brown 2011) Holger Hoos: Automated machine learning: Learning how to learn from data 9
  42. 42. AutoML: Under the hood Flexible ML framework with many algorithms, components, parameters Expose all choices as parameters Optimise parameter settings for specific use case using powerful machine learning and optimisation techniques “learn how to learn” Our approach: Use general-purpose automated algorithm configurator (Hutter, HH, Leyton-Brown 2011) https://www.cs.ubc.ca/labs/beta/Projects/autoweka Holger Hoos: Automated machine learning: Learning how to learn from data 9
  43. 43. AI is more than ML automated reasoning ( hard- / software correctness) Holger Hoos: Automated machine learning: Learning how to learn from data 10
  44. 44. AI is more than ML automated reasoning ( hard- / software correctness) knowledge representation Holger Hoos: Automated machine learning: Learning how to learn from data 10
  45. 45. AI is more than ML automated reasoning ( hard- / software correctness) knowledge representation search and optimisation Holger Hoos: Automated machine learning: Learning how to learn from data 10
  46. 46. AI is more than ML automated reasoning ( hard- / software correctness) knowledge representation search and optimisation planning & scheduling multi-agent systems Holger Hoos: Automated machine learning: Learning how to learn from data 10
  47. 47. AI is more than ML automated reasoning ( hard- / software correctness) knowledge representation search and optimisation planning & scheduling multi-agent systems natural language processing robotics computer vision Holger Hoos: Automated machine learning: Learning how to learn from data 10
  48. 48. AI is more than ML automated reasoning ( hard- / software correctness) knowledge representation search and optimisation planning & scheduling multi-agent systems natural language processing robotics computer vision ethical, legal, social issues ... Holger Hoos: Automated machine learning: Learning how to learn from data 10
  49. 49. The Age of AI Holger Hoos: Automated machine learning: Learning how to learn from data 11
  50. 50. The Age of AI Opportunities: AI technology: posed to transform industry & society; key driver of future innovation, growth, competitiveness Holger Hoos: Automated machine learning: Learning how to learn from data 11
  51. 51. The Age of AI Opportunities: AI technology: posed to transform industry & society; key driver of future innovation, growth, competitiveness AI key to solving grand problems of humanity (disease, climate change, resource limitations . . . ) Holger Hoos: Automated machine learning: Learning how to learn from data 11
  52. 52. The Age of AI Opportunities: AI technology: posed to transform industry & society; key driver of future innovation, growth, competitiveness AI key to solving grand problems of humanity (disease, climate change, resource limitations . . . ) Challenges: job loss due to automation manipulation of individual / collective behaviour Holger Hoos: Automated machine learning: Learning how to learn from data 11
  53. 53. The Age of AI Opportunities: AI technology: posed to transform industry & society; key driver of future innovation, growth, competitiveness AI key to solving grand problems of humanity (disease, climate change, resource limitations . . . ) Challenges: job loss due to automation manipulation of individual / collective behaviour increased inequity (due to barrier to entry) limited talent + tendency to overestimate AI capabilities Holger Hoos: Automated machine learning: Learning how to learn from data 11
  54. 54. The Age of AI Opportunities: AI technology: posed to transform industry & society; key driver of future innovation, growth, competitiveness AI key to solving grand problems of humanity (disease, climate change, resource limitations . . . ) Challenges: job loss due to automation manipulation of individual / collective behaviour increased inequity (due to barrier to entry) limited talent + tendency to overestimate AI capabilities poor-quality AI systems + deployment Holger Hoos: Automated machine learning: Learning how to learn from data 11
  55. 55. Key idea: Use AI to (help) construct AI systems Holger Hoos: Automated machine learning: Learning how to learn from data 12
  56. 56. Key idea: Use AI to (help) construct AI systems Key idea: AutoAI (Automated Artificial Intelligence) Holger Hoos: Automated machine learning: Learning how to learn from data 12
  57. 57. Key idea: Use AI to (help) construct AI systems Key idea: AutoAI (Automated Artificial Intelligence) AI → AutoAI ∼= ML → AutoML Holger Hoos: Automated machine learning: Learning how to learn from data 12
  58. 58. Key idea: Use AI to (help) construct AI systems Key idea: AutoAI (Automated Artificial Intelligence) AI → AutoAI ∼= ML → AutoML leverage learning, optimisation, meta-algorithmic approaches Holger Hoos: Automated machine learning: Learning how to learn from data 12
  59. 59. Key idea: Use AI to (help) construct AI systems Key idea: AutoAI (Automated Artificial Intelligence) AI → AutoAI ∼= ML → AutoML leverage learning, optimisation, meta-algorithmic approaches enable non-experts to build, deploy, maintain high-quality AI systems (predictable, robust, performant) Holger Hoos: Automated machine learning: Learning how to learn from data 12
  60. 60. Key idea: Use AI to (help) construct AI systems Key idea: AutoAI (Automated Artificial Intelligence) AI → AutoAI ∼= ML → AutoML leverage learning, optimisation, meta-algorithmic approaches enable non-experts to build, deploy, maintain high-quality AI systems (predictable, robust, performant) alleviate talent bottleneck broader access to AI, reduced barrier to entry Holger Hoos: Automated machine learning: Learning how to learn from data 12
  61. 61. Key idea: Use AI to (help) construct AI systems Key idea: AutoAI (Automated Artificial Intelligence) AI → AutoAI ∼= ML → AutoML leverage learning, optimisation, meta-algorithmic approaches enable non-experts to build, deploy, maintain high-quality AI systems (predictable, robust, performant) alleviate talent bottleneck broader access to AI, reduced barrier to entry (democratisation of AI) Holger Hoos: Automated machine learning: Learning how to learn from data 12
  62. 62. Key idea: Use AI to (help) construct AI systems Key idea: AutoAI (Automated Artificial Intelligence) AI → AutoAI ∼= ML → AutoML leverage learning, optimisation, meta-algorithmic approaches enable non-experts to build, deploy, maintain high-quality AI systems (predictable, robust, performant) alleviate talent bottleneck broader access to AI, reduced barrier to entry (democratisation of AI) increased benefits, reduced risks from use of AI Holger Hoos: Automated machine learning: Learning how to learn from data 12

×