Successfully reported this slideshow.
Your SlideShare is downloading. ×


Loading in …3

Check these out next

1 of 42 Ad

More Related Content

Recently uploaded (20)



  1. 1. Machine Learning Systems Engineering (MLSE): Retrospective of Five-Year Activities in Japan Hiroshi Maruyama* Preferred Networks / Kao Corporation / U. Tokyo Twitter; @maruyama * Presentation is done by me but this is a collective effort by the MLSE members
  2. 2. Agenda 1. Backdrop 2. Launch of MLSE 3. Activities 4. Research Results 5. Retrospective
  3. 3. Googleʼs seminal paper on “technical debt” of ML systems (2015)
  4. 4. Workshop on “Towards real world implementation of ML systems”, in conjunction with JSAI annual convention, 2016 ● How to deploy ML systems for industry applications ● Recent trends in ML businesses ● Intellectual property in ML systems ● Nikoniko-Deep Learning β ● Artificial life hackason, tried ● Evolution of communities, such as open source Business Community
  5. 5.
  6. 6. Software 2.0: Find a program rather than write it by hand Find a program Set of programs that satisfy the spec Search algorithm Write a program vs x x 🡺 Software 2.0 requires completely different set of skills
  7. 7. Concerns on the shortage of ML skills Source: METI, IT人材の最新動向と将来推計に関 する調査結果 (Study results on the latest status and the future trends of IT talents), 2016 Shortage of 48,000 talents on “big data, IOT, and AI” in 2020
  8. 8. 8 “Shortage of skills” -- doesn’t this sound familiar? Who can write software ⇒ Software Crisis (1960’s) 🡺 Dawn of Software Engineering! IBM System 360 Source: Wikipedia System 360 Instruction set Source:Quora
  9. 9. Keynote at APRES (Asia-Pacific Requirements Engineering Symposium) by Maruyama (2016)
  10. 10. My email to Mikio Aoyama “We have a number of projects going on with customers. Sometimes I feel that we are reinventing software engineering practices in ML. I wonder if the SE community can help us to establish a new engineering discipline in ML”
  11. 11. Mikio’s email to the organizer of SES2017, suggesting a panel discussion on SE for ML Annual SES (Software Engineering Symposium) is the largest event in Japan dedicated to Software Engineering, and Mikio suggested this is the best place to draw the interests from the SE community
  12. 12. Agenda 1. Backdrop 2. Launch of MLSE 3. Activities 4. Research Results 5. Retrospective
  13. 13. Panel discussion on machine learning engineering at SES 2017 “This panel discussion was the highlight of this year’s SES” Panelists ● Fuyuki Ishikawa (NII) ● Koichi Hamada (DeNA) ● Hiroshi Maruyama (PFN) Moderator ● Mikio Aoyama INanzan U.)
  14. 14. A couple of meetups among SE and ML engineers revealed new gaps ● ML engineer (ME): “my improved model now gives an erroneous output for a certain input that was ok with my previous model” ● Software engineer (SE): “What did you do with your regression test? Don’t you have one? ● ME: “...” (ah, but what does regression test mean in ML? How can we do it?) ● ME: “My customer is concerned with the safety” ● SE: “What is the invariant in your code?” ● ME: “...” (Invariant? In an ML system?) ● SE: “You are concerned with the quality. Why don’t you use stronger-typed language than Python?” ● ME: “...” (yes, I wish Python could statically check the shape of numpy ndarray)
  15. 15. Sorrow of ML Project: “Curse of infinite PoC” Develop model Evaluate Yes, we achieved xx% accuracy! Can you make a little better? ML engineer Customer How can we have a reasonable level of customer expectation? Looks good! But not enough for my customer
  16. 16. Michael Jordanʼs blog on the need for new engineering discipline on-hasnt-happened-yet-5e1d5812e1e7 “... we do have a major challenge on our hands in bringing together computers and humans in ways that enhance human life. While this challenge is viewed by some as subservient to the creation of “artificial intelligence,” it can also be viewed more prosaically — but with no less reverence — as the creation of a new branch of engineering. Much like civil engineering and chemical engineering in decades past, this new discipline aims to corral the power of a few key ideas, bringing new resources and capabilities to people, and doing so safely. “
  17. 17. The role of engineering -- my personal view Theories * Safety Factor Engineering as a form of agreement between engineers and the society Civil Engineering Handbook, p999 Why do we trust bridges? Because of the accumulated knowledge called Civil Engineering
  18. 18. In Apr. 2018, the SIG on MLSE (Machine Learning Systems Engineering, pronounced as “Mel-See”) is formed under JSSST
  19. 19. Agenda 1. Backdrop 2. Launch of MLSE 3. Activities 4. Research Results 5. Retrospective
  20. 20. MLSE kick-off meeting, Mar. 2018 (>500 participants) Source: ● Mikio Aoyama (Nanzan U), “Expectations to MLSE” ● Takuya Kudo (Accenture), “Challenges in software engineering and the new form of ML” ● Masashi Sugiyama (Riken AIP), “Current and future of ML research” ● Akimichi Ariga (Cloudara), “ML starting from business applications” ● Takahiro Kubo (TIS), “ML code design without remorse” ● Shin Nakajima (NII), “Quality assurance of ML software” Common issues in ML-in-practice have surfaced See for a report on the symposium
  21. 21. First things first: What are the challenges of MLSE? Ishikawa, Fuyuki, and Nobukazu Yoshioka. "How do engineers perceive difficulties in engineering of machine-learning systems?-questionnaire survey." 2019 IEEE/ACM Joint 7th International Workshop on Conducting Empirical Studies in Industry (CESI) and 6th International Workshop on Software Engineering Research and Industrial Practice (SER&IP). IEEE, 2019.
  22. 22. Working groups ● ML operational infrastructures / operations WG ● ML fairness WG ● ML system foundation WG ● Data quality engineering WG ● ML operations WG ● ML development process and case studies WG ● ML security WG ● : Active discussions on Discord, everybody is welcome
  23. 23. MLSE summer camps ● Main venue for community discussions ● 2-3 days, plenary / parallel sessions (workshops) + posters ● WG’s to report annual findings ● Every year new ideas are coming out Online due to Covid-19 2019 venue in Hakone hot spa
  24. 24. Cooperative gatherings Symposium on safety for AI/IoT systems Symposium on ML and fairness w/ JSAI and IBIS/ML, Jan., 2020
  25. 25. International activities -- 1/2 ● iMLSE -- International Workshop on Machine Learning Systems Engineering, in conjunction with APSEC 1st iMLSE in Nara (2018) iMLSE 2020 (online) Clark Barrett Jacomo Corbo iMLSE 2021 (online) Hironori Washizaki ● Shonan meeting, Nov. 2019
  26. 26. ● MLSE International Symposium (2019) ● Sanjit Seshua, “Towards Verified Artificial Intelligence” ● Akira Sakakibara, “Engineer's Responsibility in Machine Learning Era” ● Foutse Khomh, “Towards Debugging and Testing Deep Learning Systems” ● Lei Ma, “ Towards Testing and Analysis of Deep Learning Systems” ● Amel Bennaceur, “Requirements for Machine Learning Applications” ● Rüdiger Ehlers, “The Role of Verification in the Engineering Process of Complex Cyber-Physical Systems That Employ Machine Learning” International activities -- 2/2
  27. 27. Agenda 1. Backdrop 2. Launch of MLSE 3. Activities 4. Research Results 5. Retrospective
  28. 28. Req. Req. e.g., low risk in a specific situation? e.g., good prediction performance for rare cases? Reliable model building with small data Controllable model update for local improvement and mitigation of degradation Fine-Grained Requirements for Dependability AI researchers and SE researchers Decrease oversight of existing AI by 50% for rare cases of cancers Improve existing AI to mitigate risks over 20+ fine-grained safety metrics Healthcare Automotive “Engineerable AI” Project: Overview
  29. 29. “Engineerable AI” Project: Example of Techniques Work with Fujitsu [ Tokui+, NeuRecover: Regression-Controlled Repair of Deep Neural Networks with Training History, SANER’22 ] Target Neural Network Analyze internal behavior regarding occurrences of undesirable error pattern e.g., misclassification of nearby pedestrian to rider Identify and try to fix small part of neuron weight parameters - that affected the occurrences of the error pattern Also use hints by looking at past versions - “past: success, now: fail” 🡪 try to fix relevant parameters - “past: fail, now success” 🡪 not touch relevant parameters Avoid side-effect of causing other errors or “shuffling” of success/failure caused by retraining or baseline method
  30. 30. Testing, Debugging, Analysis, Repairing Techniques and their Integration into MLOps in a human-centered & Interactive Way
  31. 31. Continuous Quality Monitoring and Assurance of AI System & AI System Trusthworty Technique Application across Diverse Domains AI System Continuous Integration & Continuous Delivery
  32. 32. Rule-based safeguard, with output space transformation DNN Policy Filter in Rn Maruyama, Hiroshi. "Guaranteeing Deep Neural Network Outputs in a Feasible Region." Proceedings of the International Workshop on Evidence-based Security and Privacy in the Wild and the 1st International Workshop on Machine Learning Systems Engineering. 2018. Feasible Region Non-feasible solutions in feasible region
  33. 33. Machine Learning Project Canvas
  34. 34. Takeuchi, Hironori, et. al “Collecting Insights and Developing Patterns for Machine Learning Projects based on Project Practices, 14th International Joint Conference on Knowledge-Based Software Engineering (JCKBSE) Bad “smell” in the project
  35. 35. 37 ML system quality assurance guidelines in Japan AIST, Machine Learning Quality Management Guideline, 2nd Edition, signed.pdf Guideline for Quality Assurance of AI-based products and services Guidelines on Assessment of AI Reliability in the Field of Plant Safety
  36. 36. The book “Machine Learning Engineering” by the community 1. What is machine learning systems engineering? (Nakagawa, Ishikawa) 2. Project management of ML systems (Takeuchi) 3. Operation of ML systems (Horiuchi, Dobashi) 4. ML design patterns (Washizaki) 5. Quality assurance (Ishikawa) 6. Explainability of ML systems (Hara) 7. Ethics (Nakagawa) 8. Intellectual properties and contracts (Kakinuma) 9. Future of machine learning systems engineering (Ishikawa) ISBN-13 : 978-4065285862
  37. 37. Upcoming book on “Machine Learning Engineering for AI Project Managers” 1. Introduction to AI System Development (Yoshioka) 2. Requirement Engineering for AI Systems (Yoshioka) 3. Architecture and Design of ML Systems (Washizaki) 4. Project Management of AI Systems (Uchihira) 5. Cooperation with Stakeholders in AI Project (Takeuchi) 6. Future Vision of Machine Learning Engineering (Yoshioka)
  38. 38. Agenda 1. Backdrop 2. Launch of MLSE 3. Activities 4. Research Results 5. Retrospective
  39. 39. In five years, we … ● formed a very active community ○ From both SE and ML communities ○ From both industry and academia ○ Connecting people, sharing ideas, … ● produced research results ● helped the industry via many symposia and guideline documents, including the quality assurance guidelines ● published a book (with more upcoming …) Challenges ahead ● More academic activities as well as industry success stories ● Recognition as an engineering discipline by the general public To me, perhaps this is the biggest achievement
  40. 40. 42 Thank you all for making this movement possible Twitter: @maruyama