Machine Learning Systems Engineering (MLSE) is a collective effort started 5 years ago in Japan to address challenges in developing and deploying machine learning systems. Key activities included panel discussions at conferences to raise awareness among software engineers, workshops identifying gaps between ML and software engineering practices, and forming a special interest group to organize further discussions. Working groups studied challenges such as fairness, infrastructure, and development processes. International collaborations helped spread ideas to other countries. Research projects explored techniques for requirements engineering, testing, debugging and assuring quality in machine learning systems to develop the new field of machine learning systems engineering. Guidelines and books were also created to establish best practices.

1. Machine Learning Systems Engineering (MLSE):
Retrospective of Five-Year Activities in Japan
Hiroshi Maruyama*
Preferred Networks / Kao Corporation / U. Tokyo
Twitter; @maruyama
* Presentation is done by me but this is a collective effort by the MLSE members

2. Agenda
1. Backdrop
2. Launch of MLSE
3. Activities
4. Research Results
5. Retrospective

3. Googleʼs seminal paper on “technical debt” of ML systems (2015)

4. Workshop on “Towards real world implementation of ML systems”, in
conjunction with JSAI annual convention, 2016
● How to deploy ML systems for industry
applications
● Recent trends in ML businesses
● Intellectual property in ML systems
● Nikoniko-Deep Learning β
● Artificial life hackason, tried
● Evolution of communities, such as open
source
Business
Community

5. https://medium.com/@karpathy/software-2-0-a64152b37c35

6. Software 2.0: Find a program rather than write it by hand
Find a program
Set of programs that satisfy the spec
Search
algorithm
Write a program vs
ｘ
ｘ
🡺 Software 2.0 requires completely different set of skills

7. Concerns on the shortage of ML skills
Source: METI, IT人材の最新動向と将来推計に関
する調査結果 (Study results on the latest status
and the future trends of IT talents), 2016
Shortage of 48,000 talents on “big
data, IOT, and AI” in 2020

8. 8
“Shortage of skills” -- doesn’t this sound familiar?
Who can write software ⇒ Software Crisis (1960’s)
🡺 Dawn of Software Engineering!
IBM System 360
Source： Wikipedia
System 360 Instruction set
Source：Quora
https://www.quora.com/How-did-you-learn-an-assembly-language-and-which-one

9. Keynote at APRES (Asia-Pacific Requirements Engineering
Symposium) by Maruyama (2016)

10. My email to Mikio Aoyama
“We have a number of projects going on
with customers. Sometimes I feel that we
are reinventing software engineering
practices in ML.
I wonder if the SE community can help us
to establish a new engineering discipline
in ML”

11. Mikio’s email to the organizer of SES2017, suggesting a
panel discussion on SE for ML
Annual SES (Software Engineering Symposium)
is the largest event in Japan dedicated to
Software Engineering, and Mikio suggested this
is the best place to draw the interests from the
SE community

12. Agenda
1. Backdrop
2. Launch of MLSE
3. Activities
4. Research Results
5. Retrospective

13. Panel discussion on machine learning engineering at SES 2017
“This panel discussion was the
highlight of this year’s SES”
Panelists
● Fuyuki Ishikawa (NII)
● Koichi Hamada (DeNA)
● Hiroshi Maruyama (PFN)
Moderator
● Mikio Aoyama INanzan U.)
https://www.facebook.com/bonotake/posts/1504893489556668

14. A couple of meetups among SE and ML engineers revealed new gaps
● ML engineer (ME): “my improved model now gives an erroneous output for a certain input
that was ok with my previous model”
● Software engineer (SE): “What did you do with your regression test? Don’t you have one?
● ME: “...” (ah, but what does regression test mean in ML? How can we do it?)
● ME: “My customer is concerned with the safety”
● SE: “What is the invariant in your code?”
● ME: “...” (Invariant? In an ML system?)
● SE: “You are concerned with the quality. Why don’t you use stronger-typed language than
Python?”
● ME: “...” (yes, I wish Python could statically check the shape of numpy ndarray)

15. Sorrow of ML Project: “Curse of infinite PoC”
Develop
model
Evaluate
Yes, we
achieved xx%
accuracy!
Can you
make a little
better?
ML engineer Customer
How can we have a reasonable level of customer expectation?
Looks good!
But not enough for
my customer

16. Michael Jordanʼs blog on the need for new engineering discipline
https://medium.com/@mijordan3/artificial-intelligence-the-revoluti
on-hasnt-happened-yet-5e1d5812e1e7
“... we do have a major challenge on our hands in
bringing together computers and humans in ways
that enhance human life. While this challenge is
viewed by some as subservient to the creation of
“artificial intelligence,” it can also be viewed more
prosaically — but with no less reverence — as
the creation of a new branch of engineering.
Much like civil engineering and chemical
engineering in decades past, this new discipline
aims to corral the power of a few key ideas, bringing
new resources and capabilities to people, and doing
so safely. “

17. The role of engineering -- my personal view
Theories ＊ Safety Factor
Engineering as a form of agreement between engineers and the society
Civil Engineering Handbook, p999
Why do we trust bridges?
Because of the accumulated knowledge
called Civil Engineering

18. In Apr. 2018, the SIG on MLSE (Machine Learning Systems
Engineering, pronounced as “Mel-See”) is formed under JSSST
https://mlxse.connpass.com/

19. Agenda
1. Backdrop
2. Launch of MLSE
3. Activities
4. Research Results
5. Retrospective

20. MLSE kick-off meeting, Mar. 2018 (>500 participants)
Source: https://ledge.ai/mlse-symposium/
● Mikio Aoyama (Nanzan U), “Expectations to
MLSE”
● Takuya Kudo (Accenture), “Challenges in
software engineering and the new form of ML”
● Masashi Sugiyama (Riken AIP), “Current and
future of ML research”
● Akimichi Ariga (Cloudara), “ML starting from
business applications”
● Takahiro Kubo (TIS), “ML code design without
remorse”
● Shin Nakajima (NII), “Quality assurance of ML
software”
Common issues in ML-in-practice have surfaced
See https://leapmind.io/blog/2018/06/12/mlsekickoff/ for a report on the symposium

21. First things first: What are the challenges of MLSE?
Ishikawa, Fuyuki, and Nobukazu Yoshioka. "How do engineers perceive difficulties in engineering of machine-learning systems?-questionnaire survey." 2019 IEEE/ACM
Joint 7th International Workshop on Conducting Empirical Studies in Industry (CESI) and 6th International Workshop on Software Engineering Research and Industrial
Practice (SER&IP). IEEE, 2019.

22. Working groups
● ML operational infrastructures /
operations WG
● ML fairness WG
● ML system foundation WG
● Data quality engineering WG
● ML operations WG
● ML development process and case
studies WG
● ML security WG
● :
Active discussions on Discord, everybody is welcome

23. MLSE summer camps
● Main venue for community discussions
● 2-3 days, plenary / parallel sessions (workshops) + posters
● WG’s to report annual findings
● Every year new ideas are coming out
Online due to Covid-19
2019 venue in Hakone hot spa

24. Cooperative gatherings
Symposium on safety for AI/IoT systems
Symposium on ML and fairness w/ JSAI and
IBIS/ML, Jan., 2020

25. International activities -- 1/2
● iMLSE -- International Workshop on Machine Learning Systems Engineering, in conjunction with APSEC
1st iMLSE in Nara (2018) iMLSE 2020 (online)
Clark Barrett Jacomo Corbo
iMLSE 2021 (online)
Hironori Washizaki
● Shonan meeting, Nov. 2019

26. ● MLSE International Symposium (2019)
● Sanjit Seshua, “Towards Verified Artificial
Intelligence”
● Akira Sakakibara, “Engineer's Responsibility
in Machine Learning Era”
● Foutse Khomh, “Towards Debugging and
Testing Deep Learning Systems”
● Lei Ma, “ Towards Testing and Analysis of
Deep Learning Systems”
● Amel Bennaceur, “Requirements for Machine
Learning Applications”
● Rüdiger Ehlers, “The Role of Verification in the
Engineering Process of Complex
Cyber-Physical Systems That Employ Machine
Learning”
International activities -- 2/2

27. Agenda
1. Backdrop
2. Launch of MLSE
3. Activities
4. Research Results
5. Retrospective

28. Req.
Req.
e.g., low risk in a specific situation?
e.g., good prediction performance for rare cases?
Reliable model
building with
small data
Controllable model update
for local improvement and
mitigation of degradation
Fine-Grained Requirements
for Dependability AI researchers and
SE researchers
Decrease oversight of
existing AI by 50% for
rare cases of cancers
Improve existing AI to
mitigate risks over
20+ fine-grained
safety metrics
Healthcare Automotive
“Engineerable AI” Project: Overview

29. “Engineerable AI” Project: Example of Techniques
Work with Fujitsu
[ Tokui+, NeuRecover: Regression-Controlled Repair of Deep Neural Networks with Training History,
SANER’22 ]
Target Neural Network
Analyze internal behavior regarding
occurrences of undesirable error pattern
e.g., misclassification of nearby pedestrian to rider
Identify and try to fix small part of neuron weight parameters
- that affected the occurrences of the error pattern
Also use hints by looking at past versions
- “past: success, now: fail” 🡪 try to fix relevant parameters
- “past: fail, now success” 🡪 not touch relevant parameters
Avoid side-effect of causing other errors or
“shuffling” of success/failure
caused by retraining or baseline method

31. Testing, Debugging, Analysis, Repairing Techniques and their Integration
into MLOps in a human-centered & Interactive Way

32. Continuous Quality Monitoring and Assurance of AI System &
AI System Trusthworty Technique Application across Diverse Domains
AI System Continuous Integration
& Continuous Delivery

34. Rule-based safeguard, with output space transformation
DNN
Policy
Filter
in Rn
Maruyama, Hiroshi. "Guaranteeing Deep Neural Network Outputs in a Feasible Region." Proceedings of the International Workshop on
Evidence-based Security and Privacy in the Wild and the 1st International Workshop on Machine Learning Systems Engineering. 2018.
Feasible Region
Non-feasible
solutions
in feasible region

35. Machine Learning Project Canvas
https://www.mitsubishichem-hd.co.jp/news_release/pdf/190718.pdf

36. Takeuchi, Hironori, et. al “Collecting Insights and Developing Patterns for Machine Learning Projects based on Project
Practices, 14th International Joint Conference on Knowledge-Based Software Engineering (JCKBSE)
Bad “smell” in
the project

37. 37
ML system quality assurance guidelines in Japan
AIST, Machine Learning Quality Management Guideline, 2nd Edition,
https://www.digiarc.aist.go.jp/en/publication/aiqm/aiqm-guideline-en-2.1.1.0057-e26-
signed.pdf
Guideline for Quality Assurance of AI-based products and services
https://www.qa4ai.jp/download/
Guidelines on Assessment of AI Reliability in the Field of Plant Safety
https://www.meti.go.jp/english/press/2021/0330_001.html

38. The book “Machine Learning Engineering” by the community
1. What is machine learning systems engineering?
(Nakagawa, Ishikawa)
2. Project management of ML systems (Takeuchi)
3. Operation of ML systems (Horiuchi, Dobashi)
4. ML design patterns (Washizaki)
5. Quality assurance (Ishikawa)
6. Explainability of ML systems (Hara)
7. Ethics (Nakagawa)
8. Intellectual properties and contracts (Kakinuma)
9. Future of machine learning systems engineering
(Ishikawa)
ISBN-13 : 978-4065285862

39. Upcoming book on “Machine Learning Engineering for AI Project Managers”
1. Introduction to AI System Development (Yoshioka)
2. Requirement Engineering for AI Systems (Yoshioka)
3. Architecture and Design of ML Systems (Washizaki)
4. Project Management of AI Systems (Uchihira)
5. Cooperation with Stakeholders in AI Project (Takeuchi)
6. Future Vision of Machine Learning Engineering (Yoshioka)

40. Agenda
1. Backdrop
2. Launch of MLSE
3. Activities
4. Research Results
5. Retrospective

41. In five years, we …
● formed a very active community
○ From both SE and ML communities
○ From both industry and academia
○ Connecting people, sharing ideas, …
● produced research results
● helped the industry via many symposia and guideline documents, including the
quality assurance guidelines
● published a book (with more upcoming …)
Challenges ahead
● More academic activities as well as industry success stories
● Recognition as an engineering discipline by the general public
To me, perhaps this is the
biggest achievement

42. 42
Thank you all
for making this movement possible
Twitter: @maruyama
https://sites.google.com/view/sig-mlse/en