The document discusses the evolution of the Software Engineering Body of Knowledge (SWEBOK) and its upcoming version 4, highlighting new areas such as machine learning, software architecture, operations, and security. It emphasizes the need for patterns in machine learning software engineering and presents an integrated framework for modeling and workflow pipelines. Additionally, it addresses challenges and insights from practitioners on the use of patterns in software engineering for AI applications.

1. SWEBOK Guide Evolution and Its Emerging
Areas including Machine Learning Patterns
Hironori Washizaki
Waseda University
http://www.washi.cs.waseda.ac.jp/
APSEC 2023 Keynote
Seoul, December 6th, 2023

2. Hironori Washizaki
• Professor and the Associate Dean of the Research
Promotion Division at Waseda University in Tokyo
• Visiting Professor at the National Institute of Informatics
• Outside Directors of SYSTEM INFORMATION and
eXmotion
• APSEC ‘18 Program Chair, ‘22 General Chair, Steering
Committee
• IEEE Computer Society 2025 President
• ISO/IEC/JTC1/SC7/WG20 Convenor
• IPSJ-SIGSE Chair
2

3. Agenda
• SWEBOK Guide Evolution
• Machine Learning Software Engineering Patterns
• Integrated Framework ML System Modeling, Patterns, and
Workflow Pipelines
3
Special thanks to the SWEBOK editors, IEEE CS staff, and other
volunteers for their contributions to the SWEBOK development.

4. Knowledge Area
Topic Topic
Reference
Material
Body of Knowledge Skills Competencies Jobs / Roles
SWEBOK
Software Engineering Professional Certifications
SWECOM
EITBOK
Learning courses
4
Guide to the Software Engineering Body of Knowledge (SWEBOK)
https://www.computer.org/education/bodies-of-knowledge/software-engineering
• Guiding researchers and practitioners to identify and have
common understanding on “generally-accepted-knowledge”
in software engineering
• Foundations for certifications and educational curriculum
• ‘01 v1, ‘04 v2, ‘05 ISO adoption, ‘14 v3, ’24 v4 soon!

5. Mainframe
70’s –
Early 80’s
Late 80’s -
Early 90’s
Late 90’s -
Early 00’s
Late 00’s -
Early 10’s
PC,
Client &
server
Internet
Ubiquitous
computing
Late 10’s -
Early 20’s
IoT,
Big data,
AI
Structured
programming
Waterfall
Formalization
Design
Program
generation
Maturity
Management
Object-oriented
Req. eng.
Modeling
Verification
Reuse
Model-driven
Product-line
Global & open
Value-based
Systems eng.
Agile
Iterative &
incremental
DevOps
Empirical
Data-driven
Continuous
SE and IoT
SE and AI
SWEBOK V1
SWEBOK V2
SWEBOK V3
SWEBOK V4

6. SWEBOK Evolution from V3 to V4
• Modern engineering, practice update, BOK grows and recently developed areas
Requirements
Design
Construction
Testing
Maintenance
Configuration Management
Engineering Management
Process
Models and Methods
Quality
Professional Practice
Economics
Computing Foundations
Mathematical Foundations
Engineering Foundations
Requirements
Architecture
Design
Construction
Testing
Operations
Maintenance
Configuration Management
Engineering Management
Process
Models and Methods
Quality
Security
Professional Practice
Economics
Computing Foundations
Mathematical Foundations
Engineering Foundations
V3 V4
Agile,
DevOps
AI for
SE, SE
for AI

7. New: Software Architecture
7

8. New: Software Security
8
User
id
name
ProtectionObject
id
name
* *
Authorization_rule
Right
access_type
predicate
copy_flag
checkRights
Role
id
name
*
*
MemberOf
Role-Based Access Control (RBAC)
Problem: How do we assign rights to people
based on their functions or tasks?
Solution: Assign users to roles and give rights to
these roles so they can perform their tasks.
Related pattern: Authorization, ...

9. New: Operations
9

10. Agile and DevOps
• Agile
– Fit the current scenario: fast-moving and
changing times, full of uncertainty
– Paradigm shifts rather than just a new set of
practices
– Direct and indirect impacts on both the
engineering and the management level
• DevOps
– Critical thinking and judgment based on DevOps
values/principles/practices and schemes
– Affects decision-making at all levels of the
software engineering process and KAs
To Do Doing Done
Securing …
Product
Technical Debt …
10
Development Deploy & operation
Test & verification Monitoring
Dev Ops
Hironori Washizaki, Maria-Isabel Sanchez-Segura, Juan Garbajosa, Steve Tockey and Kenneth E Nidiffer, “Envisioning software engineer training needs in
the digital era through the SWEBOK V4 prism,” 35th IEEE International Conference on Software Engineering Education and Training (CSEE&T 2023)

11. AI and software engineering
• Limitations and challenges
– Uncertain and stochastic behavior
– Necessity of sufficiently labeled, structured datasets
• AI for SE
– Building high-quality software systems by replicating
human developers’ behavior
– Ranging over almost all development stages
• SE for AI
– Different from traditional software since the rules and
system behavior of AI systems are inferred from data
– There is a need for particular support of SE for AI
– Documenting practices as patterns
11
Software
engineering
AI
AI for SE
SE for AI

12. Mainframe
70’s –
Early 80’s
Late 80’s -
Early 90’s
Late 90’s -
Early 00’s
Late 00’s -
Early 10’s
PC,
Client &
server
Internet
Ubiquitous
computing
Late 10’s -
Early 20’s
IoT,
Big data,
AI
Structured
programming
Waterfall
Formalization
Design
Program
generation
Maturity
Management
Object-oriented
Req. eng.
Modeling
Verification
Reuse
Model-driven
Product-line
Global & open
Value-based
Systems eng.
Agile
Iterative &
incremental
DevOps
Empirical
Data-driven
Continuous
SE and IoT
SE and AI
SWEBOK V1
SWEBOK V2
SWEBOK V3
SWEBOK V4
？
？

13. IEEE CS Technology Predictions Report for 2023
13
IEEE CS Technology Prediction Team (Chair: Dejan Milojicic) https://www.computer.org/2023-top-technology-predictions
Chance of success
higher than impact
on humanity
Impact on humanity higher
than chance of success
(worth investing in)

14. Mainframe
70’s –
Early 80’s
Late 80’s -
Early 90’s
Late 90’s -
Early 00’s
Late 00’s -
Early 10’s
PC,
Client &
server
Internet
Ubiquitous
computing
Late 10’s -
Early 20’s
IoT,
Big data,
AI
GenAI, FM,
Autonomous,
Quantum,
Continuum
Late 20’s
Structured
programming
Waterfall
Formalization
Design
Program
generation
Maturity
Management
Object-oriented
Req. eng.
Modeling
Verification
Reuse
Model-driven
Product-line
Global & open
Value-based
Systems eng.
Agile
Iterative &
incremental
DevOps
Empirical
Data-driven
Continuous
SE and IoT
SE and AI
SE and GenAI
SE and QC
Sustainability
SE for
autonomous
and continuum
AI-assisted
DevOps
SWEBOK V1
SWEBOK V2
SWEBOK V3
SWEBOK V4

15. Summary and take-away
• History of software engineering and SWEBOK evolution
– Mainframe → C/S → Ubiquitous → IoT/AI/Big data → FM/QC/Autonomous
– SWEBOK: Common understanding on “generally-accepted-knowledge”
• SWBOK Guide V4
– New areas incl. architecture, operations, and security
– Agile and DevOps as cross-cutting concern
– AI and software engineering
– Public review will start soon!
• Based on SWEBOK, we/you may consider …
– Establishing and revising education and training program
– Improving engineering processes and activities
– Identifying research areas and topics
– V5 may address SE and GenAI/FM, QC, Sustainability, and Continuum
15

16. Agenda
• SWEBOK Evolution
• Machine Learning Software Engineering Patterns
• Integrated Framework ML System Modeling, Patterns, and
Workflow Pipelines
16
Special thanks to collaborators in ML software engineering patterns, such as
Foutse Khomh, Yann-Gael Gueheneuc, Hironori Takeuchi, Naotake Natori,
Takuo Doi, Satoshi Okuda, Weitao Pan, Nobukazu Yoshioka, and others

17. Example case of ML-based system design
• We wish to identify the type of
instrument for the sound picked up
by the phone and achieve recording
and response according to the type.
• However, the memory and
performance of the phone is
limited, and a large deep learning
model is unlikely to be loaded.
How can we do this?
17
Pretrained
Model
• Let's use Two-stage predictions where a
small model on the phone determines if a
sound is a musical instrument, and a large
model on the cloud classifies the type of
sound only if it is a musical instrument.
• For the large model, we will adopt Transfer
Learning to achieve precise classification.
Machine Learning Design Patterns
(V. Lakshmanan, et al. 2020)

18. Example of ML design patterns
Two-stage predictions
• Problem: There is a need to maintain the
performance of models that are large and
complex in nature, even when deployed
to edge or distributed devices.
• Solution: The utilization flow is divided
into two phases, with only the simple
phase performed on the edge.
Transfer Learning
• Problem: There is a lack of large data sets
needed to train complex machine learning
models.
• Solution: Some layers of the trained
model are taken out and the weights are
frozen and used in the new model to solve
similar problems without being trained.
18
Machine Learning Design Patterns (V. Lakshmanan, et al. 2020)

19. ML software engineering needs patterns!
• Bridge between abstract paradigms and concrete
cases/tools
– Documenting Know-Why, Know-What and Know-How
– Reusing solutions and problems
– Getting consistent architecture
• Common language among stakeholders
– Software engineers, data scientist, domain experts,
network engineers, …
19
Paradigm
Case Tool
FW
Instruction
？
？

20. AI/ML software engineering patterns
• Architecture and design patterns
– Software Engineering Patterns for ML
applications [SEP4MLA]
– Machine Learning Design Patterns
[MLDP]
• Safety and security patterns
– Safety Case Pattern for ML systems
[Safety]
– Security Argument Patterns for DNN
[Security]
• Responsible AI patterns
– Responsible AI System Design Patterns
[Responsible]
• Development and management
practices
– Lifecycle phase practices [Practice1]
– Issues and development practices
[Practice2]
• Prompt engineering patterns
– Prompt Pattern Catalog [Prompt]
20
[MLDP] V. Lakshmanan, et al., “Machine Learning Design Patterns,” O’Reilly, 2020
[SEP4MLA] H. Washizaki, et al. “Software Engineering Design Patterns for Machine Learning Applications,” IEEE Computer 55(3) 2022
[Safety] E. Wozniak, et al., “A Safety Case Pattern for Systems with Machine Learning Components,” SAFECOMP 2020 Workshop
[Security] M. Zeroual, et al., “Security Argument patterns for Deep Neural Network Development,” PLoP 2023
[Responsible] Q. Lu, et al., “Responsible-AI-by-Design: a Pattern Collection for Designing Responsible AI Systems,” IEEE Software, 2023
[Practice1] M. S. Rahman, et al., “Machine Learning Application Development: Practitioners’ Insights,” Software Quality Journal, 31, 2023.
[Practice2] Y. Watanabe, et al., “Preliminary Literature Review of Machine Learning System Development Practices,” COMPSAC 2021 Fast Abstract
[Prompt] J, White, et al., “A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT,” arXiv 2302.11382, 2023

21. SE Patterns for ML applications [Computer’22]
• 15 patterns extracted from around 40 scholarly and gray documents
21
Hironori Washizaki, Foutse Khomh, Yann-Gael Gueheneuc, Hironori Takeuchi, Naotake Natori, Takuo Doi, Satoshi Okuda,
“Software Engineering Design Patterns for Machine Learning Applications,” IEEE Computer, Vol. 55, No. 3, pp. 30-39, 2022.
Encapsulate ML Models within Rule-based
Safeguards
• Problem: ML models are known to be
unstable and vulnerable to adversarial
attacks, noise, and data drift.
• Solution: Encapsulate functionality provided
by ML models and deal with the inherent
uncertainty in the containing system using
deterministic and verifiable rules.
Business
Logic API
Rule-based
Safeguard
Inference
(Prediction)
Encapsulated
ML model
Input
Output
Rule
Explainable Proxy Model
• Problem: A surrogate ML model
must be built to provide
explainability.
• Solution: Run the explainable
inference pipeline in parallel with
the primary inference pipeline to
monitor prediction differences.
Input
Decoy model Data lake
Proxy model
(E.g., Decision
tree) Monitoring
and
comparison
Reproduce
and
retraining
Production
model
(E.g., DNN)

22. Practitioners’ insights on patterns
[ICSME’20]
• 118 developers answered
• Developers were unfamiliar with
most patterns.
• Most respondents indicated
considering the use of patterns
in future designs.
• As respondents become more
organized in their approach to
design problems by reuse, the
pattern usage ratio increased.
22
Knew it Didn’t know it
0 20 40 60 80 100 120
Data Flows Up, Model Flows Down
Secure Aggregation
Deployable Canary Model
Kappa Architecture for ML
Parameter-Server Abstraction
Different Workloads in Different Computing…
Encapsulate ML models within rule-base…
ML Gateway Routing Architecture
Lambda Architecture for ML
Separation of Concerns and Modularization of…
Distinguish Business Logic from ML Models
Data Lake for ML
Discard PoC code
Microservice Architecture for ML
ML Versioning
Used it Never used it Consider using it Not consider
Hironori Washizaki, Hironori Takeuchi, Foutse Khomh, Naotake Natori, Takuo Doi, Satoshi Okuda, “Practitioners’ insights on machine-learning software
engineering design patterns: a preliminary study,” 36th IEEE International Conference on Software Maintenance and Evolution (ICSME 2020), Late Breaking Ideas

23. (ML) patterns engineering
• Extraction: Identifying and formulating recurring problems and solutions
into “new” patterns to have reusable patterns
• Detection: Detecting “known” patterns in software processes and products
to comprehend and identify further improvement opportunities
• Application: Concretizing and deploying patterns on software processes
and products to resolve particular problems
23
• Problem: …
• Solution: ….
ML pattern
Extraction Application
Similar
results Detection
Pattern
instances

24. Pytorch vs. Keras
ML pattern detection: Example of pattern instances [APSEC’23]
24
Weitao Pan, Hironori Washizaki, Nobukazu Yoshioka, Yoshiaki Fukazawa, Foutse Khomh, Yann-Gaël Guéhéneuc,
“A Machine Learning Based Approach to Detect Machine Learning Design Patterns,” APSEC 2023 ERA
Embeddings
• Problem: High-cardinality features
where closeness relationships are
important to preserve.
• Solution: Learn to map high-
cardinality data into a lower
dimensional space in such a way
that the information relevant to
the learning problem is preserved.

25. ML pattern detection: Detection by ML
25
Revised Text-CNN Model
Weitao Pan, Hironori Washizaki, Nobukazu Yoshioka, Yoshiaki Fukazawa, Foutse Khomh, Yann-Gaël Guéhéneuc,
“A Machine Learning Based Approach to Detect Machine Learning Design Patterns,” APSEC 2023 ERA

26. ML pattern dection: Evaluation
26
Weitao Pan, Hironori Washizaki, Nobukazu Yoshioka, Yoshiaki Fukazawa, Foutse Khomh, Yann-Gaël Guéhéneuc,
“A Machine Learning Based Approach to Detect Machine Learning Design Patterns,” APSEC 2023 ERA
Text-CNN LSTM Text-RNN
ML pattern Precision Recall F1 Accuracy Pr Re F1 Acc Pr Re F1 Acc
Embeddings 0.75 0.84 0.79
0.94
0.59 0.96 0.73
0.75
0.41 0.84 0.55
0.66
Feature Cross 1 0.8 0.89 0.89 0.68 0.77 0.95 0.8 0.87
Multilabel 0.85 0.88 0.86 0.64 0.36 0.46 0.44 0.28 0.34
Hashed Feature 0.85 0.92 0.88 0.74 0.8 0.77 0.79 0.44 0.56
non-pattern 1 0.96 0.98 1 0.96 0.98 1 0.92 0.96

27. Summary and take-away
• ML software engineering needs patterns!
– Architecture and design, safety and security, responsible AI, prompt engineering …
– But, practitioners are unfamiliar with most patterns
– Engineering activities: Extraction, detection, and application
– ML pattern detection can be well achieved by ML
• We/you may consider …
– Extracting more patterns addressing specific quality attributes and aspects
– Research opportunities including detection and application over different
patterns and areas (such as IoT patterns and Agile/DevOps patterns)
– Incorporating pattern engineering into ML system processes
27

28. Agenda
• SWEBOK Evolution
• Machine Learning Software Engineering Patterns
• Integrated Framework ML System Modeling, Patterns, and
Workflow Pipelines
28
Special thanks to JST MIRAI grant for eAI project and its project members, such
as Jati H. Husen, Hironori Takeuchi, Hnin Thandar Tun, and Nobukazu Yoshioka

29. Business
Concerns
System
Concerns
Software
Concerns
Traditional
Software
Concerns
ML Software
Concerns
Costs
Revenues
Stakeholders
Integration
Safety
ML
performance
Data Quality
Model
architecture
Reliability
Experimentative
Definitive
29
How address multiple aspects in traceable
and consistent way?
=> Metamodel-based multi-view modeling
How to align different
natures together?
=> Pipeline integration
Responsible AI
patterns
ML patterns
ML Safety and
security
patterns
ML architecture
and design patterns
How to incorporate ML patterns
into development?
=> Application with configuration

30. Multi-view modeling for ML systems [MODELSWARD’23]
ML Canvas
AI Project Canvas Safety Case
Architectural Diagram (SysML) KAOS Goal Model
STAMP/STPA
Value
MLOps Architecture Goals
Safety
Argumentation
Jati Husen, Hironori Washizaki, Nobukazu Yoshioka, Hnin Tun, Yoshiaki Fukazawa and Hironori Takeuchi, “Metamodel-Based Multi-View Modeling Framework
for Machine Learning Systems,” 11th International Conference on Model-Based Software and Systems Engineering (MODELSWARD 2023)
30

31. Metamodel for consistency and traceability [ICEBE’23]
ML Canvas
AI Project Canvas
Safety Case
KAOS Goal Model
STAMP/STPA
Architecture (SysML)
ML workflow
pipeline
31
Hironori Takeuchi, Jati H. Husen, Hnin Thandar Tun, Hironori Washizaki and Nobukazu Yoshioka, “Enterprise Architecture-based Metamodel for a Holistic
Business – IT Alignment View on Machine Learning Projects,” IEEE International Conference on E-Business Engineering (ICEBE 2023), Best Paper Award

32. Metamodel
Requirements and architecture
modeling
ML configuration, training,
evaluation and repair pipelines
Refine
ment
Requirements analysis and design
DNN
evaluation
Visualizing issues
DNN
evaluation
Visualizing resolution
OK
OK OK
Not good
OK OK
OK
Not
good
OK OK OK
[ML.VP1🡨
AI.VP1]
Providereliable
real-timeobject
detectionsystem
fordriving
decisionmakingin
highway(incl.
trafficsign
detectionand
lane/vehicle
detection)
• [ML.DS1]Procured
datasets
• [ML.DS2]Internal
databasefrom
collectionduring
operation
• [ML.DC1]Openand
commercialdatasets
• [ML.DC2]Data
collectedduring
operation(imageand
identificationresult)
•[ML.F1🡨
AI.D1/AI.D3]
Boundingbox
forobject(incl.
othervehicles
orsigns)
•[ML.F2🡨
AI.D2]Ridge
detectionfor
lanedetection
[ML.BM1]
Modelswillbe
developed,
tested,and
deployedtocars
monthly
• [ML.PT1]Input:
imagefromsensors
• [ML.PT2←AI.D]
Output:trafficsigns,
lanemarking,
vehicles,and
pedestrians.
[ML.De1]Use
predictionresults
fordecision-
makinginself-
drivingsystem
[ML.IS1]
Usingtestdata,
achieveveryhigh
recallandhigh
precisionin
followingcondition:
night,rainy,and
generalcondition
Datasetsissplitinto
80:20ratio
[ML.MP1]
Predictionshould
bemadein
batchesreal
time.
[ML.M1]Inputdatamonitoring
[ML.VP1🡨
AI.VP1]
Providereliable
real-timeobject
detectionsystem
fordriving
decisionmakingin
highway(incl.
trafficsign
detectionand
lane/vehicle
detection)
• [ML.DS1]Procured
datasets
• [ML.DS2]Internal
databasefrom
collectionduring
operation
• [ML.DC1]Openand
commercialdatasets
• [ML.DC2]Data
collectedduring
operation(imageand
identificationresult)
•[ML.F1🡨
AI.D1/AI.D3]
Boundingbox
forobject(incl.
othervehicles
orsigns)
•[ML.F2🡨
AI.D2]Ridge
detectionfor
lanedetection
[ML.BM1]
Modelswillbe
developed,
tested,and
deployedtocars
monthly
• [ML.PT1]Input:
imagefromsensors
• [ML.PT2←AI.D]
Output:trafficsigns,
lanemarking,
vehicles,and
pedestrians.
[ML.De1]Use
predictionresults
fordecision-
makinginself-
drivingsystem
[ML.IS1]
Usingtestdata,
achieveveryhigh
recallandhigh
precisionin
followingcondition:
night,rainy,and
generalcondition
Datasetsissplitinto
80:20ratio
[ML.MP1]
Predictionshould
bemadein
batchesreal
time.
[ML.M1]Inputdatamonitoring
[ML.VP1🡨
AI.VP1]
Providereliable
real-timeobject
detectionsystem
fordriving
decisionmakingin
highway(incl.
trafficsign
detectionand
lane/vehicle
detection)
•[ML.DS1]Procured
datasets
•[ML.DS2]Internal
databasefrom
collectionduring
operation
•[ML.DC1]Openand
commercialdatasets
•[ML.DC2]Data
collectedduring
operation(imageand
identificationresult)
•[ML.F1🡨
AI.D1/AI.D3]
Boundingbox
forobject(incl.
othervehicles
orsigns)
•[ML.F2🡨
AI.D2]Ridge
detectionfor
lanedetection
[ML.BM1]
Modelswillbe
developed,
tested,and
deployedtocars
monthly
•[ML.PT1]Input:
imagefromsensors
•[ML.PT2←AI.D]
Output:trafficsigns,
lanemarking,
vehicles,and
pedestrians.
[ML.De1]Use
predictionresults
fordecision-
makinginself-
drivingsystem
[ML.IS1]
Usingtestdata,
achieveveryhigh
recallandhigh
precisionin
followingcondition:
night,rainy,and
generalcondition
Datasetsissplitinto
80:20ratio
[ML.MP1]
Predictionshould
bemadein
batchesreal
time.
[ML.M1]Inputdatamonitoring
Adding repair-strategy
DNN training
DNN repair
ML system modeling and pipeline integration
Machine Learning and Reliable System Analysis with Astah, https://astahblog.com/2023/10/11/machine-learning-and-reliable-system-analysis-with-astah/
32

33. Example case of image
classification in self-driving
City
Highway
AI Project Canvas
ML Canvas
Architecture
Data Skills
Output
Value
proposition
Integration
Stakeholders
Customer
Cost Revenue
How can we develop and revise a system based
on DNNs with acceptable recognition accuracy
considering safety in the city and on the highway?

34. DNN a DNN b DNN c
Evaluation of classification
Safety Case
Misclassified data Selection for repair
Balanced repair of DNN
Result of repair
Aggressive repair of DNN
Further revision
1. Dataset revision
2. Architecture
revision for
improving images
3. Revisiting
business goals
Misclassified data
STAMP/STPA KAOS Goal Model
34

35. 35

36. ML pattern
application:
General solution
36
Training data
sampling
• Problem: Requires a
large amount of
training and test data
to ensure reliability.
• Solution: Sampling
to reduce the training
set size.

37. ML pattern application:
Security argument
37
DNN Secure
development argument
• Problem: Compliance is
required in the secure
development processes
for the given DNN.
• Solution: Claim
decomposition to argue
satisfaction of security
requirements.
• E.g., “The specified
robustness guarantees
adversarial perturbations
will be recognizable by
humans.”
M. Zeroual, et al., “Security Argument patterns for Deep Neural Network Development,” PLoP 2023
Acceptable (L2 Norm = 2) Unacceptable (L2 Norm = 3)
Goal: Robustness >= 2

38. Summary and take-away
• ML integrated framework needed to
– Handle multi aspects from business to data in consistent and traceable way
– Ensure alignment between definitive and experimentative approaches by
modeling and workflow pipeline integration
– Incorporate pattern-oriented development for efficient development
• We/you may consider …
– Integration addressing various aspects such as responsible AI and
activities such as requirements engineering, testing, and optimization
– Further extensible architecture and domain-specific adaptation
– For LLM/FM (and By LLM/FM)
38

39. 39
Mainframe
70’s –
Early 80’s
Late 80’s -
Early 90’s
Late 90’s -
Early 00’s
Late 00’s -
Early 10’s
PC,
Client &
server
Internet
Ubiquitous
computing
Late 10’s -
Early 20’s
IoT,
Big data,
AI
GenAI, FM,
Autonomous,
Quantum,
Continuum
Late 20’s
Structured
programming
Waterfall
Formalization
Design
Program
generation
Maturity
Management
Object-oriented
Req. eng.
Modeling
Verification
Reuse
Model-driven
Product-line
Global & open
Value-based
Systems eng.
Agile
Iterative &
incremental
DevOps
Empirical
Data-driven
Continuous
SE and IoT
SE and AI
SE and GenAI
SE and QC
Sustainability
SE for
autonomous
and continuum
AI-assisted
DevOps
SWEBOK V1
SWEBOK V2
SWEBOK V3
SWEBOK V4
(ML) patterns engineering
• Extraction: Identifying and formulating recurring problems and solutions
into “new” patterns to have reusable patterns
• Detection: Detecting “known” patterns in software processes and products
to comprehend and identify further improvement opportunities
• Application: Concretizing and deploying patterns on software processes
and products to resolve particular problems
23
• Problem: …
• Solution: ….
ML pattern
Extraction Application
Similar
results Detection
Pattern
instances
Metamodel
Requirements and architecture
modeling
ML configuration, training,
evaluation and repair pipelines
Refine
ment
Requirements analysis and design
DNN
evaluation
Visualizing issues
DNN
evaluation
Visualizing resolution
OK
OK OK
Not good
OK OK
OK
Not
good
OK OK OK
[ML.VP1🡨
AI.VP1]
Providereliable
real-timeobject
detectionsystem
fordriving
decisionmakingin
highway(incl.
trafficsign
detectionand
lane/vehicle
detection)
• [ML.DS1]Procured
datasets
• [ML.DS2]Internal
databasefrom
collectionduring
operation
• [ML.DC1]Openand
commercialdatasets
• [ML.DC2]Data
collectedduring
operation(imageand
identificationresult)
•[ML.F1🡨
AI.D1/AI.D3]
Boundingbox
forobject(incl.
othervehicles
orsigns)
•[ML.F2🡨
AI.D2]Ridge
detectionfor
lanedetection
[ML.BM1]
Modelswillbe
developed,
tested,and
deployedtocars
monthly
• [ML.PT1]Input:
imagefromsensors
• [ML.PT2←AI.D]
Output:trafficsigns,
lanemarking,
vehicles,and
pedestrians.
[ML.De1]Use
predictionresults
fordecision-
makinginself-
drivingsystem
[ML.IS1]
Usingtestdata,
achieveveryhigh
recallandhigh
precisionin
followingcondition:
night,rainy,and
generalcondition
Datasetsissplitinto
80:20ratio
[ML.MP1]
Predictionshould
bemadein
batchesreal
time.
[ML.M1]Inputdatamonitoring
[ML.VP1🡨
AI.VP1]
Providereliable
real-timeobject
detectionsystem
fordriving
decisionmakingin
highway(incl.
trafficsign
detectionand
lane/vehicle
detection)
• [ML.DS1]Procured
datasets
• [ML.DS2]Internal
databasefrom
collectionduring
operation
• [ML.DC1]Openand
commercialdatasets
• [ML.DC2]Data
collectedduring
operation(imageand
identificationresult)
•[ML.F1🡨
AI.D1/AI.D3]
Boundingbox
forobject(incl.
othervehicles
orsigns)
•[ML.F2🡨
AI.D2]Ridge
detectionfor
lanedetection
[ML.BM1]
Modelswillbe
developed,
tested,and
deployedtocars
monthly
• [ML.PT1]Input:
imagefromsensors
• [ML.PT2←AI.D]
Output:trafficsigns,
lanemarking,
vehicles,and
pedestrians.
[ML.De1]Use
predictionresults
fordecision-
makinginself-
drivingsystem
[ML.IS1]
Usingtestdata,
achieveveryhigh
recallandhigh
precisionin
followingcondition:
night,rainy,and
generalcondition
Datasetsissplitinto
80:20ratio
[ML.MP1]
Predictionshould
bemadein
batchesreal
time.
[ML.M1]Inputdatamonitoring
[ML.VP1🡨
AI.VP1]
Providereliable
real-timeobject
detectionsystem
fordriving
decisionmakingin
highway(incl.
trafficsign
detectionand
lane/vehicle
detection)
•[ML.DS1]Procured
datasets
•[ML.DS2]Internal
databasefrom
collectionduring
operation
•[ML.DC1]Openand
commercialdatasets
•[ML.DC2]Data
collectedduring
operation(imageand
identificationresult)
•[ML.F1🡨
AI.D1/AI.D3]
Boundingbox
forobject(incl.
othervehicles
orsigns)
•[ML.F2🡨
AI.D2]Ridge
detectionfor
lanedetection
[ML.BM1]
Modelswillbe
developed,
tested,and
deployedtocars
monthly
•[ML.PT1]Input:
imagefromsensors
•[ML.PT2←AI.D]
Output:trafficsigns,
lanemarking,
vehicles,and
pedestrians.
[ML.De1]Use
predictionresults
fordecision-
makinginself-
drivingsystem
[ML.IS1]
Usingtestdata,
achieveveryhigh
recallandhigh
precisionin
followingcondition:
night,rainy,and
generalcondition
Datasetsissplitinto
80:20ratio
[ML.MP1]
Predictionshould
bemadein
batchesreal
time.
[ML.M1]Inputdatamonitoring
Adding repair-strategy
DNN training
DNN repair
ML system modeling and pipeline integration
Machine Learning and Reliable System Analysis with Astah, https://astahblog.com/2023/10/11/machine-learning-and-reliable-system-analysis-with-astah/
33
• SWEBOK for establishing education
program, improving engineering
processes, identifying research areas
• More patterns and research
opportunities over different areas and
pattern engineering
• Integration addressing various
aspects, activities, and LLM/FM