Hironori Washizaki, profile picture
Uploaded byHironori Washizaki
216 views

AI Software Engineering based on Multi-view Modeling and Engineering Patterns

Hironori Washizaki, "AI Software Engineering based on Multi-view Modeling and Engineering Patterns," The 25th International Conference on Software Quality, Reliability, and Security (QRS 2025), Keynote, Hangzhou and online, Jul 17, 2025.

Embed presentation

Downloaded 11 times
AI Software Engineering based on Multi-view Modeling and Engineering Patterns QRS 2025 July 17th, Hangzhou and online Hironori Washizaki, IEEE Computer Society President (Waseda University)
Waseda University, Tokyo, Japan • A top institution of higher education • 50,000 students in 13 undergraduate and 21 postgraduate schools • Founded in 1882 by Shigenobu Okuma, former Prime Minister of Japan • Strong alumni network of over 660,000 members: 8 prime ministers of Japan; 3 prime ministers of Korea; important figures of Japanese literature (incl. Haruki Murakami); founders of leading companies, incl. Fast Retailing (UNIQLO), Sony, Samsung, Ito En, Lotte, CJ Group, POSCO 2 Japanese University Life https://www.youtube.com/watch?v=qjTqeejCWY0 • Has hosted many international software engineering and computing conferences – SPLC 2013, IEEE ICST 2017, IEEE COMPSAC 2018 (partially), ACM VRST 2018, ICIAM 2023, IEEE VCIP 2024, IEEE CSEE&T 2023 – Strong software engineering team: Prof. Hironori Washizaki, Prof. Tomoji Kishi, and Prof. Naoyasu Ubayashi
Agenda • IEEE CS: SWEBOK Guide evolution • AI and software engineering: AI for SE and SE for AI • Multi-view Modeling • AI Engineering Patterns and Engineering
375,000+ Community Members 1031 Global Chapters 157 Countries with Computer Society Members For over 75 years, the Computer Society has empowered the people who advance technology by delivering resources and solutions that computing students and professionals need to achieve goals at all stages of their careers. 4 • Engage more students and early career professionals • Engage more industry individuals and organizations • Lead the way in new technical areas Goals • Empower and diversify volunteer base • Nimbleness in execution • Diversity and inclusion Themes
Does software engineering form a legitimate profession? 5
Legitimation of Professional Authority Professional’s judgment and advice are oriented toward a set of substantive values 6 Paul Starr, “The Social Transformation of American Medicine,” Basic Books, 1982. Knowledge and competence of the professional have been validated by a community Consensually validated knowledge and competence rest on rational, scientific grounds 6
Towards a Body of Knowledge Activities (and practices) Body of Knowledge Islands of Knowledge 7 Every profession is based on a body of knowledge (BOK), which is a collection of knowledge items or areas generally agreed to be essential to understanding a particular subject.
Knowledge Area Topic Topic Reference Material Body of Knowledge Skills Competencies Jobs / Roles SWEBOK Software Engineering Professional Certifications SWECOM EITBOK Learning courses 8 Guide to the Software Engineering Body of Knowledge (SWEBOK) https://www.computer.org/education/bodies-of-knowledge/software-engineering • Guiding researchers and practitioners to identify and have common understanding on “generally-accepted-knowledge” in software engineering • Foundations for certifications and educational curriculum • ‘01 v1, ‘04 v2, ‘05 ISO adoption, ‘14 v3, ’24 v4 released! H. Washizaki, eds., “Guide to the Software Engineering Body of Knowledge (SWEBOK Guide), Version 4.0,” IEEE Computer Society, 2024
Mainframe 70’s – Early 80’s Late 80’s - Early 90’s Late 90’s - Early 00’s Late 00’s - Early 10’s PC, Client & server Internet Ubiquitous computing Late 10’s - Early 20’s IoT, Big data, AI Structured programming Waterfall Formalization Design Program generation Maturity Management Object-oriented Req. eng. Modeling Verification Reuse Model-driven Product-line Global & open Value-based Systems eng. Agile Iterative & incremental DevOps Empirical Data-driven Continuous SE and IoT SE and AI SWEBOK V1 SWEBOK V2 SWEBOK V3 SWEBOK V4 9
SWEBOK Guide evolution from V3 to V4 • Modern engineering, practice update, BOK grows and recently developed areas Requirements Design Construction Testing Maintenance Configuration Management Engineering Management Process Models and Methods Quality Professional Practice Economics Computing Foundations Mathematical Foundations Engineering Foundations Requirements Architecture Design Construction Testing Operations Maintenance Configuration Management Engineering Management Process Models and Methods Quality Security Professional Practice Economics Computing Foundations Mathematical Foundations Engineering Foundations V3 V4 Agile, DevOps AI for SE, SE for AI H. Washizaki, eds., “Guide to the Software Engineering Body of Knowledge (SWEBOK Guide), Version 4.0,” IEEE Computer Society, 2024 Editor: H. Washizaki KA editors: A. Ihara, S. Ogata, N. Yoshioka, S. Munetoh, K. Shintani, E. Hayashiguchi and 15+ experts
IEEE CS Technology Prediction Team (Chair: Dejan Milojicic) https://www.computer.org/resources/2025-top-technology-predictions
Mainframe 70’s – Early 80’s Late 80’s - Early 90’s Late 90’s - Early 00’s Late 00’s - Early 10’s PC, Client & server Internet Ubiquitous computing Late 10’s - Early 20’s IoT, Big data, AI GenAI, FM, Autonomous, Quantum, Continuum Late 20’s – 30’s Structured programming Waterfall Formalization Design Program generation Maturity Management Object-oriented Req. eng. Modeling Verification Reuse Model-driven Product-line Global & open Value-based Systems eng. Agile Iterative & incremental DevOps Empirical Data-driven Continuous SE and IoT SE and AI SE and GenAI, Quantum, Sustainability Autonomous and Continuum AI-assisted DevOps/OpsDev SWEBOK V1 SWEBOK V2 SWEBOK V3 SWEBOK V4 12 Partially adopted from “The Trailer of the ACM 2030 Roadmap for Software Engineering”
Agenda • IEEE CS: SWEBOK Guide evolution • AI and software engineering: AI for SE and SE for AI • Multi-view Modeling • AI Engineering Patterns and Engineering
SWEBOK Guide topic: AI and software engineering • Limitations and challenges • Uncertain and stochastic behavior • Necessity of sufficiently labeled, structured datasets • AI for SE • Building high-quality software systems by replicating human developers’ behavior • Ranging over almost all development stages • SE for AI • Different from traditional software since the rules and system behavior of AI systems are inferred from data • Need for particular support of SE for AI • Documenting practices as patterns 14 Software engineering AI AI for SE SE for AI Hironori Washizaki, Foutse Khomh, Yann-Gael Gueheneuc, Hironori Takeuchi, Naotake Natori, Takuo Doi, Satoshi Okuda, “Software Engineering Design Patterns for Machine Learning Applications,” IEEE Computer, Vol. 55, No. 3, pp. 30-39, 2022. (Best Paper Award)
AI for SE: From AI-assisted dev to Human/AI co-creation(ref: AI-SEAL) [Feldt18] 15 Ref: Robert Feldt, et al., Ways of Applying Artificial Intelligence in Software Engineering, RAISE 2018, CoRR abs/1802.02033 Agentic action Process Automation level Target Product Runtime Low risk High risk Middle risk Completion Chat and exploration Human- centered AI-assisted Human/AI co- creation Developers Customers End users AI agents Reasoning and implementation
SE for AI: Techniques useful for AI/ML systems quality Training data Trained model Prediction, inference Infrastructure software system New data ML model repair Monitoring, goal-oriented modeling Testing oracle problem, balanced dataset and coverage Performance, robustness and explainability Architecture validity and quality assurance Suitability with objective, handling unexpected situations N. Uchihira, AI and Software Engineering, JUSE SQiP 2017 Eric Breck et al., The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction, IEEE Big Data 2017 Metamorphic testing Search-based testing Practices and patterns Quality measurement
Agenda • IEEE CS: SWEBOK Guide evolution • AI and software engineering: AI for SE and SE for AI • Multi-view Modeling • AI Engineering Patterns and Engineering
Business Concerns System Concerns Software Concerns Traditional Software Concerns ML Software Concerns Costs Revenues Stakeholders Integration Safety ML performance Data Quality Model architecture Reliability Experimentative Definitive 18 How to address multiple aspects in traceable and consistent way? => Metamodel-based multi-view modeling How to align different natures together? => Pipeline integration and MLOps Responsible AI patterns ML patterns ML Safety and security patterns ML architecture and design patterns How to incorporate ML patterns into development? => Application with configuration
Multi-view modeling for ML systems [SQJ’24] [ICEBE’23][FGCS’24] ML Canvas AI Project Canvas Safety Case Architectural Diagram (SysML) KAOS Goal Model STAMP/STPA Value MLOps Architecture Goals Safety Argumentation 20 Hironori Takeuchi, Jati H. Husen, Hnin Thandar Tun, Hironori Washizaki and Nobukazu Yoshioka, “Enterprise Architecture-based Metamodel for a Holistic Business – IT Alignment View on Machine Learning Projects,” IEEE International Conference on E-Business Engineering (ICEBE 2023), Best Paper Award Hironori Takeuchi, Jati H. Husenb, Hnin Thandar Tun, Hironori Washizaki, Nobukazu Yoshioka, “Enterprise Architecture-based Metamodel for Machine Learning Projects and its Management,” Future Generation Computer Systems, Elsevier, Vol. 161, pp. 135-145, 2024.
Corresponding process 21 Jati H. Husen, Hironori Washizaki, Jomphon Runpakprakun, Nobukazu Yoshioka, Hnin Thandar Tun, Yoshiaki Fukazawa, Hironori Takeuchi, “Integrated Multi-view Modeling for Reliable Machine Learning-Intensive Software Engineering,” Software Quality Journal, Vol. 32, pp. 1239–1285, Springer-Nature, 2024.
Metamodel for consistency and traceability [ICEBE’23][FGCS’24] ML Canvas AI Project Canvas Safety Case KAOS Goal Model STAMP/STPA Architecture (SysML) ML workflow pipeline 22 Hironori Takeuchi, Jati H. Husen, Hnin Thandar Tun, Hironori Washizaki and Nobukazu Yoshioka, “Enterprise Architecture-based Metamodel for a Holistic Business – IT Alignment View on Machine Learning Projects,” IEEE International Conference on E-Business Engineering (ICEBE 2023), Best Paper Award Hironori Takeuchi, Jati H. Husenb, Hnin Thandar Tun, Hironori Washizaki, Nobukazu Yoshioka, “Enterprise Architecture-based Metamodel for Machine Learning Projects and its Management,” Future Generation Computer Systems, Elsevier, Vol. 161, pp. 135-145, 2024.
Metamodel extension 23 Hiroshi Tanaka, Ide Masaru, Kazuki Munakata, Hironori Washizaki, and Nobukazu Yoshioka, “Activity-based modeling strategy for reliable machine learning system analysis targeting GUI-based applications,” 10th International Conference on Dependable Systems and Their Applications (DSA 2023) Metamodel for ML systems Extension for activity-based reliability analysis [DSA’23] Business-ML alignment model ML canvas AI canvas Extension for ML-system business alignment [FGCS’24] Hironori Takeuchi, Jati H. Husenb, Hnin Thandar Tun, Hironori Washizaki, Nobukazu Yoshioka, “Enterprise Architecture-based Metamodel for Machine Learning Projects and its Management,” Future Generation Computer Systems, Vol. 32, 2024
Example case of image classification in autonomous driving City Highway AI Project Canvas ML Canvas Architecture Data Skills Output Value proposition Integration Stakeholders Customer Cost Revenue How can we develop and revise a system based on DNNs with acceptable recognition accuracy considering safety in the city and on the highway? Jati H. Husen, Hironori Washizaki, Jomphon Runpakprakun, Nobukazu Yoshioka, Hnin Thandar Tun, Yoshiaki Fukazawa, Hironori Takeuchi, “Integrated Multi-view Modeling for Reliable Machine Learning-Intensive Software Engineering,” Software Quality Journal, Vol. 32, Springer, 2024.
Case of ML m1 m2 m3 Evaluation of classification Safety Case Misclassified data Selection for repair Balanced repair Result of repair Aggressive repair Further revision 1. Dataset revision 2. Architecture revision for improving images 3. Revisiting business goals Misclassified data STAMP/STPA KAOS Goal Model 25 Jati H. Husen, Hironori Washizaki, Jomphon Runpakprakun, Nobukazu Yoshioka, Hnin Thandar Tun, Yoshiaki Fukazawa, Hironori Takeuchi, “Integrated Multi-view Modeling for Reliable Machine Learning-Intensive Software Engineering,” Software Quality Journal, Vol. 32, Springer, 2024. Application of assurance patterns
Metamodel ML evaluation Visualizing issues ML evaluation Visualizing resolution OK OK OK Failed Failed OK OK OK OK OK OK OK [ML.VP1🡨 AI.VP1] Providereliable real-timeobject detectionsystem fordriving decisionmakingin highway(incl. trafficsign detectionand lane/vehicle detection) • [ML.DS1]Procured datasets • [ML.DS2]Internal databasefrom collectionduring operation • [ML.DC1]Openand commercialdatasets • [ML.DC2]Data collectedduring operation(imageand identificationresult) •[ML.F1🡨 AI.D1/AI.D3] Boundingbox forobject(incl. othervehicles orsigns) •[ML.F2🡨 AI.D2]Ridge detectionfor lanedetection [ML.BM1] Modelswillbe developed, tested,and deployedtocars monthly • [ML.PT1]Input: imagefromsensors • [ML.PT2←AI.D] Output:trafficsigns, lanemarking, vehicles,and pedestrians. [ML.De1]Use predictionresults fordecision- makinginself- drivingsystem [ML.IS1] Usingtestdata, achieveveryhigh recallandhigh precisionin followingcondition: night,rainy,and generalcondition Datasetsissplitinto 80:20ratio [ML.MP1] Predictionshould bemadein batchesreal time. [ML.M1]Inputdatamonitoring [ML.VP1🡨 AI.VP1] Providereliable real-timeobject detectionsystem fordriving decisionmakingin highway(incl. trafficsign detectionand lane/vehicle detection) • [ML.DS1]Procured datasets • [ML.DS2]Internal databasefrom collectionduring operation • [ML.DC1]Openand commercialdatasets • [ML.DC2]Data collectedduring operation(imageand identificationresult) •[ML.F1🡨 AI.D1/AI.D3] Boundingbox forobject(incl. othervehicles orsigns) •[ML.F2🡨 AI.D2]Ridge detectionfor lanedetection [ML.BM1] Modelswillbe developed, tested,and deployedtocars monthly • [ML.PT1]Input: imagefromsensors • [ML.PT2←AI.D] Output:trafficsigns, lanemarking, vehicles,and pedestrians. [ML.De1]Use predictionresults fordecision- makinginself- drivingsystem [ML.IS1] Usingtestdata, achieveveryhigh recallandhigh precisionin followingcondition: night,rainy,and generalcondition Datasetsissplitinto 80:20ratio [ML.MP1] Predictionshould bemadein batchesreal time. [ML.M1]Inputdatamonitoring [ML.VP1🡨 AI.VP1] Providereliable real-timeobject detectionsystem fordriving decisionmakingin highway(incl. trafficsign detectionand lane/vehicle detection) •[ML.DS1]Procured datasets •[ML.DS2]Internal databasefrom collectionduring operation •[ML.DC1]Openand commercialdatasets •[ML.DC2]Data collectedduring operation(imageand identificationresult) •[ML.F1🡨 AI.D1/AI.D3] Boundingbox forobject(incl. othervehicles orsigns) •[ML.F2🡨 AI.D2]Ridge detectionfor lanedetection [ML.BM1] Modelswillbe developed, tested,and deployedtocars monthly •[ML.PT1]Input: imagefromsensors •[ML.PT2←AI.D] Output:trafficsigns, lanemarking, vehicles,and pedestrians. [ML.De1]Use predictionresults fordecision- makinginself- drivingsystem [ML.IS1] Usingtestdata, achieveveryhigh recallandhigh precisionin followingcondition: night,rainy,and generalcondition Datasetsissplitinto 80:20ratio [ML.MP1] Predictionshould bemadein batchesreal time. [ML.M1]Inputdatamonitoring Adding repair-strategy ML training ML repair System modeling and MLOps integration [ICEBE’23][FGCS’24][SQJ’24] 26 “Integrated Multi-view Modeling for Reliable Machine Learning-Intensive Software Engineering,” Software Quality Journal, Vol. 32, Springer, 2024. “Enterprise Architecture-based Metamodel for a Holistic Business – IT Alignment View on Machine Learning Projects,” IEEE ICEBE 2023, Best Paper Award “Enterprise Architecture-based Metamodel for Machine Learning Projects and its Management,” Future Generation Computer Systems, 161, 2024 Requirements Construction Design Test Architecture Operations Economics Models and Methods Quality Requirements analysis and design
Agenda • IEEE CS: SWEBOK Guide evolution • AI and software engineering: AI for SE and SE for AI • Multi-view Modeling • AI Engineering Patterns and Engineering
Machine learning design patterns [Lakshmanan+ 20] • We wish to identify the type of instrument for the sound picked up by the phone and achieve recording and response according to the type. • However, the memory and performance of the phone is limited, and a large deep learning model is unlikely to be loaded. How can we do this? 28 Pretrained model • Let's use Two-stage predictions where a small model on the phone determines if a sound is a musical instrument, and a large model on the cloud classifies the type of sound only if it is a musical instrument. • For the large model, we will adopt Transfer Learning to achieve precise classification. Machine Learning Design Patterns (V. Lakshmanan, et al. 2020) Two-stage predictions Problem: There is a need to maintain the performance of models that are large and complex in nature, even when deployed to edge or distributed devices. Solution: The utilization flow is divided into two phases, with only the simple phase performed on the edge.
AI/ML software engineering patterns • Architecture and design patterns – Software Engineering Patterns for ML applications [SEP4MLA] – Machine Learning Design Patterns [MLDP] • Safety and security patterns – Safety Case Pattern for ML systems [Safety] – Security Argument Patterns for DNN [Security] • Responsible AI patterns – Responsible AI System Design Patterns [Responsible] • Development and management practices – Lifecycle phase practices [Practice1] – Issues and development practices [Practice2] • Prompt engineering patterns – Prompt Pattern Catalog [Prompt][Prompt2] 29 [MLDP] V. Lakshmanan, et al., “Machine Learning Design Patterns,” O’Reilly, 2020 [SEP4MLA] H. Washizaki, et al. “Software Engineering Design Patterns for Machine Learning Applications,” IEEE Computer 55(3) 2022, Best Paper Award [Safety] E. Wozniak, et al., “A Safety Case Pattern for Systems with Machine Learning Components,” SAFECOMP 2020 Workshop [Security] M. Zeroual, et al., “Security Argument patterns for Deep Neural Network Development,” PLoP 2023 [Responsible] Q. Lu, et al., “Responsible-AI-by-Design: a Pattern Collection for Designing Responsible AI Systems,” IEEE Software, 2023 [Practice1] M. S. Rahman, et al., “Machine Learning Application Development: Practitioners’ Insights,” Software Quality Journal, 31, 2023. [Practice2] Y. Watanabe, et al., “Preliminary Literature Review of Machine Learning System Development Practices,” COMPSAC 2021 Fast Abstract [Prompt1] J, White, et al., “A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT,” arXiv 2302.11382, 2023 [Prompt2] Y. Sasaki, H. Washizaki, Jialong Li, Nobukazu Yoshioka, Naoyasu Ubayashi, Yoshiaki Fukazawa, “Landscape and Taxonomy of Prompt Engineering Patterns in Software Engineering,” IEEE IT Professional (IT Pro), Vol. 27, pp. 41-49, 2025.
SE Patterns for ML applications [Computer’22] • 15 patterns extracted from around 40 scholarly and gray documents • Classified into three types: Topology, Programming, and Model Operation 30 Hironori Washizaki, Foutse Khomh, Yann-Gael Gueheneuc, Hironori Takeuchi, Naotake Natori, Takuo Doi, Satoshi Okuda, “Software Engineering Design Patterns for Machine Learning Applications,” IEEE Computer, Vol. 55, No. 3, pp. 30-39, 2022. (Best Paper Award) Encapsulate ML Models within Rule-based Safeguards • Problem: ML models are known to be unstable and vulnerable to adversarial attacks, noise, and data drift. • Solution: Encapsulate functionality provided by ML models and deal with the inherent uncertainty in the containing system using deterministic and verifiable rules. Business Logic API Rule-based Safeguard Inference (Prediction) Encapsulated ML model Input Output Rule Explainable Proxy Model • Problem: A surrogate ML model must be built to provide explainability. • Solution: Run the explainable inference pipeline in parallel with the primary inference pipeline to monitor prediction differences. Input Decoy model Data lake Proxy model (E.g., Decision tree) Monitoring and comparison Reproduce and retraining Production model (E.g., DNN)
Argument-based assurance case • Assurance case • Structured claims, arguments, and evidence which gives confidence that a system will possess the particular qualities or properties that need to be assured • Safety, security, resilience, … • Goal Structuring Notation (GSN) • Graphical notation for visualizing arguments that assure critical properties • Adoption for AI application risk- based assessment • E.g., UK ICO AI Explainability Guide 31 Jordi Cabot, Goal Structuring Notation – a short introduction https://modeling-languages.com/goal-structuring-notation-introduction/
ML security argument patterns [Zeroual+23] 32 DNN Secure development argument • Problem: Compliance is required in the secure development processes for the given DNN. • Solution: Claim decomposition to argue satisfaction of security requirements. • E.g., “The specified robustness guarantees adversarial perturbations will be recognizable by humans.” M. Zeroual, et al., “Security Argument patterns for Deep Neural Network Development,” PLoP 2023 Goal (Claim) Solution (Evidence) Context Strategy
33 ML assurance argument patterns (e.g., “DNN Robustness Case Verification”) [AsianPLoP’24] Goal (Claim) Solution (Evidence) Strategy M. Mutsche, et al. “Robustness-based Security Case Verification for Deep Neural Networks,” AsianPLoP 2024
Yuya Sasaki, Hironori Washizaki, Jialong Li, Nobukazu Yoshioka, Naoyasu Ubayashi, Yoshiaki Fukazawa, “Landscape and Taxonomy of Prompt Engineering Patterns in Software Engineering,” IT Pro, 27, 2025. User-Model Collaboration Refinement • Problem: Balancing between accuracy and confidence is challenging, especially when users need guidance to achieve goals. • Solution: Interactive prompts that assign LLM to simulate role, instead of user-driven only prompting. Prompt Engineering Patterns [Sasaki+25] 34 Persona Game Question From LLM Simulation Interactive Prompt Templates This is a conversation with Socrates, an eager and helpful, but humble expert automatic AI... I would like you to ask me questions to achieve X. You should ask questions until this condition is met or to achieve this goal...
• Problem: … Pattern engineering • Extraction: Identifying and formulating recurring problems and solutions into “new” patterns to have reusable patterns • Detection: Detecting “known” patterns in software processes and products to comprehend and identify further improvement opportunities • Application: Selecting, concretizing and deploying patterns on software processes and products to resolve particular problems • Organization: Organizing patterns to build a system (i.e., language) of patterns • Integration: Integrating into pattern-oriented development and management 35 • Problem: … • Solution: …. AI/ML pattern Extraction Application Similar results Detection Pattern instances Organization Process Management Integration
Pytorch vs. Keras ML pattern detection: Example of pattern instances [APSEC’23] 36 Weitao Pan, Hironori Washizaki, Nobukazu Yoshioka, Yoshiaki Fukazawa, Foutse Khomh, Yann-Gaël Guéhéneuc, “A Machine Learning Based Approach to Detect Machine Learning Design Patterns,” APSEC 2023 ERA Embeddings • Problem: High-cardinality features where closeness relationships are important to preserve. • Solution: Learn to map high- cardinality data into a lower dimensional space in such a way that the information relevant to the learning problem is preserved.
ML pattern detection by ML 37 Revised Text-CNN Model Weitao Pan, Hironori Washizaki, Nobukazu Yoshioka, Yoshiaki Fukazawa, Foutse Khomh, Yann-Gaël Guéhéneuc, “A Machine Learning Based Approach to Detect Machine Learning Design Patterns,” APSEC 2023 ERA
ML pattern application: Deployment of assurance argument patterns [AsianPLoP’24] 38 1. Identifying and showing pattern candidates: OCL constraints-based and LLM-based 2. Showing potential impact and result before deployment 3. Deploying a selected pattern T. Ayukawa, et al., “Machine Learning Design Pattern Application Support,” AsianPLoP 2024
Design patterns IoT design patterns Security and safety patterns Pattern organization: Towards a pattern language … OK, so, to attract many people to our city, Small Public Squares should be located in the center. At the SMALL PUBLIC SQUARE, make Street Cafes be Opening to the Street ... 39 Small Public Square Street Cafe Opening to the Street https://unsplash.com/photos/EdpbTj3Br-Y https://unsplash.com/photos/GqurqYbj7aU https://unsplash.com/photos/zFoRwZirFvY Responsible AI patterns AI assurance argument patterns AI architecture and design patterns
SWEBOK Guide evolution from V3 to V4 • Modern engineering, practice update, BOK grows and recently developed areas Requirements Design Construction Testing Maintenance Configuration Management Engineering Management Process Models and Methods Quality Professional Practice Economics Computing Foundations Mathematical Foundations Engineering Foundations Requirements Architecture Design Construction Testing Operations Maintenance Configuration Management Engineering Management Process Models and Methods Quality Security Professional Practice Economics Computing Foundations Mathematical Foundations Engineering Foundations V3 V4 Agile, DevOps AI for SE, SE for AI H. Washizaki, eds., “Guide to the Software Engineering Body of Knowledge (SWEBOK Guide), Version 4.0,” IEEE Computer Society, 2024 Editor: H. Washizaki KA editors: A. Ihara, S. Ogata, N. Yoshioka, S. Munetoh, K. Shintani, E. Hayashiguchi and 15+ experts Metamodel ML evaluation Visualizing issues ML evaluation Visualizing resolution OK OK OK Failed Failed OK OK OK OK OK OK OK [ML.VP1🡨 AI.VP1] Providereliable real-timeobject detectionsystem fordriving decisionmakingin highway(incl. trafficsign detectionand lane/vehicle detection) • [ML.DS1]Procured datasets • [ML.DS2]Internal databasefrom collectionduring operation • [ML.DC1]Openand commercialdatasets • [ML.DC2]Data collectedduring operation(imageand identificationresult) •[ML.F1🡨 AI.D1/AI.D3] Boundingbox forobject(incl. othervehicles orsigns) •[ML.F2🡨 AI.D2]Ridge detectionfor lanedetection [ML.BM1] Modelswillbe developed, tested,and deployedtocars monthly • [ML.PT1]Input: imagefromsensors • [ML.PT2←AI.D] Output:trafficsigns, lanemarking, vehicles,and pedestrians. [ML.De1]Use predictionresults fordecision- makinginself- drivingsystem [ML.IS1] Usingtestdata, achieveveryhigh recallandhigh precisionin followingcondition: night,rainy,and generalcondition Datasetsissplitinto 80:20ratio [ML.MP1] Predictionshould bemadein batchesreal time. [ML.M1]Inputdatamonitoring [ML.VP1🡨 AI.VP1] Providereliable real-timeobject detectionsystem fordriving decisionmakingin highway(incl. trafficsign detectionand lane/vehicle detection) • [ML.DS1]Procured datasets • [ML.DS2]Internal databasefrom collectionduring operation • [ML.DC1]Openand commercialdatasets • [ML.DC2]Data collectedduring operation(imageand identificationresult) •[ML.F1🡨 AI.D1/AI.D3] Boundingbox forobject(incl. othervehicles orsigns) •[ML.F2🡨 AI.D2]Ridge detectionfor lanedetection [ML.BM1] Modelswillbe developed, tested,and deployedtocars monthly • [ML.PT1]Input: imagefromsensors • [ML.PT2←AI.D] Output:trafficsigns, lanemarking, vehicles,and pedestrians. [ML.De1]Use predictionresults fordecision- makinginself- drivingsystem [ML.IS1] Usingtestdata, achieveveryhigh recallandhigh precisionin followingcondition: night,rainy,and generalcondition Datasetsissplitinto 80:20ratio [ML.MP1] Predictionshould bemadein batchesreal time. [ML.M1]Inputdatamonitoring [ML.VP1🡨 AI.VP1] Providereliable real-timeobject detectionsystem fordriving decisionmakingin highway(incl. trafficsign detectionand lane/vehicle detection) •[ML.DS1]Procured datasets •[ML.DS2]Internal databasefrom collectionduring operation •[ML.DC1]Openand commercialdatasets •[ML.DC2]Data collectedduring operation(imageand identificationresult) •[ML.F1🡨 AI.D1/AI.D3] Boundingbox forobject(incl. othervehicles orsigns) •[ML.F2🡨 AI.D2]Ridge detectionfor lanedetection [ML.BM1] Modelswillbe developed, tested,and deployedtocars monthly •[ML.PT1]Input: imagefromsensors •[ML.PT2←AI.D] Output:trafficsigns, lanemarking, vehicles,and pedestrians. [ML.De1]Use predictionresults fordecision- makinginself- drivingsystem [ML.IS1] Usingtestdata, achieveveryhigh recallandhigh precisionin followingcondition: night,rainy,and generalcondition Datasetsissplitinto 80:20ratio [ML.MP1] Predictionshould bemadein batchesreal time. [ML.M1]Inputdatamonitoring Adding repair-strategy ML training ML repair System modeling and MLOps integration [ICEBE’23][FGCS’24][SQJ’24] 26 “Integrated Multi-view Modeling for Reliable Machine Learning-Intensive Software Engineering,” Software Quality Journal, Vol. 32, Springer, 2024. “Enterprise Architecture-based Metamodel for a Holistic Business – IT Alignment View on Machine Learning Projects,” IEEE ICEBE 2023, Best Paper Award “Enterprise Architecture-based Metamodel for Machine Learning Projects and its Management,” Future Generation Computer Systems, 161, 2024 Requirements Construction Design Test Architecture Operations Economics Models and Methods Quality Requirements analysis and design • Problem: … Pattern engineering • Extraction: Identifying and formulating recurring problems and solutions into “new” patterns to have reusable patterns • Detection: Detecting “known” patterns in software processes and products to comprehend and identify further improvement opportunities • Application: Selecting, concretizing and deploying patterns on software processes and products to resolve particular problems • Organization: Organizing patterns to build a system (i.e., language) of patterns • Integration: Integrating into pattern-oriented development and management 35 • Problem: … • Solution: …. AI/ML pattern Extraction Application Similar results Detection Pattern instances Organization Process Management Integration SE for AI: Techniques useful for AI/ML systems quality Training data Trained model Prediction, inference Infrastructure software system New data ML model repair Monitoring, goal-oriented modeling Testing oracle problem, balanced dataset and coverage Performance, robustness and explainability Architecture validity and quality assurance Suitability with objective, handling unexpected situations N. Uchihira, AI and Software Engineering, JUSE SQiP 2017 Eric Breck et al., The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction, IEEE Big Data 2017 Metamorphic testing Search-based testing Practices and patterns Quality measurement

More Related Content

PPTX
ppt about chatgpt.pptx
bySrinivas237938
 
PPTX
QSpiders - Aptitude Assignments
byQspiders - Software Testing Training Institute
 
PPTX
Basics-of-R-programming.9625714.powerpoint.pptx
byMSANDHYARANI3
 
PPTX
Diagnosis test of diabetics and hypertension by AI
byDr. C.V. Suresh Babu
 
PPTX
End to End In-Vehicle Infotainment Test Solution - MaxEye Technologies
byMaxEye Technologies Private Limited
 
PPTX
ChatGPT.pptx
bynitindhiman53
 
PDF
ChatGPT and the Future of Work - Clark Boyd
byClark Boyd
 
PDF
Biz Model for ARM
byJeffrey Funk Business Models
 
ppt about chatgpt.pptx
bySrinivas237938
 
QSpiders - Aptitude Assignments
byQspiders - Software Testing Training Institute
 
Basics-of-R-programming.9625714.powerpoint.pptx
byMSANDHYARANI3
 
Diagnosis test of diabetics and hypertension by AI
byDr. C.V. Suresh Babu
 
End to End In-Vehicle Infotainment Test Solution - MaxEye Technologies
byMaxEye Technologies Private Limited
 
ChatGPT.pptx
bynitindhiman53
 
ChatGPT and the Future of Work - Clark Boyd
byClark Boyd
 
Biz Model for ARM
byJeffrey Funk Business Models
 

What's hot

PPTX
Car electronization trend in automotive industry
byKenji Suzuki
 
PDF
Microsoft + OpenAI: Recent Updates (Machine Learning 15minutes! Broadcast #74)
byNaoki (Neo) SATO
 
PDF
DESIGN OF LOW POWER SAR ADC FOR ECG USING 45nm CMOS TECHNOLOGY
byVLSICS Design
 
PDF
ChatGPT webinar slides
byAlireza Esmikhani
 
PDF
Embedded System -Lyla B Das.pdf
byJohnMcClaine2
 
PDF
Microsoft365-Copilot-Partner-Guide
byErol GIRAUDY
 
Car electronization trend in automotive industry
byKenji Suzuki
 
Microsoft + OpenAI: Recent Updates (Machine Learning 15minutes! Broadcast #74)
byNaoki (Neo) SATO
 
DESIGN OF LOW POWER SAR ADC FOR ECG USING 45nm CMOS TECHNOLOGY
byVLSICS Design
 
ChatGPT webinar slides
byAlireza Esmikhani
 
Embedded System -Lyla B Das.pdf
byJohnMcClaine2
 
Microsoft365-Copilot-Partner-Guide
byErol GIRAUDY
 

Similar to AI Software Engineering based on Multi-view Modeling and Engineering Patterns

PDF
Impact of IEEE Computer Society in Advancing Emerging Technologies including ...
byHironori Washizaki
 
PDF
SWEBOK Guide and Software Services Engineering Education
byHironori Washizaki
 
PDF
SWEBOK Guide Evolution and Its Emerging Areas including Machine Learning Patt...
byHironori Washizaki
 
PDF
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
byHironori Washizaki
 
PDF
IEEE-CS Tech Predictions, SWEBOK and Quantum Software: Towards Q-SWEBOK
byHironori Washizaki
 
PDF
SWEBOK and Education at FUSE Okinawa 2024
byHironori Washizaki
 
PDF
IEEE Computer Society 2025 Vision and Future
byHironori Washizaki
 
PDF
SWEBOK: the software engineering body of knowledge (SC25 Workshop Research So...
byHironori Washizaki
 
PDF
Software Engineering 10th Edition by Ian Sommerville (eBook PDF)
byaddarkemot
 
PDF
Software Engineering 10th Edition by Ian Sommerville (eBook PDF)
bydmdzkrhdtu892
 
PDF
Impact of IEEE Computer Society in Advancing Software Engineering and Emergin...
byHironori Washizaki
 
PPTX
Making Software and Software Engineering visible
byCREST
 
DOCX
software engineering notes pdf jntuh R18
byetyalamaheshwari777
 
PPTX
[DSC Europe 22] AI Ethics and AI Quality By Design - Muthu Ramachandran
byDataScienceConferenc1
 
PDF
Software Engineering 10th Edition by Ian Sommerville (eBook PDF)
byhekelhombsca
 
PPTX
Se introduction lec 1
byAmir Shahzad
 
PPTX
Software Engineering: Education and Industry in Portugal
byCarlos J. Costa
 
PDF
IEEE Software Testing Technology Development Trend
byHironori Washizaki
 
PDF
(eBook PDF) Software Engineering 10th Edition
byhectamaxmut
 
PDF
Swebok guide 2004
byMiguel Mora Escobar
 
Impact of IEEE Computer Society in Advancing Emerging Technologies including ...
byHironori Washizaki
 
SWEBOK Guide and Software Services Engineering Education
byHironori Washizaki
 
SWEBOK Guide Evolution and Its Emerging Areas including Machine Learning Patt...
byHironori Washizaki
 
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
byHironori Washizaki
 
IEEE-CS Tech Predictions, SWEBOK and Quantum Software: Towards Q-SWEBOK
byHironori Washizaki
 
SWEBOK and Education at FUSE Okinawa 2024
byHironori Washizaki
 
IEEE Computer Society 2025 Vision and Future
byHironori Washizaki
 
SWEBOK: the software engineering body of knowledge (SC25 Workshop Research So...
byHironori Washizaki
 
Software Engineering 10th Edition by Ian Sommerville (eBook PDF)
byaddarkemot
 
Software Engineering 10th Edition by Ian Sommerville (eBook PDF)
bydmdzkrhdtu892
 
Impact of IEEE Computer Society in Advancing Software Engineering and Emergin...
byHironori Washizaki
 
Making Software and Software Engineering visible
byCREST
 
software engineering notes pdf jntuh R18
byetyalamaheshwari777
 
[DSC Europe 22] AI Ethics and AI Quality By Design - Muthu Ramachandran
byDataScienceConferenc1
 
Software Engineering 10th Edition by Ian Sommerville (eBook PDF)
byhekelhombsca
 
Se introduction lec 1
byAmir Shahzad
 
Software Engineering: Education and Industry in Portugal
byCarlos J. Costa
 
IEEE Software Testing Technology Development Trend
byHironori Washizaki
 
(eBook PDF) Software Engineering 10th Edition
byhectamaxmut
 
Swebok guide 2004
byMiguel Mora Escobar
 

More from Hironori Washizaki

PDF
書籍紹介『生成AIによるソフトウェア開発: 設計からテスト、マネジメントまでを全て変革するLLM活用の実践体系』
byHironori Washizaki
 
PDF
『生成AIによるソフトウェア開発』（鷲崎弘宜, 鵜林尚靖, 中川尊雄, 増田航太, 徳本晋, 近藤将成, 石川冬樹, 竹之内啓太, 小川秀人, スマートエ...
byHironori Washizaki
 
PDF
LLM/生成AI＆エージェントによるソフトウェア開発の実践と展望（SES2025チュートリアル）
byHironori Washizaki
 
PDF
鷲崎弘宜, "AI/LLM時代のソフトウェエンジニアリング", 情報学科・専攻協議会 総会・研究会, 早稲田大学, 2025年7月26日
byHironori Washizaki
 
PDF
Landscape of Requirements Engineering for/by AI through Literature Review
byHironori Washizaki
 
PDF
鷲崎弘宜, "高品質なAIシステムの開発・運用のための"フレームワーク", eAIシンポジウム 2025年1月16日
byHironori Washizaki
 
PDF
AI/IoTをベースにしたDX人材育成の産学連携育成, 愛媛県デジタル人材育成シンポジウム, 2024年12月20日
byHironori Washizaki
 
PDF
コンピューティングおよびソフトウェア工学の潮流: IEEE-CS技術予測＆SWEBOK Guideに基づくAI・アジャイル・サステナビリティの展望
byHironori Washizaki
 
PDF
鷲崎弘宜, "機械学習システムの多面的モデリング・パイプライン統合フレームワーク", 第6回 AI/IoTシステム安全性シンポジウム, 2024
byHironori Washizaki
 
PDF
Opening, 1st International Workshop on Patterns and Practices of Reliable AI ...
byHironori Washizaki
 
PDF
The Global Impact of IEEE Computer Society in Advancing Software Engineering ...
byHironori Washizaki
 
PDF
Overview of ISO/IEC/JTC1 SC7/WG20: Certification of software and systems engi...
byHironori Washizaki
 
PDF
次世代AI時代のトレンドと高信頼AIソフトウェアシステム開発に向けたフレームワーク＆パターン
byHironori Washizaki
 
PDF
「スマートエスイー」におけるスマートシステム＆サービスおよびDX推進人材の産学連携育成ならびに参照モデルに基づく育成プログラム分析
byHironori Washizaki
 
PDF
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
byHironori Washizaki
 
PDF
Machine Learning Software Engineering Patterns and Their Engineering
byHironori Washizaki
 
PDF
IEEE Computer Society 2024 Technology Predictions Update
byHironori Washizaki
 
PDF
鷲崎弘宜, "国際規格ISO/IEC 24773とその意義", 情報処理学会 第86回全国大会
byHironori Washizaki
 
PDF
TISO/IEC JTC1におけるソフトウェア工学知識体系、技術者認証および品質の標準化と研究・教育他への活用
byHironori Washizaki
 
PDF
アジャイル品質のパターンとメトリクス Agile Quality Patterns and Metrics (QA2AQ) 20240225
byHironori Washizaki
 
書籍紹介『生成AIによるソフトウェア開発: 設計からテスト、マネジメントまでを全て変革するLLM活用の実践体系』
byHironori Washizaki
 
『生成AIによるソフトウェア開発』（鷲崎弘宜, 鵜林尚靖, 中川尊雄, 増田航太, 徳本晋, 近藤将成, 石川冬樹, 竹之内啓太, 小川秀人, スマートエ...
byHironori Washizaki
 
LLM/生成AI＆エージェントによるソフトウェア開発の実践と展望（SES2025チュートリアル）
byHironori Washizaki
 
鷲崎弘宜, "AI/LLM時代のソフトウェエンジニアリング", 情報学科・専攻協議会 総会・研究会, 早稲田大学, 2025年7月26日
byHironori Washizaki
 
Landscape of Requirements Engineering for/by AI through Literature Review
byHironori Washizaki
 
鷲崎弘宜, "高品質なAIシステムの開発・運用のための"フレームワーク", eAIシンポジウム 2025年1月16日
byHironori Washizaki
 
AI/IoTをベースにしたDX人材育成の産学連携育成, 愛媛県デジタル人材育成シンポジウム, 2024年12月20日
byHironori Washizaki
 
コンピューティングおよびソフトウェア工学の潮流: IEEE-CS技術予測＆SWEBOK Guideに基づくAI・アジャイル・サステナビリティの展望
byHironori Washizaki
 
鷲崎弘宜, "機械学習システムの多面的モデリング・パイプライン統合フレームワーク", 第6回 AI/IoTシステム安全性シンポジウム, 2024
byHironori Washizaki
 
Opening, 1st International Workshop on Patterns and Practices of Reliable AI ...
byHironori Washizaki
 
The Global Impact of IEEE Computer Society in Advancing Software Engineering ...
byHironori Washizaki
 
Overview of ISO/IEC/JTC1 SC7/WG20: Certification of software and systems engi...
byHironori Washizaki
 
次世代AI時代のトレンドと高信頼AIソフトウェアシステム開発に向けたフレームワーク＆パターン
byHironori Washizaki
 
「スマートエスイー」におけるスマートシステム＆サービスおよびDX推進人材の産学連携育成ならびに参照モデルに基づく育成プログラム分析
byHironori Washizaki
 
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
byHironori Washizaki
 
Machine Learning Software Engineering Patterns and Their Engineering
byHironori Washizaki
 
IEEE Computer Society 2024 Technology Predictions Update
byHironori Washizaki
 
鷲崎弘宜, "国際規格ISO/IEC 24773とその意義", 情報処理学会 第86回全国大会
byHironori Washizaki
 
TISO/IEC JTC1におけるソフトウェア工学知識体系、技術者認証および品質の標準化と研究・教育他への活用
byHironori Washizaki
 
アジャイル品質のパターンとメトリクス Agile Quality Patterns and Metrics (QA2AQ) 20240225
byHironori Washizaki
 

Recently uploaded

PPTX
Windows File System Monitoring Tool Using C and C++
byAkshay Muley
 
PDF
Malware_Detection_A_Framework_for_Reverse_Engineered_Android_Applications_Thr...
byKiritiyadavGoskula
 
PPTX
iufdh gugh w4i rvhdio tj43ko jdi90ft kj32whidglt
byjeromeschool1976
 
PDF
🤖🧰 SPRING AI AGENTIC PATTERNS (PART 1): AGENT SKILLS
byVincent Vauban
 
PDF
LogiNext Media Kit & Company Overview 2025
byswatishahz9031
 
PDF
Manual Testing Still Matters: 7 Areas Where Humans Beat Automation
byKiwiQA Services
 
PDF
LogiNext Media Kit & Company Overview 2025
bysmritipatil055
 
PDF
LogiNext Media Kit & Company Overview 2025
bysanikaroy72
 
PDF
GDG ON CAMPUS NETAJI SUBHASH ENGINEERING COLLEGE BUILDING INTELLIGENT WEB APP...
bydscnsecorg
 
PDF
Smart ways to se AI in your business , presentation
byAI n Dot Net
 
PDF
Introducing MySQL HeatWave Migration Assistant
byAlexander Hitrov
 
PDF
Pet Care Service App Development Company
byV3CUBE TECHNOLABS
 
PDF
Own Your Domain Before It Owns You - Domain Driven Design for Pragmaticians
byPosedio GmbH
 
DOCX
The Rise of Headless CMS in Modern Web Development.docx
bywork4noahmiller
 
PDF
AI-native development: from Prompt to Platform
byMaxim Salnikov
 
PDF
Water Delivery App Development Solutions.pdf
byeSiteWorld TechnoLabs Pvt. Ltd.
 
PDF
Accelerating Data Validation with QuerySurge AI
byRTTS
 
PDF
谷歌留痕技术教程[ 𝙩𝙤𝙥 𝟮𝟯𝟯. 𝙘 𝙤𝙢 ]
by6stynggfo
 
PDF
Physiotherapist App Development for Modern Rehabilitation
byV3CUBE TECHNOLABS
 
PPTX
Java Spring Boot Development Services in USA
bydavidjohansen1597
 
Windows File System Monitoring Tool Using C and C++
byAkshay Muley
 
Malware_Detection_A_Framework_for_Reverse_Engineered_Android_Applications_Thr...
byKiritiyadavGoskula
 
iufdh gugh w4i rvhdio tj43ko jdi90ft kj32whidglt
byjeromeschool1976
 
🤖🧰 SPRING AI AGENTIC PATTERNS (PART 1): AGENT SKILLS
byVincent Vauban
 
LogiNext Media Kit & Company Overview 2025
byswatishahz9031
 
Manual Testing Still Matters: 7 Areas Where Humans Beat Automation
byKiwiQA Services
 
LogiNext Media Kit & Company Overview 2025
bysmritipatil055
 
LogiNext Media Kit & Company Overview 2025
bysanikaroy72
 
GDG ON CAMPUS NETAJI SUBHASH ENGINEERING COLLEGE BUILDING INTELLIGENT WEB APP...
bydscnsecorg
 
Smart ways to se AI in your business , presentation
byAI n Dot Net
 
Introducing MySQL HeatWave Migration Assistant
byAlexander Hitrov
 
Pet Care Service App Development Company
byV3CUBE TECHNOLABS
 
Own Your Domain Before It Owns You - Domain Driven Design for Pragmaticians
byPosedio GmbH
 
The Rise of Headless CMS in Modern Web Development.docx
bywork4noahmiller
 
AI-native development: from Prompt to Platform
byMaxim Salnikov
 
Water Delivery App Development Solutions.pdf
byeSiteWorld TechnoLabs Pvt. Ltd.
 
Accelerating Data Validation with QuerySurge AI
byRTTS
 
谷歌留痕技术教程[ 𝙩𝙤𝙥 𝟮𝟯𝟯. 𝙘 𝙤𝙢 ]
by6stynggfo
 
Physiotherapist App Development for Modern Rehabilitation
byV3CUBE TECHNOLABS
 
Java Spring Boot Development Services in USA
bydavidjohansen1597
 

AI Software Engineering based on Multi-view Modeling and Engineering Patterns

  • 1.
    AI Software Engineeringbased on Multi-view Modeling and Engineering Patterns QRS 2025 July 17th, Hangzhou and online Hironori Washizaki, IEEE Computer Society President (Waseda University)
  • 2.
    Waseda University, Tokyo,Japan • A top institution of higher education • 50,000 students in 13 undergraduate and 21 postgraduate schools • Founded in 1882 by Shigenobu Okuma, former Prime Minister of Japan • Strong alumni network of over 660,000 members: 8 prime ministers of Japan; 3 prime ministers of Korea; important figures of Japanese literature (incl. Haruki Murakami); founders of leading companies, incl. Fast Retailing (UNIQLO), Sony, Samsung, Ito En, Lotte, CJ Group, POSCO 2 Japanese University Life https://www.youtube.com/watch?v=qjTqeejCWY0 • Has hosted many international software engineering and computing conferences – SPLC 2013, IEEE ICST 2017, IEEE COMPSAC 2018 (partially), ACM VRST 2018, ICIAM 2023, IEEE VCIP 2024, IEEE CSEE&T 2023 – Strong software engineering team: Prof. Hironori Washizaki, Prof. Tomoji Kishi, and Prof. Naoyasu Ubayashi
  • 3.
    Agenda • IEEE CS:SWEBOK Guide evolution • AI and software engineering: AI for SE and SE for AI • Multi-view Modeling • AI Engineering Patterns and Engineering
  • 4.
    375,000+ Community Members 1031 Global Chapters 157 Countrieswith Computer Society Members For over 75 years, the Computer Society has empowered the people who advance technology by delivering resources and solutions that computing students and professionals need to achieve goals at all stages of their careers. 4 • Engage more students and early career professionals • Engage more industry individuals and organizations • Lead the way in new technical areas Goals • Empower and diversify volunteer base • Nimbleness in execution • Diversity and inclusion Themes
  • 5.
    Does software engineeringform a legitimate profession? 5
  • 6.
    Legitimation of ProfessionalAuthority Professional’s judgment and advice are oriented toward a set of substantive values 6 Paul Starr, “The Social Transformation of American Medicine,” Basic Books, 1982. Knowledge and competence of the professional have been validated by a community Consensually validated knowledge and competence rest on rational, scientific grounds 6
  • 7.
    Towards a Bodyof Knowledge Activities (and practices) Body of Knowledge Islands of Knowledge 7 Every profession is based on a body of knowledge (BOK), which is a collection of knowledge items or areas generally agreed to be essential to understanding a particular subject.
  • 8.
    Knowledge Area Topic Topic Reference Material Bodyof Knowledge Skills Competencies Jobs / Roles SWEBOK Software Engineering Professional Certifications SWECOM EITBOK Learning courses 8 Guide to the Software Engineering Body of Knowledge (SWEBOK) https://www.computer.org/education/bodies-of-knowledge/software-engineering • Guiding researchers and practitioners to identify and have common understanding on “generally-accepted-knowledge” in software engineering • Foundations for certifications and educational curriculum • ‘01 v1, ‘04 v2, ‘05 ISO adoption, ‘14 v3, ’24 v4 released! H. Washizaki, eds., “Guide to the Software Engineering Body of Knowledge (SWEBOK Guide), Version 4.0,” IEEE Computer Society, 2024
  • 9.
    Mainframe 70’s – Early 80’s Late80’s - Early 90’s Late 90’s - Early 00’s Late 00’s - Early 10’s PC, Client & server Internet Ubiquitous computing Late 10’s - Early 20’s IoT, Big data, AI Structured programming Waterfall Formalization Design Program generation Maturity Management Object-oriented Req. eng. Modeling Verification Reuse Model-driven Product-line Global & open Value-based Systems eng. Agile Iterative & incremental DevOps Empirical Data-driven Continuous SE and IoT SE and AI SWEBOK V1 SWEBOK V2 SWEBOK V3 SWEBOK V4 9
  • 10.
    SWEBOK Guide evolutionfrom V3 to V4 • Modern engineering, practice update, BOK grows and recently developed areas Requirements Design Construction Testing Maintenance Configuration Management Engineering Management Process Models and Methods Quality Professional Practice Economics Computing Foundations Mathematical Foundations Engineering Foundations Requirements Architecture Design Construction Testing Operations Maintenance Configuration Management Engineering Management Process Models and Methods Quality Security Professional Practice Economics Computing Foundations Mathematical Foundations Engineering Foundations V3 V4 Agile, DevOps AI for SE, SE for AI H. Washizaki, eds., “Guide to the Software Engineering Body of Knowledge (SWEBOK Guide), Version 4.0,” IEEE Computer Society, 2024 Editor: H. Washizaki KA editors: A. Ihara, S. Ogata, N. Yoshioka, S. Munetoh, K. Shintani, E. Hayashiguchi and 15+ experts
  • 11.
    IEEE CS TechnologyPrediction Team (Chair: Dejan Milojicic) https://www.computer.org/resources/2025-top-technology-predictions
  • 12.
    Mainframe 70’s – Early 80’s Late80’s - Early 90’s Late 90’s - Early 00’s Late 00’s - Early 10’s PC, Client & server Internet Ubiquitous computing Late 10’s - Early 20’s IoT, Big data, AI GenAI, FM, Autonomous, Quantum, Continuum Late 20’s – 30’s Structured programming Waterfall Formalization Design Program generation Maturity Management Object-oriented Req. eng. Modeling Verification Reuse Model-driven Product-line Global & open Value-based Systems eng. Agile Iterative & incremental DevOps Empirical Data-driven Continuous SE and IoT SE and AI SE and GenAI, Quantum, Sustainability Autonomous and Continuum AI-assisted DevOps/OpsDev SWEBOK V1 SWEBOK V2 SWEBOK V3 SWEBOK V4 12 Partially adopted from “The Trailer of the ACM 2030 Roadmap for Software Engineering”
  • 13.
    Agenda • IEEE CS:SWEBOK Guide evolution • AI and software engineering: AI for SE and SE for AI • Multi-view Modeling • AI Engineering Patterns and Engineering
  • 14.
    SWEBOK Guide topic:AI and software engineering • Limitations and challenges • Uncertain and stochastic behavior • Necessity of sufficiently labeled, structured datasets • AI for SE • Building high-quality software systems by replicating human developers’ behavior • Ranging over almost all development stages • SE for AI • Different from traditional software since the rules and system behavior of AI systems are inferred from data • Need for particular support of SE for AI • Documenting practices as patterns 14 Software engineering AI AI for SE SE for AI Hironori Washizaki, Foutse Khomh, Yann-Gael Gueheneuc, Hironori Takeuchi, Naotake Natori, Takuo Doi, Satoshi Okuda, “Software Engineering Design Patterns for Machine Learning Applications,” IEEE Computer, Vol. 55, No. 3, pp. 30-39, 2022. (Best Paper Award)
  • 15.
    AI for SE:From AI-assisted dev to Human/AI co-creation(ref: AI-SEAL) [Feldt18] 15 Ref: Robert Feldt, et al., Ways of Applying Artificial Intelligence in Software Engineering, RAISE 2018, CoRR abs/1802.02033 Agentic action Process Automation level Target Product Runtime Low risk High risk Middle risk Completion Chat and exploration Human- centered AI-assisted Human/AI co- creation Developers Customers End users AI agents Reasoning and implementation
  • 16.
    SE for AI:Techniques useful for AI/ML systems quality Training data Trained model Prediction, inference Infrastructure software system New data ML model repair Monitoring, goal-oriented modeling Testing oracle problem, balanced dataset and coverage Performance, robustness and explainability Architecture validity and quality assurance Suitability with objective, handling unexpected situations N. Uchihira, AI and Software Engineering, JUSE SQiP 2017 Eric Breck et al., The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction, IEEE Big Data 2017 Metamorphic testing Search-based testing Practices and patterns Quality measurement
  • 17.
    Agenda • IEEE CS:SWEBOK Guide evolution • AI and software engineering: AI for SE and SE for AI • Multi-view Modeling • AI Engineering Patterns and Engineering
  • 18.
    Business Concerns System Concerns Software Concerns Traditional Software Concerns ML Software Concerns Costs Revenues Stakeholders Integration Safety ML performance Data Quality Model architecture Reliability Experimentative Definitive 18 Howto address multiple aspects in traceable and consistent way? => Metamodel-based multi-view modeling How to align different natures together? => Pipeline integration and MLOps Responsible AI patterns ML patterns ML Safety and security patterns ML architecture and design patterns How to incorporate ML patterns into development? => Application with configuration
  • 20.
    Multi-view modeling forML systems [SQJ’24] [ICEBE’23][FGCS’24] ML Canvas AI Project Canvas Safety Case Architectural Diagram (SysML) KAOS Goal Model STAMP/STPA Value MLOps Architecture Goals Safety Argumentation 20 Hironori Takeuchi, Jati H. Husen, Hnin Thandar Tun, Hironori Washizaki and Nobukazu Yoshioka, “Enterprise Architecture-based Metamodel for a Holistic Business – IT Alignment View on Machine Learning Projects,” IEEE International Conference on E-Business Engineering (ICEBE 2023), Best Paper Award Hironori Takeuchi, Jati H. Husenb, Hnin Thandar Tun, Hironori Washizaki, Nobukazu Yoshioka, “Enterprise Architecture-based Metamodel for Machine Learning Projects and its Management,” Future Generation Computer Systems, Elsevier, Vol. 161, pp. 135-145, 2024.
  • 21.
    Corresponding process 21 Jati H.Husen, Hironori Washizaki, Jomphon Runpakprakun, Nobukazu Yoshioka, Hnin Thandar Tun, Yoshiaki Fukazawa, Hironori Takeuchi, “Integrated Multi-view Modeling for Reliable Machine Learning-Intensive Software Engineering,” Software Quality Journal, Vol. 32, pp. 1239–1285, Springer-Nature, 2024.
  • 22.
    Metamodel for consistencyand traceability [ICEBE’23][FGCS’24] ML Canvas AI Project Canvas Safety Case KAOS Goal Model STAMP/STPA Architecture (SysML) ML workflow pipeline 22 Hironori Takeuchi, Jati H. Husen, Hnin Thandar Tun, Hironori Washizaki and Nobukazu Yoshioka, “Enterprise Architecture-based Metamodel for a Holistic Business – IT Alignment View on Machine Learning Projects,” IEEE International Conference on E-Business Engineering (ICEBE 2023), Best Paper Award Hironori Takeuchi, Jati H. Husenb, Hnin Thandar Tun, Hironori Washizaki, Nobukazu Yoshioka, “Enterprise Architecture-based Metamodel for Machine Learning Projects and its Management,” Future Generation Computer Systems, Elsevier, Vol. 161, pp. 135-145, 2024.
  • 23.
    Metamodel extension 23 Hiroshi Tanaka,Ide Masaru, Kazuki Munakata, Hironori Washizaki, and Nobukazu Yoshioka, “Activity-based modeling strategy for reliable machine learning system analysis targeting GUI-based applications,” 10th International Conference on Dependable Systems and Their Applications (DSA 2023) Metamodel for ML systems Extension for activity-based reliability analysis [DSA’23] Business-ML alignment model ML canvas AI canvas Extension for ML-system business alignment [FGCS’24] Hironori Takeuchi, Jati H. Husenb, Hnin Thandar Tun, Hironori Washizaki, Nobukazu Yoshioka, “Enterprise Architecture-based Metamodel for Machine Learning Projects and its Management,” Future Generation Computer Systems, Vol. 32, 2024
  • 24.
    Example case ofimage classification in autonomous driving City Highway AI Project Canvas ML Canvas Architecture Data Skills Output Value proposition Integration Stakeholders Customer Cost Revenue How can we develop and revise a system based on DNNs with acceptable recognition accuracy considering safety in the city and on the highway? Jati H. Husen, Hironori Washizaki, Jomphon Runpakprakun, Nobukazu Yoshioka, Hnin Thandar Tun, Yoshiaki Fukazawa, Hironori Takeuchi, “Integrated Multi-view Modeling for Reliable Machine Learning-Intensive Software Engineering,” Software Quality Journal, Vol. 32, Springer, 2024.
  • 25.
    Case of MLm1 m2 m3 Evaluation of classification Safety Case Misclassified data Selection for repair Balanced repair Result of repair Aggressive repair Further revision 1. Dataset revision 2. Architecture revision for improving images 3. Revisiting business goals Misclassified data STAMP/STPA KAOS Goal Model 25 Jati H. Husen, Hironori Washizaki, Jomphon Runpakprakun, Nobukazu Yoshioka, Hnin Thandar Tun, Yoshiaki Fukazawa, Hironori Takeuchi, “Integrated Multi-view Modeling for Reliable Machine Learning-Intensive Software Engineering,” Software Quality Journal, Vol. 32, Springer, 2024. Application of assurance patterns
  • 26.
    Metamodel ML evaluation Visualizing issues ML evaluation Visualizing resolution OK OKOK Failed Failed OK OK OK OK OK OK OK [ML.VP1🡨 AI.VP1] Providereliable real-timeobject detectionsystem fordriving decisionmakingin highway(incl. trafficsign detectionand lane/vehicle detection) • [ML.DS1]Procured datasets • [ML.DS2]Internal databasefrom collectionduring operation • [ML.DC1]Openand commercialdatasets • [ML.DC2]Data collectedduring operation(imageand identificationresult) •[ML.F1🡨 AI.D1/AI.D3] Boundingbox forobject(incl. othervehicles orsigns) •[ML.F2🡨 AI.D2]Ridge detectionfor lanedetection [ML.BM1] Modelswillbe developed, tested,and deployedtocars monthly • [ML.PT1]Input: imagefromsensors • [ML.PT2←AI.D] Output:trafficsigns, lanemarking, vehicles,and pedestrians. [ML.De1]Use predictionresults fordecision- makinginself- drivingsystem [ML.IS1] Usingtestdata, achieveveryhigh recallandhigh precisionin followingcondition: night,rainy,and generalcondition Datasetsissplitinto 80:20ratio [ML.MP1] Predictionshould bemadein batchesreal time. [ML.M1]Inputdatamonitoring [ML.VP1🡨 AI.VP1] Providereliable real-timeobject detectionsystem fordriving decisionmakingin highway(incl. trafficsign detectionand lane/vehicle detection) • [ML.DS1]Procured datasets • [ML.DS2]Internal databasefrom collectionduring operation • [ML.DC1]Openand commercialdatasets • [ML.DC2]Data collectedduring operation(imageand identificationresult) •[ML.F1🡨 AI.D1/AI.D3] Boundingbox forobject(incl. othervehicles orsigns) •[ML.F2🡨 AI.D2]Ridge detectionfor lanedetection [ML.BM1] Modelswillbe developed, tested,and deployedtocars monthly • [ML.PT1]Input: imagefromsensors • [ML.PT2←AI.D] Output:trafficsigns, lanemarking, vehicles,and pedestrians. [ML.De1]Use predictionresults fordecision- makinginself- drivingsystem [ML.IS1] Usingtestdata, achieveveryhigh recallandhigh precisionin followingcondition: night,rainy,and generalcondition Datasetsissplitinto 80:20ratio [ML.MP1] Predictionshould bemadein batchesreal time. [ML.M1]Inputdatamonitoring [ML.VP1🡨 AI.VP1] Providereliable real-timeobject detectionsystem fordriving decisionmakingin highway(incl. trafficsign detectionand lane/vehicle detection) •[ML.DS1]Procured datasets •[ML.DS2]Internal databasefrom collectionduring operation •[ML.DC1]Openand commercialdatasets •[ML.DC2]Data collectedduring operation(imageand identificationresult) •[ML.F1🡨 AI.D1/AI.D3] Boundingbox forobject(incl. othervehicles orsigns) •[ML.F2🡨 AI.D2]Ridge detectionfor lanedetection [ML.BM1] Modelswillbe developed, tested,and deployedtocars monthly •[ML.PT1]Input: imagefromsensors •[ML.PT2←AI.D] Output:trafficsigns, lanemarking, vehicles,and pedestrians. [ML.De1]Use predictionresults fordecision- makinginself- drivingsystem [ML.IS1] Usingtestdata, achieveveryhigh recallandhigh precisionin followingcondition: night,rainy,and generalcondition Datasetsissplitinto 80:20ratio [ML.MP1] Predictionshould bemadein batchesreal time. [ML.M1]Inputdatamonitoring Adding repair-strategy ML training ML repair System modeling and MLOps integration [ICEBE’23][FGCS’24][SQJ’24] 26 “Integrated Multi-view Modeling for Reliable Machine Learning-Intensive Software Engineering,” Software Quality Journal, Vol. 32, Springer, 2024. “Enterprise Architecture-based Metamodel for a Holistic Business – IT Alignment View on Machine Learning Projects,” IEEE ICEBE 2023, Best Paper Award “Enterprise Architecture-based Metamodel for Machine Learning Projects and its Management,” Future Generation Computer Systems, 161, 2024 Requirements Construction Design Test Architecture Operations Economics Models and Methods Quality Requirements analysis and design
  • 27.
    Agenda • IEEE CS:SWEBOK Guide evolution • AI and software engineering: AI for SE and SE for AI • Multi-view Modeling • AI Engineering Patterns and Engineering
  • 28.
    Machine learning designpatterns [Lakshmanan+ 20] • We wish to identify the type of instrument for the sound picked up by the phone and achieve recording and response according to the type. • However, the memory and performance of the phone is limited, and a large deep learning model is unlikely to be loaded. How can we do this? 28 Pretrained model • Let's use Two-stage predictions where a small model on the phone determines if a sound is a musical instrument, and a large model on the cloud classifies the type of sound only if it is a musical instrument. • For the large model, we will adopt Transfer Learning to achieve precise classification. Machine Learning Design Patterns (V. Lakshmanan, et al. 2020) Two-stage predictions Problem: There is a need to maintain the performance of models that are large and complex in nature, even when deployed to edge or distributed devices. Solution: The utilization flow is divided into two phases, with only the simple phase performed on the edge.
  • 29.
    AI/ML software engineeringpatterns • Architecture and design patterns – Software Engineering Patterns for ML applications [SEP4MLA] – Machine Learning Design Patterns [MLDP] • Safety and security patterns – Safety Case Pattern for ML systems [Safety] – Security Argument Patterns for DNN [Security] • Responsible AI patterns – Responsible AI System Design Patterns [Responsible] • Development and management practices – Lifecycle phase practices [Practice1] – Issues and development practices [Practice2] • Prompt engineering patterns – Prompt Pattern Catalog [Prompt][Prompt2] 29 [MLDP] V. Lakshmanan, et al., “Machine Learning Design Patterns,” O’Reilly, 2020 [SEP4MLA] H. Washizaki, et al. “Software Engineering Design Patterns for Machine Learning Applications,” IEEE Computer 55(3) 2022, Best Paper Award [Safety] E. Wozniak, et al., “A Safety Case Pattern for Systems with Machine Learning Components,” SAFECOMP 2020 Workshop [Security] M. Zeroual, et al., “Security Argument patterns for Deep Neural Network Development,” PLoP 2023 [Responsible] Q. Lu, et al., “Responsible-AI-by-Design: a Pattern Collection for Designing Responsible AI Systems,” IEEE Software, 2023 [Practice1] M. S. Rahman, et al., “Machine Learning Application Development: Practitioners’ Insights,” Software Quality Journal, 31, 2023. [Practice2] Y. Watanabe, et al., “Preliminary Literature Review of Machine Learning System Development Practices,” COMPSAC 2021 Fast Abstract [Prompt1] J, White, et al., “A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT,” arXiv 2302.11382, 2023 [Prompt2] Y. Sasaki, H. Washizaki, Jialong Li, Nobukazu Yoshioka, Naoyasu Ubayashi, Yoshiaki Fukazawa, “Landscape and Taxonomy of Prompt Engineering Patterns in Software Engineering,” IEEE IT Professional (IT Pro), Vol. 27, pp. 41-49, 2025.
  • 30.
    SE Patterns forML applications [Computer’22] • 15 patterns extracted from around 40 scholarly and gray documents • Classified into three types: Topology, Programming, and Model Operation 30 Hironori Washizaki, Foutse Khomh, Yann-Gael Gueheneuc, Hironori Takeuchi, Naotake Natori, Takuo Doi, Satoshi Okuda, “Software Engineering Design Patterns for Machine Learning Applications,” IEEE Computer, Vol. 55, No. 3, pp. 30-39, 2022. (Best Paper Award) Encapsulate ML Models within Rule-based Safeguards • Problem: ML models are known to be unstable and vulnerable to adversarial attacks, noise, and data drift. • Solution: Encapsulate functionality provided by ML models and deal with the inherent uncertainty in the containing system using deterministic and verifiable rules. Business Logic API Rule-based Safeguard Inference (Prediction) Encapsulated ML model Input Output Rule Explainable Proxy Model • Problem: A surrogate ML model must be built to provide explainability. • Solution: Run the explainable inference pipeline in parallel with the primary inference pipeline to monitor prediction differences. Input Decoy model Data lake Proxy model (E.g., Decision tree) Monitoring and comparison Reproduce and retraining Production model (E.g., DNN)
  • 31.
    Argument-based assurance case •Assurance case • Structured claims, arguments, and evidence which gives confidence that a system will possess the particular qualities or properties that need to be assured • Safety, security, resilience, … • Goal Structuring Notation (GSN) • Graphical notation for visualizing arguments that assure critical properties • Adoption for AI application risk- based assessment • E.g., UK ICO AI Explainability Guide 31 Jordi Cabot, Goal Structuring Notation – a short introduction https://modeling-languages.com/goal-structuring-notation-introduction/
  • 32.
    ML security argument patterns[Zeroual+23] 32 DNN Secure development argument • Problem: Compliance is required in the secure development processes for the given DNN. • Solution: Claim decomposition to argue satisfaction of security requirements. • E.g., “The specified robustness guarantees adversarial perturbations will be recognizable by humans.” M. Zeroual, et al., “Security Argument patterns for Deep Neural Network Development,” PLoP 2023 Goal (Claim) Solution (Evidence) Context Strategy
  • 33.
    33 ML assurance argumentpatterns (e.g., “DNN Robustness Case Verification”) [AsianPLoP’24] Goal (Claim) Solution (Evidence) Strategy M. Mutsche, et al. “Robustness-based Security Case Verification for Deep Neural Networks,” AsianPLoP 2024
  • 34.
    Yuya Sasaki, HironoriWashizaki, Jialong Li, Nobukazu Yoshioka, Naoyasu Ubayashi, Yoshiaki Fukazawa, “Landscape and Taxonomy of Prompt Engineering Patterns in Software Engineering,” IT Pro, 27, 2025. User-Model Collaboration Refinement • Problem: Balancing between accuracy and confidence is challenging, especially when users need guidance to achieve goals. • Solution: Interactive prompts that assign LLM to simulate role, instead of user-driven only prompting. Prompt Engineering Patterns [Sasaki+25] 34 Persona Game Question From LLM Simulation Interactive Prompt Templates This is a conversation with Socrates, an eager and helpful, but humble expert automatic AI... I would like you to ask me questions to achieve X. You should ask questions until this condition is met or to achieve this goal...
  • 35.
    • Problem: … Patternengineering • Extraction: Identifying and formulating recurring problems and solutions into “new” patterns to have reusable patterns • Detection: Detecting “known” patterns in software processes and products to comprehend and identify further improvement opportunities • Application: Selecting, concretizing and deploying patterns on software processes and products to resolve particular problems • Organization: Organizing patterns to build a system (i.e., language) of patterns • Integration: Integrating into pattern-oriented development and management 35 • Problem: … • Solution: …. AI/ML pattern Extraction Application Similar results Detection Pattern instances Organization Process Management Integration
  • 36.
    Pytorch vs. Keras MLpattern detection: Example of pattern instances [APSEC’23] 36 Weitao Pan, Hironori Washizaki, Nobukazu Yoshioka, Yoshiaki Fukazawa, Foutse Khomh, Yann-Gaël Guéhéneuc, “A Machine Learning Based Approach to Detect Machine Learning Design Patterns,” APSEC 2023 ERA Embeddings • Problem: High-cardinality features where closeness relationships are important to preserve. • Solution: Learn to map high- cardinality data into a lower dimensional space in such a way that the information relevant to the learning problem is preserved.
  • 37.
    ML pattern detection byML 37 Revised Text-CNN Model Weitao Pan, Hironori Washizaki, Nobukazu Yoshioka, Yoshiaki Fukazawa, Foutse Khomh, Yann-Gaël Guéhéneuc, “A Machine Learning Based Approach to Detect Machine Learning Design Patterns,” APSEC 2023 ERA
  • 38.
    ML pattern application:Deployment of assurance argument patterns [AsianPLoP’24] 38 1. Identifying and showing pattern candidates: OCL constraints-based and LLM-based 2. Showing potential impact and result before deployment 3. Deploying a selected pattern T. Ayukawa, et al., “Machine Learning Design Pattern Application Support,” AsianPLoP 2024
  • 39.
    Design patterns IoT design patterns Security and safetypatterns Pattern organization: Towards a pattern language … OK, so, to attract many people to our city, Small Public Squares should be located in the center. At the SMALL PUBLIC SQUARE, make Street Cafes be Opening to the Street ... 39 Small Public Square Street Cafe Opening to the Street https://unsplash.com/photos/EdpbTj3Br-Y https://unsplash.com/photos/GqurqYbj7aU https://unsplash.com/photos/zFoRwZirFvY Responsible AI patterns AI assurance argument patterns AI architecture and design patterns
  • 40.
    SWEBOK Guide evolutionfrom V3 to V4 • Modern engineering, practice update, BOK grows and recently developed areas Requirements Design Construction Testing Maintenance Configuration Management Engineering Management Process Models and Methods Quality Professional Practice Economics Computing Foundations Mathematical Foundations Engineering Foundations Requirements Architecture Design Construction Testing Operations Maintenance Configuration Management Engineering Management Process Models and Methods Quality Security Professional Practice Economics Computing Foundations Mathematical Foundations Engineering Foundations V3 V4 Agile, DevOps AI for SE, SE for AI H. Washizaki, eds., “Guide to the Software Engineering Body of Knowledge (SWEBOK Guide), Version 4.0,” IEEE Computer Society, 2024 Editor: H. Washizaki KA editors: A. Ihara, S. Ogata, N. Yoshioka, S. Munetoh, K. Shintani, E. Hayashiguchi and 15+ experts Metamodel ML evaluation Visualizing issues ML evaluation Visualizing resolution OK OK OK Failed Failed OK OK OK OK OK OK OK [ML.VP1🡨 AI.VP1] Providereliable real-timeobject detectionsystem fordriving decisionmakingin highway(incl. trafficsign detectionand lane/vehicle detection) • [ML.DS1]Procured datasets • [ML.DS2]Internal databasefrom collectionduring operation • [ML.DC1]Openand commercialdatasets • [ML.DC2]Data collectedduring operation(imageand identificationresult) •[ML.F1🡨 AI.D1/AI.D3] Boundingbox forobject(incl. othervehicles orsigns) •[ML.F2🡨 AI.D2]Ridge detectionfor lanedetection [ML.BM1] Modelswillbe developed, tested,and deployedtocars monthly • [ML.PT1]Input: imagefromsensors • [ML.PT2←AI.D] Output:trafficsigns, lanemarking, vehicles,and pedestrians. [ML.De1]Use predictionresults fordecision- makinginself- drivingsystem [ML.IS1] Usingtestdata, achieveveryhigh recallandhigh precisionin followingcondition: night,rainy,and generalcondition Datasetsissplitinto 80:20ratio [ML.MP1] Predictionshould bemadein batchesreal time. [ML.M1]Inputdatamonitoring [ML.VP1🡨 AI.VP1] Providereliable real-timeobject detectionsystem fordriving decisionmakingin highway(incl. trafficsign detectionand lane/vehicle detection) • [ML.DS1]Procured datasets • [ML.DS2]Internal databasefrom collectionduring operation • [ML.DC1]Openand commercialdatasets • [ML.DC2]Data collectedduring operation(imageand identificationresult) •[ML.F1🡨 AI.D1/AI.D3] Boundingbox forobject(incl. othervehicles orsigns) •[ML.F2🡨 AI.D2]Ridge detectionfor lanedetection [ML.BM1] Modelswillbe developed, tested,and deployedtocars monthly • [ML.PT1]Input: imagefromsensors • [ML.PT2←AI.D] Output:trafficsigns, lanemarking, vehicles,and pedestrians. [ML.De1]Use predictionresults fordecision- makinginself- drivingsystem [ML.IS1] Usingtestdata, achieveveryhigh recallandhigh precisionin followingcondition: night,rainy,and generalcondition Datasetsissplitinto 80:20ratio [ML.MP1] Predictionshould bemadein batchesreal time. [ML.M1]Inputdatamonitoring [ML.VP1🡨 AI.VP1] Providereliable real-timeobject detectionsystem fordriving decisionmakingin highway(incl. trafficsign detectionand lane/vehicle detection) •[ML.DS1]Procured datasets •[ML.DS2]Internal databasefrom collectionduring operation •[ML.DC1]Openand commercialdatasets •[ML.DC2]Data collectedduring operation(imageand identificationresult) •[ML.F1🡨 AI.D1/AI.D3] Boundingbox forobject(incl. othervehicles orsigns) •[ML.F2🡨 AI.D2]Ridge detectionfor lanedetection [ML.BM1] Modelswillbe developed, tested,and deployedtocars monthly •[ML.PT1]Input: imagefromsensors •[ML.PT2←AI.D] Output:trafficsigns, lanemarking, vehicles,and pedestrians. [ML.De1]Use predictionresults fordecision- makinginself- drivingsystem [ML.IS1] Usingtestdata, achieveveryhigh recallandhigh precisionin followingcondition: night,rainy,and generalcondition Datasetsissplitinto 80:20ratio [ML.MP1] Predictionshould bemadein batchesreal time. [ML.M1]Inputdatamonitoring Adding repair-strategy ML training ML repair System modeling and MLOps integration [ICEBE’23][FGCS’24][SQJ’24] 26 “Integrated Multi-view Modeling for Reliable Machine Learning-Intensive Software Engineering,” Software Quality Journal, Vol. 32, Springer, 2024. “Enterprise Architecture-based Metamodel for a Holistic Business – IT Alignment View on Machine Learning Projects,” IEEE ICEBE 2023, Best Paper Award “Enterprise Architecture-based Metamodel for Machine Learning Projects and its Management,” Future Generation Computer Systems, 161, 2024 Requirements Construction Design Test Architecture Operations Economics Models and Methods Quality Requirements analysis and design • Problem: … Pattern engineering • Extraction: Identifying and formulating recurring problems and solutions into “new” patterns to have reusable patterns • Detection: Detecting “known” patterns in software processes and products to comprehend and identify further improvement opportunities • Application: Selecting, concretizing and deploying patterns on software processes and products to resolve particular problems • Organization: Organizing patterns to build a system (i.e., language) of patterns • Integration: Integrating into pattern-oriented development and management 35 • Problem: … • Solution: …. AI/ML pattern Extraction Application Similar results Detection Pattern instances Organization Process Management Integration SE for AI: Techniques useful for AI/ML systems quality Training data Trained model Prediction, inference Infrastructure software system New data ML model repair Monitoring, goal-oriented modeling Testing oracle problem, balanced dataset and coverage Performance, robustness and explainability Architecture validity and quality assurance Suitability with objective, handling unexpected situations N. Uchihira, AI and Software Engineering, JUSE SQiP 2017 Eric Breck et al., The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction, IEEE Big Data 2017 Metamorphic testing Search-based testing Practices and patterns Quality measurement