SlideShare a Scribd company logo
1 of 31
SUPPORTING CHANGE
IMPACT ANALYSIS USING
A RECOMMENDATION
SYSTEM
- An Industrial Case Study in a Safety-
Critical Context
@mrksbrg
Markus Borg, Krzysztof Wnuk, Björn Regnell,
Per Runeson
Research Institutes of Sweden
Swedish Institute of Computer Science
Bug tracker
Development engineer, ABB, Malmö, Sweden
 Editor and compiler development
 Safety-critical systems
PhD student, Lund University, Sweden
 Machine learning for software engineering
 Bug reports and traceability
Senior researcher, RISE SICS AB, Lund, Sweden
 Software engineering for machine learning
 V&V for self-driving cars
5
The Challenge
The Solution
The Evaluation
Reqts. DB
Issue Repo
Code Repo
Test DB Doc. DB
Reqts.
Tests
 System evolution since
the 1980s
 ~ 2 MLOC C/C++
 Safety certification
(SIL-2)
Evolving automation system
 Formal development
process
 V model, stage-gate
process
 ~200 SW engineers
 Main sites in Sweden
and India
The Challenge
The Solution
The Evaluation
Automated Change Impact Analysis
 Intuitive tool to jump start analyses based on
historical data
 Goal: faster & more accurate analyses
Approach:
1. Mine the history
2. Recommend impact
Part 1: Construct network of
previously reported impact
Index textual data with
Calculate centrality measures
Find similar bugs using Apache Lucene
Part 2: Recommend impact
Design Doc. X.Y
Req. X.Y
Test case UTC56
Req. Z.Y
Design Doc. X.Y
Follow links to identify candidate impact set
Use centrality measures to rank candidate impact
1. Requirement X.Y
2. Design Document X.Y
3. Test case UTC56
4. Design Document X.Y
5. Requirement Z.Y
Find similar bugs using Apache Lucene
Part 2: Recommend impact
Follow links to identify candidate impact set
The Challenge
The Solution
The Evaluation
Correctness
Utility
RSSE
 Replay change impact
history
 Mine and build
ImpRec using
8 years
 Test ImpRec on
2 years of incoming
bug reports
Experiment - Correctness
Experiment - Correctness
Industrial case study - Utility
 Selected two teams
 Conducted interviews
 Deployed ImpRec
 Collected data for 3-9 months
 Logged every click
 Calculated correctness metrics
 Conducted interviews
Industrial case study - Utility
Industrial case study - Utility
Industrial case study - Utility
Conclusion
 Reuse previous change impact analysis effort
 Walk in the footsteps of previous engineers
 Inspired by MSR, IR-based tracing, RSSE
=> ImpRec
 Top-10 recommendations cover 40% of true impact
 Developers find the level of correctness useful
github.com/mrksbrg/ImpRec
Bug tracker
Bug tracker
THANKS!
markus.borg@ri.se
mrksbrg.com
@mrksbrg
Research Institutes of Sweden
Swedish Institute of Computer Science
PHOTO CREDITS
Brown stink bug
- Marlin E. Rice
Isopods
- Omoshiro Aquarium
SW dev
- Noglif, Flickr:
templetonelliot, ifl,
danburgmurmur
My wife
- My wife

More Related Content

What's hot

Icse 2011 ds_1
Icse 2011 ds_1Icse 2011 ds_1
Icse 2011 ds_1
SAIL_QU
 
Caleb Green Resume 2015
Caleb Green Resume 2015Caleb Green Resume 2015
Caleb Green Resume 2015
Caleb Green
 
JamesHunterRead-Resume
JamesHunterRead-ResumeJamesHunterRead-Resume
JamesHunterRead-Resume
Hunter Read
 

What's hot (20)

Testing throughout the software life cycle
Testing throughout the software life cycleTesting throughout the software life cycle
Testing throughout the software life cycle
 
Just-in-Time Bug Prediction in Mobile Applications: The Domain Matters!
Just-in-Time Bug Prediction in Mobile Applications: The Domain Matters!Just-in-Time Bug Prediction in Mobile Applications: The Domain Matters!
Just-in-Time Bug Prediction in Mobile Applications: The Domain Matters!
 
Software bug prediction
Software bug prediction Software bug prediction
Software bug prediction
 
Current resume
Current resumeCurrent resume
Current resume
 
Testing throughout the software life cycle
Testing throughout the software life cycleTesting throughout the software life cycle
Testing throughout the software life cycle
 
Project Cost Risk Analysis - Reporting
Project Cost Risk Analysis - ReportingProject Cost Risk Analysis - Reporting
Project Cost Risk Analysis - Reporting
 
Human factors in software reliability engineering - Research Paper
Human factors in software reliability engineering - Research PaperHuman factors in software reliability engineering - Research Paper
Human factors in software reliability engineering - Research Paper
 
7 testing principles
7 testing principles7 testing principles
7 testing principles
 
Developer-Related Factors in Change Prediction: An Empirical Assessment
Developer-Related Factors in Change Prediction: An Empirical AssessmentDeveloper-Related Factors in Change Prediction: An Empirical Assessment
Developer-Related Factors in Change Prediction: An Empirical Assessment
 
Testing throughout the software life cycle
Testing throughout the software life cycleTesting throughout the software life cycle
Testing throughout the software life cycle
 
On Adequate Behavior-based Architecture Conformance Checks
On Adequate Behavior-based Architecture Conformance ChecksOn Adequate Behavior-based Architecture Conformance Checks
On Adequate Behavior-based Architecture Conformance Checks
 
Icse 2011 ds_1
Icse 2011 ds_1Icse 2011 ds_1
Icse 2011 ds_1
 
Adoption of Software Testing in Open Source Projects - A Preliminary Study on...
Adoption of Software Testing in Open Source Projects - A Preliminary Study on...Adoption of Software Testing in Open Source Projects - A Preliminary Study on...
Adoption of Software Testing in Open Source Projects - A Preliminary Study on...
 
01 florian reil_introduction
01 florian reil_introduction01 florian reil_introduction
01 florian reil_introduction
 
Spiral mode
Spiral modeSpiral mode
Spiral mode
 
Caleb Green Resume 2015
Caleb Green Resume 2015Caleb Green Resume 2015
Caleb Green Resume 2015
 
Testing throughout the software life cycle
Testing throughout the software life cycleTesting throughout the software life cycle
Testing throughout the software life cycle
 
JamesHunterRead-Resume
JamesHunterRead-ResumeJamesHunterRead-Resume
JamesHunterRead-Resume
 
QUICKAR-ASE2016-Singapore
QUICKAR-ASE2016-SingaporeQUICKAR-ASE2016-Singapore
QUICKAR-ASE2016-Singapore
 
Understanding the Rationale for Updating a Function's Comment
Understanding the Rationale for Updating a Function's CommentUnderstanding the Rationale for Updating a Function's Comment
Understanding the Rationale for Updating a Function's Comment
 

Similar to Supporting Change Impact Analysis Using a Recommendation System - An Industrial Case Study in a Safety-Critical Context

The Good the Bad and the Ugly of Dealing with Smelly Code (ITAKE Unconference)
The Good the Bad and the Ugly of Dealing with Smelly Code (ITAKE Unconference)The Good the Bad and the Ugly of Dealing with Smelly Code (ITAKE Unconference)
The Good the Bad and the Ugly of Dealing with Smelly Code (ITAKE Unconference)
Radu Marinescu
 
Mi0033 software engineering...
Mi0033  software engineering...Mi0033  software engineering...
Mi0033 software engineering...
smumbahelp
 
Online examination system
Online examination systemOnline examination system
Online examination system
Rahul Khanwani
 
John Allen - Resume
John Allen - ResumeJohn Allen - Resume
John Allen - Resume
John Allen
 
ESEconf2011 - Hanin Makram: "Embedding Performance into Continuous Integratio...
ESEconf2011 - Hanin Makram: "Embedding Performance into Continuous Integratio...ESEconf2011 - Hanin Makram: "Embedding Performance into Continuous Integratio...
ESEconf2011 - Hanin Makram: "Embedding Performance into Continuous Integratio...
Aberla
 
AshwinSwtest 2+ (1)
AshwinSwtest 2+ (1)AshwinSwtest 2+ (1)
AshwinSwtest 2+ (1)
ashwin kumar
 

Similar to Supporting Change Impact Analysis Using a Recommendation System - An Industrial Case Study in a Safety-Critical Context (20)

Automation in the Bug Flow - Machine Learning for Triaging and Tracing
Automation in the Bug Flow - Machine Learning for Triaging and TracingAutomation in the Bug Flow - Machine Learning for Triaging and Tracing
Automation in the Bug Flow - Machine Learning for Triaging and Tracing
 
An overview of software development methodologies.
An overview of software development methodologies.An overview of software development methodologies.
An overview of software development methodologies.
 
The Good the Bad and the Ugly of Dealing with Smelly Code (ITAKE Unconference)
The Good the Bad and the Ugly of Dealing with Smelly Code (ITAKE Unconference)The Good the Bad and the Ugly of Dealing with Smelly Code (ITAKE Unconference)
The Good the Bad and the Ugly of Dealing with Smelly Code (ITAKE Unconference)
 
Slides chapter 15
Slides chapter 15Slides chapter 15
Slides chapter 15
 
software engineering
software engineering software engineering
software engineering
 
From Bugs to Decision Support - Selected Research Highlights
From Bugs to Decision Support - Selected Research HighlightsFrom Bugs to Decision Support - Selected Research Highlights
From Bugs to Decision Support - Selected Research Highlights
 
OOSE Unit 5 PPT.ppt
OOSE Unit 5 PPT.pptOOSE Unit 5 PPT.ppt
OOSE Unit 5 PPT.ppt
 
Mi0033 software engineering...
Mi0033  software engineering...Mi0033  software engineering...
Mi0033 software engineering...
 
Online examination system
Online examination systemOnline examination system
Online examination system
 
Software metrics
Software metricsSoftware metrics
Software metrics
 
Oose unit 5 ppt
Oose unit 5 pptOose unit 5 ppt
Oose unit 5 ppt
 
Parameter Estimation of GOEL-OKUMOTO Model by Comparing ACO with MLE Method
Parameter Estimation of GOEL-OKUMOTO Model by Comparing ACO with MLE MethodParameter Estimation of GOEL-OKUMOTO Model by Comparing ACO with MLE Method
Parameter Estimation of GOEL-OKUMOTO Model by Comparing ACO with MLE Method
 
Chapter 4 Software Testing_Finalised_BW.ppt
Chapter 4 Software Testing_Finalised_BW.pptChapter 4 Software Testing_Finalised_BW.ppt
Chapter 4 Software Testing_Finalised_BW.ppt
 
Introduction to Software Engineering
Introduction to Software EngineeringIntroduction to Software Engineering
Introduction to Software Engineering
 
Prakasha_Resume
Prakasha_ResumePrakasha_Resume
Prakasha_Resume
 
John Allen - Resume
John Allen - ResumeJohn Allen - Resume
John Allen - Resume
 
ESEconf2011 - Hanin Makram: "Embedding Performance into Continuous Integratio...
ESEconf2011 - Hanin Makram: "Embedding Performance into Continuous Integratio...ESEconf2011 - Hanin Makram: "Embedding Performance into Continuous Integratio...
ESEconf2011 - Hanin Makram: "Embedding Performance into Continuous Integratio...
 
Clone of an organization
Clone of an organizationClone of an organization
Clone of an organization
 
AshwinSwtest 2+ (1)
AshwinSwtest 2+ (1)AshwinSwtest 2+ (1)
AshwinSwtest 2+ (1)
 
Jean Paul Varwijk - Discussing the Future of Software Testing - EuroSTAR 2013
Jean Paul Varwijk - Discussing the Future of Software Testing - EuroSTAR 2013Jean Paul Varwijk - Discussing the Future of Software Testing - EuroSTAR 2013
Jean Paul Varwijk - Discussing the Future of Software Testing - EuroSTAR 2013
 

More from Markus Borg

Digital Twins Are Not Monozygotic - Cross-Replicating ADAS Testing in Two Ind...
Digital Twins Are Not Monozygotic - Cross-Replicating ADAS Testing in Two Ind...Digital Twins Are Not Monozygotic - Cross-Replicating ADAS Testing in Two Ind...
Digital Twins Are Not Monozygotic - Cross-Replicating ADAS Testing in Two Ind...
Markus Borg
 

More from Markus Borg (18)

Agility in Software 2.0 - Notebook Interfaces and MLOps with Buttresses and R...
Agility in Software 2.0 - Notebook Interfaces and MLOps with Buttresses and R...Agility in Software 2.0 - Notebook Interfaces and MLOps with Buttresses and R...
Agility in Software 2.0 - Notebook Interfaces and MLOps with Buttresses and R...
 
Quality Assurance Of Generative Dialog Models in an evolving Conversationa...
Quality Assurance  Of  Generative Dialog Models in an evolving  Conversationa...Quality Assurance  Of  Generative Dialog Models in an evolving  Conversationa...
Quality Assurance Of Generative Dialog Models in an evolving Conversationa...
 
Test Automation with Grad-CAM Heatmaps - A Future Pipe Segment in MLOps for V...
Test Automation with Grad-CAM Heatmaps - A Future Pipe Segment in MLOps for V...Test Automation with Grad-CAM Heatmaps - A Future Pipe Segment in MLOps for V...
Test Automation with Grad-CAM Heatmaps - A Future Pipe Segment in MLOps for V...
 
Digital Twins Are Not Monozygotic - Cross-Replicating ADAS Testing in Two Ind...
Digital Twins Are Not Monozygotic - Cross-Replicating ADAS Testing in Two Ind...Digital Twins Are Not Monozygotic - Cross-Replicating ADAS Testing in Two Ind...
Digital Twins Are Not Monozygotic - Cross-Replicating ADAS Testing in Two Ind...
 
Illuminating a Blind Spot in Digitalization - Software Development in Sweden’...
Illuminating a Blind Spot in Digitalization - Software Development in Sweden’...Illuminating a Blind Spot in Digitalization - Software Development in Sweden’...
Illuminating a Blind Spot in Digitalization - Software Development in Sweden’...
 
Trained, Not Coded - Still Safe?
Trained, Not Coded - Still Safe?Trained, Not Coded - Still Safe?
Trained, Not Coded - Still Safe?
 
SZZ Unleashed: An Open Implementation of the SZZ Algorithm
SZZ Unleashed:  An Open Implementation of the SZZ AlgorithmSZZ Unleashed:  An Open Implementation of the SZZ Algorithm
SZZ Unleashed: An Open Implementation of the SZZ Algorithm
 
Explainability First! Cousteauing the Depths of Neural Networks
Explainability First! Cousteauing the Depths of Neural NetworksExplainability First! Cousteauing the Depths of Neural Networks
Explainability First! Cousteauing the Depths of Neural Networks
 
Test Automation Research... Is That Really Needed in 2018?
Test Automation Research... Is That Really Needed in 2018?Test Automation Research... Is That Really Needed in 2018?
Test Automation Research... Is That Really Needed in 2018?
 
Component Source Origin Decisions in Practice - A Survey of Decision Making i...
Component Source Origin Decisions in Practice - A Survey of Decision Making i...Component Source Origin Decisions in Practice - A Survey of Decision Making i...
Component Source Origin Decisions in Practice - A Survey of Decision Making i...
 
Enabling Visual Analytics with Unity - Exploring Regression Test Results in A...
Enabling Visual Analytics with Unity - Exploring Regression Test Results in A...Enabling Visual Analytics with Unity - Exploring Regression Test Results in A...
Enabling Visual Analytics with Unity - Exploring Regression Test Results in A...
 
Testing Quality Requirements of a System-of-Systems in the Public Sector - Ch...
Testing Quality Requirements of a System-of-Systems in the Public Sector - Ch...Testing Quality Requirements of a System-of-Systems in the Public Sector - Ch...
Testing Quality Requirements of a System-of-Systems in the Public Sector - Ch...
 
Comparing Cousins – A Harmonized Analysis of Racket Sport Set Scores using Ra...
Comparing Cousins – A Harmonized Analysis of Racket Sport Set Scores using Ra...Comparing Cousins – A Harmonized Analysis of Racket Sport Set Scores using Ra...
Comparing Cousins – A Harmonized Analysis of Racket Sport Set Scores using Ra...
 
Revisiting the Challenges in Aligning RE and V&V: Experiences from the Public...
Revisiting the Challenges in Aligning RE and V&V: Experiences from the Public...Revisiting the Challenges in Aligning RE and V&V: Experiences from the Public...
Revisiting the Challenges in Aligning RE and V&V: Experiences from the Public...
 
Enabling Traceability Reuse for Impact Analyses - Toward a Recommendation Sys...
Enabling Traceability Reuse for Impact Analyses - Toward a Recommendation Sys...Enabling Traceability Reuse for Impact Analyses - Toward a Recommendation Sys...
Enabling Traceability Reuse for Impact Analyses - Toward a Recommendation Sys...
 
Analyzing networks of issue reports
Analyzing networks of issue reportsAnalyzing networks of issue reports
Analyzing networks of issue reports
 
Findability through Traceability - A Realistic Application of Candidate Tr...
Findability through Traceability  - A Realistic Application of Candidate Tr...Findability through Traceability  - A Realistic Application of Candidate Tr...
Findability through Traceability - A Realistic Application of Candidate Tr...
 
Recommendation Systems for Issue Management
Recommendation Systems for Issue ManagementRecommendation Systems for Issue Management
Recommendation Systems for Issue Management
 

Recently uploaded

Disentangling the origin of chemical differences using GHOST
Disentangling the origin of chemical differences using GHOSTDisentangling the origin of chemical differences using GHOST
Disentangling the origin of chemical differences using GHOST
Sérgio Sacani
 
Biopesticide (2).pptx .This slides helps to know the different types of biop...
Biopesticide (2).pptx  .This slides helps to know the different types of biop...Biopesticide (2).pptx  .This slides helps to know the different types of biop...
Biopesticide (2).pptx .This slides helps to know the different types of biop...
RohitNehra6
 
Pests of cotton_Borer_Pests_Binomics_Dr.UPR.pdf
Pests of cotton_Borer_Pests_Binomics_Dr.UPR.pdfPests of cotton_Borer_Pests_Binomics_Dr.UPR.pdf
Pests of cotton_Borer_Pests_Binomics_Dr.UPR.pdf
PirithiRaju
 
CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service 🪡
CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service  🪡CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service  🪡
CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service 🪡
anilsa9823
 

Recently uploaded (20)

Stunning ➥8448380779▻ Call Girls In Panchshil Enclave Delhi NCR
Stunning ➥8448380779▻ Call Girls In Panchshil Enclave Delhi NCRStunning ➥8448380779▻ Call Girls In Panchshil Enclave Delhi NCR
Stunning ➥8448380779▻ Call Girls In Panchshil Enclave Delhi NCR
 
Hire 💕 9907093804 Hooghly Call Girls Service Call Girls Agency
Hire 💕 9907093804 Hooghly Call Girls Service Call Girls AgencyHire 💕 9907093804 Hooghly Call Girls Service Call Girls Agency
Hire 💕 9907093804 Hooghly Call Girls Service Call Girls Agency
 
Unlocking the Potential: Deep dive into ocean of Ceramic Magnets.pptx
Unlocking  the Potential: Deep dive into ocean of Ceramic Magnets.pptxUnlocking  the Potential: Deep dive into ocean of Ceramic Magnets.pptx
Unlocking the Potential: Deep dive into ocean of Ceramic Magnets.pptx
 
Botany krishna series 2nd semester Only Mcq type questions
Botany krishna series 2nd semester Only Mcq type questionsBotany krishna series 2nd semester Only Mcq type questions
Botany krishna series 2nd semester Only Mcq type questions
 
GBSN - Microbiology (Unit 2)
GBSN - Microbiology (Unit 2)GBSN - Microbiology (Unit 2)
GBSN - Microbiology (Unit 2)
 
PossibleEoarcheanRecordsoftheGeomagneticFieldPreservedintheIsuaSupracrustalBe...
PossibleEoarcheanRecordsoftheGeomagneticFieldPreservedintheIsuaSupracrustalBe...PossibleEoarcheanRecordsoftheGeomagneticFieldPreservedintheIsuaSupracrustalBe...
PossibleEoarcheanRecordsoftheGeomagneticFieldPreservedintheIsuaSupracrustalBe...
 
Biological Classification BioHack (3).pdf
Biological Classification BioHack (3).pdfBiological Classification BioHack (3).pdf
Biological Classification BioHack (3).pdf
 
Pulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceuticsPulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceutics
 
fundamental of entomology all in one topics of entomology
fundamental of entomology all in one topics of entomologyfundamental of entomology all in one topics of entomology
fundamental of entomology all in one topics of entomology
 
Raman spectroscopy.pptx M Pharm, M Sc, Advanced Spectral Analysis
Raman spectroscopy.pptx M Pharm, M Sc, Advanced Spectral AnalysisRaman spectroscopy.pptx M Pharm, M Sc, Advanced Spectral Analysis
Raman spectroscopy.pptx M Pharm, M Sc, Advanced Spectral Analysis
 
Spermiogenesis or Spermateleosis or metamorphosis of spermatid
Spermiogenesis or Spermateleosis or metamorphosis of spermatidSpermiogenesis or Spermateleosis or metamorphosis of spermatid
Spermiogenesis or Spermateleosis or metamorphosis of spermatid
 
GBSN - Biochemistry (Unit 1)
GBSN - Biochemistry (Unit 1)GBSN - Biochemistry (Unit 1)
GBSN - Biochemistry (Unit 1)
 
Recombination DNA Technology (Nucleic Acid Hybridization )
Recombination DNA Technology (Nucleic Acid Hybridization )Recombination DNA Technology (Nucleic Acid Hybridization )
Recombination DNA Technology (Nucleic Acid Hybridization )
 
Physiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptxPhysiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptx
 
Disentangling the origin of chemical differences using GHOST
Disentangling the origin of chemical differences using GHOSTDisentangling the origin of chemical differences using GHOST
Disentangling the origin of chemical differences using GHOST
 
Biopesticide (2).pptx .This slides helps to know the different types of biop...
Biopesticide (2).pptx  .This slides helps to know the different types of biop...Biopesticide (2).pptx  .This slides helps to know the different types of biop...
Biopesticide (2).pptx .This slides helps to know the different types of biop...
 
Pests of cotton_Borer_Pests_Binomics_Dr.UPR.pdf
Pests of cotton_Borer_Pests_Binomics_Dr.UPR.pdfPests of cotton_Borer_Pests_Binomics_Dr.UPR.pdf
Pests of cotton_Borer_Pests_Binomics_Dr.UPR.pdf
 
Chemistry 4th semester series (krishna).pdf
Chemistry 4th semester series (krishna).pdfChemistry 4th semester series (krishna).pdf
Chemistry 4th semester series (krishna).pdf
 
CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service 🪡
CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service  🪡CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service  🪡
CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service 🪡
 
GBSN - Microbiology (Unit 1)
GBSN - Microbiology (Unit 1)GBSN - Microbiology (Unit 1)
GBSN - Microbiology (Unit 1)
 

Supporting Change Impact Analysis Using a Recommendation System - An Industrial Case Study in a Safety-Critical Context

Editor's Notes

  1. QUPER model Experienced benefit is not linear Important quality breakpoints
  2. QUPER model Experienced benefit is not linear Important quality breakpoints
  3. QUPER model Experienced benefit is not linear Important quality breakpoints