The document describes a study on program exploration conducted by Z ́phyrin Soh et al. It introduces program exploration and how developers' navigation between program entities can be represented as exploration graphs. It then presents two extreme exploration strategies: referenced exploration, where developers revisit a set of referenced entities, and unreferenced exploration. A user study is conducted to investigate which strategy developers follow during maintenance tasks. Developer exploration histories from four Eclipse projects are collected and randomly sampled to build an oracle to classify instances as referenced or unreferenced exploration. The goal is to understand how differences in exploration strategies may affect maintenance tasks.
The document describes a study that aimed to understand how developers spend their effort during maintenance activities. The researchers analyzed change history and interaction data from open source projects to match developers' effort to the complexity of code changes. They found that developers do not necessarily spend more effort on tasks requiring more complex changes. Additional factors like the number of files changed, bug severity, and developers' experience also affected the effort spent.
130411 francis palma - detection of process antipatterns -- a bpel perspectivePtidej Team
The document describes an approach for detecting process antipatterns in BPEL processes. It begins by discussing the motivation for detecting antipatterns, which are poor design decisions that can negatively impact quality of service. It then reviews related work on modeling antipatterns, detecting them in BPMN models, and defining patterns in BPEL. The identified gaps are that antipatterns have not been considered for BPEL processes and quality aspects were not included. The proposed approach involves specifying antipatterns as rules, transforming BPEL processes into a generic model, and then detecting antipatterns by applying the rules. A small experiment is described to detect two antipatterns in three sample BPEL processes.
The document describes research into optimizing thread schedule alignments to expose interference bug patterns. It aims to maximize interference probability between threads using optimization algorithms like stochastic hill climbing and simulated annealing. Several synthetic and real-world multithreaded programs are tested with different search space dimensions. Results show the optimization algorithms, particularly simulated annealing, can find interference bugs faster than random testing as the search space increases in size.
Instance Space Analysis for Search Based Software EngineeringAldeida Aleti
Search-Based Software Engineering is now a mature area with numerous techniques developed to tackle some of the most challenging software engineering problems, from requirements to design, testing, fault localisation, and automated program repair. SBSE techniques have shown promising results, giving us hope that one day it will be possible for the tedious and labour intensive parts of software development to be completely automated, or at least semi-automated. In this talk, I will focus on the problem of objective performance evaluation of SBSE techniques. To this end, I will introduce Instance Space Analysis (ISA), which is an approach to identify features of SBSE problems that explain why a particular instance is difficult for an SBSE technique. ISA can be used to examine the diversity and quality of the benchmark datasets used by most researchers, and analyse the strengths and weaknesses of existing SBSE techniques. The instance space is constructed to reveal areas of hard and easy problems, and enables the strengths and weaknesses of the different SBSE techniques to be identified. I will present on how ISA enabled us to identify the strengths and weaknesses of SBSE techniques in two areas: Search-Based Software Testing and Automated Program Repair. Finally, I will end my talk with future directions of the objective assessment of SBSE techniques.
Pareto-Optimal Search-Based Software Engineering (POSBSE): A Literature SurveyAbdel Salam Sayyad
Paper presented at the 2nd International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE’13), San Francisco, USA. May 2013.
Controlled experiments, Hypothesis Testing, Test Selection, Threats to Validityalessio_ferrari
Complete lecture on controlled experiments in software engineering. It explains practical guidelines on conducting controlled experiments and describes the concepts of dependent, independent, and control variables, significance, and p-value. It also explains how to select the appropriate statistic test for a hypothesis, and gives example of data for different typical tests.
Finally, it discusses threats to validity in controlled experiments and gives indications for reporting.
Find the video lectures here: https://www.youtube.com/playlist?list=PLSKM4VZcJjV-P3fFJYMu2OhlTjEr9Bjl0
Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...Abdel Salam Sayyad
This document summarizes Abdel Salam Sayyad's doctoral defense that addressed using evolutionary search techniques with strong heuristics for multi-objective feature selection in software product lines. The defense outlined modeling feature models, analyzing them automatically, formulating the multi-objective feature selection problem, and using multi-objective evolutionary algorithms. Results demonstrated scalability by increasing objectives, tuning parameters, and using heuristics like "PUSH" and "PULL" as well as population seeding. The defense concluded by discussing future work.
The document describes a study that aimed to understand how developers spend their effort during maintenance activities. The researchers analyzed change history and interaction data from open source projects to match developers' effort to the complexity of code changes. They found that developers do not necessarily spend more effort on tasks requiring more complex changes. Additional factors like the number of files changed, bug severity, and developers' experience also affected the effort spent.
130411 francis palma - detection of process antipatterns -- a bpel perspectivePtidej Team
The document describes an approach for detecting process antipatterns in BPEL processes. It begins by discussing the motivation for detecting antipatterns, which are poor design decisions that can negatively impact quality of service. It then reviews related work on modeling antipatterns, detecting them in BPMN models, and defining patterns in BPEL. The identified gaps are that antipatterns have not been considered for BPEL processes and quality aspects were not included. The proposed approach involves specifying antipatterns as rules, transforming BPEL processes into a generic model, and then detecting antipatterns by applying the rules. A small experiment is described to detect two antipatterns in three sample BPEL processes.
The document describes research into optimizing thread schedule alignments to expose interference bug patterns. It aims to maximize interference probability between threads using optimization algorithms like stochastic hill climbing and simulated annealing. Several synthetic and real-world multithreaded programs are tested with different search space dimensions. Results show the optimization algorithms, particularly simulated annealing, can find interference bugs faster than random testing as the search space increases in size.
Instance Space Analysis for Search Based Software EngineeringAldeida Aleti
Search-Based Software Engineering is now a mature area with numerous techniques developed to tackle some of the most challenging software engineering problems, from requirements to design, testing, fault localisation, and automated program repair. SBSE techniques have shown promising results, giving us hope that one day it will be possible for the tedious and labour intensive parts of software development to be completely automated, or at least semi-automated. In this talk, I will focus on the problem of objective performance evaluation of SBSE techniques. To this end, I will introduce Instance Space Analysis (ISA), which is an approach to identify features of SBSE problems that explain why a particular instance is difficult for an SBSE technique. ISA can be used to examine the diversity and quality of the benchmark datasets used by most researchers, and analyse the strengths and weaknesses of existing SBSE techniques. The instance space is constructed to reveal areas of hard and easy problems, and enables the strengths and weaknesses of the different SBSE techniques to be identified. I will present on how ISA enabled us to identify the strengths and weaknesses of SBSE techniques in two areas: Search-Based Software Testing and Automated Program Repair. Finally, I will end my talk with future directions of the objective assessment of SBSE techniques.
Pareto-Optimal Search-Based Software Engineering (POSBSE): A Literature SurveyAbdel Salam Sayyad
Paper presented at the 2nd International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE’13), San Francisco, USA. May 2013.
Controlled experiments, Hypothesis Testing, Test Selection, Threats to Validityalessio_ferrari
Complete lecture on controlled experiments in software engineering. It explains practical guidelines on conducting controlled experiments and describes the concepts of dependent, independent, and control variables, significance, and p-value. It also explains how to select the appropriate statistic test for a hypothesis, and gives example of data for different typical tests.
Finally, it discusses threats to validity in controlled experiments and gives indications for reporting.
Find the video lectures here: https://www.youtube.com/playlist?list=PLSKM4VZcJjV-P3fFJYMu2OhlTjEr9Bjl0
Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...Abdel Salam Sayyad
This document summarizes Abdel Salam Sayyad's doctoral defense that addressed using evolutionary search techniques with strong heuristics for multi-objective feature selection in software product lines. The defense outlined modeling feature models, analyzing them automatically, formulating the multi-objective feature selection problem, and using multi-objective evolutionary algorithms. Results demonstrated scalability by increasing objectives, tuning parameters, and using heuristics like "PUSH" and "PULL" as well as population seeding. The defense concluded by discussing future work.
On the application of SAT solvers for Search Based Software Testingjfrchicanog
The document discusses using SAT solvers to solve optimization problems in search-based software testing. It introduces optimization problems and techniques like metaheuristics and evolutionary algorithms. The document then focuses on applying SAT solvers to the test suite minimization problem, which aims to minimize the number of tests needed to achieve full code coverage. It describes translating the optimization problem into a SAT instance that can be solved by SAT solvers to obtain optimal solutions.
The paper presents a new language called UDITA for describing tests. UDITA is a Java-based language that includes non-deterministic choice operators and an interface for generating linked data structures. This allows for more efficient and effective test generation compared to previous approaches. The language aims to make test specification easier while generating tests that are faster, of higher quality, and less complex than traditional manually written or randomly generated tests.
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...Abdel Salam Sayyad
The document summarizes a study that replicated an earlier empirical study on parameter tuning in search-based software engineering. The original study found that different parameter settings can significantly impact performance and that default parameter settings do not always perform optimally. The replication confirmed these findings, and also found that default settings generally performed poorly compared to best tuned settings. The replication also found that IBEA's best tuned performance was generally better than NSGA-II's best tuned performance. Additionally, parameter tuning on a sample of problems did not necessarily lead to the best settings for a new problem, but was generally better than default settings.
Experiments on Design Pattern DiscoveryTim Menzies
The document describes experiments conducted to discover design patterns from source code. It outlines the approach taken by DP-Miner tool, presents experiment data on four Java systems, and evaluates results by calculating precision and recall values. Benchmarks are lacking for accurately evaluating design pattern discovery techniques.
AUTOMATIC GENERATION AND OPTIMIZATION OF TEST DATA USING HARMONY SEARCH ALGOR...csandit
Software testing is the primary phase, which is performed during software development and it is
carried by a sequence of instructions of test inputs followed by expected output. The Harmony
Search (HS) algorithm is based on the improvisation process of music. In comparison to other
algorithms, the HSA has gain popularity and superiority in the field of evolutionary
computation. When musicians compose the harmony through different possible combinations of
the music, at that time the pitches are stored in the harmony memory and the optimization can
be done by adjusting the input pitches and generate the perfect harmony. The test case
generation process is used to identify test cases with resources and also identifies critical
domain requirements. In this paper, the role of Harmony search meta-heuristic search
technique is analyzed in generating random test data and optimized those test data. Test data
are generated and optimized by applying in a case study i.e. a withdrawal task in Bank ATM
through Harmony search. It is observed that this algorithm generates suitable test cases as well
as test data and gives brief details about the Harmony search method. It is used for test data
generation and optimization
Speeding-up Software Testing With Computational IntelligenceAnnibale Panichella
Software testing is a crucial activity to assess the correct behavior of a program. However, it is also costly since it consumes a large ratio of software development time. For this reason, researchers have investigated techniques to automate the process of creating test cases. The key idea is to use meta-heuristics (e.g., Genetic Algorithms) to automatically generate test cases that reveal software failures. In this talk, I will present a case study in the automotive and showing the larger effectiveness and efficiency of meta-heuristics compared to manual testing.
Search-based testing of procedural programs:iterative single-target or multi-...Vrije Universiteit Brussel
In the context of testing of Object-Oriented (OO) software systems, researchers have recently proposed search based approaches to automatically generate whole test suites by considering simultaneously all targets (e.g., branches) defined by the coverage criterion (multi-target approach). The goal of whole suite approaches is to overcome the problem of wasting search budget that iterative single-target approaches (which iteratively generate test cases for each target) can encounter in case of infeasible targets. However, whole suite approaches have not been implemented and experimented in the context of procedural programs. In this paper we present OCELOT (Optimal Coverage sEarch-based tooL for sOftware Testing), a test data generation tool for C programs which implements both a state-of-the-art whole suite approach and an iterative single-target approach designed for a parsimonious use of the search budget. We also present an empirical study conducted on 35 open-source C programs to compare the two approaches implemented in OCELOT. The results indicate that the iterative single-target approach provides a higher efficiency while achieving the same or an even higher level of coverage than the whole suite approach.
Comprehensive Testing Tool for Automatic Test Suite Generation, Prioritizatio...CSCJournals
Testing has been an essential part of software development life cycle. Automatic test case and test data generation has attracted many researchers in the recent past. Test suite generation is the concept given importance which considers multiple objectives in mind and ensures core coverage. The test cases thus generated can have dependencies such as open dependencies and closed dependencies. When there are dependencies, it is obvious that the order of execution of test cases can have impact on the percentage of flaws detected in the software under test. Therefore test case prioritization is another important research area that complements automatic test suite generation in objects oriented systems. Prior researches on test case prioritization focused on dependency structures. However, in this paper, we automate the extraction of dependency structures. We proposed a methodology that takes care of automatic test suite generation and test case prioritization for effective testing of object oriented software. We built a tool to demonstrate the proof of concept. The empirical study with 20 case studies revealed that the proposed tool and underlying methods can have significant impact on the software industry and associated clientele.
A software fault localization technique based on program mutationsTao He
The document describes a new software fault localization technique called Muffler that uses program mutation analysis. Muffler aims to address the problem of coincidental correctness in existing coverage-based fault localization methods. It combines the Naish ranking function with a new metric called mutation impact, which measures the average number of test results that change from passed to failed when each statement is mutated. An empirical evaluation on seven programs shows that Muffler reduces the average code examination needed to find faults by 50% compared to the Naish technique.
Using Developer Information as a Prediction FactorTim Menzies
The document discusses using developer information to improve fault prediction in software systems. It analyzes data from multiple releases of four different software systems totaling over 20 years of data. Adding variables related to the number of developers who modified a file and whether developers were new improves predictions slightly, but factors like file size and whether a file was new or changed are much more important predictors of faults. The best models using these factors identified the top 20% of files that contained over 80% of the faults on average.
Data collection for software defect predictionAmmAr mobark
It is one of the important stages that software companies need it, it will be after produce the program and published, to know the reactions of the users and their impressions about the program and work on developing and improving it.
The Contents
* BACKGROUND AND RELATED WORK
* EXPERIMENTAL PLANNING
-Research Goal -Research Questions -Experimental Subjects
-Experimental Material -Tasks and Methods
-Experimental Design
عمار عبد الكريم صاحب مبارك
AmmAr Abdualkareem sahib mobark
An introduction to AI in Test EngineeringHeemeng Foo
This document provides an overview of artificial intelligence (AI) and how it can be applied to testing and DevOps. It begins with an introduction to common AI techniques like machine learning, deep learning, computer vision, and natural language processing. It then discusses how machine learning can help with challenges in test automation like keeping tests updated and dealing with flaky tests. The document suggests that AI has the potential to improve testing by making it more efficient and less reliant on manual work.
The document discusses test case optimization using genetic and tabu search algorithms for structural testing. It proposes a hybrid algorithm that combines genetic algorithm and tabu search algorithm. Genetic algorithm is initially used to generate test cases, but it can get stuck in local optima. The hybrid approach uses tabu search to improve the mutation step of genetic algorithm. This helps guide the search away from previously visited solutions and avoid local optima, improving test case optimization. An experiment on a voter validation form showed the hybrid approach produced a more efficient test suite compared to genetic algorithm alone.
With the rise of software systems ranging from personal assistance to the nation's facilities, software defects become more critical concerns as they can cost millions of dollar as well as impact human lives. Yet, at the breakneck pace of rapid software development settings (like DevOps paradigm), the Quality Assurance (QA) practices nowadays are still time-consuming. Continuous Analytics for Software Quality (i.e., defect prediction models) can help development teams prioritize their QA resources and chart better quality improvement plan to avoid pitfalls in the past that lead to future software defects. Due to the need of specialists to design and configure a large number of configurations (e.g., data quality, data preprocessing, classification techniques, interpretation techniques), a set of practical guidelines for developing accurate and interpretable defect models has not been well-developed.
The ultimate goal of my research aims to (1) provide practical guidelines on how to develop accurate and interpretable defect models for non-specialists; (2) develop an intelligible defect model that offer suggestions how to improve both software quality and processes; and (3) integrate defect models into a real-world practice of rapid development cycles like CI/CD settings. My research project is expected to provide significant benefits including the reduction of software defects and operating costs, while accelerating development productivity for building software systems in many of Australia's critical domains such as Smart Cities and e-Health.
Towards Privacy-Preserving Evaluation for Information Retrieval Models over I...Twitter Inc.
This document presents a framework called Privacy-Preserving Evaluation (PPE) that allows researchers to evaluate information retrieval models on private industrial datasets without directly accessing the raw data. PPE is designed as a level 2 evaluation platform that allows users to explore indexes, inspect results, and implement retrieval models through APIs. Experiments show that several retrieval models perform well on an industrial collection under PPE, and diagnostic analyses involving document perturbations provide insights into model behaviors without revealing private data. The level-based PPE framework enables more effective use of valuable private datasets while preserving privacy.
The adoption of machine learning techniques for software defect prediction: A...RAKESH RANA
The adoption of machine learning techniques for software defect prediction: An initial industrial validation
Presented at:
11th Joint Conference On Knowledge-Based Software Engineering, JCKBSE, Volgograd, Russia, 2014
Get full text of publication at:
http://rakeshrana.website/index.php/work/publications/
This document discusses the theory of software testing. It covers several key topics:
1) It identifies five common problems in software testing like limitations of testing teams and issues with manual testing.
2) It describes different testing processes like verification, validation, white-box testing and black-box testing.
3) It outlines three main phases of software testing - preliminary testing, testing, and user acceptance testing - to evaluate a new software system and identify any issues.
A Model To Compare The Degree Of Refactoring Opportunities Of Three Projects ...acijjournal
Refactoring is applied to the software artifacts so as to improve its internal structure, while preserving its
external behavior. Refactoring is an uncertain process and it is difficult to give some units for
measurement. The amount to refactoring that can be applied to the source-code depends upon the skills of
the developer. In this research, we have perceived refactoring as a quantified object on an ordinal scale of
measurement. We have a proposed a model for determining the degree of refactoring opportunities in the
given source-code. The model is applied on the three projects collected from a company. UML diagrams
are drawn for each project. The values for source-code metrics, that are useful in determining the quality of
code, are calculated for each UML of the projects. Based on the nominal values of metrics, each relevant
UML is represented on an ordinal scale. A machine learning tool, weka, is used to analyze the dataset,
imported in the form of arff file, produced by the three projects
This document summarizes an empirical study that analyzed static relationships between anti-patterns (ineffective code motifs) and design patterns (effective code solutions) in software systems. The study used tools to detect instances of anti-patterns and design patterns, and found relationships like use, association, aggregation and composition between certain anti-patterns and design patterns. For example, the Command design pattern had relationships with 50% of instances of the SpeculativeGenerality anti-pattern in one system. The study provides evidence that some anti-patterns are more likely than others to have relationships with design patterns.
Ptidej is a flexible reverse engineering tool suite developed at the University of Montreal to evaluate and improve object-oriented programs. It uses a PADL meta-model to parse source code and identify design patterns, motifs, and defects. The demo shows Ptidej parsing and visualizing the JHotDraw program, identifying composites and other design motifs, and integrating with external visualization tools.
SCAN is an approach that assigns meaningful labels to chunks of segmented execution traces and relates these segments. It accepts segmented traces as input and labels segments using terms from invoked method signatures. Formal concept analysis is used to group related segments and identify execution phases. The approach was evaluated on traces from JHotDraw and ArgoUML, achieving reasonable precision and recall in labeling segments and discovering relations between them. Future work includes automating phase recognition and further validation.
On the application of SAT solvers for Search Based Software Testingjfrchicanog
The document discusses using SAT solvers to solve optimization problems in search-based software testing. It introduces optimization problems and techniques like metaheuristics and evolutionary algorithms. The document then focuses on applying SAT solvers to the test suite minimization problem, which aims to minimize the number of tests needed to achieve full code coverage. It describes translating the optimization problem into a SAT instance that can be solved by SAT solvers to obtain optimal solutions.
The paper presents a new language called UDITA for describing tests. UDITA is a Java-based language that includes non-deterministic choice operators and an interface for generating linked data structures. This allows for more efficient and effective test generation compared to previous approaches. The language aims to make test specification easier while generating tests that are faster, of higher quality, and less complex than traditional manually written or randomly generated tests.
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...Abdel Salam Sayyad
The document summarizes a study that replicated an earlier empirical study on parameter tuning in search-based software engineering. The original study found that different parameter settings can significantly impact performance and that default parameter settings do not always perform optimally. The replication confirmed these findings, and also found that default settings generally performed poorly compared to best tuned settings. The replication also found that IBEA's best tuned performance was generally better than NSGA-II's best tuned performance. Additionally, parameter tuning on a sample of problems did not necessarily lead to the best settings for a new problem, but was generally better than default settings.
Experiments on Design Pattern DiscoveryTim Menzies
The document describes experiments conducted to discover design patterns from source code. It outlines the approach taken by DP-Miner tool, presents experiment data on four Java systems, and evaluates results by calculating precision and recall values. Benchmarks are lacking for accurately evaluating design pattern discovery techniques.
AUTOMATIC GENERATION AND OPTIMIZATION OF TEST DATA USING HARMONY SEARCH ALGOR...csandit
Software testing is the primary phase, which is performed during software development and it is
carried by a sequence of instructions of test inputs followed by expected output. The Harmony
Search (HS) algorithm is based on the improvisation process of music. In comparison to other
algorithms, the HSA has gain popularity and superiority in the field of evolutionary
computation. When musicians compose the harmony through different possible combinations of
the music, at that time the pitches are stored in the harmony memory and the optimization can
be done by adjusting the input pitches and generate the perfect harmony. The test case
generation process is used to identify test cases with resources and also identifies critical
domain requirements. In this paper, the role of Harmony search meta-heuristic search
technique is analyzed in generating random test data and optimized those test data. Test data
are generated and optimized by applying in a case study i.e. a withdrawal task in Bank ATM
through Harmony search. It is observed that this algorithm generates suitable test cases as well
as test data and gives brief details about the Harmony search method. It is used for test data
generation and optimization
Speeding-up Software Testing With Computational IntelligenceAnnibale Panichella
Software testing is a crucial activity to assess the correct behavior of a program. However, it is also costly since it consumes a large ratio of software development time. For this reason, researchers have investigated techniques to automate the process of creating test cases. The key idea is to use meta-heuristics (e.g., Genetic Algorithms) to automatically generate test cases that reveal software failures. In this talk, I will present a case study in the automotive and showing the larger effectiveness and efficiency of meta-heuristics compared to manual testing.
Search-based testing of procedural programs:iterative single-target or multi-...Vrije Universiteit Brussel
In the context of testing of Object-Oriented (OO) software systems, researchers have recently proposed search based approaches to automatically generate whole test suites by considering simultaneously all targets (e.g., branches) defined by the coverage criterion (multi-target approach). The goal of whole suite approaches is to overcome the problem of wasting search budget that iterative single-target approaches (which iteratively generate test cases for each target) can encounter in case of infeasible targets. However, whole suite approaches have not been implemented and experimented in the context of procedural programs. In this paper we present OCELOT (Optimal Coverage sEarch-based tooL for sOftware Testing), a test data generation tool for C programs which implements both a state-of-the-art whole suite approach and an iterative single-target approach designed for a parsimonious use of the search budget. We also present an empirical study conducted on 35 open-source C programs to compare the two approaches implemented in OCELOT. The results indicate that the iterative single-target approach provides a higher efficiency while achieving the same or an even higher level of coverage than the whole suite approach.
Comprehensive Testing Tool for Automatic Test Suite Generation, Prioritizatio...CSCJournals
Testing has been an essential part of software development life cycle. Automatic test case and test data generation has attracted many researchers in the recent past. Test suite generation is the concept given importance which considers multiple objectives in mind and ensures core coverage. The test cases thus generated can have dependencies such as open dependencies and closed dependencies. When there are dependencies, it is obvious that the order of execution of test cases can have impact on the percentage of flaws detected in the software under test. Therefore test case prioritization is another important research area that complements automatic test suite generation in objects oriented systems. Prior researches on test case prioritization focused on dependency structures. However, in this paper, we automate the extraction of dependency structures. We proposed a methodology that takes care of automatic test suite generation and test case prioritization for effective testing of object oriented software. We built a tool to demonstrate the proof of concept. The empirical study with 20 case studies revealed that the proposed tool and underlying methods can have significant impact on the software industry and associated clientele.
A software fault localization technique based on program mutationsTao He
The document describes a new software fault localization technique called Muffler that uses program mutation analysis. Muffler aims to address the problem of coincidental correctness in existing coverage-based fault localization methods. It combines the Naish ranking function with a new metric called mutation impact, which measures the average number of test results that change from passed to failed when each statement is mutated. An empirical evaluation on seven programs shows that Muffler reduces the average code examination needed to find faults by 50% compared to the Naish technique.
Using Developer Information as a Prediction FactorTim Menzies
The document discusses using developer information to improve fault prediction in software systems. It analyzes data from multiple releases of four different software systems totaling over 20 years of data. Adding variables related to the number of developers who modified a file and whether developers were new improves predictions slightly, but factors like file size and whether a file was new or changed are much more important predictors of faults. The best models using these factors identified the top 20% of files that contained over 80% of the faults on average.
Data collection for software defect predictionAmmAr mobark
It is one of the important stages that software companies need it, it will be after produce the program and published, to know the reactions of the users and their impressions about the program and work on developing and improving it.
The Contents
* BACKGROUND AND RELATED WORK
* EXPERIMENTAL PLANNING
-Research Goal -Research Questions -Experimental Subjects
-Experimental Material -Tasks and Methods
-Experimental Design
عمار عبد الكريم صاحب مبارك
AmmAr Abdualkareem sahib mobark
An introduction to AI in Test EngineeringHeemeng Foo
This document provides an overview of artificial intelligence (AI) and how it can be applied to testing and DevOps. It begins with an introduction to common AI techniques like machine learning, deep learning, computer vision, and natural language processing. It then discusses how machine learning can help with challenges in test automation like keeping tests updated and dealing with flaky tests. The document suggests that AI has the potential to improve testing by making it more efficient and less reliant on manual work.
The document discusses test case optimization using genetic and tabu search algorithms for structural testing. It proposes a hybrid algorithm that combines genetic algorithm and tabu search algorithm. Genetic algorithm is initially used to generate test cases, but it can get stuck in local optima. The hybrid approach uses tabu search to improve the mutation step of genetic algorithm. This helps guide the search away from previously visited solutions and avoid local optima, improving test case optimization. An experiment on a voter validation form showed the hybrid approach produced a more efficient test suite compared to genetic algorithm alone.
With the rise of software systems ranging from personal assistance to the nation's facilities, software defects become more critical concerns as they can cost millions of dollar as well as impact human lives. Yet, at the breakneck pace of rapid software development settings (like DevOps paradigm), the Quality Assurance (QA) practices nowadays are still time-consuming. Continuous Analytics for Software Quality (i.e., defect prediction models) can help development teams prioritize their QA resources and chart better quality improvement plan to avoid pitfalls in the past that lead to future software defects. Due to the need of specialists to design and configure a large number of configurations (e.g., data quality, data preprocessing, classification techniques, interpretation techniques), a set of practical guidelines for developing accurate and interpretable defect models has not been well-developed.
The ultimate goal of my research aims to (1) provide practical guidelines on how to develop accurate and interpretable defect models for non-specialists; (2) develop an intelligible defect model that offer suggestions how to improve both software quality and processes; and (3) integrate defect models into a real-world practice of rapid development cycles like CI/CD settings. My research project is expected to provide significant benefits including the reduction of software defects and operating costs, while accelerating development productivity for building software systems in many of Australia's critical domains such as Smart Cities and e-Health.
Towards Privacy-Preserving Evaluation for Information Retrieval Models over I...Twitter Inc.
This document presents a framework called Privacy-Preserving Evaluation (PPE) that allows researchers to evaluate information retrieval models on private industrial datasets without directly accessing the raw data. PPE is designed as a level 2 evaluation platform that allows users to explore indexes, inspect results, and implement retrieval models through APIs. Experiments show that several retrieval models perform well on an industrial collection under PPE, and diagnostic analyses involving document perturbations provide insights into model behaviors without revealing private data. The level-based PPE framework enables more effective use of valuable private datasets while preserving privacy.
The adoption of machine learning techniques for software defect prediction: A...RAKESH RANA
The adoption of machine learning techniques for software defect prediction: An initial industrial validation
Presented at:
11th Joint Conference On Knowledge-Based Software Engineering, JCKBSE, Volgograd, Russia, 2014
Get full text of publication at:
http://rakeshrana.website/index.php/work/publications/
This document discusses the theory of software testing. It covers several key topics:
1) It identifies five common problems in software testing like limitations of testing teams and issues with manual testing.
2) It describes different testing processes like verification, validation, white-box testing and black-box testing.
3) It outlines three main phases of software testing - preliminary testing, testing, and user acceptance testing - to evaluate a new software system and identify any issues.
A Model To Compare The Degree Of Refactoring Opportunities Of Three Projects ...acijjournal
Refactoring is applied to the software artifacts so as to improve its internal structure, while preserving its
external behavior. Refactoring is an uncertain process and it is difficult to give some units for
measurement. The amount to refactoring that can be applied to the source-code depends upon the skills of
the developer. In this research, we have perceived refactoring as a quantified object on an ordinal scale of
measurement. We have a proposed a model for determining the degree of refactoring opportunities in the
given source-code. The model is applied on the three projects collected from a company. UML diagrams
are drawn for each project. The values for source-code metrics, that are useful in determining the quality of
code, are calculated for each UML of the projects. Based on the nominal values of metrics, each relevant
UML is represented on an ordinal scale. A machine learning tool, weka, is used to analyze the dataset,
imported in the form of arff file, produced by the three projects
This document summarizes an empirical study that analyzed static relationships between anti-patterns (ineffective code motifs) and design patterns (effective code solutions) in software systems. The study used tools to detect instances of anti-patterns and design patterns, and found relationships like use, association, aggregation and composition between certain anti-patterns and design patterns. For example, the Command design pattern had relationships with 50% of instances of the SpeculativeGenerality anti-pattern in one system. The study provides evidence that some anti-patterns are more likely than others to have relationships with design patterns.
Ptidej is a flexible reverse engineering tool suite developed at the University of Montreal to evaluate and improve object-oriented programs. It uses a PADL meta-model to parse source code and identify design patterns, motifs, and defects. The demo shows Ptidej parsing and visualizing the JHotDraw program, identifying composites and other design motifs, and integrating with external visualization tools.
SCAN is an approach that assigns meaningful labels to chunks of segmented execution traces and relates these segments. It accepts segmented traces as input and labels segments using terms from invoked method signatures. Formal concept analysis is used to group related segments and identify execution phases. The approach was evaluated on traces from JHotDraw and ArgoUML, achieving reasonable precision and recall in labeling segments and discovering relations between them. Future work includes automating phase recognition and further validation.
This document presents a recommendation system for design patterns in software development. The system uses a knowledge base of design patterns and a Goal, Question, Metric methodology to ask users a series of weighted questions to determine the most suitable design pattern for their needs. An experiment was conducted with 8 users to evaluate the system. The system was able to correctly recommend a design pattern 50% of the time and was found to require fewer questions than similar existing systems. Future work will focus on expanding the knowledge base and further evaluating the system.
This document studies the relationship between anti-patterns, dependencies, and fault-proneness. It analyzes three open-source systems to detect anti-patterns and their static and co-change relationships with other classes. It finds that classes with static relationships or that co-change with anti-patterns like Blob, ComplexClass, and SwissArmyKnife are significantly more fault-prone than other classes, even when controlling for complexity, change history, and size. The study aims to understand how anti-patterns can impact the fault-proneness of related classes beyond just those exhibiting the anti-patterns themselves.
This document discusses combining constraint-based testing (CBT) and search-based testing (SBT) to overcome their individual limitations. It provides an example foo function with multiple targets or branches to test. CBT generates constraints from the code to test specific targets, but the constraints for one target are unsatisfiable. Relaxing some constraints allows generating test inputs. The goal is to use CBT to guide SBT's search for test inputs to improve effectiveness over SBT alone.
TRIS is a fast and accurate algorithm for splitting and expanding identifiers. It was compared to other state-of-the-art approaches on two systems and a large dataset. TRIS outperformed other approaches with medium to large effect sizes, achieving a median precision and recall of 1.0. It is also efficient with quadratic complexity in identifier length.
This document summarizes a five-day empirical software engineering course called PASED that was held in Montreal, Canada in 2011. The course aimed to provide graduate students with hands-on experience in planning and conducting empirical software engineering studies. Over the five days, participants learned about experiment design, statistical analysis, text mining, and building prediction models through lectures, labs, and an example running study. Feedback from participants showed the labs and tutorials on tools were most useful, and some wanted longer labs and guidance on writing empirical papers. The course aimed to teach empirical methods through examples and practical application to complement typical statistics courses.
This document describes SODA, a tool that supports the detection of SOA (service-oriented architecture) antipatterns. The tool uses a three step approach: 1) specifying antipatterns, 2) generating detection algorithms, and 3) detecting antipatterns. It is built on the SOFA framework which allows for static and dynamic metric computation and rule specification. The SODA tool provides a graphical user interface for importing services, detecting antipatterns, and visualizing detection results. It aims to help identify common poor solutions known as antipatterns that can hinder maintenance and evolution of service-based systems.
This document presents an approach for detecting antipatterns in BPEL processes. It begins by discussing the motivation, which is that antipatterns can lead to poor quality of service and maintainability. It then reviews related work on modeling antipatterns, detecting them, and defining patterns. The proposed approach involves 3 steps: 1) specifying antipatterns as rules, 2) transforming BPEL processes into an abstract model, and 3) detecting antipatterns by applying the rules. An experiment detects 2 antipatterns in 3 sample processes. The conclusion is that the approach helps identify antipatterns but could be improved by automating detection of more patterns and analyzing dynamic process behavior.
This document proposes an approach to formalize security patterns using formal methods like Petri nets. It describes security patterns, formal methods, and how Petri nets can be used to model the behavior of security patterns. An example of a sender-receiver pattern is analyzed by comparing its Petri net and code structures. Future work includes applying this approach to real systems and using other formal methods to provide running analysis and testing.
Following the previous lectures that focus on the practical uses of good and bad practices, we frame this practices in the context of the developers' cognitive characteristics. We first recall that developers' use sight first and foremost to acquire and reason about their systems. Then, we cast the use of patterns by developers and then discuss the visual system, memory models, and mental models.
In this first lecture, we discuss software quality, introduce the quality characteristic of maintainability, and argue that maintainability can be studied from four different points of view: (1) quality models, (2) good practices, (3) social studies, and (4) developers' studies. We discuss major works and results for these four points of view and show the last three can be used in the first one to build better quality models. We show that quality models are mandatory to make sense of any quality evaluation.
AsianPLoP'14: How and Why Design Patterns Impact Quality and Future ChallengesPtidej Team
The document discusses schemas, design patterns, and how patterns relate to schemas and learning. It notes that schemas are mental representations that allow people to recognize patterns. Design patterns aim to capture best practices and reusable solutions to common problems. When patterns are reused, it activates existing schemas and can lead to better design through experiences being shared. The document explores how pattern reuse relates to theories of schema and learning, and how patterns can be codified and shared between designers based on these theories.
The document discusses patterns in software development. It begins by defining what a pattern is, noting that a pattern describes a general reusable solution to a commonly occurring problem within a specific context. It then discusses some qualities of patterns, including that they aim to enhance reusability, encapsulate design experiences, and provide a common vocabulary among designers. The document also notes that patterns aim to capture an ineffable "quality without a name". It provides examples of patterns from different programming languages to illustrate recurring solutions.
When developers perform a maintenance task, they
follow an exploration strategy (ES) that is characterised by how
they navigate through the program entities. Studying ES can help
to assess how developers understand a program and perform
a change task. Various factors could influence how developers
explore a program and the way in which they explore a program
may affect their performance for a certain task. In this paper,
we investigate the ES followed by developers during maintenance
tasks and assess the impact of these ES on the duration and effort
spent by developers on the tasks. We want to know if developers
frequently revisit one (or a set) of program entities (referenced
exploration), or if they visit program entities with almost the
same frequency (unreferenced exploration) when performing a
maintenance task. We mine 1,705 Mylyn interaction histories
(IH) from four open-source projects (ECF, Mylyn, PDE, and
Eclipse Platform) and perform a user study to verify if both
referenced exploration (RE) and unreferenced exploration (UE)
were followed by some developers. Using the Gini inequality index
on the number of revisits of program entities, we automatically
classify interaction histories as RE and UE and perform an
empirical study to measure the effect of program exploration
on the task duration and effort. We report that, although a UE
may require more exploration effort than a RE, a UE is on
average 12.30% less time consuming than a RE.
130321 zephyrin soh - on the effect of exploration strategies on maintenanc...Ptidej Team
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
9 / 30
This document presents an empirical study that investigates developers' program exploration strategies. The goal is to understand how developers navigate through a program's entities in order to help them more efficiently. The study analyzes developers' interaction histories to identify common exploration strategies and examines relationships between strategies and other factors like task type and expertise level. The results could help evaluate developer performance, improve comprehension models, and guide less experienced developers.
The document discusses a study on how developers spend their effort during maintenance activities. The study aimed to understand if the complexity of changes made to address a task reflects the effort spent by developers. The researchers collected data on developers' interactions and corresponding patches from Eclipse projects. They defined metrics to measure developers' effort and complexity of changes. Their results found that developers do not necessarily spend more effort on tasks requiring more complex changes. Additional factors like number of additional files explored and bug severity were found to affect developers' effort.
For many years, researchers and practitioners have
strived to assess and improve the productivity of software
development teams. One key step toward achieving this goal is
the understanding of factors affecting the efficiency of developers
performing development and maintenance activities. In this paper,
we aim to understand how developers’ spend their effort during
maintenance activities and study the factors affecting developers’
effort. By knowing how developers’ spend their effort and
which factors affect their effort, software organisations will be
able to take the necessary steps to improve the efficiency of
their developers, for example, by providing them with adequate
program comprehension tools. For this preliminary study, we
mine 2,408 developers’ interaction histories and 3,395 patches
from four open-source software projects (ECF, Mylyn, PDE,
Eclipse Platform). We observe that usually, the complexity of
the implementation required for a task does not reflect the effort
spent by developers on the task. Most of the effort appears to be
spent during the exploration of the program. In average, 62%
of files explored during the implementation of a task are not
significantly relevant to the final implementation of the task.
Developers who explore a large number of files that are not
significantly relevant to the solution to their task take a longer
time to perform the task. We expect that the results of this study will pave the way for better program comprehension tools to guide developers during their explorations of software systems.
This document summarizes a study that investigated how students perceive risk in software projects. Students participated in an experiment where they assessed risks related to two project scenarios. Their responses were analyzed to determine if their risk assessment followed a linear, convex, or concave utility function. The study found differences in perceived risk between students and also investigated whether risk perception differed based on their project role or the criticality of the project. The results provide insight into how individuals evaluate and make decisions regarding project risks.
A Survey on Software Release Planning Models - Slides for the Presentation @ ...Supersede
Software release planning (SRP) is the problem of selecting which features or requirements will be included in the next release or releases. It is a crucial step in software development, which happens to be extremely complex given the need to reconcile multiple decision making criteria, (e.g., business value, effort and cost), while considering several constraints (e.g., feature precedencies, resource availa-bility). For this reason, several SRP models have been proposed in the literature. The objective of this study is to provide an updated review of SRP approaches reported in the literature.
A MODEL TO COMPARE THE DEGREE OF REFACTORING OPPORTUNITIES OF THREE PROJECTS ...acijjournal
This document presents a model for quantifying and comparing the degree of refactoring opportunities in three software projects. The model involves drawing UML diagrams for the projects, calculating source code metrics for each UML diagram, representing the diagrams on an ordinal scale based on the metrics, and using a machine learning tool (Weka) to analyze the resulting dataset. The tool uses a Naive Bayesian classifier to generate a confusion matrix for each project, allowing evaluation of the model's performance at classifying refactoring opportunities as low, medium, or high. The model is applied to three projects from a company to test its ability to measure and compare refactoring opportunities in code.
Recommendation System for Design Patterns in Software DevelopmentFrancis Palma
This document presents a recommendation system for design patterns in software development. It aims to help designers choose appropriate design patterns for their problems. The system uses a knowledge base of design patterns and a Goal, Question, Metric model to ask users a series of weighted questions. User responses are scored and patterns are ranked to make recommendations. An experiment was conducted with 8 users, and preliminary results found the system was effective at recommending patterns with less questions than other approaches. Future work will include expanding the knowledge base and more evaluation.
This study investigated the effect of PowerPoint-based quizzes on student performance and experiences in the topic of wave motion. The results showed that students who took a PowerPoint-based quiz performed better and scored higher on average than students who took a traditional oral quiz. A statistical analysis found this difference in scores to be statistically significant. Interviews with students revealed that they found the PowerPoint-based quizzes caught their attention more, allowed them to visualize questions better than oral quizzes, and provided a clearer form of communication compared to traditional oral quizzes. Therefore, the study concluded that PowerPoint-based quizzes had a positive impact on student learning and performance in wave motion.
This document presents a method for prioritizing test cases for event-driven software using a genetic algorithm. It proposes a single abstract model that can test both graphical user interface (GUI) and web applications. Test cases are executed and assigned a fitness value, then stored in a training database. When test cases have equal fitness values, prioritization criteria like fault detection, time, and code coverage are applied using the genetic algorithm to determine the optimal testing order. The approach was experimentally tested on small GUI and web apps and showed a reduction in latency time compared to other techniques. Future work could involve applying the algorithm to larger real-world software.
This document describes a recommendation system for design patterns in software development. It presents a methodology for such a system that has two main components: a knowledge base of design patterns and a Goal-Question-Metric model for selecting patterns. An experiment is described that tests the system with users in a scenario where they must select a design pattern to solve a particular problem. It finds that the system is effective at engaging with users through a question-based interface and recommending the most suitable pattern based on their responses.
Empirical research methods for software engineeringsarfraznawaz
This document outlines guidelines for empirical research methods in software engineering. It discusses case studies, experimental research, surveys, and post-mortem analysis. For each method, it provides examples and discusses how the method can be used to study software engineering problems. It also lists detailed guidelines for different aspects of empirical research, such as experimental context and design, data collection, analysis, and presentation and interpretation of results. The goal of the guidelines is to improve the quality and rigor of empirical studies in software engineering.
The spiral model is a software development process that combines elements of both design and prototyping-centric models. It involves iterating through four phases - planning, risk analysis, engineering, and evaluation - with each iteration intended to incrementally approach completion of the project. The spiral model aims to reduce risk at each phase by gathering user feedback and requirements and addressing risks before proceeding. It is suited for large, complex projects where requirements are not fully known. An example is a project creating a video mail system for illiterate users through iterative user testing and feedback.
Identifying Key Terms in Prompts for Relevance Evaluation with GPT Modelskevig
Relevance evaluation of a query and a passage is essential in Information Retrieval (IR). Recently, numerous studies have been conducted on tasks related to relevance judgment using Large Language Models (LLMs) such as GPT-4, demonstrating significant improvements. However, the efficacy of LLMs is considerably influenced by the design of the prompt. The purpose of this paper is to identify which specific terms in prompts positively or negatively impact relevance evaluation with LLMs. We employed two types of prompts: those used in previous research and generated automatically by LLMs. By comparing the performance of these prompts in both few-shot and zero-shot settings, we analyze the influence of specific terms in the prompts. We have observed two main findings from our study. First, we discovered that prompts using the term ‘answer’ lead to more effective relevance evaluations than those using ‘relevant.’ This indicates that a more direct approach, focusing on answering the query, tends to enhance performance. Second, we noted the importance of appropriately balancing the scope of ‘relevance.’ While the term ‘relevant’ can extend the scope too broadly, resulting in less precise evaluations, an optimal balance in defining relevance is crucial for accurate assessments. The inclusion of few-shot examples helps in more precisely defining this balance. By providing clearer contexts for the term ‘relevance,’ few-shot examples contribute to refine relevance criteria. In conclusion, our study highlights the significance of carefully selecting terms in prompts for relevance evaluation with LLMs.
Identifying Key Terms in Prompts for Relevance Evaluation with GPT Modelskevig
Relevance evaluation of a query and a passage is essential in Information Retrieval (IR). Recently, numerous studies have been conducted on tasks related to relevance judgment using Large Language Models (LLMs) such as GPT-4,
demonstrating significant improvements. However, the efficacy of LLMs is considerably influenced by the design of the prompt. The purpose of this paper is to
identify which specific terms in prompts positively or negatively impact relevance
evaluation with LLMs. We employed two types of prompts: those used in previous
research and generated automatically by LLMs. By comparing the performance of
these prompts in both few-shot and zero-shot settings, we analyze the influence of
specific terms in the prompts. We have observed two main findings from our study.
First, we discovered that prompts using the term ‘answer’ lead to more effective
relevance evaluations than those using ‘relevant.’ This indicates that a more direct
approach, focusing on answering the query, tends to enhance performance. Second,
we noted the importance of appropriately balancing the scope of ‘relevance.’ While
the term ‘relevant’ can extend the scope too broadly, resulting in less precise evaluations, an optimal balance in defining relevance is crucial for accurate assessments.
The inclusion of few-shot examples helps in more precisely defining this balance.
By providing clearer contexts for the term ‘relevance,’ few-shot examples contribute
to refine relevance criteria. In conclusion, our study highlights the significance of
carefully selecting terms in prompts for relevance evaluation with LLMs.
The document discusses prototypes and the spiral model of software development. It defines a prototype as a working model built to test design aspects. The spiral model combines prototyping and the waterfall model, with software development occurring in iterative loops or spirals of planning, risk analysis, engineering, and evaluation. Each spiral builds upon the previous one with increasing requirements, reduced risks, and additional features developed. The spiral model is useful for large, risky projects where requirements are unclear.
An Application-Oriented Approach for Computer Security EducationXiao Qin
In the past few years, numerous universities have incorporated computer security courses into their
undergraduate curricula. Recent studies show that students can effectively gain their knowledge and
experience in building secure computer systems by conducting course projects. However, existing
computer security laboratory exercises are comprised of small-scale, fragmented, and isolated course projects, making it inadequate to prepare undergraduate students to implement real-world secure computing systems. Conventional wisdom in designing computer security course projects pays little
attention to train students to assemble small building blocks into a large-scale secure computing and information system. To overcome students’ lack of experience in implementing large-scale secure software, we propose a novel application-oriented approach to teaching computer security courses by constructing course projects for computer security education. In this pilot project we will develop an extensible application framework for computer security course projects. The framework will provide valuable learning materials that can enable undergraduate students to gain unique experience of building large-scale trustworthy computer systems. Course projects are implemented as plugin modules of an application-based framework. After integrating all the security modules together in the framework, undergraduate students can experiment with various ways of implementing sophisticated
secure computer and information systems.
This document summarizes Zohreh Sharafi's research on the influence of representation type and gender on program comprehension. The research includes several experiments that investigate how representation type (graphical vs. textual) and gender influence developers' efficiency, effectiveness, and viewing strategies during program comprehension tasks. The results show that developers prefer graphical representations and find relevant information faster with graphical representations compared to textual. The experiments also found that men and women use different strategies to select correct answers, with women taking longer but achieving higher accuracy. Further analysis revealed differences in how men and women distribute their visual attention across source code entities during comprehension tasks.
The Meaning of Process
2.2 Software Process Models
2.3 Tools and Techniques for Process Modeling
2.4 Practical Process Modeling
2.5 Information System Example
2.6 What this Chapter Means for You
The document discusses the history and current state of software engineering and its application to IoT systems. It notes that 50 years after the earliest software projects, issues still include cost overruns, property damage, risks to life and death, and challenges ensuring quality. For IoT, fragmentation across hardware, software, APIs and standards poses significant problems. The document proposes that research into IoT software engineering could help address these issues through approaches like developing software to run across diverse IoT platforms, and automatically miniaturizing software through techniques like multi-objective optimization to suit different IoT device capabilities.
1) Issue trackers are often used to track more than just bugs, including features, enhancements, and refactoring work.
2) A manual analysis found that nearly half of issues labeled as "bugs" in issue trackers were actually not bugs.
3) Relying on issue tracker labels alone can introduce significant errors into datasets used for tasks like bug prediction and severity estimation. More work is needed to clean noisy and unreliable data.
The document discusses how to derive dependency structures for legacy J2EE applications. It proposes analyzing all application tiers together using a language-independent model and parsing various artifacts. Configuration files and limited data flow analysis are used to understand dependencies. Container dependencies are explicitly codified by studying technology specifications and codifying dependency rules to apply when certain code patterns are detected in applications. This allows completing an application's dependency graph.
The document discusses the state of practices of service identification in the industry for migrating legacy systems to service-oriented architectures (SOA). It finds that while service identification is seen as important, it remains primarily a manual process focused on identifying coarse-grained business services from source code and business processes. Wrapping and clustering functionalities are common techniques. Fully automating service identification is still challenging due to the need to understand complex legacy system dependencies. The document recommends service identification be business-driven and follow proven methodologies.
This document discusses techniques for testing advanced driver assistance systems (ADAS) through physics-based simulation. It faces challenges due to the large, complex, and multidimensional test input space as well as the computational expense of simulation. The document proposes using a genetic algorithm guided by decision trees to more efficiently search for critical test cases. Classification trees are built to partition the input space into homogeneous regions in order to better guide the selection and generation of test inputs toward more critical areas.
The document reports on the findings of a survey of 45 industrial practitioners on their experiences with legacy-to-SOA migrations. The key findings include: 1) Practitioners migrate legacy systems implemented in Cobol and Java to reduce maintenance costs and improve flexibility/interoperability; 2) Identifying services is an important step but is mostly manual and business-driven; 3) The most used techniques are functionality clustering and wrapping; 4) Desired service qualities are reusability, granularity and loose coupling; 5) Identified services prioritize domain-specific over technical services; 6) RESTful services are most targeted technology.
The document investigates the impact of linguistic anti-patterns (LAs) on program comprehension. It defines LAs as bad naming, documentation, and implementation practices. A study was conducted involving 92 students assessing programs with and without LAs. The study found that LAs negatively impact understandability by increasing time and reducing correctness. Certain LAs like A2, B4, and D1 had a stronger negative effect than others like E1. The study also found that providing knowledge about LAs can help mitigate their impact by making programs easier and faster to comprehend.
The document discusses research on identifying and analyzing the impact of patterns on the quality of multi-language systems. The objectives are to collect and categorize sets of programming languages used together, detect patterns in multi-language programs to track bugs and provide best practices, and study how patterns impact quality. The contributions will be a catalog of multi-language patterns and defects, a detection tool, and analysis of patterns' effects on quality attributes. Current work includes reviewing literature on language combinations and patterns to provide recommendations for high-quality multi-language development.
This document discusses research on change impact analysis in multi-language systems. It begins by outlining recommendations for best practices when using JNI, such as passing primitive types, minimizing calls between native and Java code, and properly handling strings. It then describes a qualitative analysis of JNI usage that identified common practices and issues. Finally, it proposes future work to survey developers on applying recommendations to facilitate change impact analysis in multi-language systems.
The document summarizes a recommendation system that suggests software processes for video game projects based on similarities to past projects. The system analyzes over 100 postmortems from previous games to build a database of development processes and project contexts. It uses principal component analysis to identify similar past projects and recommends a process by combining elements from similar projects' processes. The system was evaluated both quantitatively based on correctness and coverage metrics and qualitatively through surveys and a case study with a developer team.
Will io t trigger the next software crisisPtidej Team
This document discusses how the rise of the Internet of Things (IoT) could trigger a new software crisis due to issues like fragmentation, complexity, and lack of standards. It provides a brief history of software engineering challenges over the past 50 years such as cost overruns, safety issues, and prioritizing productivity over quality. The document then examines how these same problems are emerging in the IoT context today. It argues that IoT software engineering practices need to address issues like device software, cloud/app development, and privacy in order to avoid a major crisis.
This document discusses theories related to software design patterns. It notes that while design patterns are commonly used, there is a need for more research on how they impact software quality. The document proposes several areas for developing theories, including systematically categorizing existing patterns based on underlying principles, combining principles to identify new patterns, and developing theories of patterns from developer behavior and for building software systems. Formalizing patterns and identifying their relationships could help teaching and understanding of patterns.
Laleh M. Eshkevari defended her Ph.D dissertation on developing techniques for the automatic detection and classification of identifier renamings in software projects. Her dissertation outlined a taxonomy of renamings, described approaches for renaming detection based on line mapping, entity mapping and data flow analysis, and discussed methods for classifying renamings based on their form and semantic changes. Evaluation of the approaches on several open source projects showed high precision and recall for renaming detection and identified trends in how renamings are used in practice.
1) The document analyzes the co-occurrence of code smells like anti-patterns and clones in software systems and their impact on fault-proneness.
2) It finds that over 50% of classes with anti-patterns also have clones, and 59-78% of classes with clones also participate in anti-patterns.
3) Classes with both anti-patterns and clones are significantly more fault-prone than other classes, with the risk of faults being at least 7 times higher in one system studied.
Trustrace is an approach that uses software repository links like SVN commits to improve the trust in automatically recovered traceability links between requirements and code. It calculates an initial trust value for links based on IR techniques like VSM, and then reweights the links based on additional information from the software repository. An evaluation on two case studies found Trustrace improved precision over VSM alone and showed no significant difference in recall, supporting the hypothesis that Trustrace can improve link recovery accuracy over IR-only approaches.
The document presents a taxonomy called ProMeTA for classifying program metamodels used in program reverse engineering. ProMeTA defines characteristics such as target language, abstraction level, meta-language, and more to classify popular metamodels like AST, KDM, FAMIX. The taxonomy aims to provide a comprehensive guide for researchers and practitioners to select, design, and communicate metamodels. The paper also analyzes existing metamodels according to the ProMeTA taxonomy and identifies gaps to guide future metamodel development.
This document describes a controlled, multiple case study of software evolution and defects from industrial projects. It details the data sources used, including source code repositories, issue tracking databases, and interviews. Metrics such as code smells, size, effort, and defects were collected. Programming skills of developers were also measured. Code smell detection tools and custom scripts to analyze code changes were used to extract metrics on a variety of code issues and evolution over time. The data is available online for further analysis.
The document describes a study on detecting linguistic (anti)patterns in RESTful APIs. It presents an approach called DOLAR (Detection Of Linguistic Antipatterns in REST) that analyzes REST API URIs and detects antipatterns using heuristics-based algorithms. Experiments were conducted on 309 methods from 15 public REST APIs to test DOLAR's accuracy, the extensibility of the underlying SOFA framework, and the performance of detection algorithms. The results showed that 42% of methods exhibited contextualized resource names (a pattern) while 14% had contextless resource names (an antipattern), with detection taking under a second on average.
Webinar: Designing a schema for a Data WarehouseFederico Razzoli
Are you new to data warehouses (DWH)? Do you need to check whether your data warehouse follows the best practices for a good design? In both cases, this webinar is for you.
A data warehouse is a central relational database that contains all measurements about a business or an organisation. This data comes from a variety of heterogeneous data sources, which includes databases of any type that back the applications used by the company, data files exported by some applications, or APIs provided by internal or external services.
But designing a data warehouse correctly is a hard task, which requires gathering information about the business processes that need to be analysed in the first place. These processes must be translated into so-called star schemas, which means, denormalised databases where each table represents a dimension or facts.
We will discuss these topics:
- How to gather information about a business;
- Understanding dictionaries and how to identify business entities;
- Dimensions and facts;
- Setting a table granularity;
- Types of facts;
- Types of dimensions;
- Snowflakes and how to avoid them;
- Expanding existing dimensions and facts.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Speck&Tech
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
OpenID AuthZEN Interop Read Out - AuthorizationDavid Brossard
During Identiverse 2024 and EIC 2024, members of the OpenID AuthZEN WG got together and demoed their authorization endpoints conforming to the AuthZEN API
Generating privacy-protected synthetic data using Secludy and MilvusZilliz
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slackshyamraj55
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
1. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
On the Effect of Program Exploration on
Maintenance Tasks
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
Z´phyrin Soh, Foutse Khomh, Yann-Ga¨l Gu´h´neuc,
e
e
e e
Giuliano Antoniol, Bram Adams
Department of Computer and Software Engineering
´
Ecole Polytechnique de Montr´al, Qu´bec, Canada
e
e
October 14, 2013
Pattern Trace Identification, Detection, and Enhancement in Java
SOftware Cost-effective Change and Evolution Research Lab
2. Program
Exploration
Outline
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Introduction
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
User Study
Empirical Study
Conclusion and Future Work
2 / 23
3. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example (1/3)
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
3 / 23
Program Exploration
Developers interact with program entities, e.g., open
files, edit methods, etc.
Developers explore the program, i.e., move from
program entities to program entities.
Types of program entities.
4. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
4 / 23
Introduction
Context and Example (2/3)
5. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
4 / 23
Introduction
Context and Example (2/3)
6. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
4 / 23
Introduction
Context and Example (2/3)
7. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
4 / 23
Containment
Introduction principle:
the exploration graph
Context and Example (2/3)
Aggregate
at class level
8. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example (2/3)
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
4 / 23
Exploration graph aggregated at class level
9. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
5 / 23
Introduction
Context and Example (3/3)
10. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
5 / 23
Introduction
Context and Example (3/3)
11. Program
Exploration
Introduction
Z´phyrin Soh et al.
e
Context and Example (3/3)
A graph
represents a program exploration
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
(a) Graph 1
(b) Graph 2
Is there any difference between exploration graphs?
How to find the difference?
How does the difference affect the maintenance tasks?
5 / 23
12. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Why do we care? (1/3)
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Research on Program Exploration
Developers spend on average 35% of their time
navigating within and between source files [1]
Developers perform on average 19.31 navigation actions
between two modifications [2]
Conclusion
Threats to Validity
and Future Work
[1] Ko et al., An exploratory study of how developers seek, relate, and collect
relevant information during software maintenance tasks. TSE 2006
[2] D. R¨thlisberger, SmartGroups: Focusing on Task-Relevant Source
o
Artifacts in IDEs, ICPC 2011
6 / 23
13. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Why do we care? (2/3)
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
Reasons of Program Exploration
Developers may have several different ways to explore a
program e.g.,
Looking for relevant entities ⇒ Some developers (may)
not know where to look at.
Validate previous changes ⇒ Developers (may) know
the target entity to look at.
⇒ In some cases, developers may perform more back and
forth navigations (i.e., revisitation) on program entities
7 / 23
14. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Why do we care? (3/3)
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
Revisitation-based Exploration
Some recommendation systems (e.g., Mylyn [3],
NavClus [4]) are based on the revisitation of program
entities
We wonder if revisitation is “good“ or “bad“
⇒ Knowing that is helpful to improve revisitation-based
approach for recommendation systems.
[3] Kersten and Murphy, Mylar: a degree-of-interest model for IDEs, AOSD
2005
[4] Lee and Kang, Clustering and recommending collections of code relevant
to tasks, ICSM 2011
8 / 23
15. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Research Question
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
9 / 23
Exploration Strategies
In the revisitation-based exploration, we focus on two
extreme cases:
Referenced Exploration (RE): A developer revisits one
(or a set of) entity(ies) already visited
⇒ referenced entities
Unreferenced Exploration (UE): There is no set of
referenced entities
⇒ Almost the same revisitation of program entity(ies)
16. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Research Question
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
Exploration Strategies
In the revisitation-based exploration, we focus on two
extreme cases:
Referenced Exploration (RE): A developer revisits one
(or a set of) entity(ies) already visited
⇒ referenced entities
Unreferenced Exploration (UE): There is no set of
referenced entities
⇒ Almost the same revisitation of program entity(ies)
Do developers follow a referenced exploration when
performing maintenance tasks?
9 / 23
17. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
10 / 23
User Study
Approach (1/5)
Data Collection
Mylyn’s IHs appear as attachment (XML file with name
“mylyn-context.zip”) to a bug
18. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
User Study
Approach (1/5)
Data Collection
Mylyn’s IHs appear as attachment (XML file with name
“mylyn-context.zip”) to a bug
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
Top four Eclipse projects with more IHs (from 2,601 bugs)
ECF Mylyn PDE Platform Total
# IHs
26
1,273
131
275 1,705
10 / 23
19. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Approach (2/5)
How do we build an oracle?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
11 / 23
Random sample of IHs
(proportional among projects)
20. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Approach (2/5)
How do we build an oracle?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
11 / 23
Random sample of IHs
(proportional among projects)
21. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (2/5)
Introduction
How do we build an oracle?
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Random sample of IHs
(proportional among projects)
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
9 subjects
Java experience
- mean : 4.16 yrs
-sd : 2.80 yrs
11 / 23
22. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (2/5)
Introduction
How do we build an oracle?
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Random sample of IHs
(proportional among projects)
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
9 subjects
Java experience
- mean : 4.16 yrs
-sd : 2.80 yrs
11 / 23
23. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (2/5)
Introduction
How do we build an oracle?
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Random sample of IHs
(proportional among projects)
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
9 subjects
Java experience
- mean : 4.16 yrs
-sd : 2.80 yrs
RE
11 / 23
UE
D
24. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (2/5)
Introduction
How do we build an oracle?
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Random sample of IHs
(proportional among projects)
Empirical Study
Research Questions
RQ1
RQ2
Discussions
How much the subjects agreed?
Fliess' Kappa interater agreement coeficient
Conclusion and
Future Work
- Overall: fair agreement (0.36)
- RE: almost moderate agreement (0.38)
- UE: almost moderate agreement (0.39)
- Doubt: No agreement (-0.009)
Conclusion
Threats to Validity
and Future Work
9 subjects
Java experience
- mean : 4.16 yrs
-sd : 2.80 yrs
RE
11 / 23
UE
D
25. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (3/5)
Introduction
Context and Example
Why do we care?
How do we aggregate the results?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
RE
12 / 23
RE
UE
UE
D
26. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (3/5)
Introduction
Context and Example
Why do we care?
How do we aggregate the results?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
RE
RE
UE
>= 2/3 subjects
RE
12 / 23
UE
D
27. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (3/5)
Introduction
Context and Example
Why do we care?
How do we aggregate the results?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
RE
RE
UE
UE
>= 2/3 subjects
UE
12 / 23
D
28. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (3/5)
Introduction
Context and Example
Why do we care?
How do we aggregate the results?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
RE
RE
UE
< 2/3 subjects
< 2/3 subjects
D
12 / 23
UE
D
29. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
13 / 23
User Study
Approach (4/5)
Identification Technique
Subjects count the number of nodes and the number of in/out arrows in the graphs
⇒ look at the distribution/inequality of revisits across graphs
30. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
User Study
Approach (4/5)
Identification Technique
Subjects count the number of nodes and the number of in/out arrows in the graphs
⇒ look at the distribution/inequality of revisits across graphs
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
Gini Inequality Index [5]
We measure how much classes are (in)equally revisited
Used in econometrics to measure the inequality of
income among a population
Population = classes
Income of a class = NumRevisits
Gini(IH) =
1
2n2 µ
n
n
i=1 j=1
| NumRevisit(eni ) − NumRevisit(enj ) |
[5] K. Xu, How has the literature on Gini’s index evolved in the past 80 years?,
Technical report, Department of Economics, Dalhouse University, 2004
13 / 23
31. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (5/5)
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
14 / 23
Inequality Index Threshold
Define a threshold to know if classes are (in)equally revisited
If Gini(IH) < Threshold
⇒ Classes are almost equally revisited
⇒ UE: Unreferenced Exploration
If Gini(IH) ≥ Threshold
⇒ Some classes are more revisited than others
⇒ RE: Referenced Exploration
32. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (5/5)
Introduction
Inequality Index Threshold
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
0
Perfect
equality
14 / 23
1
Maximum
inequality
33. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (5/5)
Introduction
Inequality Index Threshold
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Threshold ?
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
0
Perfect
equality
14 / 23
1
Maximum
inequality
34. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Approach (5/5)
Inequality Index Threshold
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
14 / 23
RE
35. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (5/5)
Introduction
Inequality Index Threshold
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
RE
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
0.28
Conclusion
Threats to Validity
and Future Work
0
Perfect
equality
14 / 23
1
Maximum
inequality
36. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (5/5)
Introduction
Inequality Index Threshold
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
RE
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
0.28
Conclusion
Threats to Validity
and Future Work
0
Perfect
equality
14 / 23
0.2
0.3
- Threshold = 0.2 ==> RE
- Threshold = 0.3 ==> UE
1
Maximum
inequality
37. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (5/5)
Introduction
Inequality Index Threshold
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
RE
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
0.4
Conclusion
Threats to Validity
and Future Work
0
Perfect
equality
1
Maximum
inequality
Use F-Measure to maximize both precision and recall
14 / 23
38. Program
Exploration
Z´phyrin Soh et al.
e
User Study
Approach (5/5)
Introduction
Inequality Index Threshold
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
RE
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
0.4
Conclusion
Threats to Validity
and Future Work
0
Perfect
equality
1
UE
RE
Maximum
inequality
Use F-Measure to maximize both precision and recall
14 / 23
39. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
15 / 23
User Study
Results (1/2)
Percentage RE vs. UE
RE: 28.03% of Interaction Histories
UE: 71.96% of Interaction Histories
⇒ Developers follow mostly the unreferenced
exploration (UE) when performing a maintenance task.
40. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
User Study
Results (1/2)
Percentage RE vs. UE
RE: 28.03% of Interaction Histories
UE: 71.96% of Interaction Histories
⇒ Developers follow mostly the unreferenced
exploration (UE) when performing a maintenance task.
Methodical developers do not reinvestigate methods as
frequently as opportunistic developers [6]
RE = opportunistic developers?
UE = methodical developers?
⇒ We need more investigations
[6] Robillard et al., How effective developers investigate source code: An
exploratory study, TSE 2004
15 / 23
41. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
16 / 23
User Study
Results (2/2)
Confounding Factors
Architecture of the system
42. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
16 / 23
User Study
Results (2/2)
Confounding Factors
Architecture of the system
Number of common classes (NCC): The more A and B
have CC, the more they used the same part of the
system
43. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
16 / 23
User Study
Results (2/2)
Confounding Factors
Architecture of the system
Number of common classes (NCC): The more A and B
have CC, the more they used the same part of the
system
Compute NCC for each pair of IH.
NCC pairs with same ES vs. NCC pairs with different
ES ⇒ No difference.
44. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
16 / 23
User Study
Results (2/2)
Confounding Factors
Architecture of the system
Number of common classes (NCC): The more A and B
have CC, the more they used the same part of the
system
Compute NCC for each pair of IH.
NCC pairs with same ES vs. NCC pairs with different
ES ⇒ No difference.
Task interruption: The interruption time seems to push
developers to be concentrate on a set of (referenced)
entities (RE)
45. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
User Study
Results (2/2)
Confounding Factors
Architecture of the system
Number of common classes (NCC): The more A and B
have CC, the more they used the same part of the
system
Compute NCC for each pair of IH.
NCC pairs with same ES vs. NCC pairs with different
ES ⇒ No difference.
Task interruption: The interruption time seems to push
developers to be concentrate on a set of (referenced)
entities (RE)
Type of the task: Bug severity because developers may
have more back and forth navigation when fixing severe
bugs wrt. less severe bugs
Inequality Indexes are not significantly different for
different types of bugs.
16 / 23
46. Program
Exploration
Z´phyrin Soh et al.
e
Empirical Study
Research Questions
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Research Questions
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Does any difference exist in maintenance time between RE and UE?
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
17 / 23
Does any difference exist in edit/navigation ratio
between RE and UE?
47. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
18 / 23
Empirical Study
RQ1
Approach
Overall Time spend =
Unpaired Wilcoxon test
Duration(entity )
#entity
entity ∈IH
48. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
RQ1
Approach
Overall Time spend =
Duration(entity )
#entity
entity ∈IH
Unpaired Wilcoxon test
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
18 / 23
Results
The UE is on average 12.30% less time consuming
than the RE.
49. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
19 / 23
Empirical Study
RQ2
Approach
Edit/navigation ratio =
Unpaired Wilcoxon test
NumEdit(IH)
NumEvent(IH)
50. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
19 / 23
Empirical Study
RQ2
Approach
Edit/navigation ratio =
NumEdit(IH)
NumEvent(IH)
Unpaired Wilcoxon test
Results
An UE requires less edit/navigation ratio than a
RE.
51. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
20 / 23
Empirical Study
Discussions
Is there a good strategy?
RE (more) ⇒ The back and forth actions on refUE (less) erenced entities are time consuming
⇒ More edit actions compare to navRE (more)
igation actions. Wrong edits/modif.
UE (less)
then come back to revert/cancel?
52. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
Empirical Study
Discussions
Is there a good strategy?
RE (more) ⇒ The back and forth actions on refUE (less) erenced entities are time consuming
User Study
Research Question
Approach
Results
Empirical Study
⇒ More edit actions compare to navRE (more)
igation actions. Wrong edits/modif.
UE (less)
then come back to revert/cancel?
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
RE
F1
F2
F1
F3
F1
F4
F1
F2
F3
F1
F4
F5
UE
20 / 23
4/8 > 3/6 (RE better)
F2
3/6
F5
4/8
53. Program
Exploration
Z´phyrin Soh et al.
e
Introduction
Context and Example
Why do we care?
Empirical Study
Discussions
Is there a good strategy?
RE (more) ⇒ The back and forth actions on refUE (less) erenced entities are time consuming
User Study
Research Question
Approach
Results
Empirical Study
⇒ More edit actions compare to navRE (more)
igation actions. Wrong edits/modif.
UE (less)
then come back to revert/cancel?
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
RE
F1
F2
F1
F3
F1
F4
F1
F2
F3
F1
F4
F5
UE
20 / 23
4/8 > 3/6 (RE better)
F2
3/6
F5
4/8
2/8
2/6
2/8 < 2/6 (UE better)
54. Program
Exploration
Z´phyrin Soh et al.
e
Conclusion and Future Work
Conclusion
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
RE
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
21 / 23
0.4
0
Perfect
equality
1
UE
RE
Maximum
inequality
Use F-Measure to maximize both precision and recall
55. Program
Exploration
Z´phyrin Soh et al.
e
Conclusion and Future Work
Conclusion
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
RE
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
21 / 23
0.4
0
Perfect
equality
1
UE
RE
Maximum
inequality
Use F-Measure to maximize both precision and recall
56. Program
Exploration
Z´phyrin Soh et al.
e
Conclusion and Future Work
Conclusion
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
RE
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
21 / 23
0.4
0
Perfect
equality
1
UE
RE
Maximum
inequality
Use F-Measure to maximize both precision and recall
57. Program
Exploration
Z´phyrin Soh et al.
e
Conclusion and Future Work
Threats to Validity and Future Work
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
22 / 23
Threats to Validity and Future Work
Construct validity: An IH can be a part of developers’
work
58. Program
Exploration
Z´phyrin Soh et al.
e
Conclusion and Future Work
Threats to Validity and Future Work
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
22 / 23
Threats to Validity and Future Work
Construct validity: An IH can be a part of developers’
work
Conclusion validity: The time recorded can be
different to the “real” time spent
⇒ Perform an experiment to collect a data
59. Program
Exploration
Z´phyrin Soh et al.
e
Conclusion and Future Work
Threats to Validity and Future Work
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
22 / 23
Threats to Validity and Future Work
Construct validity: An IH can be a part of developers’
work
Conclusion validity: The time recorded can be
different to the “real” time spent
⇒ Perform an experiment to collect a data
Internal validity: We use only Mylyn’s IH
⇒ Use the IH from other tools
60. Program
Exploration
Z´phyrin Soh et al.
e
Conclusion and Future Work
Threats to Validity and Future Work
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
Threats to Validity and Future Work
Construct validity: An IH can be a part of developers’
work
Conclusion validity: The time recorded can be
different to the “real” time spent
⇒ Perform an experiment to collect a data
Internal validity: We use only Mylyn’s IH
⇒ Use the IH from other tools
External validity: Our subject projets are Eclipse-based
projects
⇒ Use other systems
22 / 23
61. Program
Exploration
Z´phyrin Soh et al.
e
Thanks for your attention!
Introduction
Context and Example
Why do we care?
User Study
Research Question
Approach
Results
RE
Empirical Study
Research Questions
RQ1
RQ2
Discussions
Conclusion and
Future Work
Conclusion
Threats to Validity
and Future Work
23 / 23
0.4
0
Perfect
equality
1
UE
RE
Maximum
inequality
Use F-Measure to maximize both precision and recall