Abstract—Combinatorial testing (also called interaction testing) is an effective specification-based test input generation technique. By now most of research work in combinatorial testing aims to propose novel approaches trying to generate test suites with minimum size that still cover all the pairwise, triple, or n-way combinations of factors. Since the difficulty of solving this problem is demonstrated to be NP-hard, existing approaches have been designed to generate optimal or near optimal combinatorial test suites in polynomial time. In this paper, we try to apply particle swarm optimization (PSO), a kind of meta-heuristic search technique, to pairwise testing (i.e. a special case of combinatorial testing aiming to cover all the pairwise combinations). To systematically build pairwise test suites, we propose two different PSO based algorithms. One algorithm is based on one-test-at-a-time strategy and the other is based on IPO-like strategy. In these two different algorithms, we use PSO to complete the construction of a single test. To successfully apply PSO to cover more uncovered pairwise combinations in this construction process, we provide a detailed description on how to formulate the search space, define the fitness function and set some heuristic settings. To verify the effectiveness of our approach, we implement these algorithms and choose some typical inputs. In our empirical study, we analyze the impact factors of our approach and compare our approach to other well-known approaches. Final empirical results show the effectiveness and efficiency of our approach.
Software testing is an important activity of the software development process. Software testing is most
efforts consuming phase in software development. One would like to minimize the effort and maximize the
number of faults detected and automated test case generation contributes to reduce cost and time effort.
Hence test case generation may be treated as an optimization problem In this paper we have used genetic
algorithm to optimize the test case that are generated applying conditional coverage on source code. Test
case data is generated automatically using genetic algorithm are optimized and outperforms the test cases
generated by random testing.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Welcome to International Journal of Engineering Research and Development (IJERD)IJERD Editor
call for paper 2012, hard copy of journal, research paper publishing, where to publish research paper,
journal publishing, how to publish research paper, Call For research paper, international journal, publishing a paper, IJERD, journal of science and technology, how to get a research paper published, publishing a paper, publishing of journal, publishing of research paper, reserach and review articles, IJERD Journal, How to publish your research paper, publish research paper, open access engineering journal, Engineering journal, Mathemetics journal, Physics journal, Chemistry journal, Computer Engineering, Computer Science journal, how to submit your paper, peer reviw journal, indexed journal, reserach and review articles, engineering journal, www.ijerd.com, research journals,
yahoo journals, bing journals, International Journal of Engineering Research and Development, google journals, hard copy of journal
Software testing is an important activity of the software development process. Software testing is most
efforts consuming phase in software development. One would like to minimize the effort and maximize the
number of faults detected and automated test case generation contributes to reduce cost and time effort.
Hence test case generation may be treated as an optimization problem In this paper we have used genetic
algorithm to optimize the test case that are generated applying conditional coverage on source code. Test
case data is generated automatically using genetic algorithm are optimized and outperforms the test cases
generated by random testing.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Welcome to International Journal of Engineering Research and Development (IJERD)IJERD Editor
call for paper 2012, hard copy of journal, research paper publishing, where to publish research paper,
journal publishing, how to publish research paper, Call For research paper, international journal, publishing a paper, IJERD, journal of science and technology, how to get a research paper published, publishing a paper, publishing of journal, publishing of research paper, reserach and review articles, IJERD Journal, How to publish your research paper, publish research paper, open access engineering journal, Engineering journal, Mathemetics journal, Physics journal, Chemistry journal, Computer Engineering, Computer Science journal, how to submit your paper, peer reviw journal, indexed journal, reserach and review articles, engineering journal, www.ijerd.com, research journals,
yahoo journals, bing journals, International Journal of Engineering Research and Development, google journals, hard copy of journal
Software Quality Analysis Using Mutation Testing SchemeEditor IJMTER
The software test coverage is used measure the safety measures. The safety critical analysis is
carried out for the source code designed in Java language. Testing provides a primary means for
assuring software in safety-critical systems. To demonstrate, particularly to a certification authority, that
sufficient testing has been performed, it is necessary to achieve the test coverage levels recommended or
mandated by safety standards and industry guidelines. Mutation testing provides an alternative or
complementary method of measuring test sufficiency, but has not been widely adopted in the safetycritical industry. The system provides an empirical evaluation of the application of mutation testing to
airborne software systems which have already satisfied the coverage requirements for certification.
The system mutation testing to safety-critical software developed using high-integrity subsets of
C and Ada, identify the most effective mutant types and analyze the root causes of failures in test cases.
Mutation testing could be effective where traditional structural coverage analysis and manual peer
review have failed. They also show that several testing issues have origins beyond the test activity and
this suggests improvements to the requirements definition and coding process. The system also
examines the relationship between program characteristics and mutation survival and considers how
program size can provide a means for targeting test areas most likely to have dormant faults. Industry
feedback is also provided, particularly on how mutation testing can be integrated into a typical
verification life cycle of airborne software. The system also covers the safety and criticality levels of
Java source code.
Software Quality Assurance (SQA) teams play a critical role in the software development process to ensure the absence of software defects. It is not feasible to perform exhaustive SQA tasks (i.e., software testing and code review) on a large software product given the limited SQA resources that are available. Thus, the prioritization of SQA efforts is an essential step in all SQA efforts. Defect prediction models are used to prioritize risky software modules and understand the impact of software metrics on the defect-proneness of software modules. The predictions and insights that are derived from defect prediction models can help software teams allocate their limited SQA resources to the modules that are most likely to be defective and avoid common past pitfalls that are associated with the defective modules of the past. However, the predictions and insights that are derived from defect prediction models may be inaccurate and unreliable if practitioners do not control for the impact of experimental components (e.g., datasets, metrics, and classifiers) on defect prediction models, which could lead to erroneous decision-making in practice. In this thesis, we investigate the impact of experimental components on the performance and interpretation of defect prediction models. More specifically, we investigate the impact of the three often overlooked experimental components (i.e., issue report mislabelling, parameter optimization of classification techniques, and model validation techniques) have on defect prediction models. Through case studies of systems that span both proprietary and open-source domains, we demonstrate that (1) issue report mislabelling does not impact the precision of defect prediction models, suggesting that researchers can rely on the predictions of defect prediction models that were trained using noisy defect datasets; (2) automated parameter optimization for classification techniques substantially improve the performance and stability of defect prediction models, as well as they change their interpretation, suggesting that researchers should no longer shy from applying parameter optimization to their models; and (3) the out-of-sample bootstrap validation technique produces a good balance between bias and variance of performance estimates, suggesting that the single holdout and cross-validation families that are commonly-used nowadays should be avoided.
Testing embedded system through optimal mining technique (OMT) based on multi...IJECEIAES
Testing embedded systems must be done carefully particularly in the significant regions of the embedded systems. Inputs from an embedded system can happen in multiple order and many relationships can exist among the input sequences. Consideration of the sequences and the relationships among the sequences is one of the most important considerations that must be tested to find the expected behavior of the embedded systems. On the other hand combinatorial approaches help determining fewer test cases that are quite enough to test the embedded systems exhaustively. In this paper, an Optimal Mining Technique that considers multi-input domain which is based on built-in combinatorial approaches has been presented. The method exploits multi-input sequences and the relationships that exist among multi-input vectors. The technique has been used for testing an embedded system that monitors and controls the temperature within the Nuclear reactors.
Model based test case prioritization using neural network classificationcseij
Model-based testing for real-life software systems often require a large number of tests, all of which cannot
exhaustively be run due to time and cost constraints. Thus, it is necessary to prioritize the test cases in
accordance with their importance the tester perceives. In this paper, this problem is solved by improving
our given previous study, namely, applying classification approach to the results of our previous study
functional relationship between the test case prioritization group membership and the two attributes:
important index and frequency for all events belonging to given group are established. A for classification
purpose, neural network (NN) that is the most advances is preferred and a data set obtained from our study
for all test cases is classified using multilayer perceptron (MLP) NN. The classification results for
commercial test prioritization application show the high classification accuracies about 96% and the
acceptable test prioritization performances are achieved.
Search-based testing of procedural programs:iterative single-target or multi-...Vrije Universiteit Brussel
In the context of testing of Object-Oriented (OO) software systems, researchers have recently proposed search based approaches to automatically generate whole test suites by considering simultaneously all targets (e.g., branches) defined by the coverage criterion (multi-target approach). The goal of whole suite approaches is to overcome the problem of wasting search budget that iterative single-target approaches (which iteratively generate test cases for each target) can encounter in case of infeasible targets. However, whole suite approaches have not been implemented and experimented in the context of procedural programs. In this paper we present OCELOT (Optimal Coverage sEarch-based tooL for sOftware Testing), a test data generation tool for C programs which implements both a state-of-the-art whole suite approach and an iterative single-target approach designed for a parsimonious use of the search budget. We also present an empirical study conducted on 35 open-source C programs to compare the two approaches implemented in OCELOT. The results indicate that the iterative single-target approach provides a higher efficiency while achieving the same or an even higher level of coverage than the whole suite approach.
Software analytics (for software quality purpose) is a statistical or machine learning classifier that is trained to identify defect-prone software modules. The goal of software analytics is to help software engineers prioritize their software testing effort on the most-risky modules and understand past pitfalls that lead to defective code. While the adoption of software analytics enables software organizations to distil actionable insights, there are still many barriers to broad and successful adoption of such analytics systems. Indeed, even if software organizations can access such invaluable software artifacts and toolkits for data analytics, researchers and practitioners often have little knowledge to properly develop analytics systems. Thus, the accuracy of the predictions and the insights that are derived from analytics systems is one of the most important challenges of data science in software engineering.
In this work, we conduct a series of empirical investigation to better understand the impact of experimental components (i.e., class mislabelling, parameter optimization of classification techniques, and model validation techniques) on the performance and interpretation of software analytics. To accelerate a large amount of compute-intensive experiment, we leverage the High-Performance-Computing (HPC) resources of Centre for Advanced Computing (CAC) from Queen’s University, Canada. Through case studies of systems that span both proprietary and open- source domains, we demonstrate that (1) realistic noise does not impact the precision of software analytics; (2) automated parameter optimization for classification techniques substantially improve the performance and stability of software analytics; and (3) the out-of- sample bootstrap validation technique produces a good balance between bias and variance of performance estimates. Our results lead us to conclude that the experimental components of analytics modelling impact the predictions and associated insights that are derived from software analytics. Empirical investigations on the impact of overlooked experimental components are needed to derive practical guidelines for analytics modelling.
Defect prediction models help software quality assurance teams to effectively allocate their limited resources to the most defect-prone software modules. Model validation techniques, such as k-fold cross-validation, use this historical data to estimate how well a model will perform in the future. However, little is known about how accurate the performance estimates of these model validation techniques tend to be. In this paper, we set out to investigate the bias and variance of model validation techniques in the domain of defect prediction. A preliminary analysis of 101 publicly available defect prediction datasets suggests that 77% of them are highly susceptible to producing unstable results. Hence, selecting an appropriate model validation technique is a critical experimental design choice. Based on an analysis of 256 studies in the defect prediction literature, we select the 12 most commonly adopted model validation techniques for evaluation. Through a case study of data from 18 systems that span both open-source and proprietary domains, we derive the following practical guidelines for future defect prediction studies: (1) the single holdout validation techniques should be avoided; and (2) researchers should use the out-of-sample bootstrap validation technique instead of holdout or the commonly-used cross-validation techniques.
AUTOMATIC GENERATION AND OPTIMIZATION OF TEST DATA USING HARMONY SEARCH ALGOR...csandit
Software testing is the primary phase, which is performed during software development and it is
carried by a sequence of instructions of test inputs followed by expected output. The Harmony
Search (HS) algorithm is based on the improvisation process of music. In comparison to other
algorithms, the HSA has gain popularity and superiority in the field of evolutionary
computation. When musicians compose the harmony through different possible combinations of
the music, at that time the pitches are stored in the harmony memory and the optimization can
be done by adjusting the input pitches and generate the perfect harmony. The test case
generation process is used to identify test cases with resources and also identifies critical
domain requirements. In this paper, the role of Harmony search meta-heuristic search
technique is analyzed in generating random test data and optimized those test data. Test data
are generated and optimized by applying in a case study i.e. a withdrawal task in Bank ATM
through Harmony search. It is observed that this algorithm generates suitable test cases as well
as test data and gives brief details about the Harmony search method. It is used for test data
generation and optimization
Instance Space Analysis for Search Based Software EngineeringAldeida Aleti
Search-Based Software Engineering is now a mature area with numerous techniques developed to tackle some of the most challenging software engineering problems, from requirements to design, testing, fault localisation, and automated program repair. SBSE techniques have shown promising results, giving us hope that one day it will be possible for the tedious and labour intensive parts of software development to be completely automated, or at least semi-automated. In this talk, I will focus on the problem of objective performance evaluation of SBSE techniques. To this end, I will introduce Instance Space Analysis (ISA), which is an approach to identify features of SBSE problems that explain why a particular instance is difficult for an SBSE technique. ISA can be used to examine the diversity and quality of the benchmark datasets used by most researchers, and analyse the strengths and weaknesses of existing SBSE techniques. The instance space is constructed to reveal areas of hard and easy problems, and enables the strengths and weaknesses of the different SBSE techniques to be identified. I will present on how ISA enabled us to identify the strengths and weaknesses of SBSE techniques in two areas: Search-Based Software Testing and Automated Program Repair. Finally, I will end my talk with future directions of the objective assessment of SBSE techniques.
30 February 2005 QUEUE rants [email protected] DARNEDTestin.docxtamicawaysmith
30 February 2005 QUEUE rants: [email protected] DARNEDTesting large systems is a daunting task, but there are steps we can take to ease the pain.
T
he increasing size and complexity of software, coupled with concurrency and dis-
tributed systems, has made apparent the ineffectiveness of using only handcrafted
tests. The misuse of code coverage and avoidance of random testing has exacer-
bated the problem. We must start again, beginning with good design (including
dependency analysis), good static checking (including model property checking), and
good unit testing (including good input selection). Code coverage can help select and
prioritize tests to make you more effi cient, as can the all-pairs technique for controlling
the number of confi gurations. Finally, testers can use models to generate test coverage
and good stochastic tests, and to act as test oracles.
HANDCRAFTED TESTS OUTPACED BY HARDWARE AND SOFTWARE
Hardware advances have followed Moore’s law for years, giving the capability for run-
ning ever-more complex software on the same cost platform. Developers have taken
advantage of this by raising their level of abstraction from assembly to fi rst-generation
compiled languages to managed code such as C# and Java, and rapid application devel-
opment environments such as Visual Basic. Although the number of lines of code per
day per programmer appears to remain relatively fi xed, the power of each line of code
has increased, allowing complex systems to be built. Moore’s law provides double the
computing power every 18 months and software code size tends to double every seven
years, but software testing does not appear to be keeping pace.
QUEUE February 2005 31 more queue: www.acmqueue.comBIGBIGBIGTO TESTTOO DARNED
Quality
AssuranceFO
C
U
S
KEITH STOBIE, MICROSOFT
32 February 2005 QUEUE rants: [email protected]
Unfortunately, the increased power has not, in gen-
eral, made testing easier. While in a few cases the more
powerful hardware means we can do more complete test-
ing, in general the testing problem is getting worse. You
can test square root for all the 32-bit values in a reason-
able time,1,2 but we are now moving to 64-bit machines
with 64-bit values (and even longer for fl oating point).
Assuming a nanosecond per test case, it would take 584
years to test all the values. Sure, you could scale to, say,
1,000 processors, but you would still need six months for
just this one test case.
Two other issues of complexity to consider are: the
number of different execution states software can go
through, and concurrency. User interfaces were originally
very rigid, letting users progress through a single fi xed
set of operations—for example, a single set of hierarchi-
cal menus and prompts. Now, good user design is event
driven with the user in control of the sequence of actions.
Further, many actions can be accomplished in multiple
manners (e.g., closing a window via the menu [File/
Close] ...
A Complexity Based Regression Test Selection StrategyCSEIJJournal
Software is unequivocally the foremost and indispensable entity in this technologically driven world.
Therefore quality assurance, and in particular, software testing is a crucial step in the software
development cycle. This paper presents an effective test selection strategy that uses a Spectrum of
Complexity Metrics (SCM). Our aim in this paper is to increase the efficiency of the testing process by
significantly reducing the number of test cases without having a significant drop in test effectiveness. The
strategy makes use of a comprehensive taxonomy of complexity metrics based on the product level (class,
method, statement) and its characteristics.We use a series of experiments based on three applications with
a significant number of mutants to demonstrate the effectiveness of our selection strategy.For further
evaluation, we compareour approach to boundary value analysis. The results show the capability of our
approach to detect mutants as well as the seeded errors.
Software Quality Analysis Using Mutation Testing SchemeEditor IJMTER
The software test coverage is used measure the safety measures. The safety critical analysis is
carried out for the source code designed in Java language. Testing provides a primary means for
assuring software in safety-critical systems. To demonstrate, particularly to a certification authority, that
sufficient testing has been performed, it is necessary to achieve the test coverage levels recommended or
mandated by safety standards and industry guidelines. Mutation testing provides an alternative or
complementary method of measuring test sufficiency, but has not been widely adopted in the safetycritical industry. The system provides an empirical evaluation of the application of mutation testing to
airborne software systems which have already satisfied the coverage requirements for certification.
The system mutation testing to safety-critical software developed using high-integrity subsets of
C and Ada, identify the most effective mutant types and analyze the root causes of failures in test cases.
Mutation testing could be effective where traditional structural coverage analysis and manual peer
review have failed. They also show that several testing issues have origins beyond the test activity and
this suggests improvements to the requirements definition and coding process. The system also
examines the relationship between program characteristics and mutation survival and considers how
program size can provide a means for targeting test areas most likely to have dormant faults. Industry
feedback is also provided, particularly on how mutation testing can be integrated into a typical
verification life cycle of airborne software. The system also covers the safety and criticality levels of
Java source code.
Software Quality Assurance (SQA) teams play a critical role in the software development process to ensure the absence of software defects. It is not feasible to perform exhaustive SQA tasks (i.e., software testing and code review) on a large software product given the limited SQA resources that are available. Thus, the prioritization of SQA efforts is an essential step in all SQA efforts. Defect prediction models are used to prioritize risky software modules and understand the impact of software metrics on the defect-proneness of software modules. The predictions and insights that are derived from defect prediction models can help software teams allocate their limited SQA resources to the modules that are most likely to be defective and avoid common past pitfalls that are associated with the defective modules of the past. However, the predictions and insights that are derived from defect prediction models may be inaccurate and unreliable if practitioners do not control for the impact of experimental components (e.g., datasets, metrics, and classifiers) on defect prediction models, which could lead to erroneous decision-making in practice. In this thesis, we investigate the impact of experimental components on the performance and interpretation of defect prediction models. More specifically, we investigate the impact of the three often overlooked experimental components (i.e., issue report mislabelling, parameter optimization of classification techniques, and model validation techniques) have on defect prediction models. Through case studies of systems that span both proprietary and open-source domains, we demonstrate that (1) issue report mislabelling does not impact the precision of defect prediction models, suggesting that researchers can rely on the predictions of defect prediction models that were trained using noisy defect datasets; (2) automated parameter optimization for classification techniques substantially improve the performance and stability of defect prediction models, as well as they change their interpretation, suggesting that researchers should no longer shy from applying parameter optimization to their models; and (3) the out-of-sample bootstrap validation technique produces a good balance between bias and variance of performance estimates, suggesting that the single holdout and cross-validation families that are commonly-used nowadays should be avoided.
Testing embedded system through optimal mining technique (OMT) based on multi...IJECEIAES
Testing embedded systems must be done carefully particularly in the significant regions of the embedded systems. Inputs from an embedded system can happen in multiple order and many relationships can exist among the input sequences. Consideration of the sequences and the relationships among the sequences is one of the most important considerations that must be tested to find the expected behavior of the embedded systems. On the other hand combinatorial approaches help determining fewer test cases that are quite enough to test the embedded systems exhaustively. In this paper, an Optimal Mining Technique that considers multi-input domain which is based on built-in combinatorial approaches has been presented. The method exploits multi-input sequences and the relationships that exist among multi-input vectors. The technique has been used for testing an embedded system that monitors and controls the temperature within the Nuclear reactors.
Model based test case prioritization using neural network classificationcseij
Model-based testing for real-life software systems often require a large number of tests, all of which cannot
exhaustively be run due to time and cost constraints. Thus, it is necessary to prioritize the test cases in
accordance with their importance the tester perceives. In this paper, this problem is solved by improving
our given previous study, namely, applying classification approach to the results of our previous study
functional relationship between the test case prioritization group membership and the two attributes:
important index and frequency for all events belonging to given group are established. A for classification
purpose, neural network (NN) that is the most advances is preferred and a data set obtained from our study
for all test cases is classified using multilayer perceptron (MLP) NN. The classification results for
commercial test prioritization application show the high classification accuracies about 96% and the
acceptable test prioritization performances are achieved.
Search-based testing of procedural programs:iterative single-target or multi-...Vrije Universiteit Brussel
In the context of testing of Object-Oriented (OO) software systems, researchers have recently proposed search based approaches to automatically generate whole test suites by considering simultaneously all targets (e.g., branches) defined by the coverage criterion (multi-target approach). The goal of whole suite approaches is to overcome the problem of wasting search budget that iterative single-target approaches (which iteratively generate test cases for each target) can encounter in case of infeasible targets. However, whole suite approaches have not been implemented and experimented in the context of procedural programs. In this paper we present OCELOT (Optimal Coverage sEarch-based tooL for sOftware Testing), a test data generation tool for C programs which implements both a state-of-the-art whole suite approach and an iterative single-target approach designed for a parsimonious use of the search budget. We also present an empirical study conducted on 35 open-source C programs to compare the two approaches implemented in OCELOT. The results indicate that the iterative single-target approach provides a higher efficiency while achieving the same or an even higher level of coverage than the whole suite approach.
Software analytics (for software quality purpose) is a statistical or machine learning classifier that is trained to identify defect-prone software modules. The goal of software analytics is to help software engineers prioritize their software testing effort on the most-risky modules and understand past pitfalls that lead to defective code. While the adoption of software analytics enables software organizations to distil actionable insights, there are still many barriers to broad and successful adoption of such analytics systems. Indeed, even if software organizations can access such invaluable software artifacts and toolkits for data analytics, researchers and practitioners often have little knowledge to properly develop analytics systems. Thus, the accuracy of the predictions and the insights that are derived from analytics systems is one of the most important challenges of data science in software engineering.
In this work, we conduct a series of empirical investigation to better understand the impact of experimental components (i.e., class mislabelling, parameter optimization of classification techniques, and model validation techniques) on the performance and interpretation of software analytics. To accelerate a large amount of compute-intensive experiment, we leverage the High-Performance-Computing (HPC) resources of Centre for Advanced Computing (CAC) from Queen’s University, Canada. Through case studies of systems that span both proprietary and open- source domains, we demonstrate that (1) realistic noise does not impact the precision of software analytics; (2) automated parameter optimization for classification techniques substantially improve the performance and stability of software analytics; and (3) the out-of- sample bootstrap validation technique produces a good balance between bias and variance of performance estimates. Our results lead us to conclude that the experimental components of analytics modelling impact the predictions and associated insights that are derived from software analytics. Empirical investigations on the impact of overlooked experimental components are needed to derive practical guidelines for analytics modelling.
Defect prediction models help software quality assurance teams to effectively allocate their limited resources to the most defect-prone software modules. Model validation techniques, such as k-fold cross-validation, use this historical data to estimate how well a model will perform in the future. However, little is known about how accurate the performance estimates of these model validation techniques tend to be. In this paper, we set out to investigate the bias and variance of model validation techniques in the domain of defect prediction. A preliminary analysis of 101 publicly available defect prediction datasets suggests that 77% of them are highly susceptible to producing unstable results. Hence, selecting an appropriate model validation technique is a critical experimental design choice. Based on an analysis of 256 studies in the defect prediction literature, we select the 12 most commonly adopted model validation techniques for evaluation. Through a case study of data from 18 systems that span both open-source and proprietary domains, we derive the following practical guidelines for future defect prediction studies: (1) the single holdout validation techniques should be avoided; and (2) researchers should use the out-of-sample bootstrap validation technique instead of holdout or the commonly-used cross-validation techniques.
AUTOMATIC GENERATION AND OPTIMIZATION OF TEST DATA USING HARMONY SEARCH ALGOR...csandit
Software testing is the primary phase, which is performed during software development and it is
carried by a sequence of instructions of test inputs followed by expected output. The Harmony
Search (HS) algorithm is based on the improvisation process of music. In comparison to other
algorithms, the HSA has gain popularity and superiority in the field of evolutionary
computation. When musicians compose the harmony through different possible combinations of
the music, at that time the pitches are stored in the harmony memory and the optimization can
be done by adjusting the input pitches and generate the perfect harmony. The test case
generation process is used to identify test cases with resources and also identifies critical
domain requirements. In this paper, the role of Harmony search meta-heuristic search
technique is analyzed in generating random test data and optimized those test data. Test data
are generated and optimized by applying in a case study i.e. a withdrawal task in Bank ATM
through Harmony search. It is observed that this algorithm generates suitable test cases as well
as test data and gives brief details about the Harmony search method. It is used for test data
generation and optimization
Instance Space Analysis for Search Based Software EngineeringAldeida Aleti
Search-Based Software Engineering is now a mature area with numerous techniques developed to tackle some of the most challenging software engineering problems, from requirements to design, testing, fault localisation, and automated program repair. SBSE techniques have shown promising results, giving us hope that one day it will be possible for the tedious and labour intensive parts of software development to be completely automated, or at least semi-automated. In this talk, I will focus on the problem of objective performance evaluation of SBSE techniques. To this end, I will introduce Instance Space Analysis (ISA), which is an approach to identify features of SBSE problems that explain why a particular instance is difficult for an SBSE technique. ISA can be used to examine the diversity and quality of the benchmark datasets used by most researchers, and analyse the strengths and weaknesses of existing SBSE techniques. The instance space is constructed to reveal areas of hard and easy problems, and enables the strengths and weaknesses of the different SBSE techniques to be identified. I will present on how ISA enabled us to identify the strengths and weaknesses of SBSE techniques in two areas: Search-Based Software Testing and Automated Program Repair. Finally, I will end my talk with future directions of the objective assessment of SBSE techniques.
30 February 2005 QUEUE rants [email protected] DARNEDTestin.docxtamicawaysmith
30 February 2005 QUEUE rants: [email protected] DARNEDTesting large systems is a daunting task, but there are steps we can take to ease the pain.
T
he increasing size and complexity of software, coupled with concurrency and dis-
tributed systems, has made apparent the ineffectiveness of using only handcrafted
tests. The misuse of code coverage and avoidance of random testing has exacer-
bated the problem. We must start again, beginning with good design (including
dependency analysis), good static checking (including model property checking), and
good unit testing (including good input selection). Code coverage can help select and
prioritize tests to make you more effi cient, as can the all-pairs technique for controlling
the number of confi gurations. Finally, testers can use models to generate test coverage
and good stochastic tests, and to act as test oracles.
HANDCRAFTED TESTS OUTPACED BY HARDWARE AND SOFTWARE
Hardware advances have followed Moore’s law for years, giving the capability for run-
ning ever-more complex software on the same cost platform. Developers have taken
advantage of this by raising their level of abstraction from assembly to fi rst-generation
compiled languages to managed code such as C# and Java, and rapid application devel-
opment environments such as Visual Basic. Although the number of lines of code per
day per programmer appears to remain relatively fi xed, the power of each line of code
has increased, allowing complex systems to be built. Moore’s law provides double the
computing power every 18 months and software code size tends to double every seven
years, but software testing does not appear to be keeping pace.
QUEUE February 2005 31 more queue: www.acmqueue.comBIGBIGBIGTO TESTTOO DARNED
Quality
AssuranceFO
C
U
S
KEITH STOBIE, MICROSOFT
32 February 2005 QUEUE rants: [email protected]
Unfortunately, the increased power has not, in gen-
eral, made testing easier. While in a few cases the more
powerful hardware means we can do more complete test-
ing, in general the testing problem is getting worse. You
can test square root for all the 32-bit values in a reason-
able time,1,2 but we are now moving to 64-bit machines
with 64-bit values (and even longer for fl oating point).
Assuming a nanosecond per test case, it would take 584
years to test all the values. Sure, you could scale to, say,
1,000 processors, but you would still need six months for
just this one test case.
Two other issues of complexity to consider are: the
number of different execution states software can go
through, and concurrency. User interfaces were originally
very rigid, letting users progress through a single fi xed
set of operations—for example, a single set of hierarchi-
cal menus and prompts. Now, good user design is event
driven with the user in control of the sequence of actions.
Further, many actions can be accomplished in multiple
manners (e.g., closing a window via the menu [File/
Close] ...
A Complexity Based Regression Test Selection StrategyCSEIJJournal
Software is unequivocally the foremost and indispensable entity in this technologically driven world.
Therefore quality assurance, and in particular, software testing is a crucial step in the software
development cycle. This paper presents an effective test selection strategy that uses a Spectrum of
Complexity Metrics (SCM). Our aim in this paper is to increase the efficiency of the testing process by
significantly reducing the number of test cases without having a significant drop in test effectiveness. The
strategy makes use of a comprehensive taxonomy of complexity metrics based on the product level (class,
method, statement) and its characteristics.We use a series of experiments based on three applications with
a significant number of mutants to demonstrate the effectiveness of our selection strategy.For further
evaluation, we compareour approach to boundary value analysis. The results show the capability of our
approach to detect mutants as well as the seeded errors.
An Adaptive Hybrid Technique approach of Test Case PrioritizationINFOGAIN PUBLICATION
Test-Case Prioritization is the method to schedule any execution order of the test with the purpose of maximizing some objects like revealing faults early. In this paper we have proposed the hybrid approach for the purpose of the test case prioritization involving Robust Genetic Algorithm to improve the parameters like APSC and execution time. This technique involves robust approach, parent generation, cross-over and mutation over each test-case and then calculates APSC and execution time.
Configuration Navigation Analysis Model for Regression Test Case Prioritizationijsrd.com
Regression testing has been receiving increasing attention nowadays. Numerous regression testing strategies have been proposed. Most of them take into account various metrics like cost as well as the ability to find faults quickly thereby saving overall testing time. In this paper, a new model called the Configuration Navigation Analysis Model is proposed which tries to consider all stakeholders and various testing aspects while prioritizing regression test cases.
Research Inventy : International Journal of Engineering and Science is published by the group of young academic and industrial researchers with 12 Issues per year. It is an online as well as print version open access journal that provides rapid publication (monthly) of articles in all areas of the subject such as: civil, mechanical, chemical, electronic and computer engineering as well as production and information technology. The Journal welcomes the submission of manuscripts that meet the general criteria of significance and scientific excellence. Papers will be published by rapid process within 20 days after acceptance and peer review process takes only 7 days. All articles published in Research Inventy will be peer-reviewed.
Annotated Bibliography
.
Guidelines: Annotated Bibliography
Purpose: Explore current literature (collection of writing on a specific topic) to increase
knowledge of leadership in nursing practice.
The annotated bibliography assignment will help students prepare to design and present a poster presentation regarding nursing leadership in practice. The focus is building student knowledge of various leadership roles in nursing (current trends). The assignment also promotes student reflection on development of their own leadership skills.
Students will read the summary of the Institute of Medicine (IOM) “The Future of Nursing: Leading Change, Advancing Health” for baseline identification of leadership roles (posted in Blackboard).
Students will then search the literature to identify and select five (5) nurse leaders, who will be the topic of the annotated bibliography summaries (students must use credible sources when searching literature).
Students may also choose to submit 2 of the 5 annotated bibliography summaries on the following topics:
1. Student Nurse Leaders (2)
2. Current Trends in Nursing Leadership (3)
Each of the five annotated bibliography summaries should be no more than one page, typed, and must include the following:
1. The identified leader’s specific roles & responsibilities
2. The identified leader’s accomplishments
3. Barriers and facilitators to leader achievement of goals
4. Knowledge gained from reading content included in the annotated bibliography summary
Annotated Bibliography Grading Rubric
Criteria
Points Possible
Points Earned
Faculty Comments
Provides a clear description of the identified leader’s role (s) and responsibilities (related to nursing)
20
Provides examples of the leader’s
accomplishments (at least 2 examples)
10
Summarizes barriers inhibiting the leader’s achievement of goals
15
Summarizes facilitators enhancing the leader’s achievement of goals
15
Summary of leadership knowledge gained from reading content included in the annotated bibliography summary
20
Correct grammar/spelling
10
APA format
10
Total
100
[Type text]
30 February 2005 QUEUE rants: [email protected] DARNEDTesting large systems is a daunting task, but there are steps we can take to ease the pain.
T
he increasing size and complexity of software, coupled with concurrency and dis-
tributed systems, has made apparent the ineffectiveness of using only handcrafted
tests. The misuse of code coverage and avoidance of random testing has exacer-
bated the problem. We must start again, beginning with good design (including
dependency analysis), good static checking (including model property checking), and
good unit testing (including good input selection). Code coverage can help select and
prioritize tests to make you more effi cient, as can the all-pairs technique for controlling
the number of confi gurations. Finally, testers can use models to generate test coverage
and good stochastic.
Characterization of Open-Source Applications and Test Suites ijseajournal
Software systems that meet the stakeholders needs and expectations is the ultimate objective of the software
provider. Software testing is a critical phase in the software development lifecycle that is used to evaluate
the software. Tests can be written by the testers or the automatic test generators in many different ways and
with different goals. Yet, there is a lack of well-defined guidelines or a methodology to direct the testers to
write tests
We want to understand how tests are written and why they may have been written that way. This work is a characterization study aimed at recognizing the factors that may have influenced the development of the test suite. We found that increasing the coverage of the test suites for applications with at least 500 test
cases can make the test suites more costly. The correlation coeffieicent obtained was 0.543. The study also found that there is a positive correlation between the mutation score and the coverage score.
One of the obstacles that hinder the usage of mutation testing is its impracticality, two main contributors of this are a large number of mutants and a large number of test cases involves in the process. Researcher usually tries to address this problem by optimizing the mutants and the test case separately. In this research, we try to tackle both of optimizing mutant and optimizing test-case simultaneousl y using a coevolution optimization method. The coevolution optimization method is chosen for the mutation testing problem because the method works by optimizing multiple collections (population) of a solution. This research found that coevolution is better suited for multiproblem optimization than other single population methods (i.e. Genetic Algorithm), we also propose new indicator to determine the optimal coevolution cycle. The experiment is done to the artificial case, laboratory, and also a real case.
This publication is to help software engineering students to understand the basis in software testing. Software testing is an inevitable process in software development.
Regression testing concentrates on finding defects after a major code change has occurred. Specifically, it
exposes software regressions or old bugs that have reappeared. It is an expensive testing process that has
been estimated to account for almost half of the cost of software maintenance. To improve the regression
testing process, test case prioritization techniques organizes the execution level of test cases. Further, it
gives an improved rate of fault identification, when test suites cannot run to completion.
A NOVEL APPROACH FOR TEST CASEPRIORITIZATIONIJCSEA Journal
Test case prioritization techniques basically schedule the execution of test cases in a definite order such that to attain an objective function with greater efficiency. This scheduling of test cases improves the results of regression testing. Test case prioritization techniques order the test cases such that the most important ones are executed first encountering the faults first and thus makes the testing effective. In this paper an approach is presented which calculates the product of statement coverage and function calls. The results illustrate the effectiveness of formula computed with the help of APFD metric.
TEST CASE PRIORITIZATION FOR OPTIMIZING A REGRESSION TESTijfcstjournal
Regression testing makes sure that upgradation of software in terms of adding new features or for bug
fixing purposes should not hamper previously working functionalities. Whenever a software is upgraded or
modified, a set of test cases are run on each of its functions to assure that the change to that function is not
affecting other parts of the software that were previously running flawlessly. For achieving this, all existing
test cases need to run as well as new test cases might be required to be created. It is not feasible to reexecute
every test case for all the functions of a given software, because if there is a large number of test
cases to be run, then a lot of time and effort would be required. This problem can be addressed by
prioritizing test cases. Test case prioritization technique reorders the priority in which test cases are
implemented, in an attempt to ensure that maximum faults are uncovered early on by the high priority test
cases implemented first. In this paper we propose an optimized test case prioritization technique using Ant
Colony Optimization (ACO) to reduce the cost, effort and time taken to perform regression testing and also
uncover maximum faults. Comparison of different techniques such as Retest All, Test Case Minimization,
Test Case Prioritization, Random Test Case Selection and Test Case Prioritization using ACO is also
depicted.
TEST CASE PRIORITIZATION FOR OPTIMIZING A REGRESSION TESTijfcstjournal
Regression testing makes sure that upgradation of software in terms of adding new features or for bug fixing purposes should not hamper previously working functionalities. Whenever a software is upgraded or modified, a set of test cases are run on each of its functions to assure that the change to that function is not affecting other parts of the software that were previously running flawlessly. For achieving this, all existing test cases need to run as well as new test cases might be required to be created. It is not feasible to re- execute every test case for all the functions of a given software, because if there is a large number of test cases to be run, then a lot of time and effort would be required. This problem can be addressed by prioritizing test cases. Test case prioritization technique reorders the priority in which test cases are implemented, in an attempt to ensure that maximum faults are uncovered early on by the high priority test cases implemented first. In this paper we propose an optimized test case prioritization technique using Ant Colony Optimization (ACO) to reduce the cost, effort and time taken to perform regression testing and also uncover maximum faults. Comparison of different techniques such as Retest All, Test Case Minimization, Test Case Prioritization, Random Test Case Selection and Test Case Prioritization using ACO is also depicted.
TEST CASE PRIORITIZATION FOR OPTIMIZING A REGRESSION TESTijfcstjournal
Regression testing makes sure that upgradation of software in terms of adding new features or for bug
fixing purposes should not hamper previously working functionalities. Whenever a software is upgraded or
modified, a set of test cases are run on each of its functions to assure that the change to that function is not
affecting other parts of the software that were previously running flawlessly. For achieving this, all existing
test cases need to run as well as new test cases might be required to be created. It is not feasible to reexecute every test case for all the functions of a given software, because if there is a large number of test
cases to be run, then a lot of time and effort would be required. This problem can be addressed by
prioritizing test cases. Test case prioritization technique reorders the priority in which test cases are
implemented, in an attempt to ensure that maximum faults are uncovered early on by the high priority test
cases implemented first. In this paper we propose an optimized test case prioritization technique using Ant
Colony Optimization (ACO) to reduce the cost, effort and time taken to perform regression testing and also
uncover maximum faults. Comparison of different techniques such as Retest All, Test Case Minimization,
Test Case Prioritization, Random Test Case Selection and Test Case Prioritization using ACO is also
depicted.
TEST CASE PRIORITIZATION FOR OPTIMIZING A REGRESSION TESTijfcstjournal
Regression testing makes sure that upgradation of software in terms of adding new features or for bug
fixing purposes should not hamper previously working functionalities. Whenever a software is upgraded or
modified, a set of test cases are run on each of its functions to assure that the change to that function is not
affecting other parts of the software that were previously running flawlessly. For achieving this, all existing
test cases need to run as well as new test cases might be required to be created. It is not feasible to reexecute
every test case for all the functions of a given software, because if there is a large number of test
cases to be run, then a lot of time and effort would be required. This problem can be addressed by
prioritizing test cases. Test case prioritization technique reorders the priority in which test cases are
implemented, in an attempt to ensure that maximum faults are uncovered early on by the high priority test
cases implemented first. In this paper we propose an optimized test case prioritization technique using Ant
Colony Optimization (ACO) to reduce the cost, effort and time taken to perform regression testing and also
uncover maximum faults. Comparison of different techniques such as Retest All, Test Case Minimization,
Test Case Prioritization, Random Test Case Selection and Test Case Prioritization using ACO is also
depicted.
Explore our comprehensive data analysis project presentation on predicting product ad campaign performance. Learn how data-driven insights can optimize your marketing strategies and enhance campaign effectiveness. Perfect for professionals and students looking to understand the power of data analysis in advertising. for more details visit: https://bostoninstituteofanalytics.org/data-science-and-artificial-intelligence/
StarCompliance is a leading firm specializing in the recovery of stolen cryptocurrency. Our comprehensive services are designed to assist individuals and organizations in navigating the complex process of fraud reporting, investigation, and fund recovery. We combine cutting-edge technology with expert legal support to provide a robust solution for victims of crypto theft.
Our Services Include:
Reporting to Tracking Authorities:
We immediately notify all relevant centralized exchanges (CEX), decentralized exchanges (DEX), and wallet providers about the stolen cryptocurrency. This ensures that the stolen assets are flagged as scam transactions, making it impossible for the thief to use them.
Assistance with Filing Police Reports:
We guide you through the process of filing a valid police report. Our support team provides detailed instructions on which police department to contact and helps you complete the necessary paperwork within the critical 72-hour window.
Launching the Refund Process:
Our team of experienced lawyers can initiate lawsuits on your behalf and represent you in various jurisdictions around the world. They work diligently to recover your stolen funds and ensure that justice is served.
At StarCompliance, we understand the urgency and stress involved in dealing with cryptocurrency theft. Our dedicated team works quickly and efficiently to provide you with the support and expertise needed to recover your assets. Trust us to be your partner in navigating the complexities of the crypto world and safeguarding your investments.
As Europe's leading economic powerhouse and the fourth-largest hashtag#economy globally, Germany stands at the forefront of innovation and industrial might. Renowned for its precision engineering and high-tech sectors, Germany's economic structure is heavily supported by a robust service industry, accounting for approximately 68% of its GDP. This economic clout and strategic geopolitical stance position Germany as a focal point in the global cyber threat landscape.
In the face of escalating global tensions, particularly those emanating from geopolitical disputes with nations like hashtag#Russia and hashtag#China, hashtag#Germany has witnessed a significant uptick in targeted cyber operations. Our analysis indicates a marked increase in hashtag#cyberattack sophistication aimed at critical infrastructure and key industrial sectors. These attacks range from ransomware campaigns to hashtag#AdvancedPersistentThreats (hashtag#APTs), threatening national security and business integrity.
🔑 Key findings include:
🔍 Increased frequency and complexity of cyber threats.
🔍 Escalation of state-sponsored and criminally motivated cyber operations.
🔍 Active dark web exchanges of malicious tools and tactics.
Our comprehensive report delves into these challenges, using a blend of open-source and proprietary data collection techniques. By monitoring activity on critical networks and analyzing attack patterns, our team provides a detailed overview of the threats facing German entities.
This report aims to equip stakeholders across public and private sectors with the knowledge to enhance their defensive strategies, reduce exposure to cyber risks, and reinforce Germany's resilience against cyber threats.
Show drafts
volume_up
Empowering the Data Analytics Ecosystem: A Laser Focus on Value
The data analytics ecosystem thrives when every component functions at its peak, unlocking the true potential of data. Here's a laser focus on key areas for an empowered ecosystem:
1. Democratize Access, Not Data:
Granular Access Controls: Provide users with self-service tools tailored to their specific needs, preventing data overload and misuse.
Data Catalogs: Implement robust data catalogs for easy discovery and understanding of available data sources.
2. Foster Collaboration with Clear Roles:
Data Mesh Architecture: Break down data silos by creating a distributed data ownership model with clear ownership and responsibilities.
Collaborative Workspaces: Utilize interactive platforms where data scientists, analysts, and domain experts can work seamlessly together.
3. Leverage Advanced Analytics Strategically:
AI-powered Automation: Automate repetitive tasks like data cleaning and feature engineering, freeing up data talent for higher-level analysis.
Right-Tool Selection: Strategically choose the most effective advanced analytics techniques (e.g., AI, ML) based on specific business problems.
4. Prioritize Data Quality with Automation:
Automated Data Validation: Implement automated data quality checks to identify and rectify errors at the source, minimizing downstream issues.
Data Lineage Tracking: Track the flow of data throughout the ecosystem, ensuring transparency and facilitating root cause analysis for errors.
5. Cultivate a Data-Driven Mindset:
Metrics-Driven Performance Management: Align KPIs and performance metrics with data-driven insights to ensure actionable decision making.
Data Storytelling Workshops: Equip stakeholders with the skills to translate complex data findings into compelling narratives that drive action.
Benefits of a Precise Ecosystem:
Sharpened Focus: Precise access and clear roles ensure everyone works with the most relevant data, maximizing efficiency.
Actionable Insights: Strategic analytics and automated quality checks lead to more reliable and actionable data insights.
Continuous Improvement: Data-driven performance management fosters a culture of learning and continuous improvement.
Sustainable Growth: Empowered by data, organizations can make informed decisions to drive sustainable growth and innovation.
By focusing on these precise actions, organizations can create an empowered data analytics ecosystem that delivers real value by driving data-driven decisions and maximizing the return on their data investment.
Techniques to optimize the pagerank algorithm usually fall in two categories. One is to try reducing the work per iteration, and the other is to try reducing the number of iterations. These goals are often at odds with one another. Skipping computation on vertices which have already converged has the potential to save iteration time. Skipping in-identical vertices, with the same in-links, helps reduce duplicate computations and thus could help reduce iteration time. Road networks often have chains which can be short-circuited before pagerank computation to improve performance. Final ranks of chain nodes can be easily calculated. This could reduce both the iteration time, and the number of iterations. If a graph has no dangling nodes, pagerank of each strongly connected component can be computed in topological order. This could help reduce the iteration time, no. of iterations, and also enable multi-iteration concurrency in pagerank computation. The combination of all of the above methods is the STICD algorithm. [sticd] For dynamic graphs, unchanged components whose ranks are unaffected can be skipped altogether.
Chatty Kathy - UNC Bootcamp Final Project Presentation - Final Version - 5.23...John Andrews
SlideShare Description for "Chatty Kathy - UNC Bootcamp Final Project Presentation"
Title: Chatty Kathy: Enhancing Physical Activity Among Older Adults
Description:
Discover how Chatty Kathy, an innovative project developed at the UNC Bootcamp, aims to tackle the challenge of low physical activity among older adults. Our AI-driven solution uses peer interaction to boost and sustain exercise levels, significantly improving health outcomes. This presentation covers our problem statement, the rationale behind Chatty Kathy, synthetic data and persona creation, model performance metrics, a visual demonstration of the project, and potential future developments. Join us for an insightful Q&A session to explore the potential of this groundbreaking project.
Project Team: Jay Requarth, Jana Avery, John Andrews, Dr. Dick Davis II, Nee Buntoum, Nam Yeongjin & Mat Nicholas
A PARTICLE SWARM OPTIMIZATION TECHNIQUE FOR GENERATING PAIRWISE TEST CASES
1. 1
CHAPTER 1
INTRODUCTION
1.1 INTRODUCTION
Software testing and debugging is an essential part in software development process but
this process is confirmed to be intensive and expensive. In research and industry
communities, researchers and practitioners focus on finding automatic cost-effective
software testing and debugging techniques while still maintaining high fault detection and
localization ability to ensure releasing high-quality software productions. Nowadays
software testing research mainly concerns such issues as test coverage criterion design,
test generation problem, test oracle problem, regression testing problem and fault
localization problem. Among these issues, test generation problem is deemed to be an
important issue in software testing research.
In numerous test generation techniques, combinatorial testing is considered to be an
important test input generation technique and it is an important complement to other black
box testing techniques such as boundary value analysis and equivalence class partition.
Combinatorial testing generates test suites according to artifacts under test. These artifacts
come from either functional requirements or detailed development specifications. Given
an input described by factors and levels after analyzing a specific artifact, to make an
exhaustive test, we should generate a set of tests to cover every possible combination of all
those factors. However the size of generated tests may be too large and the required test
cost is prohibitive. To overcome this shortcoming, Cohen et al. firstly introduced the
concept of combinatorial testing and proposed a greedy algorithm AETG. This testing
technique can extremely reduce the number of tests by only concerning combinations of a
few factors and they demonstrated that the size of the optimal test suite grows at most
logarithmically with the number of factors. When we only want to cover all the
combinations of arbitrary two factors, we call it pairwise testing. However, the reduction
of tests does not seriously weaken the effectiveness of fault detection ability.
Empirical studies found that a significant number of faults are triggered by the
combinations of a small number of factors in a variety of application domains. For
example, Kuhn et al. analyzed error reports of a large distributed system after applying
combinatorial testing and found that more than 70% bugs were found by pairwise testing.
To find all the faults, they only need 6-way combinatorial testing at most. Nowadays, most
2. 2
of research works in combinatorial testing aims to generate combinatorial test suites with
minimum size. However, the difficulty of solving this problem is demonstrated to be NP-
hard. By now many approaches have been proposed and many tools have been developed
to find optimal or near optimal combinatorial test suites in polynomial time. Existing
approaches and tools can be briefly divided into three categories: algebraic construction,
greedy algorithm and meta-heuristic search. The brief description of these approaches can
be found in section V.
1.2 MOTIVATION
Pair-wise testing is a specification based testing criterion, which requires that for each pair
number of input parameters of a system, every combination of valid values of these
parameters be covered by at least one test case. Empirical results show that pair-wise
testing is practical and effective for various types of software systems. By seeing the graph
we have found that there are 90% of errors can be covered by applying pair-wise testing.
Hence pair-wise testing can be used in web applications.
Fig: 1 Percentage of failures trigger by t-way interactions
1.3 AIM AND OBJECTIVE
Combinatorial interaction testing is an existing technique that appropriately reduces the
number of test cases by choosing pairs, triplets, etc., i.e. pair, of input values. Of course,
the effectiveness of a test suite is higher when choosing e.g. triplets of inputs rather than
pairs. Since high values of pairs are preferable, a large number of test cases could still be
generated. This approach is motivated by the observation that in many applications a
significant number of faults are caused by interactions of a smaller number of parameters.
The main aim of our project is to generate the optimized pair wise test cases test suite for
testing WEB based applications using PSO technique.
3. 3
1.4 SUMMARY OF THE PROJECT
The next chapter in this project (Chapter-2) is about survey of the Literature. In Chapter 3
we present the definition of the Problem and various feasibility analysis studies regarding
this project.Chapter-4 of this project precisely says about software Requirements
Specification (SRS) needed for developing the project. In Chapter-5 we mention various
design issues and different UML diagrams required to develop the design of the proposing
system. In Chapter-6 we mention the implementation issues of this project. Last chapter
in this report is briefly about the references, different textbooks and websites we referred
for this project.
4. 4
CHAPTER 2
SURVEY OF THE LITERATURE
2.1 INTRODUCTION
Lack of testing often leads to disastrous consequences including loss of data, fortunes and
Even lives. For these reasons, many input parameters and system conditions need to be
tested against the specifications of the system for conformance. Although desirable,
exhaustive testing is prohibitively expensive even in a moderate-sized project, due to
resources as well as timing constraints. Therefore, it is necessary to reduce the test
selection space in a systematic manner. In line with increasing consumer demands for new
functionalities and innovations, software applications grew tremendously in size over the
last 15 years. This sudden increase has a profound impact as far as testing is concerned.
Here, the test size grew significantly as a result. To address the aforementioned issues,
much research is now focusing on sampling techniques based on interaction testing
(termed t-way testing strategy) in order to derive the most optimum test suite for testing
consideration (i.e., termed as Covering Array (CA)for uniform parameter values and
Mixed Covering Array(MCA) for non-uniform parameter values respectively).
Two-way testing (also termed pair wise) appears to be adequate for achieving good
test coverage in some existing system, a counter argument suggests that such a conclusion
cannot be generalized to all (future) software system. Often, the net effect of software
growth introduces new intertwined dependency between parameters involved, thus,
justifying the need to support for high interaction strength (t).One reduction approach is
via pair wise testing. Pair wise testing helps detect faults caused by interactions between
two parameters. Indeed, earlier work demonstrates that pair wise testing achieves higher
block and decision coverage than traditional methods for a commercial email system.
While such a conclusion can be true for some system, a counter argument suggests that
some faults may also be caused by the interaction of more than two parameters (i.e. often
termed as t-way testing). For example, by applying pair-way testing to a telephone
software system demonstrates that several faults can only be detected under certain
combinations of input parameters.
A study conducted by The National Institute of Standards and Technology (NIST)
has shown that 95% of actual faults are caused by 4-wayinteractions in some system. In
5. 5
fact, only after considering up to 6-way interactions can all the faults be found. Given that
software applications grew tremendously in the last 15years, there is clearly a need to
consider the support for high interaction strength, that is, to cater for the possibility of new
intertwined dependencies between involved parameters. Considering more than two
parameter interactions is not without difficulties. When the number of parameter coverage
increases, the size of pair-way test sets also increases exponentially. As such, for a large
system, considering a higher order t-way test set can lead to a combinatorial explosion
problem. Here, computational efforts required in search of an optimum test set, termed as
Covering Array (CA)(in which the parameter values are uniform) and Mixed Covering
Array (MCA) (in which the parameter values are non-uniform), can be expensive
especially when each interaction is to be covered optimally by the minimum number of
test cases for a given interaction strength (t).Addressing the aforementioned issues, this
paper proposes a new strategy, called Modified IPOG for pair-way testing.
Some of pair-wise test techniques are available to generate the pair-wise test cases like,
ACO (Ant Colony Optimization), BCO (Bee colony Optimization), and Simulated
Annealing etc. For a system with two or more input parameters, the GA strategy generates
a pair-wise test set for the first pair parameters, extends the test and continues to do so for
each additional parameter.
2.2 LITERATURE SURVEY
2.2.1 Using Organizational Evolutionary Particle Swarm Techniques to Generate
TestCases for Combinatorial Testing
Based on the analysis of the characteristics of combinatorial testing, an organizational
evolutionary particle swarm algorithm (OEPST) to generate test cases for combinatorial
testing is proposed. This algorithm is used to select the test cases of local optimal coverage
in current environment based on these test cases, and then a test suite satisfying the pair-
wise coverage criterion is built. The empirical results show that their approach can
effectively reduce the number of test case.
2.2.2 An Improved Algorithm for Test Data Generation Based on Particle Swarm
Optimization
The test case generation is one of key issues of combinatorial testing. In this paper, a new
algorithm for test data generation based on Particle Swarm Optimization PSO is presented.
Based on Particle Swarm Optimization, the optimization base and extended parameters are
6. 6
introduced. The number of the current output test data is adjusted dynamically according
to the data generated before. The efficiency of the test data generation is improved
effectively on the premise of ensuring the optimization of the data generated.
2.2.3 PSTG: A T-Way Strategy Adopting Particle Swarm Optimization
As an activity to ensure quality and conformance, testing is one of the most important
activities in any software or hardware product development cycle. Often, the challenge in
testing is that the system may support a wide range of configurations. Ideally, it is
desirable to test all of these configurations exhaustively. However, exhaustive testing is
practically impossible due to time and resource limitations. To address this issue, there is a
need for a sampling strategy that can select a subset of inputs as test data from an
inherently large search space. Recent findings demonstrate that t-way interaction testing
strategies based on artificial intelligence (i.e. where t indicates interaction strength) have
been successful to obtain a near optimal solution resulting into smaller test set to be
considered. Motivated by such findings, we have developed a new test generation strategy,
called Particle Swarm Test Generator (PSTG). In this paper, we discuss the design of
PSTG and demonstrate our preliminary test size reduction results against other competing
t-way strategies including IPOG, WHITCH, Jenny, TConfig, and TVG.
2.2.4 T-Way Test Data Generation Strategy Based on Particle Swarm
Optimization
Due to market demands, software has grown tremendously in size and functionalities over
the years. As side effects of such growth, there tend to be more and more unwanted
interaction between software and system parameters. These unwanted interactions can
sometimes lead to nasty and difficult bugs to detect. In order to address these issues, t-way
strategies (i.e. where t indicates interaction strength) are helpful to generate a set of test
cases (i.e. to form a complete suite) that cover the required interaction strength as least
once from a typically large space of possible test values. In this paper, we highlight a new
t-way strategy based on Particle Swarm Optimization, called PSTG. Preliminary results
demonstrated that PSTG compares well against other existing t-way strategies.
7. 7
2.3 EXISTING SYSTEM
The research area of search based software engineering (SBSE) was firstly proposed by
arman and Jones . They suggest that many difficult software engineering problems, which
can be abstracted as combinatorial optimization problems, can be successfully solved by
meta-heuristic search techniques. Common adopted techniques include hill climbing,
simulated annealing and genetic algorithm. In this paper, we want to use particle swarm
optimization (PSO), a global search algorithm, to build pair wise test suites. PSO is a
relative new swarm-based meta-heuristic search technique and is inspired by social
behavior of bird flocking, animals herding or fishes schooling where these swarms search
for food in a cooperative way. It was firstly formalized into a meta-heuristic search
technique by Eberhart and Kennedy in 1995 . PSO can be easily implemented and have a
few design parameters to set compared to other swarm-based metaheuristic search
techniques, such as genetic algorithm and ant colony optimization.
In the rest of this section, we give some definitions of PSO and briefly describe its
evolution process. Definition 7 (Particle): Each individual in the swarm is referred to as a
particle. The motion of particles is based on the following principal: They accelerate
toward the best individual location and the best global location while checking their
current location. Definition 8 (The Position of the Particle): The search space can be
modeled as an N-dimensional space. A position of the particle (i.e. a solution of the
problem) in iteration t can be expressed as a coordinate. Definition 9 (The Fitness
Function of the Particle): The setting of the fitness function is an important part in all 109
meta-heuristic search techniques. Here, this function is used to evaluate the goodness of a
candidate solution generated by a particle. The fitness function takes the position as an
input and outputs a fitness value representing the goodness of this candidate solution .The
main evolution process of PSO is described as follows: The system represented by a
swarm is initialized first and then the swarm is evolved to find an optimal solution. In the
initialization phase, the position and the velocity of all the particles in the swarm are
generated in random. In the iteration process, particle i keeps record of the best individual
location of the particle (i.e. pbesti) and the best global location of the swarm (i.e. gbest).
To evolve to the next generation, the particle changes its velocity toward its pbest and
gbest location and then changes its position according to current position and velocity.
8. 8
2.4 PROPOSED SYSTEM
In this paper, we propose two different algorithms to systematically generate final pairwise
test suites. One algorithm is based on one-test-at-a-time strategy, the other is based on
IPO-like strategy. In both algorithms, we use PSO to complete the construction of a single
test (i.e. choose proper levels for the unfixed factors) aiming to cover more new
combinations. In this section, we first show how to use PSO to guide one test construction
process. Then we will introduce our two different algorithms.
A. Using PSO to Complete the Construction of a Single Test
In this subsection, we will show how to use PSO to complete the construction of a single
test. During the process of our two different algorithms, they often generate some tests. In
these tests, some factors are fixed to specific levels, while the other factors remain free to
take any possible valid level. Here we use PSO to choose appropriate valid level for these
unfixed factors with the aim of covering more new combinations.
9. 9
B. Two PSO Based Approaches
Based on Algorithm 1 used to complete the construction of a single test, in this paper we
propose two different algorithms to guide the systematically generation process of pair
wise test suites. One algorithm is based on one-test-at-a-time strategy. The other algorithm
is based on IPO-like strategy. 1) Strategy A, One-test-at-a-time strategy: One-test-at-atime
strategy was firstly adopted by AETG approach and was further used by Bryce et al..
Using this strategy, it firstly generates an empty test suite TS, then generate a test t
according to some strategies, remove the combinations covered by t and add t to the test
suite TS. When all the combinations are covered, it terminates the loop and return
combinatorial test suite TS. We modified this strategy in the single test generation phase.
Here we randomly choose a pairwise combination from uncovered combination set Q and
fix the corresponding factors according to the chosen combination (in Lines 4-6). Then we
10. 10
will use Algorithm 1 to choose proper levels for other unfixed factors. The pseudo code is
shown in the Algorithm 2.
2) Strategy B, IPO-like strategy: IPO is firstly proposed by Tai and Yu [21]. Unlike one-
test-at-a-time strategy, this strategy includes horizontal growth phase and vertical growth
phase. But our approach has some difference compared to IPO. The pseudo-code is shown
in Algorithm 3. First, we sort the factors in non-increasing order according to their level
size. Then we suppose that after the sorting, the first factor has l_ 1 levels and the second
factor has l_ 2 levels. We will generate l_ 1 × l_ 2 tests for the first two factors and then
immediately extend the test suite to all the factors. In these tests, other value of (k − 2) free
factors will be determined by PSO one by one (in Lines 4-7). This phase is familiar with
horizontal growth phase in IPO to a certain extent. Lastly, we will append some tests to
cover all the remaining combinations (in Lines 8-13). This phase is familiar with vertical
growth phase in IPO.
2.5 FEASIBILITY ANALYSIS
Feasibility steady ensures that whether to proceed with the development of the project or
to stop by conducting study on primary areas such as economy, technical and operational
environments. This analysis confirms the feasibility of achieving the product or the
system. It is necessary and prudent to evaluate the feasibility of a project at the earliest
possible time. It can be measured in different scales. The following feasibility studies were
performed to gauge the feasibility of the system.
Operational Feasibility
Technical Feasibility
Economic Feasibility
11. 11
Operational Feasibility
Operational feasibility checks the operational scope of the system. The system under
consideration should have enough operational reach. It is observed that the design of
auditing system is flexible and ease to develop in terms of language dependent and
language independent components reduces the work of the designer, as only resource
sharing utilization should be recorded. Also, since we need dynamic feedback mechanism
on timely manner to achieve required goal of auditing in easy way and displays all the
information of shared resources. Hence the operational feasibility of the proposed system
is found to be high.
Technical Feasibility
Technical feasibility checks the technical possibilities of the system to be developed and
includes a study of function, performance and constraints to achieve the goals of the
system.
1. Hardware Resources
The proposed system needs to be developed over the Windows platform, the machines
required are Pentium IV class or compatible processor and a minimum of 512MB RAM.
2. Software Resources
The software required are Windows XP, Borland or Turbo C compiler,Umbrello.
3. Books
A good number of books and reference manuals are required to develop the system. Most
of the necessary books are available in the Library and the necessary reference manuals
are found on INTENET also. As the developers have made provisions for hardware and
software resources, the proposed system is technically feasible.
Economical Feasibility
An evaluation of development cost weighed against the ultimate income or benefit derived
from the development of the proposed system is made. It should be seen to that the cost of
production does not exceed the income from the system. As the developers have made
enough provisions for the required resources, the cost incurred is minimal. As Windows
XP and Borland C editors are easily available and at the feasible cost, this project will
suitable as per the project cost. Hence, the entire system is feasible.
12. 12
CHAPTER 3
DESIGN ISSUES
3.1 SYSTEM DESIGN
Software design is an interactive process through which requirements are translated into a
blue print for constructing software. Design has been described as a multi-step process in
13. 13
which representations of data structure program structure interface characteristics and
procedure details are synthesized from information requirements. The objective is to
provide the systematic approach for the derivation of the design. The Blueprint from
which software is to be constructed.
Design is a phase where the requirements are actually translated into a finished
software product or system. Preliminary design is concerned with the transformation of
requirements into data and software architecture detailed design focuses on refinements to
the architectural implementation that lead to detailed data structure and algorithmic
representation for software.
Fig: 3.1.1 System Design
3.1.1 Description about Modules
The system after careful analysis has been identified to be presented with the following
modules:
1. Input Module:
In this module it takes the inputs as number of parameters and the number of values for
those parameters for generating the test cases.
2. Internal Module: In this module, a particle swarm optimization technique is being
implemented for generating the optimized pair wise tests cases. Initially this module, the
14. 14
PSO takes the number of parameters and their values takes as input and sends the
optimized result to the output module.
3. Output Module:
The Output Module displays the optimal pair wise test suite as output.
3.2 DETAILED DESIGN OF THE PROJECT
Unified Modeling Language (UML) is a standard language for writing software blue
prints. The language which provides a vocabulary and the rules for combining words for
the purpose of communication. A modeling language is a language whose vocabulary
and rules focus on the conceptual and physical representation of a system. A modeling
language such as the UML is thus a standard language for software blueprints. Modeling
is a central part of all the activities that lead up to the deployment of good software. We
build models to communicate the desired structure and behavior of our system.
UML is a general purpose visual modeling language that is used to specify, visualize,
construct, and document the artifacts of the software system. UML will provide
vocabulary and rules for communications and functions on conceptual and physical
representation. So it is called as modeling language. The UML is applicable to object-
orient problem solving. A model is an abstraction of the underlying problem. The
domain is the actual world from which the problem comes. Models consist of Object that
interacts by sending each other message. Think of an object as “alive”. Objects have
things they know (attributes) and things they can do (behaviors or operations). The
values of an object’s attributes determine its state. Classes are the “blueprint” for objects
UML Diagrams
A diagram is the graphical presentation of a set of elements, most often rendered as a
connected graph of vertices (things) and arcs (relationships).There are two types of
diagrams.
They are:
1. Structural Diagrams
2. Behavioral Diagrams
Structural Diagrams
15. 15
The UML‘s four structural diagrams exist to visualize, specify, construct and document
the static aspects of a system. We can View the static parts of a system using one of the
following diagrams.
1. Component diagram
Behavioral Diagrams
The UML’s five behavioral diagrams are used to visualize, specify, construct, and
document the dynamic aspects of a system. The UML’s behavioral diagrams are roughly
organized around the major ways which can model the dynamics of a system.
1. Use case diagram
2. Sequence diagram
3. Activity diagram
Use case Diagram
A use case diagram shows a set of use cases and actors and their relationships. Use case
diagrams address the static use case view of a system. These diagrams are especially
important in organizing and modeling the behaviors of a system.
5.2.1.2.1 Use case diagram
Activity Diagram:
An Activity Diagram shows the flow from activity to activity. The activity diagram
emphasizes the dynamic view of a system. Activity Diagram commonly contains,
• Activity States and Action States
• Transitions
16. 16
• Objects
Activity diagram
Sequence Diagram
The sequence diagram is an interaction diagram that emphasizes the time ordering of
messages. Graphically, a sequence diagram is a table that shows objects arranged along
the X axis and messages, ordered in increasing time, along the Y axis.
Common Uses:
We use sequence diagram to illustrate the dynamic view of a system.
Sequence diagram show explicit sequence of messages.
To model the message sequence.
17. 17
Sequence diagram
Component Diagram
A component diagram is a diagram that shows relationships among software building
blocks. A component diagram shows the organizations and dependencies among a set of
components.
The component diagram emphasizes the static implementation view of a system.
Component Diagram consists of components, interfaces & dependency relationship.
Component:A component is a physical and replaceable part of a system that
conforms to and provides the realization of a set of interfaces.
Interface:An interface is a collection of operations that are used to specify a
service of a class or component.
Component diagram
18. 18
CHAPTER 4
IMPLEMENTATION ISSUES
4.1 INTRODUCTION
Implementation phase is an important phase in the software development process. After
having a clear idea on the elements that comprised the final model; the next step was to
take the design from a conceptual idea into a practical model. The goal of the phase is to
translate the design of the system produced during the design phase into source code in a
given programming language. A well-written code can then reduce the testing and
maintenance effort.
4.2 REQUIREMENTS
The system requirements analysis aims at studying the available resources and estimating
how to use each of them. As far as computer projects are concerned two types of
requirements are essential.
4.2.1 Hardware Requirements
Minimum requirements of hardware include any processor with core2duo, 2 GB RAM, 80
GB hard disk space.
4.2.2 Software Requirements
Software requirements are Operating system: have windows XP/7Language: Language:
C.
4.3 Testing
INTRODUCTION
Software testing is a critical element of software quality assurance and represents the
ultimate review of specification, design and coding. In fact, testing is the one step in the
software engineering process that could be viewed as destructive rather than constructive.
A strategy for software testing integrates software test case design methods into a well-
planned series of steps that result in the successful construction of software. Testing is the
set of activities that can be planned in advance and conducted systematically. The
underlying motivation of program testing is to affirm software quality with methods that
can economically and effectively apply to both strategic to both large and small-scale
systems.
19. 19
Psychology of Testing
The aim of the testing is often to demonstrate that a program works by showing that it has
no errors. This is the opposite of what testing should be viewed as. The basic purpose of
testing phase is to detect the errors that may be present in the program. Hence, one should
not start testing with the intent of showing that a program works; but the intent should be
show that a program does not work. With this in mind we define testing as follows.
Testing is the process of executing a program with the intent of finding errors.
TESTING ISSUES
Testing Strategies
As quality assurance is the review of software products and related documentation for
completeness, correctness, reliability and maintainability. Quality assurance can be done by
Testing
Verification and Validation
This developed software is allowed to undergo testing in different strategies. The software,
which has been developed, has to be tested to prove its validity. Testing is considered
the least creative phase of the whole cycle of system design.
20. 20
Testing Methods
Black box Testing
It is the testing process in which tester can perform testing on an application without
having any internal structural knowledge of application. Usually Test Engineers are
involved in the black box testing.
White box Testing
It is the testing process in which tester can perform testing on an application with having
internal structural knowledge. Usually the developers are involved in white box testing.
Gray Box Testing
It is the process in which the combination of black box and white box techniques are used.
Types of Testing
This project was tested along the following guide lines to prove its validity. It was tested
using the following techniques of software testing.
White Box Testing
21. 21
By using this technique, it was tested that all the individual logical paths
were executed at least once. All the logical decisions were tested on both their true and false
sides. All the loops were tested with data in between the ranges and especially at
the boundary values.
Black Box Testing
By the use of this technique the missing functions were identified and placed their
positions. The errors in the interfaces were identified and corrected. This technique was
also used to identify the initialization and termination.
Unit Testing
Unit testing focus verifications on the smallest unit of software designs the module. Using
the detailed description as a guide, important control paths are tested uncover errors within
the boundary of the module. The relative complexity of tests and the errors detected as
result is limited by the constrained scope established for unit testing. The unit test is
always white box oriented, and the step can be deducted in parallel for multiple
modules. In the lines of this strategy, all the individual function and modules were put to
test independently.
Verification and Validation Testing
There are two basic approaches to software testing: verification and validation.
Verification refers to the set of activities that ensure that software correctly implements
a specific function. Validation refers to a different set of activities that ensure that the
software that has been built is to customer requirements. Validation begins as soon as
the project starts, but verification can begin only after a specification has been accepted
.verification and validation are independent of each other. It is possible to have a
product that corresponds to the specification, but if the specification proves to be
incorrect, one could not have the right product.
Verification:" are we building the product right?"
Validation:" are we building right product?"
Although testing plays an extremely important role in verification and validation,
many other activities are necessary.
Test data and test output
22. 22
Taking various types of data we do the above testing. Preparation of test data plays
a vital role in system testing. After preparing the test the system under study is
tested using the test data. While testing the system by using the test data, errors
are again uncovered and corrected by using the above testing; errors are again
uncovered by using the above testing and correction methods.
DEBUGGING
The faults can be found by starting from an unplanned failure. The developer moves the
system through a succession of states, ultimately arriving at and identifying the erroneous
state. Once this state is identified, the algorithm or mechanical fault causing this state
needs to be determined.
Correctness Debugging
It addresses the deviation between the observed and specified functional requirements.
Performance debugging
It addresses the deviation between the observed and specified nonfunctional requirements
such as response time.
TEST PLAN
Test Plan is defined as a strategic document which describes the procedure how to perform
various testing on the total application in the most efficient way.
This document involves the scope of testing,
Objective of testing,
Areas that need to be tested,
Areas that should not be tested,
Scheduling Resource Planning,
Areas to be automated, various testing tools Used….
4.4 I/O WINDOWS
EXAMPLE 1:
Window 1:
23. 23
Fig: Parameters and values for pair wise testing.
Window2:
Fig: Reading the values of the parameters
Window 3:
Fig: Combinations of all parameter value pairs
24. 24
Window4
Fig: Generated the optimized pairwise test suit
EXAMPLE 2:
Fig: Parameters and values for pair wise testing.
25. 25
Fig: Parameters and values for pair wise testing.
Fig: Generated the optimized pair wise test suit
26. 26
CHAPTER 5
CONCLUSION AND FUTURE ENHANCEMENTS
Hence we concluded that a PSO technique was implemented to generate the optimized
pair wise test cases for testing WEB based Applications.
We have implemented this test generation algorithm and have shown some empirical
results. When used properly, pair wise test set generation is an important technique
that can helps to produce minimal test cases for testing WEB applications. The
experiment results illustrated that our proposed PSO technique will yields better results
when comparing to other deterministic techniques like IPOAETG, ACO, and SBC etc.
As mentioned earlier, pair wise testing (or 2-way testing) is a special case of n-way
testing. Our proposed strategy presented in this project can be easily extended for n-way
testing. We are investigating possible improvements of our algorithm without increasing
time complexity.
27. 27
REFERENCES
TEXT BOOKS
1. Paul Ammann and Jeff Offutt, “Introduction to Software Testing”, Cambridge Press,
2008
2. Srinivasa Desikan & Gopalaswamy Ramesh, “Software Testing – Principles and
Practices”, Pearson, 2007.
3. Pressman R S, Software Engineering-A Practitioner’s Approach, 6th edition, McGraw-
Hill, 2005.
4. Sommerville I, Software Engineering, 5th edition, Pearson Education, 1996.
5. Jawadekar W S, Software Engineering – Principles and Practice, Tata McGraw-Hill,
2004.
6. Behforooz A, and Hudson F J, Software Engineering Fundamentals, Oxford University
Press, 1996.
WEBSITES
1. www.slideshare.net
2. www.mendeley.com
3. www.cs.binghamton.edu.com
4. www.wikipedia.org
5. www.chetanasprojects.com
6. www.simmine.com
7. www.google.com