The document discusses a framework for automatically testing agent-based systems using fault models. It proposes defining fault models that specify assumptions about when faults are likely to occur in the system under test. This allows an automated testing process to generate test cases, execute them, and identify any failures as existing faults. The framework extracts information from design documents to test individual units like plans without understanding their internal logic. It aims to provide comprehensive test coverage while reducing costs compared to manual testing. Defining fault models is meant to make the testing process more effective at revealing faults compared to existing techniques.

1. PLAN ORIENTED TESTING OF AGENT BASED SYSTEMS USING FAULT
MODELS
1.INTRODUCTION
Softwaretesting remainsthe most widelyusedapproachtoverification inindustry
today,
consumingbetween30-50%oftheentire
developmentcost.
inputselectionforintelligentagentspresentsaproblem due totheveryfactthat
agentsare
intendedtooperaterobustly
under
wouldthereforebeunlikely
Usingmethodstoautomaticallygenerate
andexecutetestsisone
conditions
oneproblem
problem:how
without
the
conditions
inwhichdevelopersdidnotconsider,and
coverageofmany
Test
totest.
waytoprovide
significantlyincreasingcost.However,
usingautomaticgenerationandexecutionoftestsistheoracle
canweautomaticallydecideifobservedprogrambehavioriscorrect
withrespecttoitsspecification.
2. RESEARCH QUESTIONS
Based on the issues associated with agent testing and the limitations of existing work,
thefollowing research questions addressed in my projects are:
What is the scope of unit testing for agent systems?This involves the
identification of the basic units in agent systems as units to be tested.
How can an automated testing framework be developed to effectively capture
errors? There are three parts to this question: How do dependencies between
units/agents affect testing? An agent system usually contains multiple agents and
an agent contains multiple units. Do the dependencies between these components
affect testing?
Our approach for testing a plan does not involve the understanding of internal
logic of the plan; therefore we do not test the coverage of the execution paths and
code branches of the plan. The tester may however specify value ranges of

2. inputvariables as certain values in order to cover a plan’s internal logic
thoroughly, but this requires manual effort and understanding of the plan’s logic
There is another limitation in the algorithm of test input generation. Although combination of
input variable values and comparisons between variables have been taken into account in test
input generation, combinations of comparative relationships (e.g “x>y” and “y>z”) are not
considered
An effective way for revealing faults is to define for a SUT a fault model, which specifies
the assumptions about under what situation a fault is likely to be found in the SUT .Each
assumption introduces the occurrence of a software failure and such an occurrence in the
SUT can be identified as a fault that exists in the system. In this work we using a test case
but it is not covering the whole test process so we are introducing a fault model to
overcome the existing process to identify the total number faults present in the agent
system.
3.TESTINGFRAMEWORK
In thissection, we providean overviewof
resides.Wenote
that
oncethe
descriptorsaddedtocapture those
inputgenerationand
the test frameworkinwhich our oracle
designdocuments
aspects
have
had
testing
of implementation necessaryfor test
appropriatecodeaugmentation,
thetesting
process
isautomatic,supportingthegeneration, executionand reportingofcomprehensive test suites
withoutinvolvementfrom the test engineer.
Thesystemmodel thatweuseastheoracleconsistsof
the artifacts
producedduringthe
detaileddesign phase ofthe Prometheusmethodology.These arethe
overviewdiagramsof agents and their capabilities, along with the detailed descriptorsof
the internal components. Itisofcourse thecasethat errors found may bethe result oferrors
inthe
designdocuments,
which
arecorrectedintheimplementation.However,
intheinterestsofmaintainingup todate and correct documentation
important to correct these astocorrect implementationerrors.
of a system, it is as

3. 3.1 General Working Procedure of Testing Framework
Thetestingtoolperformsanautomatedprocessfortestinganagentsystem,asoutlined
h er e .Themainstepsinthetestingprocessareasfollows:
•Step1:
When
thetestingprocess
starts,thetestingtoolaccessesthePrometheus
designdocumentationofthesystemand extracts thelistofallagents..
•Step2: For each agent,thetestingtoolidentifies thetestunits withintheagent(plans andevents),and
determines theorder inwhichtheseunits are tested
•Step3: For eachunit,thetestingtoolimplements atestautomationvia aprocess
ofcodeaugmentation.Thedetailsregarding testautomation ofdifferenttypesofunitsare discussed.
•Step4:Thetestingtoolthenexecutesthetestharnesstoperformaunittestingprocess
fortestingtheassociatedunit.
Inaunittestingprocess,
initializestheruntimeenvironmentthatisneeded
executesa
setoftestcases,
thetestharness
fortheexecutionoftestcases,generatesand
collectstestoutcomesand
compares
themagainstthe
informationextractedfromthedesigndocumentationandidentifiesfaultsif theyexist.
•Step6:
Ifanyfaultis
detectedintheunittestingprocess,one
of
thefollowingthreeactionswillbeperformeddepending onthedifferentlevelsof the fault:
•Step7:
Attheendof
thetestingprocess,thetestingtoolgeneratesatestreportin
HTMLformat.Thereportindicatesallthe
detailsof
the
testingprocessandthetest
resultsofalltheunitsthathavebeentested
Compared
withotherexistingtechniques
testingframework
fortestingagentsystems
performs
,
thetestingtoolinour
acompleteautomatedprocessforunit
testinganagentsystem.Intheprocess,userinterventionisnotmandatoryastestautomation
areautomaticallyimplementedand
eachtestcase
associatedunit.Userinterventionisonlynecessary
isautomaticallyexecutedtotestthe
insomepreparatorytasksfortheauto-

4. matedtestingprocess,suchasdefinitionofinputvariables andspecification ofinitializationprocedures .
The usercan alsomanually add additional testcasesifnecessary.
Figure 3.1 Testing Process
4.FAULT MODELS
An effective way for revealing faults is to define for a SUT a fault model, which specifies the
assumptions about under what situation a fault is likely to be found in the SUT. Each assumption
introduces the occurrence of a software failure and such an occurrence in the SUT can be

5. identified as a fault that exists in the system. So by writing all the faults in the db as the rules and
by leveling all the rules such as level 1 as exception, level 2 is errorand level 3 is errors so by
defining it all the test cases will be passing it so all the test cases will be tested and also the test
engineer knows what behavior it will exhibit using the test process using fault models.
So I conclude it by adding the fault models in the plan oriented testing of agent systems will be
more effective testing process and the result also will be compared with existing testing
approach.
.
Agents in MAS
Units in Agents
ie.Plan
Final Result
Plan Oriented
Testing
Fault Modes
Figure 4.1 Overall System Architecture
Test Report