Software reliability
•Download as PPTX, PDF•
0 likes•410 views
reliability and reliability factors in sofware project development...
Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Viewers also liked
Viewers also liked (7)
Similar to Software reliability
Similar to Software reliability (20)
Recently uploaded
Recently uploaded (20)
Software reliability
- 4. WHATISSOFTWARERELIABILITY? • Theprobabilityoffailurefreeoperationofacomputerprogramina specifiedenvironmentforaspecifiedtime. • Reliabilityistheprobabilityofnotfailinginaspecifiedlengthoftime • Itisaqualityfactorthatcanbedirectlymeasuredandestimatedusinghistorical anddevelopmentdata. • Itmeasures howoftens/wencountersadatainputorotherconditionthatitdoes notcorrectlyprocesstoproducecorrectanswer • IfprogramXhasreliabilityof0.96(over8processinghours)thenitmeans,if programXruns100times–itwilloperatecorrectly96times
- 5. SOFTWARERELIABILITY • Theperceivedsoftwarereliabilityisanobservantdependent – Ifyoudonotfacetheproblem,youdonotreportit – Differentusershavedifferentviewsofthesystemsandthusdifferentquality andreliabilityassessments • Thesoftwarereliabilitykeepschangingasthedefectsaredetectedand fixed
- 6. ReliabilityMetrics • Probabilityoffailureondemand – Thisisameasureofthelikelihoodthatthesystemwillfailwhenaservice requestismade – POFOD=0.001means1outof1000servicerequestsresultinfailure – Relevantforsafety-criticalornon-stopsystems • Rateoffaultoccurrence(ROCOF) – Frequencyofoccurrenceofunexpectedbehaviour – ROCOFof0.02means2failuresarelikelyineach100operationaltimeunits – Relevantforoperatingsystems,transactionprocessingsystems
- 7. ReliabilityMetrics • Meantimetofailure – Measureofthetimebetweenobservedfailures – MTTFof500meansthatthetimebetweenfailuresis500timeunits – Relevantforsystemswithlongtransactionse.g.CADsystems • Availability – Measureofhowlikelythesystemisavailableforuse.Takes repair/restarttimeintoaccount – Availabilityof0.998meanssoftwareisavailablefor998outof1000timeunits – Relevantforcontinuouslyrunningsystemse.g.telephoneswitchingsystems
- 8. RelationshipofReliability&Failure • Whenwediscussreliability,weconsidertheterm‘failure’. • Non-conformancetos/wrequirementsleadstofailures • Negativeresultsorinworstcasenooutputisfailure • Failuremaybeofdifferentlevelsofcomplexity. • Somefailurescanbecorrectedinseconds,someinweeksandothersin months • Correctionofafailuremayintroducenewerrors(andeventuallynew failures).
- 9. RelationshipofReliability&Failure • Mathematicalrepresentation –F(n)=1- R(n) • where, –F(n)=probabilityoffailinginaspecifiedlength oftime –R(n)=probabilityofreliability(i.e.notfailing) –n =no.oftimeunits, –iftimeunitisassumedindaysthenprobabilityofnotfailingin1day isR(1)
- 11. FailureClassification • Temporary-onlyoccurswithcertaininputs • Permanent-occursonallinputs • Recoverable-systemcanrecoverwithoutoperatorhelp • Unrecoverable-operatorhastohelp • Non-corrupting-failuredoesnotcorruptsystemstateordata • Corrupting-systemstateordataarealtered
- 12. MeasuresofReliability&Availability • Earlyworkinsoftwarereliabilityattemptedtoextrapolatethe mathematicsofhardwarereliabilitytheorytopredictionofsoftware reliability.But, – Mosthardwarereliabilitymodelsarebasedonfailureduetophysicalwear (decompositioneffects,shock,temperature,etc.)ratherthandesigndefects. – Theoppositeistrueforsoftwares-Allsoftwarefailurescanbetracedtodesign orimplementationproblems. – Thefollowingconceptsapplytobothsystems
- 13. MeasuresofReliability&Availability • MeasureofReliability – Consideracomputer-based system. – Asimplemeasureofreliabilityforsuchasystemismean-time-between-failure (MTBF) • MTBF=MTTF+MTTR MTTF=Mean-time-to-failure MTTR=Mean-time-to-repair Manyresearchers arguethatMTBFismoreusefultermthandefects/KLOCor defects/FPasuserismoreconcernedwithfailurerateascomparedtodefect count. – Eachdefectdoesnothavesamefailurerateandthetotaldefectcountgives littleindicationofthereliabilityofasystem • MeasureofReliability – Consideracomputer-based system. – Asimplemeasureofreliabilityforsuchasystemismean-time-between-failure (MTBF) • MTBF=MTTF+MTTR MTTF=Mean-time-to-failure MTTR=Mean-time-to-repair Manyresearchers arguethatMTBFismoreusefultermthandefects/KLOCor defects/FPasuserismoreconcernedwithfailurerateascomparedtodefect count. – Eachdefectdoesnothavesamefailurerateandthetotaldefectcountgives littleindicationofthereliabilityofasystem
- 14. MeasuresofReliability&Availability • MeasureofAvailability – Softwareavailabilityistheprobabilitythataprogramisoperatingaccording torequirements atagivenpointintime. – Itisdefinedas Availability=[MTTF/(MTTF+MTTR)]*100% – AvailabilitymeasureissensitivetoMTTR
- 15. Reliabilitymodels Manysoftwaremodelscontain: • Assumptions • Factors • Mathematicalfunction Softwarereliabilitycanbedividedintocategories PredictionModel:Thismodeluseshistoricaldata. EstimationModel:Estimationmodelusesthecurrentdatafromthe currentsoftwaredevelopmenteffort
- 16. ReliabilityGrowthModeling • StepFunctionModel – Thesimplestreliabilitygrowthmodel: • astepfunctionmodel – Thebasicassumption: • reliabilityincreasesbyaconstantamounteachtimeanerroris detectedandrepaired. – Assumes: • allerrorscontributeequallytoreliabilitygrowth • highlyunrealistic: – wealreadyknowthatdifferenterrorscontributedifferentlytoreliabilitygrowth.
- 18. ReliabilityGrowthModeling • LittlewoodandVerall’sModel – Assumesdifferentfaulthavedifferentsizes,therebycontributingunequallyto failures. – Allowsfornegativereliabilitygrowth – Largesizedfaultstendstobedetectedandfixedearlier – Asnumberoferrorsisdrivendownwiththeprogressintest,soistheaverage errorsize,causingalawofdiminishingreturnindebugging
- 19. ReliabilityGrowthModeling • Applicabilityofmodels: – Thereisnouniversallyapplicablereliabilitygrowthmodel. – Reliabilitygrowthisnotindependentofapplication. – Fitobserveddatatoseveralgrowthmodels. • Taketheonethatbestfitsthedata.
- 20. StatisticalTesting • Testingforreliabilityratherthanfaultdetection • Test data selection should follow the predicted usage profile for thesoftware • Measuring the number of errors allows the reliability of the softwaretobepredicted • An acceptable level of reliability should be specifiedandthesoftwaretestedandamendeduntilthatlevelof reliabilityisreached
- 21. StatisticalTesting • Differentusershavedifferentoperationalprofile: – i.e.theyusethesystemindifferentways – formally,operationalprofile: • probabilitydistributionofinput • Dividetheinputdataintoanumberofinputclasses: – e.g.create,edit,print,fileoperations,etc. • Assignaprobabilityvaluetoeachinputclass: – aprobabilityforaninputvaluefromthatclasstobeselected.
- 22. StatisticalTesting • Determinetheoperationalprofileofthesoftware: – Thiscanbedeterminedbyanalyzingtheusagepattern. • Manuallyselectorautomaticallygenerateasetoftestdata: – correspondingtotheoperationalprofile. • Applytestcasestotheprogram: – recordexecutiontimebetweeneachfailure – itmaynotbeappropriatetouserawexecutiontime • Afterastatisticallysignificantnumberoffailureshavebeenobserved: – reliabilitycanbecomputed
- 23. StatisticalTesting • Reliesonusinglargetestdataset. • Assumesthatonlyasmallpercentageoftestinputs: – likelytocausesystemfailure. • Itisstraightforwardtogeneratetestscorrespondingtothemostcommoninputs: – butastatisticallysignificant percentageofunlikelyinputsshouldalsobeincluded. • Creatingthesemaybedifficult: – especiallyiftestgeneratorsareused. • Prosandcons -Saybyyourself
- 24. CASESTUDY
- 25. BANKAUTO-TELLERSYSTEM • Eachmachine inanetworkisused300timesaday • Bankhas1000machines • Lifetimeofsoftwarereleaseis2years • Eachmachine handlesabout200,000transactions • About300,000databasetransactionsintotalperday
- 26. EXAMPLESOFARELIABILITYSPEC. Failure class Example Reliability metric Permanent, non-corrupting. The system fails to operate with any card which is input. Software must berestarted to correct failure. ROCOF 1 occurrence/1000 days Transient, non- corrupting The magnetic stripe data cannot be read on an undamaged card which is input. POFOD 1 in 1000 transactions Transient, corrupting A pattern of transactions acrossthe network causes database corruption. Unquantifiable! Should never happen in the lifetime of the system
- 27. SPECIFICATIONVALIDATION • Itisimpossibletoempiricallyvalidateveryhighreliabilityspecifications • NodatabasecorruptionsmeansPOFODoflessthan1in200million • If a transaction takes 1 second, then simulating one day’s transactions takes 3.5 days • Itwouldtakelongerthanthesystem’slifetimetotestitforreliability
- 29. SOFTWAREQUALITY
- 31. SoftwareQualityCriteria • Correctness • Efficiency • Flexibility • Robustness • Interoperability • Maintainability
- 32. SoftwareQualityCriteria • Performance • Portability • Reliability • Reusability • Testability • Usability • Availability • Understandability
- 33. SoftwareQualityManagementSystem • Aqualitymanagementsystemisaprincipalmethodologyusedby organizationstoensurethattheproductstheydevelophavethedesired quality • Aqualitysystemistheresponsibilityofthesystemasawhole,andthe fullsupportofthetopmanagementisamust • Agoodqualitysystemmustbewelldocumented
- 34. SoftwareQualityManagementSystem • Thequalitysystemactivitiesencompassthefollowing: – Auditingofprojects – Reviewofthequalitysystem – Developmentofstandards,proceduresandguidelines…etc – Productionofreportsforthetopmanagement summarizingtheeffectiveness ofthequalitysystemintheorganization
- 35. ProductMetricsversus ProcessMetrics • Productmetricshelpmeasurethecharacteristicsofaproductbeing developed,whereasprocessmetrichelpmeasurehowaprocessis performing • Productmetrics:LOC,FP,PM,timetodeveloptheproduct,the complexityofthesystem…etc • Processmetrics:revieweffectiveness,inspectionefficiency…etc
- 36. CONCLUSIONS • Softwarereliabilityisakeypartinsoftwarequality • Softwarereliabilityimprovementishard • Therearenogenericmodels. • Measurementisveryimportantforfindingthecorrectmodel. • Statisticaltestingshouldbeusedbutitisnoteasyagain… • SoftwareReliabilityModellingisnotassimpleasitlooks.
- 37. THANKYOU