Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
42
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. 1cs205: engineering softwareuniversity of virginia fall 2006ValidationDavid Evanswww.cs.virginia.edu/cs2052cs205: engineering softwareDictionary Definitionval i date1. To declare or make legally valid.2. To mark with an indication ofofficial sanction.3. To establish the soundness of;corroborate.Can we do any of these with software?3cs205: engineering softwareJava’s LicenseREAD THE TERMS OF THIS AGREEMENT AND ANY PROVIDEDSUPPLEMENTAL LICENSE TERMS (COLLECTIVELY "AGREEMENT")CAREFULLY BEFORE OPENING THE SOFTWARE MEDIAPACKAGE. BY OPENING THE SOFTWARE MEDIA PACKAGE, YOUAGREE TO THE TERMS OF THIS AGREEMENT. IF YOU AREACCESSING THE SOFTWARE ELECTRONICALLY, INDICATE YOURACCEPTANCE OF THESE TERMS BY SELECTING THE "ACCEPT"BUTTON AT THE END OF THIS AGREEMENT. IF YOU DO NOTAGREE TO ALL THESE TERMS, PROMPTLY RETURN THE UNUSEDSOFTWARE TO YOUR PLACE OF PURCHASE FOR A REFUND OR,IF THE SOFTWARE IS ACCESSED ELECTRONICALLY, SELECT THE"DECLINE" BUTTON AT THE END OF THIS AGREEMENT.4cs205: engineering softwareJava’s License5. LIMITATION OF LIABILITY. TO THEEXTENT NOT PROHIBITED BY LAW, IN NOEVENT WILL SUN OR ITS LICENSORS BELIABLE FOR ANY LOST REVENUE, PROFIT ORDATA, OR FOR SPECIAL, INDIRECT,CONSEQUENTIAL, INCIDENTAL OR PUNITIVEDAMAGES, HOWEVER CAUSED REGARDLESSOF THE THEORY OF LIABILITY, ARISING OUTOF OR RELATED TO THE USE OF ORINABILITY TO USE SOFTWARE, EVEN IF SUNHAS BEEN ADVISED OF THE POSSIBILITY OFSUCH DAMAGES. …5cs205: engineering softwareJava’s License2. RESTRICTIONS. … Unlessenforcement is prohibited by applicablelaw, you may not modify, decompile, orreverse engineer Software. Youacknowledge that Software is notdesigned, licensed or intended for usein the design, construction, operation ormaintenance of any nuclearfacility. Sun disclaims any express orimplied warranty of fitness for suchuses.6cs205: engineering softwareSoftware Validation• Process designed to increase ourconfidence that a program works asintended• For complex programs, cannot oftenmake guarantees• This is why typical software licensesdon’t make any claims about theirprogram working
  • 2. 27cs205: engineering softwareIncreasing Confidence• Testing– Run the program on set of inputs andcheck the results• Verification– Argue formally or informally that theprogram always works as intended• Analysis– Poor programmer’s verification:examine the source code to increaseconfidence that it works as intended8cs205: engineering softwareTesting and FishingUsing somesuccessful tests toconclude that aprogram has nobugs, is likeconcluding there areno fish in the lakebecause you didn’tcatch one!9cs205: engineering softwareExhaustive Testing• Test all possible inputs• PS1: 50x50 grid, all cells can beeither dead or alive before starting22500 =375828023454801203683362418972386504867736551759258677056523839782231681498337708535732725752658844333702457749526057760309227891351617765651907310968780236464694043316236562146724416478591131832593729111221580180531749232777515579969899075142213969117994877343802049421624954402214529390781647563339535024772584901607666862982567918622849636160208877365834950163790188523026247440507390382032188892386109905869706753143243921198482212075444022433366554786856559389689585638126582377224037721702239991441466026185752651502936472280911018500320375496336749951569521541850441747925844066295279671872605285792552660130702047998218334749356321677469529682551765858267502715894007887727250070780350262952377214028842297486263597879792176338220932619489509376But that’s not all: all possible start stop step clicks,different platforms, how long to you need to run it, etc.10cs205: engineering softwareSelective Testing• We can’t test everything, pick testcases with high probability of findingflaws• Black-Box Testing: design testslooking only at specification• Glass-Box Testing: design testslooking at code– Path-complete: at least one test toexercise each path through code11cs205: engineering softwareBlack-Box TestingTest all paths through the specificationpublic CellState getNextState ()// MODIFIES: this// EFFECTS: Returns the next state for this cell. If a cell is currently// dead cell and has three live neighbors, then it becomes a live cell.// If a cell is currently alive and has two or three live neighbors it// remains alive. Otherwise, the cell dies.12cs205: engineering softwareTest all paths through the specification:1. currently dead, three live neighbors2. currently alive, two live neighbors3. currently alive, three live neighbors4. currently dead, < 3 live neighbors5. currently dead, > 3 live neighbors6. currently alive, < 2 live neighbors7. currently alive, > 3 live neighborspublic CellState getNextState ()// MODIFIES: this// EFFECTS: Returns the next state for this cell. If a cell is currently// dead cell and has three live neighbors, then it becomes a live cell.// If a cell is currently alive and has two or three live neighbors it// remains alive. Otherwise, the cell dies.
  • 3. 313cs205: engineering softwareBlack-Box TestingTest all (7) paths through the specificationTest boundary conditions1. all neighbors are dead2. all neighbors are alive3. cell is at a corner of the grid4. cell is at an edge of the gridpublic CellState getNextState ()// MODIFIES: this// EFFECTS: Returns the next state for this cell. If a cell is currently// dead cell and has three live neighbors, then it becomes a live cell.// If a cell is currently alive and has two or three live neighbors it// remains alive. Otherwise, the cell dies.14cs205: engineering softwareGlass-Box Testingpublic CellState getNextState(){int countalive = 0;Enumeration<SimObject> neighbors = getNeighbors();while (neighbors.hasMoreElements()) {SimObject neighbor = neighbors.nextElement();if (neighbor instanceof Cell) {Cell cell = (Cell) neighbor;if (cell.isAlive()) { countalive++; }}if (countalive == 3) {return CellState.createAlive ();} else if (getState ().isAlive () && countalive == 2) {return CellState.createAlive ();} else { return CellState.createDead (); }}How many paths arethere through this code?15cs205: engineering softwarePath-Complete Testing• Insufficient– Often, bugs are missing paths• Impossible– Most programs have an infinite numberof paths– Loops and recursion• Test with zero, one and several iterations– Branching• Can test all paths16cs205: engineering softwareHow many paths?if (countalive == 3) {return CellState.createAlive ();} else if (getState ().isAlive () && countalive == 2) {return CellState.createAlive ();} else {return CellState.createDead ();}}17cs205: engineering softwareTesting Recap• Testing can find problems, not to proveyour program works– Since exhaustive testing is impossible,select test cases with maximumprobability of finding bugs– A successful test case is one that reveals abug in your program!• Typically at least 40% of cost ofsoftware project is testing, often ~80%of cost for safety-critical software18cs205: engineering softwareQuizzing
  • 4. 419cs205: engineering softwareTesting Recap• Testing can find problems, but can’tprove your program works– Since exhaustive testing is impossible,select test cases with maximumprobability of finding bugs– A successful test case is one that revealsa bug in your program!• If we can’t test all possible pathsthrough a program, how can weincrease our confidence that it works?20cs205: engineering softwareAnalysis• Make claims about all possible pathsby examining the program codedirectly, not executing it• Use formal semantics of programminglanguage to know what things mean• Use formal specifications ofprocedures to know that they do21cs205: engineering softwareHopelessness of AnalysisIt is impossible to correctlydetermine if any interesting propertyis true for an arbitrary program!The Halting Problem: it isimpossible to write aprogram that determines ifan arbitrary program halts.22cs205: engineering softwareCompromises• Use imperfect automated tools:– Accept unsoundness and incompleteness– False positives: sometimes an analysis tool willreport warnings for a program, when theprogram is actually okay (unsoundness)– False negatives: sometimes an analysis toolwill report no warnings for a program, evenwhen the program violates properties it checks(incompleteness)• Use informal reasoning• Design programs to modularizereasoning23cs205: engineering softwareCharge• Next class:– ps2 hints– Exceptions, programming defensively