Your SlideShare is downloading. ×
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. CS 160: Software Engineering April 8 Class Meeting Department of Computer Science San Jose State University Spring 2008 Instructor: Prof. Ron Mak http://
  • 2. Unofficial Field Trip?
    • Computer History Museum in Mt. View.
      • See
      • You provide your own transportation to and from the museum.
      • Completely voluntary, no credit.
    • Approximately 2 hours during a Saturday afternoon later this month or early next month.
      • Docent-led tour of various computer systems and artifacts from the 1940s through the 1990s.
        • Eniac, Enigma, SAGE, IBM 360, Cray supercomputers, etc.
      • See, hear, and touch an actual IBM 1401 computer system (c. 1959-1965) that has been restored and is operational:
        • 800 cpm card reader
        • flashing console lights
        • spinning tape drives with vacuum columns
        • 600 lpm line printer
  • 3. Software Reliability
    • Reliable software has a low probability of failure while operating for a specified period of time under specified operating conditions.
      • The specified time period and the specified operating conditions are part of the nonfunctional requirements.
      • For some applications, reliability may be more important than the other functional and nonfunctional requirements.
        • mission-critical applications
        • medical applications
  • 4. Software Reliability (cont’d)
    • Reliable software is the result of good software quality assurance (SQA) throughout an application’s life cycle.
      • Design
        • good requirements elicitation
        • good object-oriented design and analysis
        • good architecture
      • Development
        • good management (e.g., source control, reasonable schedules)
        • good coding practices (e.g., design patterns)
        • good testing practices
      • Deployment
        • good preventive maintenance (e.g., training)
        • good monitoring
        • good failure analysis (when failures do occur)
  • 5. Software Testing
    • Some key questions:
      • What is testing?
      • What is a successful test?
      • Who does testing?
      • When does testing occur?
      • What are the different types of testing?
      • What testing tools are available?
      • How do you know your tests covered everything?
      • When can you stop testing?
  • 6. What is testing?
    • Testing is a systematic procedure to discover faults in software in order to prevent failure .
      • Failure: A deviation of the software’s behavior from its specified behavior (as per its requirements)
        • Can be minor to major (such as a crash)
      • Erroneous state: A state that the operating software is in that will lead to a failure.
        • Example: low on memory
      • Fault: What caused the software to enter an erroneous state.
        • Also known as: defect, bug
        • Example: a memory leak
  • 7. What is a successful test?
    • Testing is the opposite of coding.
      • Coding: Create software and try to get it to work.
      • Testing: Break the software and demonstrate that it doesn’t work.
    • Testing and coding require different mind sets.
      • It can be very difficult for developers to test their own code.
      • If you wrote the code, you psychologically want it to work and not see it break.
      • Since you know how the code should be used, you may not think to try using it in ways other than as you intended.
    • A successful test is one that finds bugs.
  • 8. Who does testing?
    • Developers
      • As difficult as it may be, you must test your own code .
      • Test each other’s code (peer testing)
        • test interfaces
    • Users
    • Testers
      • Members of the Testing or Quality Assurance (QA) department.
      • Software engineers who did not write the code.
      • Manual writers and trainers who create examples and demos.
  • 9. When does testing occur?
    • Recall the Old Waterfall Model :
    • In the new Agile Methodology, testing is part of each and every iteration .
      • Therefore, testing occurs throughout development, not just at the end.
    XXX Requirements Design Implementation Testing
  • 10. What are the different types of testing?
    • Usability testing
      • Developers and users test the user interface.
    • Unit testing
      • Developers test an individual “unit”.
      • Unit: A small set of related components, such as to implement a use case.
    • Integration testing
      • Developers test how their units work together with other units.
    • System testing
      • Test how an entire system works.
      • Includes performance testing and stress testing.
  • 11. Alpha Testing vs. Beta Testing
    • Alpha testing
      • Usability and system testing of a nearly complete application in the development environment .
    • Beta testing
      • Usability and system testing of a complete or nearly complete application in the user’s environment .
    • Today, it is not uncommon to release an application to the public for beta testing.
      • New releases of web-based applications are put “into beta” by software companies.
  • 12. Usability Testing: The German Newton
    • The Apple Newton was an early PDA developed in the early 1990s.
      • Besides the English version, there was a German and a Japanese version.
      • It was too far ahead of its time and was killed by Steve Jobs after he returned to Apple.
    • A key feature of the Newton was handwriting recognition : ‹‹ Handschrifterkennung ››
      • The Newton recognized successive words in a sentence using an algorithm that tracked the movement of the stylus.
    • Cultural differences between Americans and Germans
      • The way the Germans write their letters (e.g., the letter h).
      • The way the Germans write words (e.g., ‹‹ Gesch ä ftsreise ›› “business trip”).
      • Philosophy about personal handwriting.
  • 13. Usability Testing: NASA Mars Rover Mission
    • Usability testing for the Collaborative Information Portal (CIP) software system.
    • NASA’s Jet Propulsion Laboratory (JPL) conducted several Operational Readiness Tests (ORTs) before the actual rovers landed.
      • A working rover was inside of a large “sandbox” in a separate warehouse building at JPL.
      • Mission control personnel communicated with and operated the sandbox rover as if it were on Mars.
        • A built-in delay simulated the signal travel time between Earth and Mars.
      • Mission scientists accessed and analyzed the data and images downloaded by the sandbox rover.
    • All mission software, including CIP, was intensely tested in this simulated environment.
  • 14. Unit Testing
    • Each unit test focuses on components created by a developer.
      • Should be done by the developer before checking in the code.
      • Easier to find and fix bugs when there are fewer components.
      • Bottom-up testing.
    • Developers create test cases to do unit tests.
  • 15. Unit Testing: Test Cases
    • Test case: A set of input values for the unit and a corresponding set of expected output values.
      • Use case -> test case
      • Do unit testing on the participating objects of the use case.
    • A test case can be run within a testing framework (AKA test bed, test harness) consisting of a test driver and one or more test stubs.
      • Test driver: Simulates the part of the system that calls the unit. Calls the unit and passes in the input values.
      • Test stub: Simulates the components that the unit depends on.
        • When called by the unit, a test stub responds in a “reasonable” manner.
  • 16. Unit Testing: Testing Framework UNIT TEST DRIVER TEST STUB TEST STUB TEST STUB
  • 17. Black Box Testing vs. White Box Testing
    • Black box testing
      • Deals only with the input/output behavior of the unit.
      • The internals of the unit are not considered.
    • White box testing
      • Tests the internal behavior of the unit.
        • execution paths
        • state transitions
  • 18. Unit Testing: Equivalence Testing
    • Minimize the number of test cases by partitioning the possible inputs into equivalence classes .
      • Assumption: For each equivalence class, the unit behaves the same way for any input from the class.
      • Example: A calendar unit that takes a year and a month as input.
        • Year equivalence classes: leap years and non-leap years
        • Month equivalence classes: months with 30 days, months with 31 days, and February (28 or 29 days)
    • Black box test.
  • 19. Unit Testing: Boundary Testing
    • Test the boundaries (the extremes, or “edges”) of the equivalence classes.
    • Calendar component examples:
      • Month 0 and month 13.
      • Months with the incorrect number of days.
      • Invalid leap years (e.g., not a multiple of 4)
    • Black box test.
  • 20. Unit Testing: Monte Carlo Testing
    • Monte Carlo (a famous casino in Monaco) refers to the random generation of input values .
      • Used when the input values for a unit are too numerous and cannot be partitioned into equivalence classes.
      • The test driver generates random values from the input domain and passes them to the unit.
      • Any erroneous output narrows your search for the bug.
    • Black box test.
  • 21. Unit Testing: Path Testing
    • Analyze the execution paths within the unit.
      • Draw a flow graph.
    • Generate input values to force the unit to follow each and every path at least once .
    • White box test.
  • 22. Unit Testing: State Transition Testing
    • Similar to path testing.
    • Analyze the states that the unit can be in.
      • Draw a UML statechart diagram.
    • Generate input data that forces the unit to transition into each and every state at least once .
    • White box test.
  • 23. Unit Testing Tool: JUnit
    • JUnit: A unit testing framework for Java components.
      • A member of the xUnit family of testing frameworks for various programming languages.
    • Open source: Download from
    • JUnit demo.
  • 24. JUnit Demo
    • Class to test:
    package cs160.util; public class Calculator { public double add (double first, double second) { return first + second; } public double subtract (double first, double second) { return second - first; // oops! } public double multiply (double first, double second) { return first * second; } }
  • 25. JUnit Demo (cont’d)
    • Unit test cases:
    package cs160.test; import cs160.util.Calculator; import junit.framework.TestCase ; public class CalculatorTester extends TestCase { public void testAdd () { Calculator calculator = new Calculator(); double result = calculator.add(40, 30); assertEquals (70.0, result, 0.0); } public void testSubtract () { Calculator calculator = new Calculator(); double result = calculator.subtract(40, 30); assertEquals (10.0, result, 0.0); } }
  • 26. Unit Testing Tool: Clover
    • Clover: A unit testing tool that tells you what parts of your code you have actually executed (“covered”).
      • Unfortunately, not open source.
      • Download a 30-day trial from
    • Clover demo.
  • 27. Integration Testing
    • After your code has passed all the unit tests, you must do integration testing to see how your unit works with other units
      • Other units either written by you or by other developers.
    • Create an Ant script that builds the part of the application that contains:
      • Your unit
      • The units that call your unit
      • The units that your unit calls
  • 28. Regression Tests
    • Sad fact of life: When you fix one bug, you might very well introduce one or more new bugs.
      • Tests that have passed previously may now show failures – the system has regressed .
    • Regression tests: Re-run earlier integration tests to make sure they still pass.
      • Do this every time you make a change.
      • Create Ant scripts to run the tests.
      • Schedule regression tests to run periodically automatically, e.g., overnight every night.
  • 29. System Testing
    • Test the entire application within its operating environment.
      • Installation testing
      • Functional testing
      • Performance testing
    • Part of beta testing.
    • Acceptance testing .
      • Signoff by stakeholders, clients, and customers.
  • 30. When can you stop testing?
    • “ Testing can prove the presence of bugs, but never their absence .”
    • Stop testing when:
      • All the regression tests pass.
      • Testing finds only “acceptable” bugs.
        • put on the Known Bugs list
        • have workarounds
      • When you’ve run out of time.
  • 31. Next class meeting…
    • Logging and monitoring
    • Stress testing
    • Test-Driven Development (TDD)
    • Failure analysis
    • Creating test plans