The Test Case as Executable Example
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

The Test Case as Executable Example

on

  • 1,648 views

CWI Lectures on Understanding Software, in honor of Paul Klint. June 14, 2012.

CWI Lectures on Understanding Software, in honor of Paul Klint. June 14, 2012.

Statistics

Views

Total Views
1,648
Views on SlideShare
1,637
Embed Views
11

Actions

Likes
1
Downloads
11
Comments
0

1 Embed 11

http://tweetedtimes.com 11

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

The Test Case as Executable Example Presentation Transcript

  • 1. The Test Case as Executable ExampleCWI Lectures – Understanding Software June 14, 2012 Arie van Deursen @avandeursen 1
  • 2. 20th International Conference on Program Comprehension Study ofthe human activity of comprehending software;the processes and technologies for supporting it 2
  • 3. Programming by ExampleProgramming by Example, Spreadsheet Data Manipulation UsingDaniel Halbert, 1984, Xerox Examples, CACM, 2012 3
  • 4. The TestingPerspective• A test case is an executable example of system behavior• Stakeholder communication needs examples• Throughout the full development cycle 4
  • 5. webrat_steps.rb:Cucumber Then /^I should see "([^"]*)"$/ do |text| response.should contain(text) endFeature: Log in and out As an administrator I want to restrict access to certain portions of my site In order to prevent users from changing the content Scenario: Logging in Given I am not logged in as an administrator When I go to the administrative page And I fill in the fields | Username | admin | | Password | secret | And I press "Log in" Then I should be on the administrative page And I should see "Log out" Scenario: Logging out ... 5
  • 6. Plugin / Junit Testing in Eclipse. 1000s of tests 6
  • 7. Test Suite Information Needs Test Confessions: A Study of Testing Practices for Plug-In SystemsMichaela Greiler, Arie van Deursen, Margaret-Anne Storey, ICSE 2012 7
  • 8. Challenges• Set of examples is incomplete• Set of examples is too large – Some grouping needed – Redundancy Test Similarity• Set of examples is disconnected – Requirements traceability – Backward coverage 8
  • 9. Requirements End-to-End Analysis Test ... ... High Level Integration Design Test Code Unit TestMeasuring Test Case Similarity to Support Test Suite Understanding Michaela Greiler, Arie van Deursen, Andy Zaidman, TOOLS 2012 9
  • 10. Measuring Test Case Similarity Shared word count Test-aware Test Tests rankedinstrumentation Traces by similarity 10
  • 11. Preliminary Evaluation• 75 unit tests • 37 unit• 14 acceptance tests • 6 end to end• Ground truth by hand • Ground truth from book• Most similar match: #1 or #2 in ground truth• Top 10: all (but one) explainable & useful• Surprises: mostly point to issue in test suite.• Work in progress: Analyzing Eclipse test suites. 11
  • 12. Rethinking “Test Cases as Examples” • Languages for writing examples – Different levels of abstraction / stakeholders – Executable – Analyzable – Connected to execution traces• Modularization and grouping of examples – Different levels of abstraction / stakeholders – Requirements / design / code coverage – Multiple views – Connected via execution traces 12
  • 13. In Conclusion• A test case is an executable example• Never enough, yet also too many• Test case connection via trace similarities• Language engineering opportunities ahead 13
  • 14. Photo credits• Passau: Andreas Brandl http://icpc12.sosy-lab.org/• Programming by Example, picture JanHeering (CWI). Photo Eelco Visser. http://eelcovisser.org/post/91/example-driven-research• The Testing Perspective: Photo “Reaching for Understanding”, Flickr http://www.flickr.com/photos/elizabethsalib/2741807789/• Andromeda Galaxy http://www.nasa.gov/mission_pages/spitzer/multimedia/spitzer- 20070604.html• Colliding galaxies http://hubblesite.org/newscenter/archive/releases/2007/08/• Paul Klint, lecturing. http://www.frankwatching.com/archive/2009/05/18/het-nationaal-ict- event/ 14