Your SlideShare is downloading. ×
The Test Case as Executable Example
The Test Case as Executable Example
The Test Case as Executable Example
The Test Case as Executable Example
The Test Case as Executable Example
The Test Case as Executable Example
The Test Case as Executable Example
The Test Case as Executable Example
The Test Case as Executable Example
The Test Case as Executable Example
The Test Case as Executable Example
The Test Case as Executable Example
The Test Case as Executable Example
The Test Case as Executable Example
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

The Test Case as Executable Example

1,318

Published on

CWI Lectures on Understanding Software, in honor of Paul Klint. June 14, 2012.

CWI Lectures on Understanding Software, in honor of Paul Klint. June 14, 2012.

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,318
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
14
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. The Test Case as Executable ExampleCWI Lectures – Understanding Software June 14, 2012 Arie van Deursen @avandeursen 1
  • 2. 20th International Conference on Program Comprehension Study ofthe human activity of comprehending software;the processes and technologies for supporting it 2
  • 3. Programming by ExampleProgramming by Example, Spreadsheet Data Manipulation UsingDaniel Halbert, 1984, Xerox Examples, CACM, 2012 3
  • 4. The TestingPerspective• A test case is an executable example of system behavior• Stakeholder communication needs examples• Throughout the full development cycle 4
  • 5. webrat_steps.rb:Cucumber Then /^I should see "([^"]*)"$/ do |text| response.should contain(text) endFeature: Log in and out As an administrator I want to restrict access to certain portions of my site In order to prevent users from changing the content Scenario: Logging in Given I am not logged in as an administrator When I go to the administrative page And I fill in the fields | Username | admin | | Password | secret | And I press "Log in" Then I should be on the administrative page And I should see "Log out" Scenario: Logging out ... 5
  • 6. Plugin / Junit Testing in Eclipse. 1000s of tests 6
  • 7. Test Suite Information Needs Test Confessions: A Study of Testing Practices for Plug-In SystemsMichaela Greiler, Arie van Deursen, Margaret-Anne Storey, ICSE 2012 7
  • 8. Challenges• Set of examples is incomplete• Set of examples is too large – Some grouping needed – Redundancy Test Similarity• Set of examples is disconnected – Requirements traceability – Backward coverage 8
  • 9. Requirements End-to-End Analysis Test ... ... High Level Integration Design Test Code Unit TestMeasuring Test Case Similarity to Support Test Suite Understanding Michaela Greiler, Arie van Deursen, Andy Zaidman, TOOLS 2012 9
  • 10. Measuring Test Case Similarity Shared word count Test-aware Test Tests rankedinstrumentation Traces by similarity 10
  • 11. Preliminary Evaluation• 75 unit tests • 37 unit• 14 acceptance tests • 6 end to end• Ground truth by hand • Ground truth from book• Most similar match: #1 or #2 in ground truth• Top 10: all (but one) explainable & useful• Surprises: mostly point to issue in test suite.• Work in progress: Analyzing Eclipse test suites. 11
  • 12. Rethinking “Test Cases as Examples” • Languages for writing examples – Different levels of abstraction / stakeholders – Executable – Analyzable – Connected to execution traces• Modularization and grouping of examples – Different levels of abstraction / stakeholders – Requirements / design / code coverage – Multiple views – Connected via execution traces 12
  • 13. In Conclusion• A test case is an executable example• Never enough, yet also too many• Test case connection via trace similarities• Language engineering opportunities ahead 13
  • 14. Photo credits• Passau: Andreas Brandl http://icpc12.sosy-lab.org/• Programming by Example, picture JanHeering (CWI). Photo Eelco Visser. http://eelcovisser.org/post/91/example-driven-research• The Testing Perspective: Photo “Reaching for Understanding”, Flickr http://www.flickr.com/photos/elizabethsalib/2741807789/• Andromeda Galaxy http://www.nasa.gov/mission_pages/spitzer/multimedia/spitzer- 20070604.html• Colliding galaxies http://hubblesite.org/newscenter/archive/releases/2007/08/• Paul Klint, lecturing. http://www.frankwatching.com/archive/2009/05/18/het-nationaal-ict- event/ 14

×