Help, my tests are killing me!!
Upcoming SlideShare
Loading in...5
×
 

Help, my tests are killing me!!

on

  • 328 views

Since the early days of eXtreme Programming, tests have been touted as "executable documentation". ...

Since the early days of eXtreme Programming, tests have been touted as "executable documentation".
In practice, however, many tests fail to live up to that ideal and worse become a time sink when trying to decipher test failures.
Using examples and focusing specifically on unit or developer tests, this talk will examine the characteristics of readability in tests and offer advice on making your tests more readable.

Statistics

Views

Total Views
328
Views on SlideShare
324
Embed Views
4

Actions

Likes
0
Downloads
1
Comments
0

1 Embed 4

https://twitter.com 4

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Help, my tests are killing me!! Help, my tests are killing me!! Presentation Transcript

  • HELP MY Tests ARE KILLING ME!! brian swan @bgswan follow me on twitter
  • Contains code some people may find offensive
  • Testsuite: org.springframework.util.StopWatchTests Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.177 sec Testcase: testValidUsage took 0.171 sec FAILED Unexpected timing 167 junit.framework.AssertionFailedError: Unexpected timing 167 at org.springframework.util.StopWatchTests.testValidUsage(StopWatchTests.java:48)
  • public void testValidUsage() throws Exception { StopWatch sw = new StopWatch(); long int1 = 166L; long int2 = 45L; String name1 = "Task 1"; String name2 = "Task 2"; assertFalse(sw.isRunning()); sw.start(name1); Thread.sleep(int1); assertTrue(sw.isRunning()); sw.stop(); long fudgeFactor = 0L; assertTrue("Unexpected assertTrue("Unexpected sw.start(name2); Thread.sleep(int2); sw.stop(); assertTrue("Unexpected assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + fudgeFactor); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1 + int2); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + int2 + fudgeFactor); assertTrue(sw.getTaskCount() == 2); String pp = sw.prettyPrint(); assertTrue(pp.indexOf(name1) != -1); assertTrue(pp.indexOf(name2) != -1); StopWatch.TaskInfo[] tasks = sw.getTaskInfo(); assertTrue(tasks.length == 2); assertTrue(tasks[0].getTaskName().equals(name1)); assertTrue(tasks[1].getTaskName().equals(name2)); sw.toString(); }
  • HH GG AR A
  • public void testValidUsage() throws Exception { StopWatch sw = new StopWatch(); long int1 = 166L; long int2 = 45L; String name1 = "Task 1"; String name2 = "Task 2"; assertFalse(sw.isRunning()); sw.start(name1); Thread.sleep(int1); assertTrue(sw.isRunning()); sw.stop(); // TODO are timings off in JUnit? Why do these assertions sometimes fail // under both Ant and Eclipse? //long fudgeFactor = 5L; //assertTrue("Unexpected //assertTrue("Unexpected sw.start(name2); Thread.sleep(int2); sw.stop(); //assertTrue("Unexpected //assertTrue("Unexpected timing " + sw.getTotalTime(), sw.getTotalTime() >= int1); timing " + sw.getTotalTime(), sw.getTotalTime() <= int1 + fudgeFactor); timing " + sw.getTotalTime(), sw.getTotalTime() >= int1 + int2); timing " + sw.getTotalTime(), sw.getTotalTime() <= int1 + int2 + fudgeFactor); assertTrue(sw.getTaskCount() == 2); String pp = sw.prettyPrint(); assertTrue(pp.indexOf(name1) != -1); assertTrue(pp.indexOf(name2) != -1); StopWatch.TaskInfo[] tasks = sw.getTaskInfo(); assertTrue(tasks.length == 2); assertTrue(tasks[0].getTaskName().equals(name1)); assertTrue(tasks[1].getTaskName().equals(name2)); sw.toString(); } https://github.com/SpringSource/spring-framework/blob/master/spring-core/src/test/java/org/springframework/util/ StopWatchTests.java
  • The UnREADABLE TEST Long Test Methods featuring Undecipherable Failures introducing Developer Confusion and Wasted Developer Time produced by
  • Flesch Reading Ease http://en.wikipedia.org/wiki/Flesch–Kincaid_Readability_Test
  • MORE READABILITY Produced By SMALL WORDS and SHORT SENTENCES
  • http://www.arrestedcomputing.com/pubs/readability-issta.pdf
  • http://www.arrestedcomputing.com/pubs/readability-issta.pdf
  • “ THE LINE LENGTH IDENTIFIER MASSACRE”
  • EDIBLE HE INCR T
  • it "adds items to top" do stack = Stack.new stack.add "thing" assert_equal "thing", stack.top stack = nil end # # # # Setup Exercise Verify Teardown
  • it "adds items to top" do stack = Stack.new stack.add "thing" assert_equal "thing", stack.top end # Setup # Exercise # Verify
  • it "adds items to top" do stack = Stack.new stack.add "thing" assert_equal "thing", stack.top end # Arrange # Act # Assert
  • it "adds items to top" do stack = Stack.new stack.add "thing" assert_equal "thing", stack.top end # Given # When # Then
  • before do @stack = Stack.new end # Given it "adds items to top" do @stack.add "thing" assert_equal "thing", @stack.top end # When # Then
  • before do @stack = Stack.new end # Given it "is initially empty" do assert_empty @stack # Then end
  • it "returns the total timesheet hours worked of the timesheet" do timesheet = Timesheet.new task_items = [stub(start_time: 1359291600, end_time: 1359295200), stub(start_time: 1359295200, end_time: 1359298900)] timesheet.stub(:task_items) { task_items } timesheet.calculate_total_hours.should == 2.0 end http://www.superpumpup.com/sub-ms-test
  • it "returns the total timesheet hours worked of the timesheet" do timesheet = Timesheet.new task_items = [stub(start_time: 1359291600, end_time: 1359295200), stub(start_time: 1359295200, end_time: 1359298900)] timesheet.stub(:task_items) { task_items } timesheet.calculate_total_hours.should == 2.0 end
  • Extract Given You have a long, confusing or poorly named test setup. Turn the fragment into a method or member variable whose name describes the “given”.
  • http://martinfowler.com/bliki/ObjectMother.html
  • http://nat.truemesh.com/archives/000714.html
  • def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new task_items = [stub(start_time: 1359291600, end_time: 1359295200), stub(start_time: 1359295200, end_time: 1359298900)] timesheet.stub(:task_items) { task_items } timesheet end it "returns the total timesheet hours worked of the timesheet" do timesheet_with_two_one_hour_tasks.calculate_total_hours.should == 2.0 end
  • def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new task_items = [stub(start_time: 1359291600, end_time: 1359295200), stub(start_time: 1359295200, end_time: 1359298900)] timesheet.stub(:task_items) { task_items } timesheet end it "returns the total timesheet hours worked of the timesheet" do timesheet_with_two_one_hour_tasks.calculate_total_hours.should == 2.0 end
  • Extract Collaborator An object your test depends on is confusing or poorly named. Extract the collaborating object into a method or member variable whose name describes its purpose.
  • def one_hour_task stub(start_time: 1359291600, end_time: 1359295200) end def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new task_items = [one_hour_task, one_hour_task] timesheet.stub(:task_items) { task_items } timesheet end it "returns the total timesheet hours worked of the timesheet" do timesheet_with_two_one_hour_tasks.calculate_total_hours.should == 2.0 end
  • def one_hour_task stub(start_time: 1359291600, end_time: 1359295200) end def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new task_items = [one_hour_task, one_hour_task] timesheet.stub(:task_items) { task_items } timesheet end it "returns the total timesheet hours worked of the timesheet" do timesheet_with_two_one_hour_tasks.calculate_total_hours.should == 2.0 end
  • Inline Temp You have a temp that is assigned to once with a simple expression, and the temp is getting in the way of other refactorings. Replace all references to that temp with the expression. http://www.refactoring.com/catalog/inlineTemp.html
  • def one_hour_task stub(start_time: 1359291600, end_time: 1359295200) end def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new timesheet.stub(:task_items) { [one_hour_task, one_hour_task] } timesheet end it "returns the total timesheet hours worked of the timesheet" do timesheet_with_two_one_hour_tasks.calculate_total_hours.should == 2.0 end
  • def one_hour_task stub(start_time: 1359291600, end_time: 1359295200) end def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new timesheet.stub(:task_items) { [one_hour_task, one_hour_task] } timesheet end it "returns the total timesheet hours worked of the timesheet" do timesheet_with_two_one_hour_tasks.calculate_total_hours.should == 2.0 end
  • Use meaningful data Your test uses arbitrary literal values as test data. Replace arbitrary values with values that have meaning in the domain of the test.
  • ONE_HOUR_IN_SECONDS = (60*60*1) def one_hour_task stub(start_time: 0, end_time: ONE_HOUR_IN_SECONDS) end def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new timesheet.stub(:task_items) { [one_hour_task, one_hour_task] } timesheet end it "returns the total timesheet hours worked of the timesheet" do timesheet_with_two_one_hour_tasks.calculate_total_hours.should == 2.0 end
  • ONE_HOUR_IN_SECONDS = (60*60*1) def one_hour_task stub(start_time: 0, end_time: ONE_HOUR_IN_SECONDS) end def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new timesheet.stub(:task_items) { [one_hour_task, one_hour_task] } timesheet end it "returns the total timesheet hours worked of the timesheet" do timesheet_with_two_one_hour_tasks.calculate_total_hours.should == 2.0 end
  • Extract Test Your test method is checking multiple behaviours, or checking behaviour of a collaborating object. Extract a new test method for each behaviour.
  • describe Task do it "knows duration in hours" do task = Task.new(start_time: 0, end_time: 3600) task.duration.should == 1.0 end end
  • def one_hour_task stub(duration: 1) end def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new timesheet.stub(:task_items) { [one_hour_task, one_hour_task] } timesheet end it "returns the total timesheet hours worked of the timesheet" do timesheet_with_two_one_hour_tasks.calculate_total_hours.should == 2.0 end
  • it "returns the total timesheet hours worked of the timesheet" do one_hour_task = stub(duration: 1) timesheet = Timesheet.new timesheet.stub(:task_items) { [one_hour_task, one_hour_task] } timesheet.calculate_total_hours.should == 2.0 end
  • it "returns the total timesheet hours worked of the timesheet" do timesheet = Timesheet.new task_items = [stub(start_time: 1359291600, end_time: 1359295200), stub(start_time: 1359295200, end_time: 1359298900)] timesheet.stub(:task_items) { task_items } timesheet.calculate_total_hours.should == 2.0 end http://www.superpumpup.com/sub-ms-test
  • it "returns the total timesheet hours worked of the timesheet" do one_hour_task = stub(duration: 1) timesheet = Timesheet.new timesheet.stub(:task_items) { [one_hour_task, one_hour_task] } timesheet.calculate_total_hours.should == 2.0 end
  • The UnREADABLE TEST II
  • public void testValidUsage() throws Exception { StopWatch sw = new StopWatch(); long int1 = 166L; long int2 = 45L; String name1 = "Task 1"; String name2 = "Task 2"; assertFalse(sw.isRunning()); sw.start(name1); Thread.sleep(int1); assertTrue(sw.isRunning()); sw.stop(); long fudgeFactor = 0L; assertTrue("Unexpected assertTrue("Unexpected sw.start(name2); Thread.sleep(int2); sw.stop(); assertTrue("Unexpected assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + fudgeFactor); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1 + int2); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + int2 + fudgeFactor); assertTrue(sw.getTaskCount() == 2); String pp = sw.prettyPrint(); assertTrue(pp.indexOf(name1) != -1); assertTrue(pp.indexOf(name2) != -1); StopWatch.TaskInfo[] tasks = sw.getTaskInfo(); assertTrue(tasks.length == 2); assertTrue(tasks[0].getTaskName().equals(name1)); assertTrue(tasks[1].getTaskName().equals(name2)); sw.toString(); }
  • X ... the Unknown test case
  • “ clarity trumps brevity in test names https://www.facebook.com/notes/kent-beck/shorts-not-always-sweet-the-case-for-long-test-names/ 564493423583526
  • Rename Method The name of a method does not reveal its purpose. Change the name of the method. http://www.refactoring.com/catalog/renameMethod.html
  • public void testIsRunningWhenStartedAndTimeMultipleTasksAndTaskCountAndPrettyPrintAndTaskInfo() throws Exception { StopWatch sw = new StopWatch(); long int1 = 166L; long int2 = 45L; String name1 = "Task 1"; String name2 = "Task 2"; assertFalse(sw.isRunning()); sw.start(name1); Thread.sleep(int1); assertTrue(sw.isRunning()); sw.stop(); long fudgeFactor = 5L; assertTrue("Unexpected assertTrue("Unexpected sw.start(name2); Thread.sleep(int2); sw.stop(); assertTrue("Unexpected assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + fudgeFactor); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1 + int2); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + int2 + fudgeFactor); assertTrue(sw.getTaskCount() == 2); String pp = sw.prettyPrint(); assertTrue(pp.indexOf(name1) != -1); assertTrue(pp.indexOf(name2) != -1); StopWatch.TaskInfo[] tasks = sw.getTaskInfo(); assertTrue(tasks.length == 2); assertTrue(tasks[0].getTaskName().equals(name1)); assertTrue(tasks[1].getTaskName().equals(name2)); sw.toString(); }
  • public void testTimeMultipleTasks() throws Exception { StopWatch sw = new StopWatch(); long int1 = 166L; long int2 = 45L; String name1 = "Task 1"; String name2 = "Task 2"; sw.start(name1); Thread.sleep(int1); sw.stop(); long fudgeFactor = 5L; assertTrue("Unexpected assertTrue("Unexpected sw.start(name2); Thread.sleep(int2); sw.stop(); assertTrue("Unexpected assertTrue("Unexpected } timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + fudgeFactor); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1 + int2); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + int2 + fudgeFactor);
  • public void testTimeMultipleTasks() throws Exception { StopWatch sw = new StopWatch(); StopWatch.TaskInfo task1 = new StopWatch.TaskInfo("Task 1", 166L); StopWatch.TaskInfo task2 = new StopWatch.TaskInfo("Task 2", 45L); sw.start(task1.getTaskName()); Thread.sleep(task1.getTimeMillis()); sw.stop(); long fudgeFactor = 10L; assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= task1.getTimeMillis()); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= task1.getTimeMillis() + fudgeFactor); sw.start(task2.getTaskName()); Thread.sleep(task2.getTimeMillis()); sw.stop(); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= task1.getTimeMillis() + task2.getTimeMillis()); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= task1.getTimeMillis() + task2.getTimeMillis() + fudgeFactor); }
  • public void testTimeMultipleTasks() throws Exception { StopWatch sw = new StopWatch(); StopWatch.TaskInfo task1 = new StopWatch.TaskInfo("Task 1", 166L); StopWatch.TaskInfo task2 = new StopWatch.TaskInfo("Task 2", 45L); timeTask(sw, task1); long fudgeFactor = 10L; assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= task1.getTimeMillis()); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= task1.getTimeMillis() + fudgeFactor); timeTask(sw, task2); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= task1.getTimeMillis() + task2.getTimeMillis()); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= task1.getTimeMillis() + task2.getTimeMillis() + fudgeFactor); } private void timeTask(StopWatch sw, StopWatch.TaskInfo task) throws Exception { sw.start(task.getTaskName()); Thread.sleep(task.getTimeMillis()); sw.stop(); }
  • Replace Assertion Your test method uses a basic assertion to check for a specific condition. Use a more specific assertion provided by your test framework, or create a custom assert method.
  • public void testTimeMultipleTasks() throws Exception { StopWatch sw = new StopWatch(); StopWatch.TaskInfo task1 = new StopWatch.TaskInfo("Task 1", 166L); StopWatch.TaskInfo task2 = new StopWatch.TaskInfo("Task 2", 45L); timeTask(sw, task1); assertElapsedTime(sw, task1.getTimeMillis()); timeTask(sw, task2); assertElapsedTime(sw, task1.getTimeMillis() + task2.getTimeMillis()); } private void assertElapsedTime(StopWatch sw, int expectedDuration) { double tolerance = 10.0; assertThat((double)sw.getTotalTimeMillis(), is(closeTo((double)expectedDuration, tolerance))); } private void timeTask(StopWatch sw, StopWatch.TaskInfo task) throws Exception { sw.start(task.getTaskName()); Thread.sleep(task.getTimeMillis()); sw.stop(); }
  • public void testValidUsage() throws Exception { StopWatch sw = new StopWatch(); long int1 = 166L; long int2 = 45L; String name1 = "Task 1"; String name2 = "Task 2"; assertFalse(sw.isRunning()); sw.start(name1); Thread.sleep(int1); assertTrue(sw.isRunning()); sw.stop(); long fudgeFactor = 0L; assertTrue("Unexpected assertTrue("Unexpected sw.start(name2); Thread.sleep(int2); sw.stop(); assertTrue("Unexpected assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + fudgeFactor); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1 + int2); timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + int2 + fudgeFactor); assertTrue(sw.getTaskCount() == 2); String pp = sw.prettyPrint(); assertTrue(pp.indexOf(name1) != -1); assertTrue(pp.indexOf(name2) != -1); StopWatch.TaskInfo[] tasks = sw.getTaskInfo(); assertTrue(tasks.length == 2); assertTrue(tasks[0].getTaskName().equals(name1)); assertTrue(tasks[1].getTaskName().equals(name2)); sw.toString(); }
  • public void testTimeMultipleTasks() throws Exception { StopWatch sw = new StopWatch(); StopWatch.TaskInfo task1 = new StopWatch.TaskInfo("Task 1", 166L); StopWatch.TaskInfo task2 = new StopWatch.TaskInfo("Task 2", 45L); timeTask(sw, task1); assertElapsedTime(sw, task1.getTimeMillis()); timeTask(sw, task2); assertElapsedTime(sw, task1.getTimeMillis() + task2.getTimeMillis()); }
  • Readability Refactorings Short Test Methods Shorter Line Length Fewer Identifiers Clear Test Method Names Extract Given Extract Collaborator Use Meaningful Data Extract Test Replace Assertion