Plugin Testing

1,531 views

Published on

A talk from AtlasCamp 2009.

Published in: Technology
1 Comment
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total views
1,531
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
5
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide

Plugin Testing

  1. 1. Plugin TestingThursday, October 22, 2009
  2. 2. Plugin TestingThursday, October 22, 2009
  3. 3. Plugin Testing her stuff gins & ot PluThursday, October 22, 2009
  4. 4. Why Test? • Correctness • Performance • Knowing Your Code • DocumentationThursday, October 22, 2009
  5. 5. Cost of Change Development Feature Testing Regression Testing Time v1.0 v2.0 v3.0 v4.0Thursday, October 22, 2009
  6. 6. Feedback Loops Development JIRA TestingThursday, October 22, 2009
  7. 7. Feedback Loops Development JIRA TestingThursday, October 22, 2009
  8. 8. Development Feedback Loops JIRA TestingThursday, October 22, 2009
  9. 9. Cost of Change Development Writing Tests Executing Tests Time v1.0 v2.0 v3.0 v4.0Thursday, October 22, 2009
  10. 10. Eat Your VegetablesThursday, October 22, 2009
  11. 11. Varietyhttp://www.flickr.com/photos/sharontroy/3606495880/Thursday, October 22, 2009
  12. 12. Ingredients Matterhttp://www.flickr.com/photos/sharontroy/3938831210/Thursday, October 22, 2009
  13. 13. Ingredients + Technique = Arthttp://www.flickr.com/photos/sharontroy/3606264108/Thursday, October 22, 2009
  14. 14. No Regretshttp://www.redflagdeals.com/forums/costco-west-deals-dove-nailer-lays-tuna-gain-pantene-734644/Thursday, October 22, 2009
  15. 15. Test-Driven Developmenthttp://www.agiledata.org/essays/tdd.htmlThursday, October 22, 2009
  16. 16. Types of Tests Unit IntegrationThursday, October 22, 2009
  17. 17. Unit TestsThursday, October 22, 2009
  18. 18. What is Unit Testing? • Tests individual components • Avoids state & external systems • Runs in development environmentThursday, October 22, 2009
  19. 19. Why Unit Test? • Correctness • Improve Design • Design by ContractThursday, October 22, 2009
  20. 20. How to Unit Test • Set up • Execute • Verify • Clean upThursday, October 22, 2009
  21. 21. How to Unit Test an Atlassian PluginThursday, October 22, 2009
  22. 22. Unit Testing ToolsThursday, October 22, 2009
  23. 23. JUnit public class UriTest { @Test public void absoluteUriParamIsReturned() { URI absoluteUri = URI.create("http://www.example.com"); URI resolvedUri = resolveUriAgainstBase( "http://localhost:8080", absoluteUri) assertEquals(absoluteUri, resolvedUri); } }Thursday, October 22, 2009
  24. 24. JUnit @Test(expected= GadgetSpecUriNotAllowedException.class) public void blankLocationThrowsException() { urlBuilder.parseGadgetSpecUrl( BASE_GADGET_SPEC_PATH + PLUGIN_KEY + "/" + ""); }Thursday, October 22, 2009
  25. 25. Mockito Dashboard dashboard = mock(Dashboard.class); DashboardStore store = mock(DashboardStore.class); when(dashboard.getId()) .thenReturn(DASHBOARD_ID); when(store.update(dashboard)) .thenReturn(DashboardStore.SUCCESS); repository.save(dashboard); verify(store) .update(DASHBOARD_ID, dashboard); verifyNoMoreInteractions(store);Thursday, October 22, 2009
  26. 26. Hamcrest Iterable<ColumnIndex> range = ColumnIndex.range(ColumnIndex.ZERO, ColumnIndex.ONE); Iterator<ColumnIndex> columnIt = range.iterator(); assertThat(columnIt.next(), is(equalTo(ColumnIndex.ZERO))); assertThat(columnIt.next(), is(equalTo(ColumnIndex.ONE))); assertFalse(columnIt.hasNext());Thursday, October 22, 2009
  27. 27. Hamcrest assertThat(iterator.next().getState(), is(sameInstance(gadget.getState()))); assertThat(store.entries(), hasItem(GADGET_SPEC_URI)); assertThat(staticContentTab.asText(), containsString("Static Content")); assertThat(changesList.get(1), is(deeplyEqualTo( new RemoveGadgetChange(gadgetId))));Thursday, October 22, 2009
  28. 28. CloverThursday, October 22, 2009
  29. 29. Unit Testing Traps • False Positives • False Negatives • FlappingThursday, October 22, 2009
  30. 30. Non-Test • Doesn’t test what it appears to test • No assertion • Passes when code is broken • Solutions: • Test-Driven Development • IDE inspections, PMD, FindBugsThursday, October 22, 2009
  31. 31. Missing Test • Doesn’t test full range of inputs • Boundary conditions, special cases • Passes when code is broken • Solutions: • Clover • JUnit TheoriesThursday, October 22, 2009
  32. 32. Coverage as a Crutch • Over-reliance on Clover • Misses code paths, boundary cases • Passes when code is broken • Solutions: • Your brain • Peer reviewThursday, October 22, 2009
  33. 33. Side-Effects • Produces or relies on side-effects • Maintains state between runs • Fails inconsistently • Solutions: • Mock objects • @Before/@AfterThursday, October 22, 2009
  34. 34. Over Constraint • Tests implementation details • Overuse of mocks • Fails when code is not broken • Solutions: • Stubs/fakes • RefactoringThursday, October 22, 2009
  35. 35. Unit Testing Traps • False Positives ←More tests • False Negatives ←Reduce coupling • Flapping ←IsolateThursday, October 22, 2009
  36. 36. Integration TestsThursday, October 22, 2009
  37. 37. What is Integration Testing? • Tests full system • Tests each supported configuration • Runs in simulated production environment • Many types: functional, load, stress, etc.Thursday, October 22, 2009
  38. 38. Why Integration Test? • Correctness • Robustness • PerformanceThursday, October 22, 2009
  39. 39. How to Integration Test • Set up • Execute • Verify • Clean upThursday, October 22, 2009
  40. 40. How to Integration Test a Web Application • Deploy your plugin or application • Load data • Simulate or script a web browser • Parse the HTML to verify results • Clean upThursday, October 22, 2009
  41. 41. How to Integration Test an Atlassian PluginThursday, October 22, 2009
  42. 42. Integration Testing ToolsThursday, October 22, 2009
  43. 43. JWebUnit beginAt("/home"); clickLink("login"); assertTitleEquals("Login"); setTextField("username", "test"); setTextField("password", "test123"); submit(); assertTitleEquals("Welcome, test!");Thursday, October 22, 2009
  44. 44. HtmlUnit WebClient webClient = new WebClient(); HtmlPage page = webClient.getPage("http://example.net"); HtmlDivision div = page.getHtmlElementById("some_div_id"); HtmlAnchor anchor = page.getAnchorByName("anchor_name"); List<?> divs = page.getByXPath("//div"); HtmlDivision div = page.getByXPath("//div[@name=John]") .get(0);Thursday, October 22, 2009
  45. 45. JIRA/Confluence Test Frameworks PageHelper helper = getPageHelper(); helper.setSpaceKey(spaceKey); helper.setParentId(parentId); helper.setTitle(title); helper.setContent(content); helper.setCreationDate(new Date()); helper.setLabels(labels); assertTrue(helper.create());Thursday, October 22, 2009
  46. 46. JIRA/Confluence Test Frameworks addUser("john.doe"); addUserToGroup("john.doe", JIRA_DEV); activateTimeTracking(); String issueKey = addIssue(TEST_PROJECT_NAME, TEST_PROJECT_KEY, "Bug", "First Bug", "Major", null, null, null, ADMIN_USERNAME, null, null, "1w", null, null); logWorkOnIssue(issueKey,"1d"); gotoProjectBrowse(TEST_PROJECT_KEY);Thursday, October 22, 2009
  47. 47. Atlassian Plugin SDK $ atlas-unit-test $ atlas-clover $ atlas-integration-test --version --container --productThursday, October 22, 2009
  48. 48. SeleniumThursday, October 22, 2009
  49. 49. Integration Testing Traps • False Positives • False Negatives • FlappingThursday, October 22, 2009
  50. 50. Incomplete • Doesn’t test the full system • JavaScript disabled, ignores browsers or app servers • Passes when code is broken • Solutions: • HtmlUnit • Selenium • Cargo • atlas-integration-test --containerThursday, October 22, 2009
  51. 51. Entangled • Each test covers too much functionality • Requires lots of setup to test each feature • Bugs cause failures in tests for other features • Solutions: • Data loaders • Remote APIs • More specific test methods • Domain-driven test frameworksThursday, October 22, 2009
  52. 52. Fragile • Tests break due to UI changes • Screen-scraping, complicated XPath • Fails when not broken • Solutions: • Semantic markup (class and id attributes) • Domain-driven test frameworksThursday, October 22, 2009
  53. 53. Timing or State Dependencies • Inconsistent results from one test run to another • Concurrency bugs, no clean up, environment issues • Solutions: • @Before/@After • @BeforeClass/@AfterClass • Continuous Integration • Blood, sweat, and tearsThursday, October 22, 2009
  54. 54. Slow • Tests take too long to run • Increases length of feedback loop • Solutions: • Clover test optimization • Maven profiles • Bamboo build treesThursday, October 22, 2009
  55. 55. Build Trees Clover-Optimized Faster Unit Tests All Unit Tests All Unit Tests Java 5 Java 6 Slower Integration Integration Java 5/Tomcat 6 Java 6/Tomcat 6 Integration Integration Integration Integration Java 5 Java 5 Java 6 Java 6 JBoss Tomcat 5.5 JBoss Tomcat 5.5Thursday, October 22, 2009
  56. 56. Integration Testing Traps • False Positives ←Test more • False Negatives ←Framework • Flapping ←Get to workThursday, October 22, 2009
  57. 57. Resources http://j.mp/plugin-testingThursday, October 22, 2009

×