Plugin Testing

Plugin Testing

Thursday, October 22, 2009

Plugin Testing
her stuff
gins & ot
Plu


Why Test?

• Correctness
• Performance
• Knowing Your Code
• Documentation


Cost of Change
Development Feature Testing Regression Testing
Time

v1.0 v2.0 v3.0 v4.0


Feedback Loops

Development

JIRA Testing


Feedback Loops
Development

JIRA Testing


Development
Feedback Loops

JIRA Testing


Cost of Change
Development Writing Tests Executing Tests
Time

v1.0 v2.0 v3.0 v4.0


Eat Your Vegetables

Variety

http://www.ﬂickr.com/photos/sharontroy/3606495880/

Ingredients Matter


Ingredients + Technique
= Art


No Regrets
http://www.redﬂagdeals.com/forums/costco-west-deals-dove-nailer-lays-tuna-gain-pantene-734644/

Test-Driven Development
http://www.agiledata.org/essays/tdd.html

Types of Tests
Unit Integration


Unit Tests


What is Unit Testing?

• Tests individual components
• Avoids state & external systems
• Runs in development environment


Why Unit Test?

• Correctness
• Improve Design
• Design by Contract


How to Unit Test

• Set up
• Execute
• Verify
• Clean up


How to Unit Test an Atlassian Plugin


Unit Testing Tools


JUnit

public class UriTest {
@Test
public void absoluteUriParamIsReturned() {
URI absoluteUri =
URI.create("http://www.example.com");
URI resolvedUri = resolveUriAgainstBase(
"http://localhost:8080", absoluteUri)
assertEquals(absoluteUri, resolvedUri);
}
}


JUnit

@Test(expected=
GadgetSpecUriNotAllowedException.class)
public void blankLocationThrowsException() {
urlBuilder.parseGadgetSpecUrl(
BASE_GADGET_SPEC_PATH
+ PLUGIN_KEY
+ "/"
+ "");
}


Mockito
Dashboard dashboard = mock(Dashboard.class);
DashboardStore store =
mock(DashboardStore.class);
when(dashboard.getId())
.thenReturn(DASHBOARD_ID);
when(store.update(dashboard))
.thenReturn(DashboardStore.SUCCESS);

repository.save(dashboard);

verify(store)
.update(DASHBOARD_ID, dashboard);
verifyNoMoreInteractions(store);

Hamcrest

Iterable<ColumnIndex> range =
ColumnIndex.range(ColumnIndex.ZERO,
ColumnIndex.ONE);
Iterator<ColumnIndex> columnIt =
range.iterator();
assertThat(columnIt.next(),
is(equalTo(ColumnIndex.ZERO)));
assertThat(columnIt.next(),
is(equalTo(ColumnIndex.ONE)));
assertFalse(columnIt.hasNext());


Hamcrest
assertThat(iterator.next().getState(),
is(sameInstance(gadget.getState())));

assertThat(store.entries(),
hasItem(GADGET_SPEC_URI));

assertThat(staticContentTab.asText(),
containsString("Static Content"));

assertThat(changesList.get(1),
is(deeplyEqualTo(
new RemoveGadgetChange(gadgetId))));

Clover


Unit Testing Traps

• False Positives
• False Negatives
• Flapping


Non-Test
• Doesn’t test what it appears to test
• No assertion
• Passes when code is broken
• Solutions:
• Test-Driven Development
• IDE inspections, PMD, FindBugs

Missing Test
• Doesn’t test full range of inputs
• Boundary conditions, special cases
• Solutions:
• Clover
• JUnit Theories

Coverage as a Crutch
• Over-reliance on Clover
• Misses code paths, boundary cases
• Solutions:
• Your brain
• Peer review

Side-Effects
• Produces or relies on side-effects
• Maintains state between runs
• Fails inconsistently
• Solutions:
• Mock objects
• @Before/@After

Over Constraint
• Tests implementation details
• Overuse of mocks
• Fails when code is not broken
• Solutions:
• Stubs/fakes
• Refactoring

Unit Testing Traps

• False Positives ←More tests
• False Negatives ←Reduce coupling
• Flapping ←Isolate


Integration Tests


What is Integration
Testing?

• Tests full system
• Tests each supported conﬁguration
• Runs in simulated production environment
• Many types: functional, load, stress, etc.


Why Integration Test?

• Correctness
• Robustness
• Performance


How to Integration Test

• Set up
• Execute
• Verify
• Clean up


How to Integration Test
a Web Application
• Deploy your plugin or application
• Load data
• Simulate or script a web browser
• Parse the HTML to verify results
• Clean up


How to Integration Test an
Atlassian Plugin

Integration Testing Tools


JWebUnit

beginAt("/home");
clickLink("login");
assertTitleEquals("Login");
setTextField("username", "test");
setTextField("password", "test123");
submit();
assertTitleEquals("Welcome, test!");


HtmlUnit
WebClient webClient = new WebClient();
HtmlPage page =
webClient.getPage("http://example.net");
HtmlDivision div =
page.getHtmlElementById("some_div_id");
HtmlAnchor anchor =
page.getAnchorByName("anchor_name");
List<?> divs = page.getByXPath("//div");
HtmlDivision div =
page.getByXPath("//div[@name='John']")
.get(0);


JIRA/Conﬂuence Test
Frameworks
PageHelper helper = getPageHelper();

helper.setSpaceKey(spaceKey);
helper.setParentId(parentId);
helper.setTitle(title);
helper.setContent(content);
helper.setCreationDate(new Date());
helper.setLabels(labels);
assertTrue(helper.create());


JIRA/Conﬂuence Test
Frameworks
addUser("john.doe");
addUserToGroup("john.doe", JIRA_DEV);
activateTimeTracking();
String issueKey =
addIssue(TEST_PROJECT_NAME,
TEST_PROJECT_KEY,
"Bug", "First Bug", "Major", null,
null, null, ADMIN_USERNAME, null,
null, "1w", null, null);
logWorkOnIssue(issueKey,"1d");
gotoProjectBrowse(TEST_PROJECT_KEY);


Atlassian Plugin SDK
$ atlas-unit-test
$ atlas-clover
$ atlas-integration-test
--version
--container
--product


Selenium


Integration Testing Traps

• False Positives
• False Negatives
• Flapping


Incomplete
• Doesn’t test the full system
• JavaScript disabled, ignores browsers or app servers
• Solutions:
• HtmlUnit
• Selenium
• Cargo
• atlas-integration-test --container


Entangled
• Each test covers too much functionality
• Requires lots of setup to test each feature
• Bugs cause failures in tests for other features
• Solutions:
• Data loaders
• Remote APIs
• More speciﬁc test methods
• Domain-driven test frameworks


Fragile
• Tests break due to UI changes
• Screen-scraping, complicated XPath
• Fails when not broken
• Solutions:
• Semantic markup (class and id attributes)
• Domain-driven test frameworks

Timing or State
Dependencies
• Inconsistent results from one test run to another
• Concurrency bugs, no clean up, environment issues
• Solutions:
• @Before/@After
• @BeforeClass/@AfterClass
• Continuous Integration
• Blood, sweat, and tears


Slow
• Tests take too long to run
• Increases length of feedback loop
• Solutions:
• Clover test optimization
• Maven proﬁles
• Bamboo build trees

Build Trees
Clover-Optimized Faster

Unit Tests

All Unit Tests All Unit Tests
Java 5 Java 6 Slower

Integration Integration
Java 5/Tomcat 6 Java 6/Tomcat 6

Integration Integration Integration Integration
Java 5 Java 5 Java 6 Java 6
JBoss Tomcat 5.5 JBoss Tomcat 5.5

Integration Testing Traps

• False Positives ←Test more
• False Negatives ←Framework
• Flapping ←Get to work


Resources
http://j.mp/plugin-testing


Plugin Testing

Recommended

Recommended

More Related Content

Similar to Plugin Testing

Similar to Plugin Testing (20)

Recently uploaded

Recently uploaded (20)

Plugin Testing