More on Fitnesse and Continuous Integration (Silicon Valley code camp 2012)

2,368 views
2,137 views

Published on

FitNesse is a wiki-based software testing tool that can be a powerful addition to your Continuous Integration Environments. Its greatest advantages include providing visibility into tests and results, and providing access to test-writing by non-technical team members. We will:
* look at specific examples and code,
* discuss the advantages and drawbacks of using FitNesse as a test framework
* implement, deploy, and use a simple fixture in a fitnesse test
* review different kinds of fixtures, including decision table, script, query, html, and selenium webtest fixtures
* discuss some of the more interesting fixture extensions we've implemented, including JSON-based verification and the ability to pass in javascript code for dynamic verification
* use Hudson/Jenkins to run your FitNesse tests as a step in your Continuous Integration/Deployment process

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,368
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
35
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Outline for this deck:Who we areWhat challenge we are addressing (high level)Our platform = Ingenuity Knowledge Base Content (3 slides) Ontology (1 slide)Products and Solutions Overview Research and Analysis Solutions The challenge IPA addresses IPA overview The challenge Ingenuity Answers addresses Additional Solutions eCommerce EnterpriseWhat Sets Ingenuity Apart (USPs)
  • More on Fitnesse and Continuous Integration (Silicon Valley code camp 2012)

    1. 1. Fitnesse and Continuous Integration Jennifer Wong | Staff SQE Engineer | twitter: @jenlwong Proprietary and Confidential
    2. 2. Overview►Intro► What is FitNesse (Not an exercise program!)► The Details► Integration: putting the pieces together► Lessons Learned Proprietary and Confidential
    3. 3. Intro• Who: – Jennifer Wong – Scrum Master for Tools Team, Staff SQE Engineer• Where: – Ingenuity Systems: A leading provider of information and analytics solutions for life science researchers• What: – FitNesse as a key element of Continuous Integration workflow• Why: – Follow up to last year’s presentation (http://www.slideshare.net/jenlwong/ingenuity-svcc-ci-presentation-20111007 ) Proprietary and Confidential
    4. 4. FitNesse(Not an exercise program!) Proprietary and Confidential
    5. 5. FitNesse• FitNesse is a wiki-based web server test tool – Helps abstract test definition from technical implementation – Provides visual reporting and result history tracking Proprietary and Confidential
    6. 6. Test Types: FitNesse• We use it for: – Integration tests – Acceptance and Functional tests – UI Tests (com.jbergin.HtmlFixture, webtest) – DB Tests (dbfit) – Backward Compatibility tests• What is it good for? – Framework and visibility – accessibility to non-technical people• What is it bad for? – Unit tests – Complicated ui tests – Extensive performance testing Proprietary and Confidential
    7. 7. FitNesse: The Details Proprietary and Confidential
    8. 8. Downloading and Installing FitNesse• Get the jar file and run it – http://fitnesse.org/FrontPage.FitNesseDevelopment.DownLoad – java –jar fitnesse.jar• Or use the demo package: • Get the file: on the svcc web site, attached to the session, or on Github • unzip the file and cd to lib dir • start the fitnesse server: java -jar fitnesse.jar -p 8080 • open web browser and access http://localhost:8080 • click the link at the top for "CodeCampDemoPage“ • click the test button Proprietary and Confidential
    9. 9. Setting your classpath• Classpath statements – Fitnesse needs to know where to find your fixture code• This kind of error :• Means you need to add this kind of statement: !path – You should usually have this kind of stuff: !contents !path fitnesse.jar !path C:/eclipse/fit_demo/eclipse-bin !define TEST_SYSTEM {slim} Proprietary and Confidential
    10. 10. Some Basic Test Tables• Script table – Good for procedural/workflow tests – Flexible – Use syntax: check, reject, ensure, show• Decision table – Good for data-driven tests – Specific workflow • Input methods  execute()  output methods – Special method name interpretation – Automatically calls reset() and execute() methods• Query table – Good for validating lists or tables of data Proprietary and Confidential
    11. 11. Variable Usage• Defining a static variable – !define ROOT_URL {http://myserver.com} – !define TESTUSER {testuser1@something.com}• Storing a value in a variable on the fly – Store variable: $X= – Use variable: $X• Using a variable – To use this variable, enclose the variable name in ${ } – Example: • this: ${URL_ROOT}/context/index.html • Will resolve to this: http://myserver.com/context/index.html Proprietary and Confidential
    12. 12. Naming and Parameter passing (Methods)• (Un) Graceful Naming – Automatically concatenates space-separated words • isHalloween  isHalloween() • Is Halloween  isHalloween() • is halloween  isHalloween() • Is halloWeen  error – When using methods with multiple parameters, tries to intersperse method name and paremeters Proprietary and Confidential
    13. 13. Parameter passing (cont)• Multi-parameter methods: isHalloween(int, String) – |ensure|is|31|Halloween|October| – |ensure|is Halloween|31||October|• Single parameter: setCostume(String) – |set costume|Clark Kent| – |set|Clark Kent|costume|• Constructors with parameters – This constructor: • public Halloween(String month, int day, String costume) – Translates to this usage in a fitnesse table: |Halloween|October|31|Cat| |isHalloween?|get surprise?| Proprietary and Confidential
    14. 14. UI Test Fixtures• com.jbergin.HtmlFixture – an adapter between FitNesse and HtmlUnit for use in testing web applications – Need to use !define TEST_SYSTEM {fit} – http://htmlfixture.sourceforge.net/ – http://uebuild5:8084/FrontPage.UmaFitNesse.IngsecuritySuite.ConcurrentUserS essionTest.AcceptanceTests• webtest selenium – an extension to FIT/FitNesse that uses Selenium Remote Control. WebTest runs inside FitNesse. – http://www.fitnesse.info/webtest – http://uebuild5:8084/FrontPage.ReportsFitNesse.IsoformView.IsoformVi ewWebTestSuite.IsoformViewWebTests Proprietary and Confidential
    15. 15. Real world usage is more complex• What it looks like in the real (ie, complicated)world – Session handling • http://uebuild5.ingenuity.com:8084/FrontPage.UmaFitNesse.IngsecuritySuite.Concurren tUserSessionTest.AcceptanceTests – Static objects to provide data access • http://uebuild5.ingenuity.com:8084/FrontPage.ContentserviceFitNesse.TestSuiteForCurrentContent.Ec sMappingDataProviderTestSuite.P1Tests – Complex checking of validity • Unmarshaling JSON to check special conditions in a non order dependent way – http://uebuild5.ingenuity.com:8084/FrontPage.FaFitNesse.Test SuiteForBaselineContentSpecific.FaProviderTestSuite.Execute FaQuery.LfaQueryTestSuite.AcceptanceTests Proprietary and Confidential
    16. 16. Fixture code can get complicated very quicklypublic static boolean matchGFAResult(JSONObject jsonResult, GFAResult actualResult, boolean allowSubset,boolean allowPvalueVerification,boolean geneCountVerification ,boolean allowZscoreVerification, boolean allowGeneEffectVerification) throws JSONException {JSONArray jsonItems = jsonResult.getJSONArray("items");logger.info("expected item size = " + jsonItems.length());logger.info("actual item size = " + actualResult.getFAResultItems().size());if (jsonItems.length() > actualResult.getFAResultItems().size()) {return false;}Map<String, GFAResultItem> itemMap = buildGFAResult(jsonItems);if (allowSubset) {for (Map.Entry<String, GFAResultItem> entry : itemMap.entrySet()) {logger.info("Look for " + entry.getKey() + " in actual result");GFAResultItem item = entry.getValue();if (!containsItem(item, actualResult.getFAResultItems(), allowSubset, allowPvalueVerification,geneCountVerification,allowZscoreVerification, allowGeneEffectVerification)) {logger.info(item.getId().getAsString() + " is expected but couldnt be found in actual result");return false;}}} else {return equalGFAItems(itemMap, actualResult.getFAResultItems(), allowSubset,allowPvalueVerification,geneCountVerification,allowZscoreVerification, allowGeneEffectVerification);}return true;}private static boolean equalGFAItems(Map<String, GFAResultItem> itemMap,Collection<GFAResultItem> actualResultItems, boolean allowSubset,boolean allowPvalueVerification,boolean geneCountVerification ,boolean allowZscoreVerification, boolean allowGeneEffectVerification) {if (itemMap.size() != actualResultItems.size()) {return false;} Proprietary and Confidential
    17. 17. Tips and tricks• Search in your FitNesse wiki• Use Includes – Use includes as templates – http://uebuild5.ingenuity.com:8084/FrontPage.IngtestFitNesse.StableSui te.FaStableCompat – http://uebuild5.ingenuity.com:8084/FrontPage.IngtestFitNesse.StableSui te.ContentserviceStableCompat• Comments• Escaping special characters – Start tables with ! to avoid unwanted interpretation of graceful names, etc – Surround special chars with !- -! • Example: !-gobbledeygook ~!@#$%^&*(){}| as plain string-! Proprietary and Confidential
    18. 18. Fancy fixtures and other nifty stuff• JSON – http://uebuild5:8084/FrontPage.MgFitNesse.TestSuiteForBaselineContentSpecific.GraphProviderTestSuite.GetNeighb orhoodGraph.P1Tests• Javascript validation – http://localhost:8080/FrontPage.AutocompFitNesse.FunctionalTests.FitTests.GeneralTests.P1Tests EVAL { void execute(Parse row, JSONFixture fixture) { Parse textCell = row.parts.more;// row.parts.more; String evalText = textCell.text(); String text = fixture.page.getWebResponse().getContentAsString(); try { jsEngine.eval("result = " + text + ";"); Object evaluationResult = jsEngine.eval(evalText); if (evaluationResult instanceof Boolean) { if ((Boolean)evaluationResult){ fixture.right(textCell);} else {fixture.wrong(textCell); }• Running tests based on tag • Include: http://<host>:<port>/<suite path and test name>?responder=suite&suiteFilter=smoke,critical • Exclude: http://<host>:<port>/<suite path and test name>?responder=suite&excludeSuiteFilter=NotRunningOnHudson Proprietary and Confidential
    19. 19. Test Variations• What we’ve done with it that is different – Use as execution framework for more complex tests – Extension of fitnesse server for data-driven tests – json fixture – pass in javascript – Execution of Selenium tests – Backwards Compatibility tests Proprietary and Confidential
    20. 20. Best practices• Test robustness• Test organization• Test readability• Fixture design – Tradeoff between flexibility and readability, usability Proprietary and Confidential
    21. 21. Integration: putting the pieces together Proprietary and Confidential
    22. 22. Our Environment• Multiple products (3 external, plus internal tools)• Services-based• Builds: – Produce multiple artifacts, including a fitnesse package – Example: A build of contentservice produces: • contentservice-1.2.179886.clover.tar.gz • contentservice-1.2.179886.tar.gz • contentservicedb-1.2.179886.tar.gz • contentservice_fitnesse-1.2.179886.tar.gz – Fitnesse package contains wiki page tests, libs, config files Proprietary and Confidential
    23. 23. FitNesse as part of our Continuous Integration Workflow Application Deploy Bundle Application Run FitnesseNightly Run Junit, Tests Build Javascript (Nightly suite)(Clover) Tests Fitnesse Deploy Bundle Fitnesse publish publish Hudson Dashboard Fitnesse Wiki Link (Test history, Details, (JUnit, Fitnesse summary, Code Coverage) Test Case Management) Commit SVN (Test Cases) Proprietary and Confidential
    24. 24. Integration with Hudson/Jenkins servers• Fitnesse plugin: Proprietary and Confidential
    25. 25. Lessons learned• Adds a lot of value for our team – Visibility into results and test history – Accessible to non-technical people – FitNesse is very good for visibility and straightforward verification of data• Not good for everything – Easy to do it wrong – Requires maintenance – Not as flexible• To do more, you have to get creative• Fixture and test ownership needs to be a shared responsibility Proprietary and Confidential
    26. 26. • Demo files will be posted to github under jwong-github• Slides are on slideshare• Demo and slides are attached to session• Q&A Proprietary and Confidential
    27. 27. The “As Seen By” Matrix Proprietary and Confidential

    ×