Your SlideShare is downloading. ×
Continuous Integration using Hudson and Fitnesse at Ingenuity Systems (Silicon Valley code camp 2011)
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Continuous Integration using Hudson and Fitnesse at Ingenuity Systems (Silicon Valley code camp 2011)

5,768
views

Published on

Continuous Integration using Hudson and Fitnesse …

Continuous Integration using Hudson and Fitnesse

Speaker: Vasu Durgavarjhula , Jennifer Wong , Norman Boccone
Level: Intermediate | Room: 4221 | 11:15 AM Saturday


Learn about Continuous Integration (CI) and Continuous Deployment(CD) and how Ingenuity Systems moved from a traditional release process to a more agile frequent release model. In this talk we will discuss specifics and show demos on:

using Hudson as a framework for continuous integration, deployment, and build promotion
deployment and configuration management
changes we made to make our architecture more service-oriented
our automated test strategy using JUnit, FitNesse, and Selenium
migrating our build and deployment process from Ant to Maven
challenges to overcome and lessons learned in implementing a successful CI system

Published in: Business, Technology

8 Comments
6 Likes
Statistics
Notes
  • Our main ci hudson server has a number of slaves and administers the process from the point of building the code to the point of deployment to Stable (ie, it covers the promotion levels dev, ci, stable, beta, rc). The same hudson server/cluster is used for all software products and services that participate in the continuous integration process.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • So where is/are the Hudson servers? Does each quality gate server has its own, or is there a master one, moving tested software to the next quality gate? Also, the Fitness server/s?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @Dennis: Our Beta system is used for beta testing of new features. Sometimes the same builds of services or products might get deployed to both Beta and production systems, but when it's a customer-facing app (as opposed to a backend service), for the most part this system is used to get feedback from early access users. We then use that feedback to make final changes and polish up the features that we push to production. So, if you're talking about the actual builds, often the Beta flow path ends in Beta.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hi Martin,
    The horizontal view on slide 62 is an in-house plugin. I'm currently working on management and the developer who wrote it to get them to make it publicly available, I'll do my best to hurry up its release :)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • 2 things.
    A/ I read where Jenkins is becoming the replacement for Hudson, by community interest and investment. So Installed it, and it literally REPLACED AND UNINSTALLED Hudson with no wizard questions or anything. Guess it *IS* the replacement, LOL!
    B/ Where does the 'Beta' flow path go? It branches out of 'Stable' and then what?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
5,768
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
96
Comments
8
Likes
6
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Outline for this deck:Who we areWhat challenge we are addressing (high level)Our platform = Ingenuity Knowledge Base Content (3 slides) Ontology (1 slide)Products and Solutions Overview Research and Analysis Solutions The challenge IPA addresses IPA overview The challenge Ingenuity Answers addresses Additional Solutions eCommerce EnterpriseWhat Sets Ingenuity Apart (USPs)
  • What is the challenge Ingenuity is addressing? Speaking points:Helping researchers access detailed and high-quality knowledge and then USING it to design experiments, generate hypothesis, answer targeted questions, and analyze dataAll of Ingenuity's products and services have one common goal — to help life science researchers generate maximum value from all types of biological and chemical knowledge.Ingenuity offers a broad and flexible range of solutions that can be tailored to needs of various clients, including academic and therapeutic area researchers, computational biologists and informatics departments, and suppliers in the life sciences industry.
  • Speaking Points:What is it? [high level description]The Ingenuity Knowledge Base is the core technology of Ingenuity Systems Is the engine behind all of Ingenuity’s products – (IPA) It is a repository of biological interactions and functional annotations created from millions of individually modeled relationships between proteins, genes, complexes, cells, tissues, drugs, and diseasesThe IKB enables scientific understanding – through IPA and other services, researchers canDiscover biological relationships and functionsBuild testable hypothesesLastly, they can answer targeted biological questions – and get back relevant answersSo, what is unique about the IKB – what have we done differently - next slideInference example:For example – can I discover a molecular path connecting a compound known to treat mammary tumors to angiogenesis? Types of contextual detail:SpeciesSynonymsExperimental methodSite of post-translational modificationDirection of changeTissue contextCell line contextOriginal source
  • Speaking Points:How is our approach different? With the information we capture (publications, public knowledge, etc. - not JUST publications) we do THREE important things:We STRUCUTURE the information using the Ingenuity Ontology – statements are modeled into Findings. Findings are not just EXPERT REVIEW PROCESS We capture a lot of detail and ensure it is correct.The information we cover is COMPREHENSIVE – IKB covers and extensive body of biological and chemical info, including publications and third party sources.The information is TIMELY – weekly updates as of IPA 8 and IPA 8.5
  • Small teams empowered to release to production without management approvalBetter iteration and release planning. Learning to build and release features incrementally; show/hide features in betaFocus on Unit testing first and then integration testingStreamline the build environment to do automated builds, deployments and build promotionRefactor the code base to create more modular and service oriented code base
  • --title: --list items
  • --only one repository
  • --mention fingerprinting--all plugins in parent
  • --artifactsinhudson repo--nodpeloyment/test pohase other than junit--just build, not CI
  • --release master is the CI one--remove jenkinsrefernces
  • --cconfigurabel, difficult to maintain
  • --describe what puppet is--erb use--cofnigmgt standardized--moreenforcable
  • --standard vmconfig
  • --hudson build succesnoyl
  • --status app page --hudson
  • --service discovery mgr, router for services
  • Transcript

    • 1. Proprietary and Confidential
      Continuous Integration using Hudson and Fitnesse
      Norman Boccone | Lead Engineer | twitter: @dropslowly
      Jennifer Wong | Lead QE Engineer | twitter: @jenlwong
      Vasu Durgavarjhula | Director of Development | twitter: @d_vasu
    • 2. Proprietary and Confidential
      2
      Overview
      ► Who we are ?
      ► The Challenge
      ► Tools and Processes
      ► Test Strategy
      ► Conclusion
    • 3. Ingenuity Systems
      We are a leading provider of information and analytics solutions for life science researchers
      Our goal is to help life science researchers get biological insights from their data quickly and reliably
      Our products are used by thousands of researchers worldwide
      Partial Customer List
      Proprietary and Confidential
      3
    • 4. ?
      Scientists Need a Tool to Make Good Decisions From Complex Biological Data
      Toxicology
      Biomarkers
      Pharmacogenomics
      Discovery
      Use understanding of disease mechanisms to identify and validate targets
      Elucidate biological mechanisms of drug action and toxicity
      Identify and prioritize novel biomarkers by understanding role in disease pathways
      Understand mechanisms behind differential response to R(x)
      Hard to synthesize the picture of what that data means in a broader biological context, and how the pieces work together…
      Traditional publishing models are good for learning but not applying knowledge …
      Enormous volume and complexity of biological and chemical data…
      Cancer
      Publishing and Collaboration
      Experimental Data
      Search
      Proprietary and Confidential
      4
    • 5. The Ingenuity® Knowledge Base
      Is the core technology behind all of Ingenuity’s products and services
      THE INGENUITY KNOWLEDGE BASE
      ANALYSIS
      HYPOTHESIS GENERATION
      VISUALIZATION
      ECOMMERCE ENABLEMENT
      PATHWAY, REAGENT & GENE SEARCH
      ENABLES SCIENTIFIC UNDERSTANDING
      Discover existing relationships and function, as well as inference of relationships that may not have been studied in a particular context yet
      Generatetestable hypotheses, build pathways, ability to inference
      Get answers by asking detailed and in-depth scientific questions
    • 6. The Ingenuity® Knowledge Base
      How is it different?
      THE INGENUITY KNOWLEDGE BASE
      ► Structured-to capture relevant details
      Scientific statements are modeled into Findings using the Ingenuity Ontology
      ► Expert Review Process- for accuracy
      Findings go through QC process
      ► Comprehensive- leverage knowledge in one place
      Largest scientific knowledge base of its kind with modeled relationships between proteins, genes, complexes, cells, tissues, drugs, pathways and diseases
      ► Timely- for access to up-to-date knowledge
      Findings are added weekly
    • 7. Proprietary and Confidential
      The Challenge
      7
    • 8. Proprietary and Confidential
      8
      Single product
    • 9. Proprietary and Confidential
      9
      multiple products
      Ingenuity Variant Analysis
      Next Generation Sequencing Analysis Reports
      Ingenuity Product Search
      Reagent Portals for BD, Sigma, Thermo Fischer
    • 10. Proprietary and Confidential
      10
      Large monolithic code base
    • 11. Proprietary and Confidential
      11
      Modular Services
      Single Sign On
      User
      Management
      Views & Reports
      User Data
      Content
      Service
      iReport
      Master Graph
    • 12. Proprietary and Confidential
      12
      Long release cycles
    • 13. Proprietary and Confidential
      13
      Faster release cycles
    • 14. Proprietary and Confidential
      14
      Speeding up the feedback loop
      Developer
      Stakeholders
    • 15. Proprietary and Confidential
      15
      Do all of this while supporting the existing product
    • 16. Proprietary and Confidential
      16
      Approach
      Move to a Service Oriented Architecture
      Don’t miss Jeff’s talk on Sunday 10:45 am “Learning to Love Services: Experiences from the Transition to a Service-oriented Architecture”
      Develop a test strategy focused on automated testing
      Build a continuous integration system
      Adopt agile product management
      Improve infrastructure to create and deploy services
      Foster a Dev/Ops culture
    • 17. Proprietary and Confidential
      17
      Continuous Integration Workflow
      Application
      Bundle
      Deploy Application
      Run Fitnesse Tests
      (Nightly suite)
      Nightly Build
      (Clover)
      Run JUnit
      Tests
      publish summary
      Fitnesse
      Bundle
      Deploy Fitnesse
      publish
      publish
      Hudson Dashboard
      (JUnit, Fitnesse summary, Code Coverage)
      Fitnesse Wiki
      (Test history, Details, Test Case Management)
      Link
      SVN
      Commit
      (Test Cases)
    • 18. Proprietary and Confidential
      18
      Tools and Processes
    • 19. Proprietary and Confidential
      19
      Service Oriented Architecture
    • 20. Proprietary and Confidential
      20
      SOA: Before
      lots of modules, separate but dependent
      easy to develop on but also easy to add circular dependencies
      hard to keep track of purpose/value of all the modules
      product consisted of bunches of modules put together
      reference was typically to latest devcode
      to ensure "latest," individual modules of a product were rebuilt for every release
      junittests were run every time as well
      new product version meant build everything, test everything
      builds were slow
    • 21. Proprietary and Confidential
      21
      SOA: “Current”
      components: module grouped together.  Access through api module only
      less chance of circular dependencies
      easier to comprehend modules
      more flexibility to change code: just do not change api
      components built/tested separately
      faster builds
      less repetitive testing
      some shared modules still exist (they are not part of any component)
    • 22. Proprietary and Confidential
      22
      SOA: Conversion Notes
      find a logical group of modules
      Convert groupput group into hierarchical structure, hide things behind an apilayer
      convert outside modules to use apis
      see Jeff's talk
    • 23. Proprietary and Confidential
      23
      Build System
    • 24. Proprietary and Confidential
      24
      Build System: Before
      in-house ant scripts
      lots of control but it required lots of resources
      no real industry standard
      global scripts/property files, overrides allowed
      offered consistency but (busy) people would override instead of follow pattern
      scripts got complicated
      anyone could add a new feature
      if anyone wanted a new feature, just tell them to add it
      inefficient additions, different code styles made scripts hard to read
      very little dependency management
      multiple repositories for binary files
    • 25. Proprietary and Confidential
      25
      Build System: “Current”
      maven
      not much control, and very rigid  (a good thing, in a way)
      steep learning curve
      different from previous way of doing things
      standard structure; difficult to not follow structure
      hierarchical structure easily followed
      versions for all dependencies well defined
      easy to add new features (after a short burst of intense agony)
      only one repository
    • 26. Proprietary and Confidential
      26
      Build System: Conversion Notes
      get a maven repository manager (nexus? Turn on search!)
      add antrun plugin to publish artifacts to old system
      choose a simple component to start, make lots of little submodules(1 module per artifact, plus assembly)
      manually add needed dependencies to repository
      build up company parent pom, use hierarchy
      Put all plugin management into parent
      Key maven plugins: buildnumber, buildhelper, release, assembly,
    • 27. Proprietary and Confidential
      27
      Build System: TODO
      cleaner release plugin
      continue learning maven
      contribute to plugins
      Improve internal maven FAQ
      Look into git
    • 28. Proprietary and Confidential
      28
      Continuous Builds
    • 29. Proprietary and Confidential
      29
      Continuous Builds: Before
      Hudson (no plugins)
      auto build start for each module, with artifacts published
      manual build start for products
      shell scripts to copy artifacts to release dir
    • 30. Proprietary and Confidential
      30
      Continuous Builds: “Current”
      Hudson
      devmaster for auto build of modules/submodules, auto published to repository manager (dev area)
      release master for components/products, auto published to repository manager (release area)
      modules/submodulesbuilt on checkin
      components/products built on schedule or on demand
      process includes various tests and deployments (Jen)
    • 31. Proprietary and Confidential
      31
      Continuous Builds: Conversion Notes
      use Jenkins, not Hudson (more activity for Jenkins)
      startup new, clean servers
      copy/create job that we want
      Try to make Jenkins be the portal to everything
      fingerprinting
      Plugins we use: clover, fitnesse, findbugs, checkstyle, disk-usage, downstream build view, (custom) dashboard view, promoted builds, ssh slaves
    • 32. Proprietary and Confidential
      32
      Continuous Builds: TODO
      better job rollup
      job templates
      more maven/hudson plugins
      submit hudson plugin(s) to community
    • 33. Proprietary and Confidential
      33
      Deployment: Before
      proprietary shell/perl scripts
      targeted to install app only (not machine set up)
      multiple proprietary property config scripts
    • 34. Proprietary and Confidential
      34
      Deployment: “Current”
      Puppet
      Configures machines and apps
      Work on our machines and EC2
      Use of .erb files everywhere for property configuration
    • 35. Proprietary and Confidential
      35
      Deployment: Conversion Notes
      modify tar format
      moved to one tomcat
      used erb files
      split properties into app
    • 36. Proprietary and Confidential
      36
      Deployment: TODO
      standard server configuration types (e.g. “build machine”, “app machine”, …)
      more puppet scripts
    • 37. Proprietary and Confidential
      37
      Visibility
    • 38. Proprietary and Confidential
      38
      Visibility: Before
      cfmap (IT server diagnostic tool)
      Hudson status page
      FitNesse server
    • 39. Proprietary and Confidential
      39
      Visibility: “Current”
      cfmap
      Hudson status page
      Hudson dashboard
      extra tests built in to Hudson
      FitNesse server
      Service status page
    • 40. Proprietary and Confidential
      40
      Visibility: Conversion Notes
      use Hudson plugins, with drill down
      get everything to report in a standard way (junit xml)
    • 41. Proprietary and Confidential
      41
      Visibility: TODO
      more widgets for the dashboard
      Integrate outside reports into dashboard
      cfmap
      status page
      sdm? (service discovery manager)
    • 42. Proprietary and Confidential
      42
      Test Strategy
    • 43. Proprietary and Confidential
      43
      Testing For CI/CD
      Test Automation is key
      How and what to automate
      Mezaros
      Cohn
      Crispin
    • 44. Proprietary and Confidential
      44
      Test Challenges
      Testability: some products are difficult to test at lower levels
      Legacy Apps: one of our main products was a ‘Legacy App’. Tests used to look like this:
    • 45. Proprietary and Confidential
      45
      Test Challenges
      varying levels of coverage
      some of our newer components have great coverage
      other components have lower coverage: legacy, proof of concept
    • 46. Proprietary and Confidential
      46
      Solutions
      Test and code refactoring
      Legacy Apps: Strangulation approach
      automate new and refactored features
      incremental work on tests: reserve time in each iteration for adding to tests
      Change in culture: team ownership of tests and status
      Cycle time for ipa was 2.5 weeks
      Now for most of our services, it is 20-40 minutes
    • 47. Proprietary and Confidential
      47
      Test Infrastructure and Environments
    • 48. Proprietary and Confidential
      48
      Process: Build Promotion
      Build Promotion through Quality Gates
      Beta
      • Deployed to Beta
    • Proprietary and Confidential
      49
      Test Environments
      Stable Environment
      • Multiple nodes
      • 49. Deployment Tests
      • 50. System and performance tests
      • 51. Manual/exploratory
      CI Environment
      • Single nodes
      • 52. functional tests
      Service1
      Service1
      Service2
      Service2
      Service3
      Service3
      Service4
      Service4
      Service5
      Service5
      Service6
    • 53. Proprietary and Confidential
      50
      Test Types
    • 54. Proprietary and Confidential
      51
      Test Types: Deployment and Health
      Each of our components has a built-in status page
    • 55. Proprietary and Confidential
      52
      Test Types: Deployment and Health
      Status page
      Reflects app status
      Resource availability: DB Connections, Services
    • 56. Proprietary and Confidential
      53
      Test Types: Deployment and Health
      Information is used for:
      Deployment
      Health Monitoring
      Service Compatibility and Dependency checking
    • 57. Proprietary and Confidential
      54
      Test Types: FitNesse
      FitNesse is a wiki-based web server test tool
      Helps abstract test definition from technical implementation
      Provides visual reporting and result history tracking
    • 58. Proprietary and Confidential
      55
      Test Types: FitNesse
      The FitNesse Server
      http://localhost:8080/FitNesse.UserGuide.TwoMinuteExamplehttp://localhost:8080/FitNesse.UserGuide.TwoMinuteExample?pageHistory
      Wiki view Editing the wiki Test Results
    • 59. Proprietary and Confidential
      56
      Test Types: FitNesse
      • Integration with Hudson
      - Views, drilldown, configuration
      • Test results converted to junit format
      View of fitnesse test job in Hudson
    • 60. Proprietary and Confidential
      57
      Drill downs on fitnesse test results in Hudson
    • 61. Proprietary and Confidential
      58
      FitNessePlugin Configuration
      Convert results to Junit format
    • 62. Proprietary and Confidential
      59
      Publish FitNesse Results and Coverage
    • 63. Proprietary and Confidential
      60
      Test Types: FitNesse
      • What we use it for:
      • 64. Functional tests
      • 65. Integration tests
      • 66. UI Tests (com.jbergin.HtmlFixture)
      • 67. DB Tests (dbfit)
      What we’ve done with it that is different
      Use as execution framework for more complex tests
      Extension of fitnesse server for data-driven tests
      json fixture – pass in javascript
      Execution of Selenium tests
    • 68. Proprietary and Confidential
      61
      Test Types: FitNesse
      Lessons learned
      FitNesse is good for straightforward verification of data
      To do more, you have to get creative
      Fixture and test ownership needs to be a shared responsibility
    • 69. Proprietary and Confidential
      62
      Test Types: Selenium
      We fire off Selenium tests in two ways: via Hudson job or through FitNesse.
      • Test results converted to junit format for display in hudson
      Hudson job in dashboard:
      Hudson job view:
    • 70. Proprietary and Confidential
      63
      Test Types: Selenium
      Hudson job configuration: trigger execution on Selenium grid using Ant
    • 71. Proprietary and Confidential
      64
      Test Types: Selenium
      Second way we fire off Selenium tests is using a FitNesse fixture: WebTest
    • 72. Proprietary and Confidential
      65
      Test Types: Backward Compatibility
      No staging environment
      How do we know that when we release a new service version to prod that it won’t break?
      Service version compatibility
      Leverage Service Discover Manager (SDM) and Status pages to check for availability of services
      Jar compatibility
      static code analysis could detect changes in method signature but not underlying object or serialization changes
    • 73. Proprietary and Confidential
      66
      What’s Next: Getting to the End of the Road
    • 74. Proprietary and Confidential
      67
      What’s Next
      What do we need to do to get to the end of the path to Continuous Integration and Deployment?
      Continue to invest in standardized tools and processes
      Maven
      Puppet
      Jenkins
      Metrics and Immune Systems
      • Continue to improve our test coverage and practices
      • 75. Culture:
      Continue to support, reinforce, educate
    • 76. Proprietary and Confidential
      68
      The Finish Line
      Continuous deployment
      Reduced Cycle Time
      Improved quality
      Predictable metrics
      Reduced risk
    • 77. Proprietary and Confidential
      69
      Slides are posted here: http://www.slideshare.net/jenlwong/ingenuity-svcc-ci-presentation-20111007
      Plugs
      Learning to Love Services: Experiences from the Transition to a Service-oriented Architecture
      Speaker: Jeff Green 10:45 AM Sunday
      Ingenuity is hiring: http://www.ingenuity.com/company/careers.html
      Q&A
      References
      Todd. Test Automation Pyramid – review. Retrieved September 29, 2011 from: http://blog.goneopen.com/2010/08/test-automation-pyramid-review/
      Humble, Jez, and Farley, David. Continuous Delivery. Boston: Pearson Education Inc, 2011. Print.
      Crispin, Lisa, and Gregory, Janet. Agile Testing: A Practical Guide for Testers and Agile Teams. Boston: Pearson Education Inc, 2011. Print.

    ×