• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Continuous Integration using Hudson and Fitnesse at Ingenuity Systems (Silicon Valley code camp 2011)
 

Continuous Integration using Hudson and Fitnesse at Ingenuity Systems (Silicon Valley code camp 2011)

on

  • 5,312 views

Continuous Integration using Hudson and Fitnesse ...

Continuous Integration using Hudson and Fitnesse

Speaker: Vasu Durgavarjhula , Jennifer Wong , Norman Boccone
Level: Intermediate | Room: 4221 | 11:15 AM Saturday


Learn about Continuous Integration (CI) and Continuous Deployment(CD) and how Ingenuity Systems moved from a traditional release process to a more agile frequent release model. In this talk we will discuss specifics and show demos on:

using Hudson as a framework for continuous integration, deployment, and build promotion
deployment and configuration management
changes we made to make our architecture more service-oriented
our automated test strategy using JUnit, FitNesse, and Selenium
migrating our build and deployment process from Ant to Maven
challenges to overcome and lessons learned in implementing a successful CI system

Statistics

Views

Total Views
5,312
Views on SlideShare
5,311
Embed Views
1

Actions

Likes
6
Downloads
91
Comments
8

1 Embed 1

http://a0.twimg.com 1

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

18 of 8 previous next Post a comment

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Our main ci hudson server has a number of slaves and administers the process from the point of building the code to the point of deployment to Stable (ie, it covers the promotion levels dev, ci, stable, beta, rc). The same hudson server/cluster is used for all software products and services that participate in the continuous integration process.
    Are you sure you want to
    Your message goes here
    Processing…
  • So where is/are the Hudson servers? Does each quality gate server has its own, or is there a master one, moving tested software to the next quality gate? Also, the Fitness server/s?
    Are you sure you want to
    Your message goes here
    Processing…
  • @Dennis: Our Beta system is used for beta testing of new features. Sometimes the same builds of services or products might get deployed to both Beta and production systems, but when it's a customer-facing app (as opposed to a backend service), for the most part this system is used to get feedback from early access users. We then use that feedback to make final changes and polish up the features that we push to production. So, if you're talking about the actual builds, often the Beta flow path ends in Beta.
    Are you sure you want to
    Your message goes here
    Processing…
  • Hi Martin,
    The horizontal view on slide 62 is an in-house plugin. I'm currently working on management and the developer who wrote it to get them to make it publicly available, I'll do my best to hurry up its release :)
    Are you sure you want to
    Your message goes here
    Processing…
  • 2 things.
    A/ I read where Jenkins is becoming the replacement for Hudson, by community interest and investment. So Installed it, and it literally REPLACED AND UNINSTALLED Hudson with no wizard questions or anything. Guess it *IS* the replacement, LOL!
    B/ Where does the 'Beta' flow path go? It branches out of 'Stable' and then what?
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Outline for this deck:Who we areWhat challenge we are addressing (high level)Our platform = Ingenuity Knowledge Base Content (3 slides) Ontology (1 slide)Products and Solutions Overview Research and Analysis Solutions The challenge IPA addresses IPA overview The challenge Ingenuity Answers addresses Additional Solutions eCommerce EnterpriseWhat Sets Ingenuity Apart (USPs)
  • What is the challenge Ingenuity is addressing? Speaking points:Helping researchers access detailed and high-quality knowledge and then USING it to design experiments, generate hypothesis, answer targeted questions, and analyze dataAll of Ingenuity's products and services have one common goal — to help life science researchers generate maximum value from all types of biological and chemical knowledge.Ingenuity offers a broad and flexible range of solutions that can be tailored to needs of various clients, including academic and therapeutic area researchers, computational biologists and informatics departments, and suppliers in the life sciences industry.
  • Speaking Points:What is it? [high level description]The Ingenuity Knowledge Base is the core technology of Ingenuity Systems Is the engine behind all of Ingenuity’s products – (IPA) It is a repository of biological interactions and functional annotations created from millions of individually modeled relationships between proteins, genes, complexes, cells, tissues, drugs, and diseasesThe IKB enables scientific understanding – through IPA and other services, researchers canDiscover biological relationships and functionsBuild testable hypothesesLastly, they can answer targeted biological questions – and get back relevant answersSo, what is unique about the IKB – what have we done differently - next slideInference example:For example – can I discover a molecular path connecting a compound known to treat mammary tumors to angiogenesis? Types of contextual detail:SpeciesSynonymsExperimental methodSite of post-translational modificationDirection of changeTissue contextCell line contextOriginal source
  • Speaking Points:How is our approach different? With the information we capture (publications, public knowledge, etc. - not JUST publications) we do THREE important things:We STRUCUTURE the information using the Ingenuity Ontology – statements are modeled into Findings. Findings are not just EXPERT REVIEW PROCESS We capture a lot of detail and ensure it is correct.The information we cover is COMPREHENSIVE – IKB covers and extensive body of biological and chemical info, including publications and third party sources.The information is TIMELY – weekly updates as of IPA 8 and IPA 8.5
  • Small teams empowered to release to production without management approvalBetter iteration and release planning. Learning to build and release features incrementally; show/hide features in betaFocus on Unit testing first and then integration testingStreamline the build environment to do automated builds, deployments and build promotionRefactor the code base to create more modular and service oriented code base
  • --title: --list items
  • --only one repository
  • --mention fingerprinting--all plugins in parent
  • --artifactsinhudson repo--nodpeloyment/test pohase other than junit--just build, not CI
  • --release master is the CI one--remove jenkinsrefernces
  • --cconfigurabel, difficult to maintain
  • --describe what puppet is--erb use--cofnigmgt standardized--moreenforcable
  • --standard vmconfig
  • --hudson build succesnoyl
  • --status app page --hudson
  • --service discovery mgr, router for services

Continuous Integration using Hudson and Fitnesse at Ingenuity Systems (Silicon Valley code camp 2011) Continuous Integration using Hudson and Fitnesse at Ingenuity Systems (Silicon Valley code camp 2011) Presentation Transcript

  • Proprietary and Confidential
    Continuous Integration using Hudson and Fitnesse
    Norman Boccone | Lead Engineer | twitter: @dropslowly
    Jennifer Wong | Lead QE Engineer | twitter: @jenlwong
    Vasu Durgavarjhula | Director of Development | twitter: @d_vasu
  • Proprietary and Confidential
    2
    Overview
    ► Who we are ?
    ► The Challenge
    ► Tools and Processes
    ► Test Strategy
    ► Conclusion
  • Ingenuity Systems
    We are a leading provider of information and analytics solutions for life science researchers
    Our goal is to help life science researchers get biological insights from their data quickly and reliably
    Our products are used by thousands of researchers worldwide
    Partial Customer List
    Proprietary and Confidential
    3
  • ?
    Scientists Need a Tool to Make Good Decisions From Complex Biological Data
    Toxicology
    Biomarkers
    Pharmacogenomics
    Discovery
    Use understanding of disease mechanisms to identify and validate targets
    Elucidate biological mechanisms of drug action and toxicity
    Identify and prioritize novel biomarkers by understanding role in disease pathways
    Understand mechanisms behind differential response to R(x)
    Hard to synthesize the picture of what that data means in a broader biological context, and how the pieces work together…
    Traditional publishing models are good for learning but not applying knowledge …
    Enormous volume and complexity of biological and chemical data…
    Cancer
    Publishing and Collaboration
    Experimental Data
    Search
    Proprietary and Confidential
    4
  • The Ingenuity® Knowledge Base
    Is the core technology behind all of Ingenuity’s products and services
    THE INGENUITY KNOWLEDGE BASE
    ANALYSIS
    HYPOTHESIS GENERATION
    VISUALIZATION
    ECOMMERCE ENABLEMENT
    PATHWAY, REAGENT & GENE SEARCH
    ENABLES SCIENTIFIC UNDERSTANDING
    Discover existing relationships and function, as well as inference of relationships that may not have been studied in a particular context yet
    Generatetestable hypotheses, build pathways, ability to inference
    Get answers by asking detailed and in-depth scientific questions
  • The Ingenuity® Knowledge Base
    How is it different?
    THE INGENUITY KNOWLEDGE BASE
    ► Structured-to capture relevant details
    Scientific statements are modeled into Findings using the Ingenuity Ontology
    ► Expert Review Process- for accuracy
    Findings go through QC process
    ► Comprehensive- leverage knowledge in one place
    Largest scientific knowledge base of its kind with modeled relationships between proteins, genes, complexes, cells, tissues, drugs, pathways and diseases
    ► Timely- for access to up-to-date knowledge
    Findings are added weekly
  • Proprietary and Confidential
    The Challenge
    7
  • Proprietary and Confidential
    8
    Single product
  • Proprietary and Confidential
    9
    multiple products
    Ingenuity Variant Analysis
    Next Generation Sequencing Analysis Reports
    Ingenuity Product Search
    Reagent Portals for BD, Sigma, Thermo Fischer
  • Proprietary and Confidential
    10
    Large monolithic code base
  • Proprietary and Confidential
    11
    Modular Services
    Single Sign On
    User
    Management
    Views & Reports
    User Data
    Content
    Service
    iReport
    Master Graph
  • Proprietary and Confidential
    12
    Long release cycles
  • Proprietary and Confidential
    13
    Faster release cycles
  • Proprietary and Confidential
    14
    Speeding up the feedback loop
    Developer
    Stakeholders
  • Proprietary and Confidential
    15
    Do all of this while supporting the existing product
  • Proprietary and Confidential
    16
    Approach
    Move to a Service Oriented Architecture
    Don’t miss Jeff’s talk on Sunday 10:45 am “Learning to Love Services: Experiences from the Transition to a Service-oriented Architecture”
    Develop a test strategy focused on automated testing
    Build a continuous integration system
    Adopt agile product management
    Improve infrastructure to create and deploy services
    Foster a Dev/Ops culture
  • Proprietary and Confidential
    17
    Continuous Integration Workflow
    Application
    Bundle
    Deploy Application
    Run Fitnesse Tests
    (Nightly suite)
    Nightly Build
    (Clover)
    Run JUnit
    Tests
    publish summary
    Fitnesse
    Bundle
    Deploy Fitnesse
    publish
    publish
    Hudson Dashboard
    (JUnit, Fitnesse summary, Code Coverage)
    Fitnesse Wiki
    (Test history, Details, Test Case Management)
    Link
    SVN
    Commit
    (Test Cases)
  • Proprietary and Confidential
    18
    Tools and Processes
  • Proprietary and Confidential
    19
    Service Oriented Architecture
  • Proprietary and Confidential
    20
    SOA: Before
    lots of modules, separate but dependent
    easy to develop on but also easy to add circular dependencies
    hard to keep track of purpose/value of all the modules
    product consisted of bunches of modules put together
    reference was typically to latest devcode
    to ensure "latest," individual modules of a product were rebuilt for every release
    junittests were run every time as well
    new product version meant build everything, test everything
    builds were slow
  • Proprietary and Confidential
    21
    SOA: “Current”
    components: module grouped together.  Access through api module only
    less chance of circular dependencies
    easier to comprehend modules
    more flexibility to change code: just do not change api
    components built/tested separately
    faster builds
    less repetitive testing
    some shared modules still exist (they are not part of any component)
  • Proprietary and Confidential
    22
    SOA: Conversion Notes
    find a logical group of modules
    Convert groupput group into hierarchical structure, hide things behind an apilayer
    convert outside modules to use apis
    see Jeff's talk
  • Proprietary and Confidential
    23
    Build System
  • Proprietary and Confidential
    24
    Build System: Before
    in-house ant scripts
    lots of control but it required lots of resources
    no real industry standard
    global scripts/property files, overrides allowed
    offered consistency but (busy) people would override instead of follow pattern
    scripts got complicated
    anyone could add a new feature
    if anyone wanted a new feature, just tell them to add it
    inefficient additions, different code styles made scripts hard to read
    very little dependency management
    multiple repositories for binary files
  • Proprietary and Confidential
    25
    Build System: “Current”
    maven
    not much control, and very rigid  (a good thing, in a way)
    steep learning curve
    different from previous way of doing things
    standard structure; difficult to not follow structure
    hierarchical structure easily followed
    versions for all dependencies well defined
    easy to add new features (after a short burst of intense agony)
    only one repository
  • Proprietary and Confidential
    26
    Build System: Conversion Notes
    get a maven repository manager (nexus? Turn on search!)
    add antrun plugin to publish artifacts to old system
    choose a simple component to start, make lots of little submodules(1 module per artifact, plus assembly)
    manually add needed dependencies to repository
    build up company parent pom, use hierarchy
    Put all plugin management into parent
    Key maven plugins: buildnumber, buildhelper, release, assembly,
  • Proprietary and Confidential
    27
    Build System: TODO
    cleaner release plugin
    continue learning maven
    contribute to plugins
    Improve internal maven FAQ
    Look into git
  • Proprietary and Confidential
    28
    Continuous Builds
  • Proprietary and Confidential
    29
    Continuous Builds: Before
    Hudson (no plugins)
    auto build start for each module, with artifacts published
    manual build start for products
    shell scripts to copy artifacts to release dir
  • Proprietary and Confidential
    30
    Continuous Builds: “Current”
    Hudson
    devmaster for auto build of modules/submodules, auto published to repository manager (dev area)
    release master for components/products, auto published to repository manager (release area)
    modules/submodulesbuilt on checkin
    components/products built on schedule or on demand
    process includes various tests and deployments (Jen)
  • Proprietary and Confidential
    31
    Continuous Builds: Conversion Notes
    use Jenkins, not Hudson (more activity for Jenkins)
    startup new, clean servers
    copy/create job that we want
    Try to make Jenkins be the portal to everything
    fingerprinting
    Plugins we use: clover, fitnesse, findbugs, checkstyle, disk-usage, downstream build view, (custom) dashboard view, promoted builds, ssh slaves
  • Proprietary and Confidential
    32
    Continuous Builds: TODO
    better job rollup
    job templates
    more maven/hudson plugins
    submit hudson plugin(s) to community
  • Proprietary and Confidential
    33
    Deployment: Before
    proprietary shell/perl scripts
    targeted to install app only (not machine set up)
    multiple proprietary property config scripts
  • Proprietary and Confidential
    34
    Deployment: “Current”
    Puppet
    Configures machines and apps
    Work on our machines and EC2
    Use of .erb files everywhere for property configuration
  • Proprietary and Confidential
    35
    Deployment: Conversion Notes
    modify tar format
    moved to one tomcat
    used erb files
    split properties into app
  • Proprietary and Confidential
    36
    Deployment: TODO
    standard server configuration types (e.g. “build machine”, “app machine”, …)
    more puppet scripts
  • Proprietary and Confidential
    37
    Visibility
  • Proprietary and Confidential
    38
    Visibility: Before
    cfmap (IT server diagnostic tool)
    Hudson status page
    FitNesse server
  • Proprietary and Confidential
    39
    Visibility: “Current”
    cfmap
    Hudson status page
    Hudson dashboard
    extra tests built in to Hudson
    FitNesse server
    Service status page
  • Proprietary and Confidential
    40
    Visibility: Conversion Notes
    use Hudson plugins, with drill down
    get everything to report in a standard way (junit xml)
  • Proprietary and Confidential
    41
    Visibility: TODO
    more widgets for the dashboard
    Integrate outside reports into dashboard
    cfmap
    status page
    sdm? (service discovery manager)
  • Proprietary and Confidential
    42
    Test Strategy
  • Proprietary and Confidential
    43
    Testing For CI/CD
    Test Automation is key
    How and what to automate
    Mezaros
    Cohn
    Crispin
  • Proprietary and Confidential
    44
    Test Challenges
    Testability: some products are difficult to test at lower levels
    Legacy Apps: one of our main products was a ‘Legacy App’. Tests used to look like this:
  • Proprietary and Confidential
    45
    Test Challenges
    varying levels of coverage
    some of our newer components have great coverage
    other components have lower coverage: legacy, proof of concept
  • Proprietary and Confidential
    46
    Solutions
    Test and code refactoring
    Legacy Apps: Strangulation approach
    automate new and refactored features
    incremental work on tests: reserve time in each iteration for adding to tests
    Change in culture: team ownership of tests and status
    Cycle time for ipa was 2.5 weeks
    Now for most of our services, it is 20-40 minutes
  • Proprietary and Confidential
    47
    Test Infrastructure and Environments
  • Proprietary and Confidential
    48
    Process: Build Promotion
    Build Promotion through Quality Gates
    Beta
    • Deployed to Beta
  • Proprietary and Confidential
    49
    Test Environments
    Stable Environment
    • Multiple nodes
    • Deployment Tests
    • System and performance tests
    • Manual/exploratory
    CI Environment
    • Single nodes
    • functional tests
    Service1
    Service1
    Service2
    Service2
    Service3
    Service3
    Service4
    Service4
    Service5
    Service5
    Service6
  • Proprietary and Confidential
    50
    Test Types
  • Proprietary and Confidential
    51
    Test Types: Deployment and Health
    Each of our components has a built-in status page
  • Proprietary and Confidential
    52
    Test Types: Deployment and Health
    Status page
    Reflects app status
    Resource availability: DB Connections, Services
  • Proprietary and Confidential
    53
    Test Types: Deployment and Health
    Information is used for:
    Deployment
    Health Monitoring
    Service Compatibility and Dependency checking
  • Proprietary and Confidential
    54
    Test Types: FitNesse
    FitNesse is a wiki-based web server test tool
    Helps abstract test definition from technical implementation
    Provides visual reporting and result history tracking
  • Proprietary and Confidential
    55
    Test Types: FitNesse
    The FitNesse Server
    http://localhost:8080/FitNesse.UserGuide.TwoMinuteExamplehttp://localhost:8080/FitNesse.UserGuide.TwoMinuteExample?pageHistory
    Wiki view Editing the wiki Test Results
  • Proprietary and Confidential
    56
    Test Types: FitNesse
    • Integration with Hudson
    - Views, drilldown, configuration
    • Test results converted to junit format
    View of fitnesse test job in Hudson
  • Proprietary and Confidential
    57
    Drill downs on fitnesse test results in Hudson
  • Proprietary and Confidential
    58
    FitNessePlugin Configuration
    Convert results to Junit format
  • Proprietary and Confidential
    59
    Publish FitNesse Results and Coverage
  • Proprietary and Confidential
    60
    Test Types: FitNesse
    • What we use it for:
    • Functional tests
    • Integration tests
    • UI Tests (com.jbergin.HtmlFixture)
    • DB Tests (dbfit)
    What we’ve done with it that is different
    Use as execution framework for more complex tests
    Extension of fitnesse server for data-driven tests
    json fixture – pass in javascript
    Execution of Selenium tests
  • Proprietary and Confidential
    61
    Test Types: FitNesse
    Lessons learned
    FitNesse is good for straightforward verification of data
    To do more, you have to get creative
    Fixture and test ownership needs to be a shared responsibility
  • Proprietary and Confidential
    62
    Test Types: Selenium
    We fire off Selenium tests in two ways: via Hudson job or through FitNesse.
    • Test results converted to junit format for display in hudson
    Hudson job in dashboard:
    Hudson job view:
  • Proprietary and Confidential
    63
    Test Types: Selenium
    Hudson job configuration: trigger execution on Selenium grid using Ant
  • Proprietary and Confidential
    64
    Test Types: Selenium
    Second way we fire off Selenium tests is using a FitNesse fixture: WebTest
  • Proprietary and Confidential
    65
    Test Types: Backward Compatibility
    No staging environment
    How do we know that when we release a new service version to prod that it won’t break?
    Service version compatibility
    Leverage Service Discover Manager (SDM) and Status pages to check for availability of services
    Jar compatibility
    static code analysis could detect changes in method signature but not underlying object or serialization changes
  • Proprietary and Confidential
    66
    What’s Next: Getting to the End of the Road
  • Proprietary and Confidential
    67
    What’s Next
    What do we need to do to get to the end of the path to Continuous Integration and Deployment?
    Continue to invest in standardized tools and processes
    Maven
    Puppet
    Jenkins
    Metrics and Immune Systems
    • Continue to improve our test coverage and practices
    • Culture:
    Continue to support, reinforce, educate
  • Proprietary and Confidential
    68
    The Finish Line
    Continuous deployment
    Reduced Cycle Time
    Improved quality
    Predictable metrics
    Reduced risk
  • Proprietary and Confidential
    69
    Slides are posted here: http://www.slideshare.net/jenlwong/ingenuity-svcc-ci-presentation-20111007
    Plugs
    Learning to Love Services: Experiences from the Transition to a Service-oriented Architecture
    Speaker: Jeff Green 10:45 AM Sunday
    Ingenuity is hiring: http://www.ingenuity.com/company/careers.html
    Q&A
    References
    Todd. Test Automation Pyramid – review. Retrieved September 29, 2011 from: http://blog.goneopen.com/2010/08/test-automation-pyramid-review/
    Humble, Jez, and Farley, David. Continuous Delivery. Boston: Pearson Education Inc, 2011. Print.
    Crispin, Lisa, and Gregory, Janet. Agile Testing: A Practical Guide for Testers and Agile Teams. Boston: Pearson Education Inc, 2011. Print.