Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Upstream Testing Collaboration


Published on

Jose Lausuch, Ericsson

OPNFV provides different test frameworks which help developers to write new test cases. Those frameworks also borrow and integrate a variety of testing tools from other open source communities (OpenStack, OpenDaylight, Open-O, ...).

This session will go through all the tools that have been integrated so far in OPNFV and the cross community collaboration that has already started in Danube time frame.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Upstream Testing Collaboration

  1. 1. Upstream testing collaboration Jose Lausuch Ericsson, Functest PTL
  2. 2. OPNFV = Integration & testing Enhancements Additions
  3. 3. OPNFV = Integration & testing OPNFV integrates several upstream components • OpenStack, OVS, ODL, ONOS, … Single components might work as expected when isolated The challenge is integration e.g. OpenStack + OVS + ODL + Tacker + SFC
  4. 4. Things might go wrong
  5. 5. e.g. OPNFV SFC The SFC feature in OPNFV SFC needs different components: • Fuel, Apex • OpenStack • OVS • ODL + SFC feature • Tacker Some bugs are found only when putting them all together
  6. 6. The community decided to use Robot as the test framework There are a lot of tests dealing with each module in the architecture • netvirt, genius, Netconf, ovsdb, opensack, etc … OPNFV is mainly interested about ODL integration with OpenStack
  7. 7. • Basic & Neutron.Basic.Restconf Modules • Get the complete list of networks • Get the complete list of subnets • Get the complete list of ports • Check OpenStack Networks • Check OpenDaylight Networks • Create new network in OpenStack Check network created in OpenDaylight • Check network created in OpenDaylight • Delete Network • Check network deleted in OpenDaylight• Check OpenStack Subnets • Check OpenDaylight subnets • Create new subnet in OpenStack • Check new subnet created in OpenDaylight • Delete New subnet • Check subnet deleted in OpenDaylight • Check OpenStack ports • Check OpenDaylight ports • Create new port in OpenStack • Check new port created in OpenDaylight • Delete New Port • Check port deleted in OpenDaylight
  8. 8. New Patchset (Commit) Create Virtual Testing Platform Download/Install new artifact Use standard testing framework e.g. Functest Build ODL artifact 10 min 15 min 35 min Unit test Installer Project e.g. Apex Snapshot 5 min Positive -> Merge commit Negative -> Prevent merge Investigate the logs/Find bugs ODL OPNFV Feedback
  9. 9. What next? OPNFV could collaborate with ODL community to define test needs OPNFV folks contribute with code to ODL Use OPNFV as a tool to give quick feedback to ODL  XCI
  10. 10. Tempest Set of integration tests exercising the OpenStack APIs In OPNFV: • Tempest smoke • Tempest full OPNFV scenarios are complex and some tests might must blacklisted
  11. 11. Rally Benchmarking tool of the OpenStack services In OPNFV: • Rally sanity • Rally full OPNFV has collaborated upstream with some bug fixes
  12. 12. Refstack Set of Tempest test cases defined by the Interop Working Group Toolchain used for OpenStack Powered platform trademark Web platform to upload test results In OPNFV: • Set of tempest test cases belonging to guidelines (2016.08) are valid for Kilo, Liberty, Mitaka and Newton releases of OpenStack
  13. 13. Refstack OpenStack is interested about NFV certification track We have the NFV knowledge They have the experience of running certification programs Why don’t we upstream our NFV use cases to e.g. Tempest?
  14. 14. Cross CI Integrate in OPNFV the latest OpenStack master Automated integration and testing Quick feedback to upstream Projects
  15. 15. VNF managers OPNFV’s mission is not to to test the functionality of a VNF manager OPNFV cares about • successful integration of the VNF manager • it can deploy a VNF • VNF lifecycle, etc… Examples • Tacker for SFC test case • Cloudify for vIMS in Functest
  16. 16. ONAP ONAP is going to have a lot of presence in OPNFV OPNFV probably doesn’t want to run the exact same tests but it should check how their tools integrate in OPNFV Needde collaboration in both directions
  17. 17. In short Integration of upstream components makes thing complex OPNFV needs to collaborate with upstream to provide: • Feedback • Bug fixes • Requirements, blueprints
  18. 18. In short Don’t wait to get something done Engage in upstream communities
  19. 19. How to engage in upstream? IRC Mailing lists Attend project meetings
  20. 20. Thank You