Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
June 20–23, 2016 | Berlin, Germany
Automated Platform for
Testing VNF Performance and Interoperability with
Variable Flavors
Takashi Torii NEC Corporation
Ta...
Introduction of
Okinawa Open Laboratory(OOL)
Vision: Okinawa Smart Hub
• Okinawa Prefectural Government vision and policy for industrial development
and promotion
• At...
Okinawa Smart Hub and OOL
• Contribute to realizing Okinawa Smart Hub concept
• Based in Okinawa with support from Okinawa...
Mission
• Research and development activities for practical application and dissemination of
advanced Information Communic...
Activity Model
• OOL connects OSS development communities and market via use cases and
reference models to expand user bas...
Member List
Board
(5)
Member
(29)
Fellow
(14)
Total	(48)
Project Formations
Full	Time	Engineers
Organizer
Companies
Research	Institute
Academia
• Planning	projects	and	goals
• Com...
Dev Project Example: OF-Patch
• Patch panel function implemented using OpenFlow
technology
OFPM
User
Change network
topolo...
OPNFV Test Lab
• VNF Test Automation
• Service Function
Chaining
• Joint POC with oversea
members
Internet
Jump	Server
POD...
VNF Test Automation Project
Background
• Market of VNF is now emerging
• ETSI proposed NFV architecture
• Open Source VNF
• VyOS (Vyatta)
• Open vSwit...
Background
• Problem of VNF testing
• There is no standard
• Performance and stability is depend on flavors
• Many test pa...
How to test VNF?
• Workflow
• Deploy
• Deploy VNF on VIM with flavor
• Initialize VNF
• Configuration
• Configure and conf...
Test workflow
Test
Deploy	
VNF
Initialize
Initialize
VNF
Config
Delete
VNF
Log
Archive
Analysis	
Report
Check
Test workflow
• Benefit of automation
• Engineer can focus on troubleshooting
• Engineers who don’t have much skills can o...
Architecture
PoC User
Orchestrator
Web-Portal
Web-Portal
Report generationManagement of Test
REST API
Interoperability Tra...
Architecture
PoC User
Orchestrator
Web-Portal
Web-Portal
Report generationManagement of Test
REST API
Interoperability Tra...
Web Portal
• Set the test parameters
• Select the test scenario from drop down lists
Orchestrator
Web-Portal
REST API
PoC ...
Architecture
PoC User
Orchestrator
Web-Portal
Report generationManagement of Test
REST API
Interoperability Traffic Test
R...
Orchestrator
• Controller management
• Controls each components based on the configuration of test scenarios.
• Creating t...
Architecture
PoC User
Orchestrator
Web-Portal
Report generationManagement of Test
REST API
Interoperability Traffic Test
R...
Architecture
PoC User
Orchestrator
Web-Portal
Report generationManagement of Test
REST API
Interoperability Traffic Test
R...
Virtual Appliance Controller
• Multi virtualized platform control
• Deploy virtual appliance on selected virtualized platf...
Architecture
PoC User
Orchestrator
Web-Portal
Report generationManagement of Test
REST API
Interoperability Traffic Test
R...
NW Device Controller
• Multi vendor control
• Provides the ability to control multiple vendor VNF
Orchestrator
NW Device C...
Architecture
PoC User
Orchestrator
Web-Portal
Report generationManagement of Test
REST API
Interoperability Traffic Test
R...
Tester Controller
• Generating Tcl script
• Based on the parameters set by REST API from Orchestrator
• Ixia Driver
• Load...
Test workflow
Repeat
Virtual	Appliance	Controller
Tester	ControllerNW	Device	Controller
Orchestrator
Test
Deploy	
VNF
Init...
Our environment
VyOS
vSwitch
vnic0
vSwitch
InteroperabilityL2SW
TrafficTestL2SW
vnic1
vnic1
OpenStack (node2)
VMware Esxi ...
Our environment for interoperability
VyOS
vSwitch
vnic0
vSwitch
InteroperabilityL2SW
TrafficTestL2SW
vnic1
vnic1
OpenStack...
Our environment for interoperability
VyOS
vSwitch
vnic0
vSwitch
InteroperabilityL2SW
TrafficTestL2SW
vnic1
vnic1
OpenStack...
Our environment for interoperability
VyOS
vSwitch
vnic0
vSwitch
InteroperabilityL2SW
TrafficTestL2SW
vnic1
vnic1
OpenStack...
Our environment for interoperability
VyOS
vSwitch
vnic0
vSwitch
InteroperabilityL2SW
TrafficTestL2SW
vnic1
vnic1
OpenStack...
Our environment for Traffic Test
VyOS
vSwitch
vnic0
vSwitch
InteroperabilityL2SW
TrafficTestL2SW
vnic1
vnic1
OpenStack (no...
Our environment for Traffic Test
VyOS
vSwitch
vnic0
vSwitch
InteroperabilityL2SW
TrafficTestL2SW
vnic1
vnic1
OpenStack (no...
Our environment for Traffic Test
VyOS
vSwitch
vnic0
vSwitch
InteroperabilityL2SW
TrafficTestL2SW
vnic1
vnic1
OpenStack (no...
Our environment for Traffic Test
VyOS
vSwitch
vnic0
vSwitch
InteroperabilityL2SW
TrafficTestL2SW
vnic1
vnic1
OpenStack (no...
Demonstration (Movie)
Result of test automation
• Enables multiple patterns of test runs which are difficult by manual
Automate	the	combination	...
Current Activities
• We have just joined OPNFV, and deploy Community Lab.
• Refactoring to align NFV Architecture of ETSI
...
TesterTester
Collaborate with Functest
Web	Portal
Functest
Web	Portal
VNF
Tester
VNFM
VNF
VNF
Functest
Any Questions?
Upcoming SlideShare
Loading in …5
×

Summit 16: Automated Platform for Testing VNF Performance and Interoperability with Variable Flavors

1,526 views

Published on

VNF sizing is one of the big issues for users, integrators and also venders/providers of VNF. In legacy system, network functions are integrated in the systems with carefully sizing by estimated performance metrics, but in NFV, performance and metrics of VNF are depends on flavors. To decide which flavor is suitable for requirements, we have to know the basic data of each metrics with variable flavors. But there are many parameters, so automatic testing is necessary. We had developed the test automation system of VNF that can automatically conducting the sequence of tests with variable parameters such as number of core, memory size, interface speed, etc. We already created the test cases that evaluate BGP interoperability of virtual router and performance test of Cisco, Juniper, Brocade, and so on. We think the activity is similar to Yardstick, so we would like to collaborate with them. In this talk, we summarize our activities and development result, and introducing our latest plan to collaborate with OPNFV.

Published in: Technology
  • Be the first to comment

Summit 16: Automated Platform for Testing VNF Performance and Interoperability with Variable Flavors

  1. 1. June 20–23, 2016 | Berlin, Germany
  2. 2. Automated Platform for Testing VNF Performance and Interoperability with Variable Flavors Takashi Torii NEC Corporation Tatsuya Tohyama, Okinawa Open Laboratory/OKIT
  3. 3. Introduction of Okinawa Open Laboratory(OOL)
  4. 4. Vision: Okinawa Smart Hub • Okinawa Prefectural Government vision and policy for industrial development and promotion • Attract companies, talents and knowledge from other prefectures in Japan and overseas, especially Asian countries, and promote new value creation in Okinawa • Form major international ICT hub in Asia
  5. 5. Okinawa Smart Hub and OOL • Contribute to realizing Okinawa Smart Hub concept • Based in Okinawa with support from Okinawa Prefectural Government, attracting leading companies, organizations, educational institutions, and research bodies from other prefectures in Japan and overseas, especially Asia, and promoting international R&D activities • Contributes to promoting concentration of ICT industries and advancement of regional companies SDN technology research and standardization base OSS cloud development community Leading Linux and OSS organization OSS SDN development project Okinawa Open Laboratory Practical-use research base that integrates cloud technology and SDN technology International R&D organization Leading companies and organizations from other prefectures in Japan and overseas, especially Asia Technology cooperation Cooperation, attraction Activity support ・Support for R&D activities ・Promotion of cloud DC facility use
  6. 6. Mission • Research and development activities for practical application and dissemination of advanced Information Communications Technologies (ICT), especially next-generation ICT platform technologies, such as • Software-Defined Networking (SDN) • Cloud Computing using Open Source Software (OSS, especially Open Stack) • Open Innovations • Collaboration, Joint Work • Bridging the gaps between users and providers
  7. 7. Activity Model • OOL connects OSS development communities and market via use cases and reference models to expand user base of next-generation ICT platform technologies in cooperation with MEMBER and partners • OOL Value Chain: Use Case à Verification à Ref. Model à Promotion • Feedback Loops: (1) OOL ó Market, (2) OOL ó OSS Dev. Communities OSS Bodies (Community/Consortium) Develop Use Case Verify Promote Issues /Needs Apply /Use OOL and its Members Market (Providers and End Users) Expand User Base Reference Model Testbed (OpenStack with SDN)
  8. 8. Member List Board (5) Member (29) Fellow (14) Total (48)
  9. 9. Project Formations Full Time Engineers Organizer Companies Research Institute Academia • Planning projects and goals • Communicate with companies • Communicate with Boards, Staffs and Stakeholders (Government etc) • Manage the activities • Promote and collaborate Staffs Board
  10. 10. Dev Project Example: OF-Patch • Patch panel function implemented using OpenFlow technology OFPM User Change network topology by remote operation OF-Patch GUI OFC OFS Patch by flow Host1 Host2 Host3
  11. 11. OPNFV Test Lab • VNF Test Automation • Service Function Chaining • Joint POC with oversea members Internet Jump Server POD#1 (Fuel) POD#2 (RDO)
  12. 12. VNF Test Automation Project
  13. 13. Background • Market of VNF is now emerging • ETSI proposed NFV architecture • Open Source VNF • VyOS (Vyatta) • Open vSwitch …etc • Vender VNF • Cisco CSR1000V • Juniper FireFly, vSRX • Brocade Vyatta Core …etc • But we don‘t have test automation platform of VNF
  14. 14. Background • Problem of VNF testing • There is no standard • Performance and stability is depend on flavors • Many test patterns are required than H/W appliance • Impossible to cover all test patterns by manual • Risk of human error or miss configuration • Configurations are depend on vendors • Need support from venders or skilled engineers in all phase • To solve these problems, we have developed VNF automation testing system
  15. 15. How to test VNF? • Workflow • Deploy • Deploy VNF on VIM with flavor • Initialize VNF • Configuration • Configure and confirm • Check the network connectivity with other devices • Run • Subscribe test patterns and run on traffic generator. • Check VNF status and behavior • Reporting • Writing report based on result
  16. 16. Test workflow Test Deploy VNF Initialize Initialize VNF Config Delete VNF Log Archive Analysis Report Check
  17. 17. Test workflow • Benefit of automation • Engineer can focus on troubleshooting • Engineers who don’t have much skills can operate the tests • High repeatability Repeat Test Deploy VNF Initialize Initialize VNF Config Delete VNF Log Archive Analysis ReportCheck
  18. 18. Architecture PoC User Orchestrator Web-Portal Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA)
  19. 19. Architecture PoC User Orchestrator Web-Portal Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA)
  20. 20. Web Portal • Set the test parameters • Select the test scenario from drop down lists Orchestrator Web-Portal REST API PoC User Test Parameter
  21. 21. Architecture PoC User Orchestrator Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA) Web-Portal
  22. 22. Orchestrator • Controller management • Controls each components based on the configuration of test scenarios. • Creating test scenario • Create test scenarios by GUI • Reporting • Output PDF format Scenario generate tool
  23. 23. Architecture PoC User Orchestrator Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA) Web-Portal
  24. 24. Architecture PoC User Orchestrator Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA) Web-Portal
  25. 25. Virtual Appliance Controller • Multi virtualized platform control • Deploy virtual appliance on selected virtualized platform. Orchestrator REST-API REST-Message OpenStack Link Virtual Appliance Controller REST-API REST-API
  26. 26. Architecture PoC User Orchestrator Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA) Web-Portal
  27. 27. NW Device Controller • Multi vendor control • Provides the ability to control multiple vendor VNF Orchestrator NW Device Controller REST-API Telnet/SSH/Netconf REST-Message Parameter Ex ・Vender Name ・Vender Version ・Test scenario other…
  28. 28. Architecture PoC User Orchestrator Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA) Web-Portal
  29. 29. Tester Controller • Generating Tcl script • Based on the parameters set by REST API from Orchestrator • Ixia Driver • Load Tcl Script by IXIAAPI. Orchestrator Tester Controller REST-Message Parameter Ex ・Interface Information ・Packet Size(Short,Long,Imix) ・Frame rate other... Test Traffic IXIA API REST API Tcl Script
  30. 30. Test workflow Repeat Virtual Appliance Controller Tester ControllerNW Device Controller Orchestrator Test Deploy VNF Initialize Initialize VNF Config Delete VNF Log Archive Analysis Report Check
  31. 31. Our environment VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600
  32. 32. Our environment for interoperability VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600
  33. 33. Our environment for interoperability VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600 VNF appliance test environment VNF appliance test environment
  34. 34. Our environment for interoperability VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600 VNF appliance for interoperability testing VNF appliance for interoperability testing
  35. 35. Our environment for interoperability VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600
  36. 36. Our environment for Traffic Test VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600
  37. 37. Our environment for Traffic Test VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600 VNF appliance test environment VNF appliance test environment
  38. 38. Our environment for Traffic Test VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600 Traffic Generator
  39. 39. Our environment for Traffic Test VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600
  40. 40. Demonstration (Movie)
  41. 41. Result of test automation • Enables multiple patterns of test runs which are difficult by manual Automate the combination test of 2736 patterns by 40 hours 【CSR1000v,Firefly,Vyatta,VyOS】VNF appliance 【VMware ESXi, OpenStack】VIM 【CPU,Mem,HDD,VNIC】 【CSR1000v,Firefly,Vyatta,VyOS】Interoperability with reference router 【VMware ESXi, OpenStack】VIM of reference router 【BGP:5 Pattern,OSPF:7 Pattern】 【Test Pattern:3 Pattern ,FrameRate:6 Pattern】 Flavor of VNF Appliance Routing protocol Traffic test pattern
  42. 42. Current Activities • We have just joined OPNFV, and deploy Community Lab. • Refactoring to align NFV Architecture of ETSI • Using TOSCA based orchestrator (we are considering Tacker) • Collaborate with Functest
  43. 43. TesterTester Collaborate with Functest Web Portal Functest Web Portal VNF Tester VNFM VNF VNF Functest
  44. 44. Any Questions?

×