QualiTest hosts a webinar: Ensuring Successful OPNFV-based NFV Deployments QualiTest and QualiSystems team up to bring you an overview of OPNFV-based NFV Deployments. We'll discuss the fundamentals and capability of the carrier-grade, integrated platform. We'll cover just how to anticipate NFV testing challenges and explore the solutions. Hosted by: Hans Ashhlock - QualiSystems Benny Sand - QualiTest Group Hosted on: November 3rd, 2015 QualiTest is the world’s second largest pure play software testing and QA company. Testing and QA is all that we do! visit us at: www.QualiTestGroup.com

1. Ensuring Successful
OPNFV-based NFV
Deployments

2. Hans Ashlock
QualiSystems
Benny Sand
NFV Strategy Lead at QualiTest
Webinar Presenters
3

3. Agenda
NFV Testing
Challenges & Solutions
Automated NFV Testing
Environment as a Service
Summary & Q&A
QualiTest & QualiSystems
Partnership
4
High Level OPNFV

4. Agenda
NFV Testing
Challenges & Solutions
Automated NFV Testing
Environment as a Service
Summary & Q&A
QualiTest & QualiSystems
Partnership
5
High Level OPNFV

5. QualiTest &
QualiSystems
partnership
| Pure Play Testing
focus
| Passion for testing
| World’s 2nd largest
| Telco expertise
| Automation software
| Cloud management
| Test & CI automation
| Carrier DevOps
6

6. Agenda
NFV Testing
Challenges & Solutions
Automated NFV Testing
Environment as a Service
Summary & Q&A
QualiTest & QualiSystems
Partnership
7
High Level OPNFV

7. OPNFV
Targets
|OPNFV aims to be a carrier-grade, integrated platform that
introduces new products and services to the industry more
quickly.
|Create an integrated and verified open source platform that
can investigate and showcase foundational NFV
functionality
|Provide proactive cooperation of end users to validate
OPNFV’s strides to address community needs
|Form an open environment for NFV products founded on
open standards and open source software
|Contribute and engage in open source projects that will be
influenced in the OPNFV reference platform
8

8. OPNFV
Advantages
|Higher reliability, more flexibility
|Faster, lower cost, and higher quality development
|Collaborative decisions about new features and roadmaps
|A common environment for uses and app developers
|Ability for users to focus resources on differentiating
development
|Opportunity to drive open standards
9

9. Agenda
NFV Testing
Challenges & Solutions
Automated NFV Testing
Environment as a Service
Summary & Q&A
QualiTest & QualiSystems
Partnership
10
High Level OPNFV

10. Strategic
Testing
Challenges
| NFV involves very complex integration of various software elements
| Everything known becomes unknown
| Network testing expertise verses Virtualization and Software testing
| Agile orientation will be introduced in Network testing
| Frequent updates in the Network services will require more management and orchestration
testing
| More test automation is required to support agility , impact testing and regression testing
| NFV test strategies are needed to accelerate and ensure the delivery of NFV solutions with
guaranteed quality
| NFV is not only a technological shift it involves a culture change which impacts testing
directly
11

11. Test Strategy
Goals
|Build an NFV Ecosystem
|Drive Quality and governance
|Continuous integration / Dev Ops model
|Reduce cost of integration.
12

12. QualiTest
NFV
Validation
Process
|QualiTest aims to create a unified end-to-end approach to
service assurance and network performance
NFV
Management
Orchestration Infrastructure Migration
13

13. QualiTest NFV validation process
Assure full
compliance to ETSI
ISG Standards.
•Define NFV Testing
process
•Test Requirements
•NFV Test
Environment
•Risk Assessment
•Roles and
Responsibilities
•Team Skills and
Training
•Test Inputs / Basis
Assure full
compliance to NFV
requirements for
any network
equipment provider
and streamline the
ability of vendors
and users to be able
to implement
compatible NFV
solutions.
Test Environment:
simulation tools,
network analytics,
QoS and QoE
probes.
Automated
Certification Process
for each VNF before
it is introduced to
the network
Test management
and Orchestration
prior to VNF
migration, including
each component
(e.g. Orchestrator,
Vnfm, VIM).
Test the VNF
architecture and
implementation to
assure smooth
transition.
NFV
Test Policy
ETSI NFV ISG
Compliance
(NFV-Co)
Selecting
Mano Vendors
(NFV-Ven)
NFV Staging
Environment
(NFV-SE)
MANO Testing
(MANO-T)
VNF Migration
(VNFM-T)
14

14. What is
DevOps?
|Who: Developers, QA and Operations
|What (Goal): Clarity, Consistency and Collaboration to
understand the changes that each release brings to the
IT and customer environments
|Why: Increase business benefits by reducing the
transaction cost associated with delivering incremental
change.
|How: Methodology and tools that continue to evolve
15

15. Where does
testing fit in
DevOps?
|Addressing a misconception
|Testing is not a bottleneck
|Testing is a junction point between development
and operations
|Testing is a change agent
|DevOps is about closing the distance between
development and operations through:
|Meaningful collaboration
|Early engagement
|Continuous processes
16

16. Shift Left Principle and implementations
|Engage QA early
|Push tests to lower levels
|Test continuously
|Benefits:
|Fewer defects created
|Defects discovered earlier
|Cost per defect drastically reduced
17

17. Positioning QA with
Development and Operations
|Development
|Engage as early as possible in the
life cycle
|Generate test automation at
lower levels within the product
|Reduce the overall demand for
manual testing and testing at the
GUI level
| Operations
|Testing and Monitoring in all
environments
• Environments should be as
production-like as possible
• QA needs more control and access
to environments
|Continuous feedback and
communication
|Testing of Operations processes
18

18. The Evolution of the Modern Tester
|Traditional Tester Skills
• Analytical and Logical Thinking
• Intellectual Curiosity
• Critical Thinking and Rational
Deduction
• Ability to Identify and Apply
Fundamental Knowledge
| Modern Tester Skills
• Grasping the Bigger Picture
• Ability to Recognize and Address
Design Challenges
• Ability to Communicate on both
Technical and Business Needs
• Ability to identify appropriate use
cases for manual testing
(exploratory testing for learning
and test design for example)
• Ability to generate automation
19

19. Agenda
NFV Testing
Challenges & Solutions
Automated NFV Testing
Environment as a Service
Summary & Q&A
QualiTest & QualiSystems
Partnership
20
High Level OPNFV

20. 21
Automated BuildCode Commit Automated Unit Test
Automated Sanity TestAutomated & Scaled Full Regression TestAutomated Test Results
Collection
Automated BI Report Generation
Automated Build, Deploy, Test
Auto-scaled regression tests
Applied to service & network changes
Goal: Agile and Continuous Service Deployments

21. Step 1: Automate all Infrastructure
Physical Labs
MaaS
Virtual Labs
IaaS & PaaS
• Build virtual labs for VNF-only scenarios
• Consolidate or federate virtual & physical labs into private/hybrid clouds

22. Ensure Orchestration can Handle Network
Topologies
=
Carriers adopting infrastructure orchestration to build DevOps have found
that most cloud management platforms only support compute concepts
CloudShell orchestration is capable of handling E2E NS topologies

23. Step 2: Create Lab as a Service Cloud
• Create self-service access to full network
environments built and provisioned on
demand
• Make available to internal and external
users (including vendors, contractors)
Key goals:
• Increase efficiency utilization of test lab
resources
• Reduce ratio of setup to test time
Select
environment
Verify
availability
Reserve
Setup
Testing
activities
Collaborate
Teardown

24. • “One-Click” is first step toward continuous
• Self-service users can easily launch entire certification from UI
• Integrate with ALM to ensure consistent reporting
• Certification can be reused for many stages:
• New technology and early stage service proof of concept
testing
• New service introduction
• Vendor interoperability testing
• Contractor code qualification
Step 3: Create “One-Click” Certification

25. Tight Integration between TestShell & CloudShell
Create an
Environment
Develop a Test
Create the
Automation
Suite
Execute and
Monitor the
Suite
Develop test in TestShell Studio in conjunction
with a live CloudShell environment for greater accuracy,
easier debugging
Suite = reservations & associated tests

26. What Happened when a Cable MSO Built an
“As-a-Service” Approach
Overall: Complete change in the way devtest team conducted business
Measured:
• 15% annual operations cost savings
• 20% immediate reduction in certification cycles
• 33% immediate increased in certification coverage and improving
• 100% increase in resource utilization due to 24/7 service availability
Ongoing observations:
• Earlier certification by internal departments and vendors moves quality to the left
in the cycle—still measuring outcomes
• Eliminated need for extensive travel time and costs on part of vendors and
contractors
• Increased compliance with their methodology and standards via automation
• Increased pace of innovation

27. Agenda
NFV Testing
Challenges & Solutions
Automated NFV Testing
Environment as a Service
Summary & Q&A
QualiTest & QualiSystems
Partnership
28
High Level OPNFV

28. Summary
| Automation of both physical and virtual is key to leveraging the
opportunity for agility that NFV represents
| On-Demand Test Environments and Lab-as-a-Service enable
DevOps agility for NFV validation and certification
| Adopting new technologies like On-Demand Environments with
Test Automation will enable your organization to maximize revenue
and speed innovation
| Testing in the new world of NFV in general and OPNFV in particular,
changes dramatically and becomes a core discipline that is central
to the overall goal of pleasing customers in a cost effective way
| DevOps improve times to NFV solution implementation
| NFV Testing is not only matter of technology, it’s involves cultural
changes and testing is the appropriate change agent
29

29. 30
www.QualiTestGroup.com/NFV | www.qualisystems.com
Visit us at
November 9-12, San Francisco, CA

30. www.QualiTestGroup.com