NFV Testing & DevOps | QualiTest
•Download as PPTX, PDF•
2 likes•1,448 views
QualiTest hosts a webinar NFV Testing and DevOps. Hosted by Benny Sand and Byron Lown Hosted on: 9/24/2015 QualiTest is the world’s second largest pure play software testing and QA company. Testing and QA is all that we do! visit us at: www.QualiTestGroup.com
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (18)
Similar to NFV Testing & DevOps | QualiTest
Similar to NFV Testing & DevOps | QualiTest (20)
More from Qualitest
More from Qualitest (18)
Recently uploaded
Recently uploaded (20)
NFV Testing & DevOps | QualiTest
- 1. NFV Testing & DevOps
- 2. Agenda NFV – Challenges NFV Testing & DevOps Enablement Summary & Questions Network DevOps Test Approaches Network DevOps Introduction 3
- 3. Agenda NFV – Challenges NFV Testing & DevOps Enablement Summary & Questions Network DevOps Test Approaches Network DevOps Introduction 4
- 4. 5 Bryon Lowen Vice President Telco at QualiTest Benny Sand NFV Strategy Lead at QualiTest Webinar Presenters
- 5. About QualiTest Group | World’s 2nd Largest Independent Testing Company | QualiTest was recognized as the only visionary pure play QA and testing Service Company in | the “ Magic Quadrant” 6
- 7. Agenda NFV – Challenges NFV Testing & DevOps Enablement Summary & Questions Network DevOps Test Approaches Network DevOps Introduction 8
- 8. NFV Predications |NVP planning phase is shifting gear |Operators are moving along the NFV curve |Financial forecasts indicate a significant growth in NFV investments |NFV will start to influence budgets and purchasing decisions |NFV adaption pace is progressing faster then expected 9
- 9. NFV impacts on OPEX |OPEX Redactions |Shared computing resources |Reduced power, space, cooling requirements |Executing trials of new services, without incurring much risk |Proof of Concepts (POCs) and trials can be run faster |Dynamic recovery from failures |Accelerated time -to-market |Stronger capability to bring up services anywhere in the world at any time. 10
- 10. NFV impacts on CAPEX |The use of commodity servers reduces hardware costs |No dependence on specialized hardware |A single, common server architecture can be used within the data center environment. |Migration to a distributed cloud architecture allows operators to deploy backup in a cost effective way |Cycling the hardware more often improves the overall performance of the network 11
- 11. Strategic challenges I |NFV involves very complex integration of various software elements |Everything known becomes unknown |Network testing expertise versus Virtualization and Software testing |Agile orientation will be introduced in Network testing 12
- 12. Strategic challenges II |Variety of NFV infrastructures means that more testing will be required by the carriers |Frequent updates in the Network services will require more management and orchestration testing |More test automation is required to support agility, impact testing and regression testing |NFV test strategies are needed to accelerate and ensure the delivery of NFV solutions with guaranteed quality 13
- 13. Agenda NFV – Challenges NFV Testing & DevOps Enablement Summary & Questions Network DevOps Test Approaches Network DevOps Introduction 14
- 14. What is DevOps? |Who: Developers, QA and Operations |What (Goal): Clarity, Consistency and Collaboration to understand the changes that each release brings to the IT and customer environments |Why: Increase business benefits by reducing the transaction cost associated with delivering incremental change. |How: Methodology and tools that continue to evolve 15
- 15. Where does testing fit in DevOps? |Addressing a misconception |Testing is not a bottleneck |Testing is a junction point between development and operations |Testing is a change agent |DevOps is about closing the distance between development and operations through: |Meaningful collaboration |Early engagement |Continuous processes 16
- 16. Shift Left Principle and implementations |Engage QA early |Push tests to lower levels |Test continuously |Benefits: |Fewer defects created |Defects discovered earlier |Cost per defect drastically reduced 17
- 17. Why Network DevOps is Key to SDN and NFV “ There remains much to do before this vision [Domain 2.0] can be implemented, including pivots from networking craft to software engineering, and from carrier operations models to cloud “DevOps” models. We also see an important pivot to embrace agile development in preference to existing waterfall models. ” 18 - AT&T’s Domain 2.0 initiative white paper
- 18. Positioning QA with Development and Operations |Development |Engage as early as possible in the life cycle |Generate test automation at lower levels within the product |Reduce the overall demand for manual testing and testing at the GUI level | Operations |Testing and Monitoring in all environments • Environments should be as production-like as possible • QA needs more control and access to environments |Continuous feedback and communication |Testing of Operations processes 19
- 19. Agenda NFV – Challenges NFV Testing & DevOps Enablement Summary & Questions Network DevOps Test Approaches Network DevOps Introduction 20
- 20. Building SDN / NFV Network Dev Ops assurance process 21 Build strong assurance process to allow Network DevOps to capitalize SDN / NFV potential |Renegotiate contracts |Right Price |Good SLAs |Increasingly Complex |Renegotiate contracts |Not affect QOS
- 21. QualiTest NFV validation process QualiTest aims to create a unified end-to-end approach to service assurance and network performance 22 NFV Management Orchestration Infrastructure Migration
- 22. The Evolution of the Modern Tester |Traditional Tester Skills | Analytical and Logical Thinking | Intellectual Curiosity | Critical Thinking and Rational Deduction | Ability to Identify and Apply Fundamental Knowledge | Modern Tester Skills | Analytical and Logical Thinking | Reading and Writing Code | Grasping the Bigger Picture | Ability to Recognize and Address Design Challenges | Ability to Communicate on both Technical and Business Needs | Ability to identify appropriate use cases for manual testing (for example, exploratory testing for learning and test design) 23
- 23. Agenda NFV – Challenges NFV Testing & DevOps Enablement Summary & Questions Network DevOps Test Approaches Network DevOps Introduction 24
- 24. NFV framework – Test areas Overall, the NFV framework consists of three main components: 1.Virtualized Network Functions (VNF) 2.NFV Infrastructure (NFVI) 3.Network Functions Virtualization Management and Orchestration Architectural Framework (NFV-MANO Architectural Framework) 25
- 25. Example: MMS VNF Test Strategy Multimedia Messaging Service (MMS) is similar to the Short Message Service (SMS) in that a message is passed between an Originating Party (OP) and a Destination Party or Parties (DP). The core Network Element is the Multimedia Messaging Service Centre (MMSC) which acts as a central store and forward server between the OP and DP, as shown below. 26
- 26. MMS VNF Network Dev Ops Test Approach |Static testing (drive defect curve) |Scope (requirements/functionality of VNF) |Shape test (risk based / ISO9126) |Define levels of testing |Create Test environment (NFV-TE): | Simulation tools | Network analytics | QoS / QoE probes | Onboarding Automated Certification Process 27
- 27. QualiTest NFV validation process Assure full compliance to ETSI ISG Standards. •Define NFV Testing process •Test Requirements •NFV Test Environment •Risk Assessment •Roles and Responsibilities •Team Skills and Training •Test Inputs / Basis Assure full compliance to NFV requirements for any network equipment provider and streamline the ability of vendors and users to be able to implement compatible NFV solutions. Test Environment: simulation tools, network analytics, QoS and QoE probes. Automated Certification Process for each VNF before it is introduced to the network Test management and Orchestration prior to VNF migration, including each component (e.g. Orchestrator, Vnfm, VIM). Test the VNF architecture and implementation to assure smooth transition. NFV Test Policy ETSI NFV ISG Compliance (NFV-Co) Selecting Mano Vendors (NFV-Ven) NFV Staging Environment (NFV-SE) MANO Testing (MANO-T) VNF Migration (VNFM-T) 28
- 28. 29 The Need for Automation | DevOps produces Up to x30 Code Shipments | Completing deployments up to 8,000 times faster | Automation becomes a must have | Coverage to reach 100%
- 29. 30 Traditional vs. DevOps Automation Element Traditional DevOps What to automate? Mainly sanity and regression New features Automation Trigger Manual CI tool Staff Automation experts All-around with deep business knowledge Tools Commercial Open-source Automation Owner QA Dev + Ops + QA Automation techniques GUI + API API Dependency between scripts Flow Independent automation units Where to automate Test Environment Dev + Test + Production environments
- 30. DevOps Automation 31 |Every layer of testing should be automated |The concept of executing all regression test cases for a test pass is soon becoming obsolete. |Strategy around testing new features needs to be formalized and interim builds can be supplied to QA
- 31. Test strategy goals |Build an NFV Ecosystem |Drive Quality and governance |Continuous integration / Dev Ops model |Reduce cost of integration 32
- 32. Agenda NFV – Challenges NFV Testing & DevOps Enablement Summary & Questions Network DevOps Test Approaches Network DevOps Introduction 33
- 33. Summary |NFV allows agile real time shifting of assets to where they are needed |Testing, in the new world of NFV, changes dramatically and becomes a core discipline that is central to the overall goal of pleasing customers in a cost effective way |NFV testing is the change agent from the physical networks to the virtualized ones |DevOps leveraging On-Demand Environments is the industry direction |Adapting new technologies like On-Demand Environments with Test Automation will enable your organization to MAXIMIZE REVENUE |DevOps improves time to NFV solution implementation |Network DevOps is the future in NFV 34
Editor's Notes
- Good morning.afternoon,evening, depending on where you are in the world. We have a good count of people registered and 100+ on the the Qualitest webinar “ NFV Testing DevOps This webinar is recorded and we We invite you ask questions via the goto webinar.... Before we dive into presentation, i would like to introduce myself and colleague Bryon Lown who will co host with me the wwebinar. My name is Benny Sand , I am a director of business development in Qualitest around 18 years in the sw testing arena in various technical and business capacities leading the our NFV imitate form the strategic perspective Bryon can you present your self? Bryon intro: Hello - I am Vice-President of Telco at QualiTest UK, over 17 years’ experience in the Telecommunications responsible for some of QualiTests largest telco clients over the last 5years. I am leading the QualiTest NFV initiative, and I am a member of the ETSI NFV forum, helping to shape the future of the technology.
- Introduction – a little bit about QualiTest NFV present and Future addressing the OPEX and CAPEX IMPACT AS WELL PREFCIATIONS REGARDING THE nfv Market. Then I will shift the baton to Bryon who is going to talk through the QualiTest approach and best practises in how you should start to build an nfv/sdn assurance process After this then he is going to talk through, how you should consider building an NFV test strategy and using an example from a tier 1 telco operator we at qualitest are working with. Intros BS – qualitest overview BS – nfv challenges BS – NFV enables network dev ops BDL – talk about network dev ops. BDL – talk about how the test approach will differ under network dev ops and NFV. BS – summary?
- Introduction – a little bit about QualiTest NFV present and Future addressing the OPEX and CAPEX IMPACT AS WELL PREFCIATIONS REGARDING THE nfv Market. Then I will shift the baton to Bryon who is going to talk through the QualiTest approach and best practises in how you should start to build an nfv/sdn assurance process After this then he is going to talk through, how you should consider building an NFV test strategy and using an example from a tier 1 telco operator we at qualitest are working with. Intros BS – qualitest overview BS – nfv challenges BS – NFV enables network dev ops BDL – talk about network dev ops. BDL – talk about how the test approach will differ under network dev ops and NFV. BS – summary?
- Good morning.afternoon,evening, depending on where you are in the world. We have a good count of people registered and 100+ on the the Qualitest webinar “ Testing the Gate Keeper for Quality in NFV This webinar is recorded and we We invite you ask questions via the goto webinar.... Before we dive into presentation, i would like to introduce myself and colleague Bryon Lown who will co host with me the wwebinar. My name is Benny Sand , I am a director of business development in Qualitest around 18 years in the sw testing arena in various technical and business capacities leading the our NFV imitate form the strategic perspective Bryon can you present your self? Bryon intro: Hello - I am Vice-President of Telco at QualiTest UK, over 17 years’ experience in the Telecommunications responsible for some of QualiTests largest telco clients over the last 5years. I am leading the QualiTest NFV initiative, and I am a member of the ETSI NFV forum, helping to shape the future of the technology.
- The world's second largest Independent pure play testing specialist. we have subsidiaries in the US Europe and Asia Testing is all we do. We have a real passion for testing, ensuring things work! And we have recognized by Gartner as the only visionary pure play QA and testing services company
- One of our largest sector is telecomunications – we have years of experience working for operators and vendors, from detailed network integration to high level UAT handset testing. We are now working with a number of our telco clients looking at how we can help them transition to an NFV / SDN environment, and building testing best practices that go along with that.
- Introduction – a little bit about QualiTest NFV present and Future addressing the OPEX and CAPEX IMPACT AS WELL PREFCIATIONS REGARDING THE nfv Market. Then I will shift the baton to Bryon who is going to talk through the QualiTest approach and best practises in how you should start to build an nfv/sdn assurance process After this then he is going to talk through, how you should consider building an NFV test strategy and using an example from a tier 1 telco operator we at qualitest are working with. Intros BS – qualitest overview BS – nfv challenges BS – NFV enables network dev ops BDL – talk about network dev ops. BDL – talk about how the test approach will differ under network dev ops and NFV. BS – summary?
- NFV is in the learning phase we are shifting gear to the stategic planing The SDxCentral predicts the turnover of SDN, NFV and other next-generation networking initiatives (SDx Networking) will exceed $105B per annum by 2020. The ACAGR of NFV till 2020 is approximately 45% According to SDxCentral in 2015 /16 many operators will move from PoC's to collaborating with vendors to develop and produce software solutions that will furnish a foundation for commercial deployments. It’s expected these technologies will influence almost 80% of the purchasing decisions associated with all networking revenue by the end of 2020, affecting virtually every customer segment within the networking space.
- OPEX reduction is being demonstared by Share computing resources between functions using hypervisor technology A significant point is the capability if Executing trials of new services, without incurring much risk The ability to dynamically recover from failures by using an orchestrating framework dramatically decrease the risk of deploying new products from vendors. Moving and scaling functionality, subject to specific requirements , drives service innovation. So Proof of Concepts (POCs) and trials can be run faster, in smaller scale environments; CAPEX The use of commodity servers reduces hardware costs. A single, common server architecture can be used to build in the redundancy and availability organizations require within their data center environment. Migration to a distributed cloud architecture allows operators to deploy backup in software using an N+1 configuration, versus having to invest in and warehouse lots of extra equipment. Cycling the hardware more often to improve the overall performance of the network A side benefit of using less expensive commodity hardware is that an organization can potentially cycle the hardware more often to improve the overall performance of the network. By upgrading the network every 2 to 3 years, instead of the traditional 5 to 7, an organization can continue to effectively address the changing demands placed on their network and increase the value captured throughout the lifetime of those servers. The use of commodity servers reduces hardware costs. No dependence on specialized hardware By delivering the services in software, organizations are no longer forced to rely on specialized hardware to run network functions. This means the premium that a handful of vendors could charge for their proprietary hardware is no longer applicable or justifiable.
- OPEX reduction is being demonstared by Share computing resources between functions using hypervisor technology A significant point is the capability if Executing trials of new services, without incurring much risk The ability to dynamically recover from failures by using an orchestrating framework dramatically decrease the risk of deploying new products from vendors. Moving and scaling functionality, subject to specific requirements , drives service innovation. So Proof of Concepts (POCs) and trials can be run faster, in smaller scale environments; CAPEX The use of commodity servers reduces hardware costs. A single, common server architecture can be used to build in the redundancy and availability organizations require within their data center environment. Migration to a distributed cloud architecture allows operators to deploy backup in software using an N+1 configuration, versus having to invest in and warehouse lots of extra equipment. Cycling the hardware more often to improve the overall performance of the network A side benefit of using less expensive commodity hardware is that an organization can potentially cycle the hardware more often to improve the overall performance of the network. By upgrading the network every 2 to 3 years, instead of the traditional 5 to 7, an organization can continue to effectively address the changing demands placed on their network and increase the value captured throughout the lifetime of those servers. The use of commodity servers reduces hardware costs. No dependence on specialized hardware By delivering the services in software, organizations are no longer forced to rely on specialized hardware to run network functions. This means the premium that a handful of vendors could charge for their proprietary hardware is no longer applicable or justifiable.
- NFV enables to accelerate time time-to-market by delivering the flexibility and agility they need to move and scale functionality to address changing business requirements. NFV enables organizations the ability to bring up services anywhere in the world, any time. Having said that it involves very complex integration of various software elements, piece by piece, from the most granular to the most complex. גרנולרי Which Network functions should be virtualized and when So the big risk Everything known becomes unknown Last but defiantly not least NFV is more then shift of technologies , it’s a cultural transformation in Telecom organizations
- NFV enables to accelerate time time-to-market by delivering the flexibility and agility they need to move and scale functionality to address changing business requirements. NFV enables organizations the ability to bring up services anywhere in the world, any time. Having said that it involves very complex integration of various software elements, piece by piece, from the most granular to the most complex. גרנולרי Which Network functions should be virtualized and when So the big risk Everything known becomes unknown Last but defiantly not least NFV is more then shift of technologies , it’s a cultural transformation in Telecom organizations
- Introduction – a little bit about QualiTest NFV present and Future addressing the OPEX and CAPEX IMPACT AS WELL PREFCIATIONS REGARDING THE nfv Market. Then I will shift the baton to Bryon who is going to talk through the QualiTest approach and best practises in how you should start to build an nfv/sdn assurance process After this then he is going to talk through, how you should consider building an NFV test strategy and using an example from a tier 1 telco operator we at qualitest are working with. Intros BS – qualitest overview BS – nfv challenges BS – NFV enables network dev ops BDL – talk about network dev ops. BDL – talk about how the test approach will differ under network dev ops and NFV. BS – summary?
- A common misconception in the past has been to view QA as a bottleneck between dev and ops. This was never truly in line with reality and DevOps makes it even less so. QA is and should be talked about as an enabler of DevOps and faster time to market. What DevOps is really about is a combination of meaningful collaboration and early engagement supported by continuous processes linking all facets of software development and maintenance.
- With the primary high level goal of DevOps being to shorten the gap between dev and ops, we arrive at the Shift Left Principle. The idea is to engage everything that would traditionally have resided downstream from development at much earlier stages in the SDLC. This obviously includes testing. QA needs to be engaged far earlier in the process, preferably long before development actually starts. Tests need to be pushed to lower levels within the product with a strong emphasis on unit and API level testing. And testing needs to be performed continuously, at every stage of the lifecycle. With an early engagement model to product creation, projects will create fewer defects, find defects earlier, and drastically reduce the overall cost of defects in general.
- BDL – I am now going to talk through the QT network devops model, and how we recommend you approach Dev ops.
- So what is network dev ops? QualiTest see NFV/SDN as a huge enabler for Dev ops in the networks space. The current slow model for development and changes in the network space will accelarate, and testing must adapt to add value in this process or will become the problem. QualiTest has developed its own assurance process, which starts to look at how a network dev ops testing model can be implemented So why network dev ops, so If an operator is to find benefits in the new technology, they primarily want reliable services, at the right price, supported by strong SLAs. ALL FAST PACED. And overall new technology cannot be adopted at the expense of the quality of service (QoS), and if customers do notice a disruption in service, they could seek to renegotiate their contracts.
- QualiTest aims to create a unified end-to-end approach to service assurance and network performance While new technology is obviously a big part of this transformation, one of our biggest challenges most companies will have will be the cultural shift. The move to a software-centric architecture means becoming more software focused. Companies will need to make a "skills pivot" QualiTest help our customers transform their physical network into a visualized network, with innovative strategies This will also include handling the fast changes in the network and testing at a pace.
- Next I am going to give you an example of how we have applied the approach for a network operator.
- Virtualized Network Functions (VNF) are software implementations of network functions NFV Infrastructure (NFVI) is the totality of all hardware and software components which build up the environment in which VNFs are deployed. Network Functions Virtualization Management and Orchestration Architectural Framework (NFV-MANO Architectural Framework) these functional blocks exchange information for the purpose of managing and orchestrating NFVI and VNFs. For the purpose of this webinar – iam going to talk through how best to approach a network devops Test strategy for an example VNF.
- Mention QualiSystems In network dev ops automation is the key. QualiTest have automated a number of the ETSI/validation process.,. The key to successful dev ops is every step/layer of testing is automed, from on boarding a new VNF, checking it works with mano, interfaces, and even end2end testing. The aim will be for 100% automation of code.
- The QualiTest validation process is segment into 6 key sub-processes: This approach has been established around the guidelines of the ETSI NFV standards. QualiTest have automated a number of these procesess so they are ready for dev ops delivery.
- Prioritise testing
- Need to build in from day one a clear automation framework. We don’t want testing to become the bottleneck in this fast paced new world.
- Introduction – a little bit about QualiTest NFV present and Future addressing the OPEX and CAPEX IMPACT AS WELL PREFCIATIONS REGARDING THE nfv Market. Then I will shift the baton to Bryon who is going to talk through the QualiTest approach and best practises in how you should start to build an nfv/sdn assurance process After this then he is going to talk through, how you should consider building an NFV test strategy and using an example from a tier 1 telco operator we at qualitest are working with. Intros BS – qualitest overview BS – nfv challenges BS – NFV enables network dev ops BDL – talk about network dev ops. BDL – talk about how the test approach will differ under network dev ops and NFV. BS – summary?
- Bryon do his questions first Then benny.. Questions What are the current issues that are being raided form the field Is thee a difference between the testing requirements of operators and vendors Then benny wrap it up…. Thankyou for attending, we will send out a recording of the webinar. Please let us know if you have any requirements we can support.