SlideShare a Scribd company logo
1 of 11
Download to read offline
PrimeSoft Solutions, Inc.


   Multi Access Gateway: Case Study

                            August 26, 2010




This document contains information proprietary to PrimeSoft Solutions, Inc.
Reproduction, release, or use of the information contained in this document is
strictly prohibited without prior written permission from PrimeSoft Solutions,
Inc.
Multi Access Gateway : Case Study


Multi Access Gateway

Client Offshore Development Center (ODC)

This Santa Clara, CA, company develops and markets industry's first multi-access
convergence gateway designed for stateful management of tens of thousands of secure,
reliable, high-quality subscriber sessions over multiple network access types including Wi-
Fi and WiMAX.

To date, the company has raised over $50M and is backed by Sequoia Capital and Kleiner
Perkins among other venture capital firms.

Relationship Model
PrimeSoft is the chosen Outsourced Product Development partner for the client. PrimeSoft
developed this relationship by providing the following at our offshore development center

   1. Building testing, automation and development teams
   2. Identifying an Project Manager
   3. Setting a Lab and setting it up with Client specific software and hardware
      components
   4. Providing supported from HR, Admin and Facilities Management functions
The offshore team consists of 18 engineering resources divided into QA, Automation and
Development. The teams function as an extension to the client’s team in USA, and
participate in various projects during a release.

Weekly status reports are provided by the Project Manager on all active offshore projects,
followed by conference calls to discuss any open issues.

UMA Handset Simulator
Technology Brief
Unlicensed Mobile Access (UMA) technology provides access to GSM and GPRS mobile
services over unlicensed spectrum technologies, including Bluetooth and 802.11. By
deploying UMA technology, service providers can enable subscribers to roam and perform
handover between cellular networks and unlicensed wireless networks using dual-mode
mobile handsets. With UMA, subscribers receive a consistent user experience for their
mobile voice and data services as they transition between networks.

Project Objectives
The objective of this project was to develop a UMA Handset Simulator. UMA simulator
came out as a requirement from the development and automation engineers to help them
perform unit testing and automation of UMA functionality.




PrimeSoft Confidential                                                         Page 2 of 11
Multi Access Gateway : Case Study


UMA simulator was required support functional, to help automate more than 90% of all
functional tests and help development engineers run pre-commit unit tests to ensure basic
functionality.




PrimeSoft Confidential                                                       Page 3 of 11
Multi Access Gateway : Case Study



Our Approach
The Team

PrimeSoft identified a technical lead, development & test engineers to staff this project.
The team had prior experience in working with telecom and data communications
protocols and call processing.

Project Execution

       Requirements Analysis
       A team from PrimeSoft traveled to US client site for gathering requirements. The
       team had brainstorming sessions with client’s technical team and end users of the
       UMA simulator.

       The requirements were documented in a Functional Requirements Document
       Template and shared with all the stake holders, followed by a formal review.
       Review comments were incorporated and the updated document was checked into
       CVS and posted on internal Wiki.

       Functional Specifications & Software Design
       Functional Requirements Document was an input for developing a Functional
       Specifications Document. The HLD (High Level Design) and LLD (Low Level
       Design) documents were developed based on the functional specifications. The
       design document identified modules, interfaces, and presented pseudo code.

       Coding, Unit Testing & Acceptance Testing
       Coding of the tool involved setting up the environment and developing code based
       on the low level design. The code was unit tested by development engineers.

       System testing of the simulator was carried out by an independent team that
       developed system test plan document based on the requirements document and
       executed multiple test cycles and logged bugs using Bugzilla.

       User acceptance test plan was developed by PrimeSoft QA team and shared with
       the client. Acceptance testing was performed jointly by PrimeSoft and client
       teams.

Salient Features

       Object oriented Design approach provides well defined programming interfaces to
       the scripting layer
       Provides interfaces to build and parse and UMA and GSM Layer3 messages
       Multithreaded implementation for RTP sends and receives for multiple handset
       instances
       Python extensions provided for abstracting the complex scripting functionality and
       reduce scripting effort
       Supports complex handover scenarios by simulating two UMA connections from a
       single Handset




PrimeSoft Confidential                                                        Page 4 of 11
Multi Access Gateway : Case Study




Project Deliverables
 Deliverable                     Details
                                 Functional Requiremnts Document
                                 Functional Specifications Document
 Documents                       Design Document
                                 Users Guide
                                 Release Notes
                                 UMA Simulator Source Code (C/C++, Makefiles, Shell
 Source Code
                                 Scripts)
                                 Unit Test Plan
 Test Documents
                                 Unit Test Matrix
 Bugs                            BUGS Logged into Bugzilla
 Sample Scripts                  Sample Automated Scripts
 Users Guide                     Automation suite and scripts users guide

                                 Traces all the requirements against Functional
 Requirements Traceability
                                 Specifications, Design, System Testing, Unit Testing
 Matrix
                                 and Users Guide


Tools & Technologies
 Title                          Details
 Protocols                      UMA, GSM, TCP/IP
 Operating Systems              Redhat Linux
 Programming & Scripting        C, C++, Python, Shell Scripts, Makefiles
 Configuration Management       CVS




PrimeSoft Confidential                                                            Page 5 of 11
Multi Access Gateway : Case Study



Multi Access Gateway Testing
PrimeSoft houses the largest testing team for our client. Our testing team is involved in the
entire test life cycle of the product including test strategy development, test plans
development, tests cases development, manual testing, filing bugs and running regression
cycles.

Project Objectives
The main objective of the testing group is to work as an extension to client’s onsite
engineering teams, and be responsible for test and automation activities involved in the
project.

The offshore team is responsible of the following project activities
   1. Defining QA and Automation Strategy for a identified release
   2. Developing QA plan for the identified release
   3. Developing Test Plans and Test Case documents based on
           a. Marketing and Product Requirements Documents
           b. Functional Specifications
           c. Design Documents
           d. RFC’s Supported
   4. Performing Functional, System, Sanity, Regression, Load and Stress, Security and
      Interop testing
   5. Identifying and logging bugs, coordinating with developers in re-producing and
      analyzing bugs
   6. Developing test matrix and reporting on manual tests progress on a daily/weekly
      basis

Testing Projects
PrimeSoft team worked with client on multiple releases of this product and was involved in
testing and owning QA and Automation for the following modules

    IPSec IKE v2, IKEv1
    MOBIKE (Mobile IKE)
    RIP/OSPF
    VLAN (Virtual LAN)
    Session Initiation Protocol (SIP)
    QoS (Quality of Service)
    AAA (Authentication, Authorization and Accounting) and RADIUS
    Change of Authorization (CoA)
    Internet Protocol Slow Path


PrimeSoft Confidential                                                           Page 6 of 11
Multi Access Gateway : Case Study


    Point to Point over Ethernet (PPPoE)
    Dynamic Host Control Protocol (DHCP)
    PDIF (Protocol Data Interworking Function)
    CDR (Call Detail Records)
    IP-IP Encapsulation
    DOS Attacks
    ACL (Access Control List)
    SNMP (Simple Network Management Protocol)
    RTC (Real Time Charging)
    HA (High Availability)
    System Testing
    Performance Testing


Our Approach
The Team
We created a dedicated test team who were technically competent in networking
technologies. Project Manager and a core group of engineers started the project by
performing manual testing on one of the releases and logged bugs. The team developed
its competency on the product by organizing
       Internal training seminars
       Technical hands on sessions
       Developing internal wiki pages
       Going through past bugs reports
       Communicating with developers and PLM teams
Test Beds

A Site-to-Site VPN was setup between PrimeSoft, and Client location in USA, allowing
engineers to reserve, configure and access remote test setups.

PrimeSoft also has a lab setup with client specific software and hardware available locally
for engineers.

Project Execution

Test Projects execution followed the following phases.

       Analysis
       The aim of this phase was to technically be competent in the technology and
       enhance product understanding and gather all the information necessary to
       develop a test plan document and completely define the scope of testing activities.

       The team initiated the projects by going through RFC’s and product documentation
       including, Requirements, Functional Specifications and High Level Design. Email



PrimeSoft Confidential                                                         Page 7 of 11
Multi Access Gateway : Case Study


       exchanges and conference calls were conducted to understand the nuances of
       implementation on client’s product.

       Test Plan Development
       Test plan development involved, identification and documenting

           1. Test Topology
           2. Software and Hardware Tools
           3. Entry and Exit Criteria
           4. Conformance scenarios from RFC’s
           5. Test cases against Requirements
           6. CLI Related test cases
           7. Functional test cases
           8. Negative test cases
           9. End-to-End tests
           10. SNMP related test cases
           11. High availability test cases

       Internal & External Reviews
       Internal peer reviews (board review) is performed and review comments are
       updated into the document. An internal review checklist is used to ensure the
       quality of the test plans is consistent.

       The update document is sent out for an external/client review followed by a walk
       through of the test plan through a conference call. Review comments received
       from architects, designers, implementers and PLM are consolidated and updated
       into the document.

       Updated document is checked into CVS and posted on internal Wiki.

       Manual Test Cycle
       Manual test cycles involved setting up test bed and executing test cases against
       the SUT (System Under Test) and tracking progress.

       A Test Matrix is filled for each cycle of manual test cycles across multiple builds,
       the test matrix captures testing results documented as PASS, FAIL or BLOCKED.
       The matrix also provides key metrics of the testing cycle including testing effort,
       bugs logged and passes percentage.

       Bugs were logged into client maintained bugzilla, with information including
           1. Bug description
           2. Build version
           3. Steps to reproduce



PrimeSoft Confidential                                                         Page 8 of 11
Multi Access Gateway : Case Study


             4. Logs and core files path
             5. Initial diagnosis findings
             6. Observation and Comments


Tools & Technologies
 Title                             Detail
 Protocols                         IPSec, IKE v1/v2, AAA, CoA, PPPoE, DHCP, SIP, SNMP

 Operating Systems                 REDHAT Linux, QNX
 Scripting                         Python, Shell Scripts
                                   Ethereal (Network Analyzer), Scapy (Packet Generator),
 Tools                             Silvercreek, SIPp, QA Robot, Smartbits, Landslide
                                   Avalanche, IXIA, IKEv1/v2 clients, DOS Attacks tools
 Configuration Management          CVS, Bugzilla


Project Deliverables

 Deliverable                       Details

 Test Plan                         Test Plan Document

 Test Matrix                       Test Cycle Summary in a Test Matrix Document

 Bugs                              BUGS Logged into Bugzilla

 Checklist                         Test Plan Release Checklist
 Requirements Traceability         Traces all the requirements against the test cases and
 Matrix                            bugs


Multi Access Gateway Automation
With a growing feature list in the product we sent out with a target of automating 50% of
our test cases, and this could significantly reduce the manual test cycle timeline.

PrimeSoft was involved in test automation from the very beginning and participated in
developing an automation framework, developing automated test scripts and suites.

Project Objectives
The objective of the automation group at PrimeSoft is to
         Design, Develop and Enhance an Object Oriented Automated Framework
         Develop automated test scripts
         Run automated regression on daily/weekly builds



PrimeSoft Confidential                                                               Page 9 of 11
Multi Access Gateway : Case Study


       Develop test suites for pre-commit, sanity, weekly regression and feature specific
       suites
       Identify and log bugs
       Fix scripts and maintain them across release and builds


Our Approach
The Team

We created a dedicated automation group within PrimeSoft who were technically
competent in scripting languages and had experience with relevant networking
technologies.

       Framework Enhancements
       Our team participated in design and development activities of the automation
       framework. An object oriented framework was designed using Python as the
       scripting language.

       Enhancements to the framework were performed and tested (using automated unit
       tests); and the framework was released to carry out the development of the scripts.

       Automated Scripts
       A team of automation engineers was selected to automate the test plans using the
       enhanced automation framework. Automation engineers focused on developing
       automated scripts and scripts development involved the following scripts
             Manual testing
             Develop and automated test script
             Running the test on a stable build to verify the behavior of the script
             Fixing all issues with the scripts
             Adding the script in a suite and running the suite

       Every developed script went through a through review with at least two reviewers
       and focused on functionality and coding guidelines.
              Functional Review: Review focused on the functionality of the scripts and
              compared them with what is described in the test cases
              Coding Guidelines Review: The review focused on usability, maintainability
              and adherence to coding guidelines
              Checked into CVS

Project Salient Features
       Python based Object Oriented Automation Framework
          o Can load/clear and check configuration on system under test
          o Can load/clear and check configuration on Juniper Netscreen hardware
          o Can initiate IKE v1 and v2 sessions from end clients
          o Can initiate PPPoE sessions from Linux based end clients
          o Can handle High Availability cases
       Exposes library functions for script developers



PrimeSoft Confidential                                                       Page 10 of 11
Multi Access Gateway : Case Study


         Automated Tests
              o   Topology File: Can be changed to point to a new topology
              o   Configuration File: Captures all common configuration for a test
              o   Suite file: Can include multiple test cases to be run in a suite

Tools & Technologies
 Title                             Detail
                                   SIP, SDP, RTP
 Protocols                         IPSec IKE v1/v2
                                   RIP, OSPF, BGP, DHCP, DNS, Radius, TCP/IP, PPPoE
 Clients Hardware/Software         Netscreen 5GT, XpressVPN, PPPoE Linux Client, SIPp
 Used                              test tool, Cisco Terminal Server
 Operating Systems                 Redhat Linux, QNX
 Programming & Scripting           Python, Perl, Shell Scripts
 Configuration Management          CVS, Bugzilla


Project Deliverables

 Deliverable                       Details
                                   Automated Scripts Developed as part of each
 Scripts
                                   functionality
 Test Suits                        Automated test suites for various types of testing

 Framework Libraries               Framework Libraries and Documentation

 Checklist                         Scripts Development and Review Checklist
 Requirements Traceability         Traces all the requirements against the test cases and
 Matrix                            bugs




PrimeSoft Confidential                                                                  Page 11 of 11

More Related Content

What's hot

Model Based Systems and Software Engineering an overview of the IBM Rational ...
Model Based Systems and Software Engineering an overview of the IBM Rational ...Model Based Systems and Software Engineering an overview of the IBM Rational ...
Model Based Systems and Software Engineering an overview of the IBM Rational ...
Real-Time Innovations (RTI)
 
Manoj resume
Manoj resumeManoj resume
Manoj resume
tekwissen
 
Maniteja_Professional_Resume
Maniteja_Professional_ResumeManiteja_Professional_Resume
Maniteja_Professional_Resume
Vaddi Maniteja
 

What's hot (20)

Caps Professional Services Diagnostic
Caps Professional Services DiagnosticCaps Professional Services Diagnostic
Caps Professional Services Diagnostic
 
VenutoResume
VenutoResumeVenutoResume
VenutoResume
 
Kahuna Systems : Product Engineering Services
Kahuna Systems : Product Engineering ServicesKahuna Systems : Product Engineering Services
Kahuna Systems : Product Engineering Services
 
Neil Tompson - SoftTest Ireland
Neil Tompson - SoftTest IrelandNeil Tompson - SoftTest Ireland
Neil Tompson - SoftTest Ireland
 
Objectif cloud
Objectif cloudObjectif cloud
Objectif cloud
 
Ast 2012 - Practices for Test Automation in Scrum Projects
Ast 2012 - Practices for Test Automation in Scrum ProjectsAst 2012 - Practices for Test Automation in Scrum Projects
Ast 2012 - Practices for Test Automation in Scrum Projects
 
Model Based Systems and Software Engineering an overview of the IBM Rational ...
Model Based Systems and Software Engineering an overview of the IBM Rational ...Model Based Systems and Software Engineering an overview of the IBM Rational ...
Model Based Systems and Software Engineering an overview of the IBM Rational ...
 
DineshCV (1)
DineshCV (1)DineshCV (1)
DineshCV (1)
 
Manoj resume
Manoj resumeManoj resume
Manoj resume
 
Resume
ResumeResume
Resume
 
Embedded services by Faststream Technologies
Embedded services by Faststream TechnologiesEmbedded services by Faststream Technologies
Embedded services by Faststream Technologies
 
Qtp sample resume
Qtp sample resumeQtp sample resume
Qtp sample resume
 
Safe and Reliable Embedded Linux Programming: How to Get There
Safe and Reliable Embedded Linux Programming: How to Get ThereSafe and Reliable Embedded Linux Programming: How to Get There
Safe and Reliable Embedded Linux Programming: How to Get There
 
Madhurima_Resume
Madhurima_ResumeMadhurima_Resume
Madhurima_Resume
 
Maniteja_Professional_Resume
Maniteja_Professional_ResumeManiteja_Professional_Resume
Maniteja_Professional_Resume
 
Rashmi_Resume
Rashmi_ResumeRashmi_Resume
Rashmi_Resume
 
Richlong2013Modified
Richlong2013ModifiedRichlong2013Modified
Richlong2013Modified
 
ResumeFQ2016
ResumeFQ2016ResumeFQ2016
ResumeFQ2016
 
EclipseEmbeddedDay2009-OSGi: Best Tool In Your Embedded Systems Toolbox
EclipseEmbeddedDay2009-OSGi: Best Tool In Your Embedded Systems ToolboxEclipseEmbeddedDay2009-OSGi: Best Tool In Your Embedded Systems Toolbox
EclipseEmbeddedDay2009-OSGi: Best Tool In Your Embedded Systems Toolbox
 
[ITAS.VN]CxSuite Enterprise Edition
[ITAS.VN]CxSuite Enterprise Edition[ITAS.VN]CxSuite Enterprise Edition
[ITAS.VN]CxSuite Enterprise Edition
 

Similar to Psi multi accessgateway_casestudy

Resume_Appaji
Resume_AppajiResume_Appaji
Resume_Appaji
Appaji K
 
Sagar_Ware_Resume
Sagar_Ware_ResumeSagar_Ware_Resume
Sagar_Ware_Resume
Sagar Ware
 
Felix Ramos_Automation Programmer Analyst_QA
Felix Ramos_Automation Programmer Analyst_QAFelix Ramos_Automation Programmer Analyst_QA
Felix Ramos_Automation Programmer Analyst_QA
Felix Ramos
 
Michael_Joshua_Validation
Michael_Joshua_ValidationMichael_Joshua_Validation
Michael_Joshua_Validation
MichaelJoshua
 
9 yrs of Testing Exp_STB and DSL gateway products
9 yrs of Testing Exp_STB and DSL gateway products9 yrs of Testing Exp_STB and DSL gateway products
9 yrs of Testing Exp_STB and DSL gateway products
Prakash S M
 

Similar to Psi multi accessgateway_casestudy (20)

Prasad_Meduri
Prasad_MeduriPrasad_Meduri
Prasad_Meduri
 
David Robinson's On Line CV
David Robinson's On Line CVDavid Robinson's On Line CV
David Robinson's On Line CV
 
SpencerKellerResume
SpencerKellerResumeSpencerKellerResume
SpencerKellerResume
 
Abhishek_cv
Abhishek_cvAbhishek_cv
Abhishek_cv
 
Kavita resume startup
Kavita resume startupKavita resume startup
Kavita resume startup
 
Resume_Appaji
Resume_AppajiResume_Appaji
Resume_Appaji
 
Kavita resume
Kavita resume Kavita resume
Kavita resume
 
2011 NASA Open Source Summit - Forge.mil
2011 NASA Open Source Summit - Forge.mil2011 NASA Open Source Summit - Forge.mil
2011 NASA Open Source Summit - Forge.mil
 
Sagar_Ware_Resume
Sagar_Ware_ResumeSagar_Ware_Resume
Sagar_Ware_Resume
 
Felix Ramos_Automation Programmer Analyst_QA
Felix Ramos_Automation Programmer Analyst_QAFelix Ramos_Automation Programmer Analyst_QA
Felix Ramos_Automation Programmer Analyst_QA
 
kavita_resume_3
kavita_resume_3kavita_resume_3
kavita_resume_3
 
Modernizing Testing as Apps Re-Architect
Modernizing Testing as Apps Re-ArchitectModernizing Testing as Apps Re-Architect
Modernizing Testing as Apps Re-Architect
 
NAGESH B KALAL
NAGESH B KALALNAGESH B KALAL
NAGESH B KALAL
 
SwagatDash
SwagatDashSwagatDash
SwagatDash
 
Michael_Joshua_Validation
Michael_Joshua_ValidationMichael_Joshua_Validation
Michael_Joshua_Validation
 
9 yrs of Testing Exp_STB and DSL gateway products
9 yrs of Testing Exp_STB and DSL gateway products9 yrs of Testing Exp_STB and DSL gateway products
9 yrs of Testing Exp_STB and DSL gateway products
 
Spellman Resume
Spellman ResumeSpellman Resume
Spellman Resume
 
Balaji Resume
Balaji ResumeBalaji Resume
Balaji Resume
 
Resume
ResumeResume
Resume
 
Let's banish "it works on my machine"
Let's banish "it works on my machine"Let's banish "it works on my machine"
Let's banish "it works on my machine"
 

Recently uploaded

Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
UK Journal
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
FIDO Alliance
 

Recently uploaded (20)

Collecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The InsideCollecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
Google I/O Extended 2024 Warsaw
Google I/O Extended 2024 WarsawGoogle I/O Extended 2024 Warsaw
Google I/O Extended 2024 Warsaw
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
Your enemies use GenAI too - staying ahead of fraud with Neo4j
Your enemies use GenAI too - staying ahead of fraud with Neo4jYour enemies use GenAI too - staying ahead of fraud with Neo4j
Your enemies use GenAI too - staying ahead of fraud with Neo4j
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 

Psi multi accessgateway_casestudy

  • 1. PrimeSoft Solutions, Inc. Multi Access Gateway: Case Study August 26, 2010 This document contains information proprietary to PrimeSoft Solutions, Inc. Reproduction, release, or use of the information contained in this document is strictly prohibited without prior written permission from PrimeSoft Solutions, Inc.
  • 2. Multi Access Gateway : Case Study Multi Access Gateway Client Offshore Development Center (ODC) This Santa Clara, CA, company develops and markets industry's first multi-access convergence gateway designed for stateful management of tens of thousands of secure, reliable, high-quality subscriber sessions over multiple network access types including Wi- Fi and WiMAX. To date, the company has raised over $50M and is backed by Sequoia Capital and Kleiner Perkins among other venture capital firms. Relationship Model PrimeSoft is the chosen Outsourced Product Development partner for the client. PrimeSoft developed this relationship by providing the following at our offshore development center 1. Building testing, automation and development teams 2. Identifying an Project Manager 3. Setting a Lab and setting it up with Client specific software and hardware components 4. Providing supported from HR, Admin and Facilities Management functions The offshore team consists of 18 engineering resources divided into QA, Automation and Development. The teams function as an extension to the client’s team in USA, and participate in various projects during a release. Weekly status reports are provided by the Project Manager on all active offshore projects, followed by conference calls to discuss any open issues. UMA Handset Simulator Technology Brief Unlicensed Mobile Access (UMA) technology provides access to GSM and GPRS mobile services over unlicensed spectrum technologies, including Bluetooth and 802.11. By deploying UMA technology, service providers can enable subscribers to roam and perform handover between cellular networks and unlicensed wireless networks using dual-mode mobile handsets. With UMA, subscribers receive a consistent user experience for their mobile voice and data services as they transition between networks. Project Objectives The objective of this project was to develop a UMA Handset Simulator. UMA simulator came out as a requirement from the development and automation engineers to help them perform unit testing and automation of UMA functionality. PrimeSoft Confidential Page 2 of 11
  • 3. Multi Access Gateway : Case Study UMA simulator was required support functional, to help automate more than 90% of all functional tests and help development engineers run pre-commit unit tests to ensure basic functionality. PrimeSoft Confidential Page 3 of 11
  • 4. Multi Access Gateway : Case Study Our Approach The Team PrimeSoft identified a technical lead, development & test engineers to staff this project. The team had prior experience in working with telecom and data communications protocols and call processing. Project Execution Requirements Analysis A team from PrimeSoft traveled to US client site for gathering requirements. The team had brainstorming sessions with client’s technical team and end users of the UMA simulator. The requirements were documented in a Functional Requirements Document Template and shared with all the stake holders, followed by a formal review. Review comments were incorporated and the updated document was checked into CVS and posted on internal Wiki. Functional Specifications & Software Design Functional Requirements Document was an input for developing a Functional Specifications Document. The HLD (High Level Design) and LLD (Low Level Design) documents were developed based on the functional specifications. The design document identified modules, interfaces, and presented pseudo code. Coding, Unit Testing & Acceptance Testing Coding of the tool involved setting up the environment and developing code based on the low level design. The code was unit tested by development engineers. System testing of the simulator was carried out by an independent team that developed system test plan document based on the requirements document and executed multiple test cycles and logged bugs using Bugzilla. User acceptance test plan was developed by PrimeSoft QA team and shared with the client. Acceptance testing was performed jointly by PrimeSoft and client teams. Salient Features Object oriented Design approach provides well defined programming interfaces to the scripting layer Provides interfaces to build and parse and UMA and GSM Layer3 messages Multithreaded implementation for RTP sends and receives for multiple handset instances Python extensions provided for abstracting the complex scripting functionality and reduce scripting effort Supports complex handover scenarios by simulating two UMA connections from a single Handset PrimeSoft Confidential Page 4 of 11
  • 5. Multi Access Gateway : Case Study Project Deliverables Deliverable Details Functional Requiremnts Document Functional Specifications Document Documents Design Document Users Guide Release Notes UMA Simulator Source Code (C/C++, Makefiles, Shell Source Code Scripts) Unit Test Plan Test Documents Unit Test Matrix Bugs BUGS Logged into Bugzilla Sample Scripts Sample Automated Scripts Users Guide Automation suite and scripts users guide Traces all the requirements against Functional Requirements Traceability Specifications, Design, System Testing, Unit Testing Matrix and Users Guide Tools & Technologies Title Details Protocols UMA, GSM, TCP/IP Operating Systems Redhat Linux Programming & Scripting C, C++, Python, Shell Scripts, Makefiles Configuration Management CVS PrimeSoft Confidential Page 5 of 11
  • 6. Multi Access Gateway : Case Study Multi Access Gateway Testing PrimeSoft houses the largest testing team for our client. Our testing team is involved in the entire test life cycle of the product including test strategy development, test plans development, tests cases development, manual testing, filing bugs and running regression cycles. Project Objectives The main objective of the testing group is to work as an extension to client’s onsite engineering teams, and be responsible for test and automation activities involved in the project. The offshore team is responsible of the following project activities 1. Defining QA and Automation Strategy for a identified release 2. Developing QA plan for the identified release 3. Developing Test Plans and Test Case documents based on a. Marketing and Product Requirements Documents b. Functional Specifications c. Design Documents d. RFC’s Supported 4. Performing Functional, System, Sanity, Regression, Load and Stress, Security and Interop testing 5. Identifying and logging bugs, coordinating with developers in re-producing and analyzing bugs 6. Developing test matrix and reporting on manual tests progress on a daily/weekly basis Testing Projects PrimeSoft team worked with client on multiple releases of this product and was involved in testing and owning QA and Automation for the following modules  IPSec IKE v2, IKEv1  MOBIKE (Mobile IKE)  RIP/OSPF  VLAN (Virtual LAN)  Session Initiation Protocol (SIP)  QoS (Quality of Service)  AAA (Authentication, Authorization and Accounting) and RADIUS  Change of Authorization (CoA)  Internet Protocol Slow Path PrimeSoft Confidential Page 6 of 11
  • 7. Multi Access Gateway : Case Study  Point to Point over Ethernet (PPPoE)  Dynamic Host Control Protocol (DHCP)  PDIF (Protocol Data Interworking Function)  CDR (Call Detail Records)  IP-IP Encapsulation  DOS Attacks  ACL (Access Control List)  SNMP (Simple Network Management Protocol)  RTC (Real Time Charging)  HA (High Availability)  System Testing  Performance Testing Our Approach The Team We created a dedicated test team who were technically competent in networking technologies. Project Manager and a core group of engineers started the project by performing manual testing on one of the releases and logged bugs. The team developed its competency on the product by organizing Internal training seminars Technical hands on sessions Developing internal wiki pages Going through past bugs reports Communicating with developers and PLM teams Test Beds A Site-to-Site VPN was setup between PrimeSoft, and Client location in USA, allowing engineers to reserve, configure and access remote test setups. PrimeSoft also has a lab setup with client specific software and hardware available locally for engineers. Project Execution Test Projects execution followed the following phases. Analysis The aim of this phase was to technically be competent in the technology and enhance product understanding and gather all the information necessary to develop a test plan document and completely define the scope of testing activities. The team initiated the projects by going through RFC’s and product documentation including, Requirements, Functional Specifications and High Level Design. Email PrimeSoft Confidential Page 7 of 11
  • 8. Multi Access Gateway : Case Study exchanges and conference calls were conducted to understand the nuances of implementation on client’s product. Test Plan Development Test plan development involved, identification and documenting 1. Test Topology 2. Software and Hardware Tools 3. Entry and Exit Criteria 4. Conformance scenarios from RFC’s 5. Test cases against Requirements 6. CLI Related test cases 7. Functional test cases 8. Negative test cases 9. End-to-End tests 10. SNMP related test cases 11. High availability test cases Internal & External Reviews Internal peer reviews (board review) is performed and review comments are updated into the document. An internal review checklist is used to ensure the quality of the test plans is consistent. The update document is sent out for an external/client review followed by a walk through of the test plan through a conference call. Review comments received from architects, designers, implementers and PLM are consolidated and updated into the document. Updated document is checked into CVS and posted on internal Wiki. Manual Test Cycle Manual test cycles involved setting up test bed and executing test cases against the SUT (System Under Test) and tracking progress. A Test Matrix is filled for each cycle of manual test cycles across multiple builds, the test matrix captures testing results documented as PASS, FAIL or BLOCKED. The matrix also provides key metrics of the testing cycle including testing effort, bugs logged and passes percentage. Bugs were logged into client maintained bugzilla, with information including 1. Bug description 2. Build version 3. Steps to reproduce PrimeSoft Confidential Page 8 of 11
  • 9. Multi Access Gateway : Case Study 4. Logs and core files path 5. Initial diagnosis findings 6. Observation and Comments Tools & Technologies Title Detail Protocols IPSec, IKE v1/v2, AAA, CoA, PPPoE, DHCP, SIP, SNMP Operating Systems REDHAT Linux, QNX Scripting Python, Shell Scripts Ethereal (Network Analyzer), Scapy (Packet Generator), Tools Silvercreek, SIPp, QA Robot, Smartbits, Landslide Avalanche, IXIA, IKEv1/v2 clients, DOS Attacks tools Configuration Management CVS, Bugzilla Project Deliverables Deliverable Details Test Plan Test Plan Document Test Matrix Test Cycle Summary in a Test Matrix Document Bugs BUGS Logged into Bugzilla Checklist Test Plan Release Checklist Requirements Traceability Traces all the requirements against the test cases and Matrix bugs Multi Access Gateway Automation With a growing feature list in the product we sent out with a target of automating 50% of our test cases, and this could significantly reduce the manual test cycle timeline. PrimeSoft was involved in test automation from the very beginning and participated in developing an automation framework, developing automated test scripts and suites. Project Objectives The objective of the automation group at PrimeSoft is to Design, Develop and Enhance an Object Oriented Automated Framework Develop automated test scripts Run automated regression on daily/weekly builds PrimeSoft Confidential Page 9 of 11
  • 10. Multi Access Gateway : Case Study Develop test suites for pre-commit, sanity, weekly regression and feature specific suites Identify and log bugs Fix scripts and maintain them across release and builds Our Approach The Team We created a dedicated automation group within PrimeSoft who were technically competent in scripting languages and had experience with relevant networking technologies. Framework Enhancements Our team participated in design and development activities of the automation framework. An object oriented framework was designed using Python as the scripting language. Enhancements to the framework were performed and tested (using automated unit tests); and the framework was released to carry out the development of the scripts. Automated Scripts A team of automation engineers was selected to automate the test plans using the enhanced automation framework. Automation engineers focused on developing automated scripts and scripts development involved the following scripts Manual testing Develop and automated test script Running the test on a stable build to verify the behavior of the script Fixing all issues with the scripts Adding the script in a suite and running the suite Every developed script went through a through review with at least two reviewers and focused on functionality and coding guidelines. Functional Review: Review focused on the functionality of the scripts and compared them with what is described in the test cases Coding Guidelines Review: The review focused on usability, maintainability and adherence to coding guidelines Checked into CVS Project Salient Features Python based Object Oriented Automation Framework o Can load/clear and check configuration on system under test o Can load/clear and check configuration on Juniper Netscreen hardware o Can initiate IKE v1 and v2 sessions from end clients o Can initiate PPPoE sessions from Linux based end clients o Can handle High Availability cases Exposes library functions for script developers PrimeSoft Confidential Page 10 of 11
  • 11. Multi Access Gateway : Case Study Automated Tests o Topology File: Can be changed to point to a new topology o Configuration File: Captures all common configuration for a test o Suite file: Can include multiple test cases to be run in a suite Tools & Technologies Title Detail SIP, SDP, RTP Protocols IPSec IKE v1/v2 RIP, OSPF, BGP, DHCP, DNS, Radius, TCP/IP, PPPoE Clients Hardware/Software Netscreen 5GT, XpressVPN, PPPoE Linux Client, SIPp Used test tool, Cisco Terminal Server Operating Systems Redhat Linux, QNX Programming & Scripting Python, Perl, Shell Scripts Configuration Management CVS, Bugzilla Project Deliverables Deliverable Details Automated Scripts Developed as part of each Scripts functionality Test Suits Automated test suites for various types of testing Framework Libraries Framework Libraries and Documentation Checklist Scripts Development and Review Checklist Requirements Traceability Traces all the requirements against the test cases and Matrix bugs PrimeSoft Confidential Page 11 of 11