Automation FrameworkDesign and Implementation          Liang Gao
AgendaWhy we need an automation frameworkElements of automation frameworkDesign PrinciplesImplementation Principles.Popula...
The Goal of AutomationIncrease productivity on repetitive testingAvoid human errorsUse night and weekendsUltimate goal is ...
Automation Framework: Why?I can develop script without any frameworkEach script can run itselfScripts execution results ca...
Automation Framework: Reason杀鸡不用宰牛刀If you have multiple developers to develop scripts inparallelIf you have thousands of t...
Remember the goalAll we want is a full version report (manual or auto)                                     This material i...
Elements of an Automation         FrameworkControl libraryCommon libraryCoding guidelineExecution engineTest harnessResult...
Control libraryAll equipment in the network need to be controlledby control libraryCLI remote control  Power of the expect...
Common LibraryCommon APIs to use when automate command tasksPing libraryWiFi libraryMPLS libraryDHPC library……..          ...
Coding GuidelineCommon style for ease of maintenanceCommon headers20% + commentsEnough logging and debuggingDocumentation ...
Execution engineConcept of Test Bed, Jobs and suitesControlled batch run  Exception catch (both software and hardware)  Wi...
Test HarnessThe end product: full version test reportCollection of the running results in high levelEasy to present and ea...
Design PrinciplesCode controlExtract layer of library designScript and test bed independencyResource allocationExecution p...
Code ControlJust like any product development, pick your choiceSubversion, Clearcase etc.                                 ...
Abstract Layer of Library Design   Equipment library layer to high the complexity of   different equipment A Calls B      ...
Script and Test Bed Independency   Any scripts can run on any test bed without change   inside the script                 ...
Resource AllocationEach test bed can only run one job?Test bed designDynamic resource allocation                          ...
Execution parallelizationPer framework instance per test bed?Central framework?                                  This mate...
Result StorageLogs need to be archived (very important)Full version report need to be storedNeed a database?  Script info ...
Dynamic TopologyEach script need different topologyFixed test bed topo vs dynamic test bed topo                           ...
Distribution MechanismCentralizedPer engineer per copy                        This material is for training purpose only. ...
Implementation PrinciplesScript language of choicePerformance consideration                            This material is fo...
Script Language of ChoiceTCLPerlRubyPythonEXPECT                  This material is for training purpose only.             ...
Performance ConsiderationCentral Framework VS Distributed                                   This material is for training ...
Framework Case StudyC company’s - largest network equipment vendorJ company’s - largest network equipment vendorH-1 compan...
CTCLPer engineer per frameworkNo GUINo dynamic resource allocationNo dynamic topology on FrameworkOpen source style librar...
JTCL/PerlDynamic topology generation by controlling theswitch Vlans.Per engineer per frameworkCLI onlyPer release code bas...
H-1Key word driven                        This material is for training purpose only.                        No duplicatio...
H-2Key word drivenScript and test bed independentIDEStatic suites                                  This material is for tr...
TRuby + AutoIT + WitrStrong support for GUI automation on both MSWindows and BrowserC/S structure                         ...
APerlHeavily use of Selemium                              This material is for training purpose only.                     ...
NPythonAll CLI              This material is for training purpose only.              No duplication without consent of Sig...
Automation framework design and implementation
Upcoming SlideShare
Loading in …5
×

Automation framework design and implementation

850 views

Published on

introduction on how to build an test automation framework

  • Be the first to comment

Automation framework design and implementation

  1. 1. Automation FrameworkDesign and Implementation Liang Gao
  2. 2. AgendaWhy we need an automation frameworkElements of automation frameworkDesign PrinciplesImplementation Principles.Popular automation framework case study This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  3. 3. The Goal of AutomationIncrease productivity on repetitive testingAvoid human errorsUse night and weekendsUltimate goal is to get the full version report (auto ormanual) This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  4. 4. Automation Framework: Why?I can develop script without any frameworkEach script can run itselfScripts execution results can be collected and put intoa spreadsheetI can do everything I want for automation without a“Framework” This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  5. 5. Automation Framework: Reason杀鸡不用宰牛刀If you have multiple developers to develop scripts inparallelIf you have thousands of thousands of scripts need torunIf your product release cycle is short and demanding This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  6. 6. Remember the goalAll we want is a full version report (manual or auto) This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  7. 7. Elements of an Automation FrameworkControl libraryCommon libraryCoding guidelineExecution engineTest harnessResult database This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  8. 8. Control libraryAll equipment in the network need to be controlledby control libraryCLI remote control Power of the expectGUI control SeleniumAPI based control Ixia, Smartbit, Ameritec, Abacus etc. This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  9. 9. Common LibraryCommon APIs to use when automate command tasksPing libraryWiFi libraryMPLS libraryDHPC library…….. This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  10. 10. Coding GuidelineCommon style for ease of maintenanceCommon headers20% + commentsEnough logging and debuggingDocumentation This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  11. 11. Execution engineConcept of Test Bed, Jobs and suitesControlled batch run Exception catch (both software and hardware) Will run to the end no matter what Abortion on demandLive status displayDebugging support (not script development debugging)Dynamic suites generationEmail notification This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  12. 12. Test HarnessThe end product: full version test reportCollection of the running results in high levelEasy to present and easy to searchEasy to debug base on the reportEasy to compare between different version’s results This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  13. 13. Design PrinciplesCode controlExtract layer of library designScript and test bed independencyResource allocationExecution parallelizationResult storageDynamic topologyDistribution mechanism This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  14. 14. Code ControlJust like any product development, pick your choiceSubversion, Clearcase etc. This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  15. 15. Abstract Layer of Library Design Equipment library layer to high the complexity of different equipment A Calls B Scripts: Common APIs with are equipment independent Abstract API Layer Device dependent layerAbacus Ameritec SIPX Others This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  16. 16. Script and Test Bed Independency Any scripts can run on any test bed without change inside the script Scripts: A Calls B Best case: all you need to change is the test bed name in the framework Logical Test Bed Test Bed Map TB1 TB2 TB3 TB4 This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  17. 17. Resource AllocationEach test bed can only run one job?Test bed designDynamic resource allocation This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  18. 18. Execution parallelizationPer framework instance per test bed?Central framework? This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  19. 19. Result StorageLogs need to be archived (very important)Full version report need to be storedNeed a database? Script info database? Log database? Report database? Suite-version database? This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  20. 20. Dynamic TopologyEach script need different topologyFixed test bed topo vs dynamic test bed topo This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  21. 21. Distribution MechanismCentralizedPer engineer per copy This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  22. 22. Implementation PrinciplesScript language of choicePerformance consideration This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  23. 23. Script Language of ChoiceTCLPerlRubyPythonEXPECT This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  24. 24. Performance ConsiderationCentral Framework VS Distributed This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  25. 25. Framework Case StudyC company’s - largest network equipment vendorJ company’s - largest network equipment vendorH-1 company’s - largest network equipment vendorH-2 company’s - largest network equipment vendorT company’s – windows based security companyA company’s – Silicon valley security appliance companyN company’s – Silicon Valley startup.Sigma’s – first commercial framework in the market This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  26. 26. CTCLPer engineer per frameworkNo GUINo dynamic resource allocationNo dynamic topology on FrameworkOpen source style library developmentCLI onlyDynamic suites This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  27. 27. JTCL/PerlDynamic topology generation by controlling theswitch Vlans.Per engineer per frameworkCLI onlyPer release code base (on scripts) This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  28. 28. H-1Key word driven This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  29. 29. H-2Key word drivenScript and test bed independentIDEStatic suites This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  30. 30. TRuby + AutoIT + WitrStrong support for GUI automation on both MSWindows and BrowserC/S structure This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  31. 31. APerlHeavily use of Selemium This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.
  32. 32. NPythonAll CLI This material is for training purpose only. No duplication without consent of Sigma-RT is permitted.

×