Laying the Foundation
for Enduring Success
Part2: Building An Automation Framework
Software Development Life Cycle
Requirements Tests Defects
What do we
make and how
should it
behave
Make sure it
works as
stated in the
Requirement
Definition Sets Environments
Actual Results
do not equal
Expected
Results
What is a Test Framework?
ATestFramework:
• LinksteststootherSDLCitems
• IsNOTaTestAutomationFrameworkbutoftencontainsone
• Allowsforrapidcreationoftestsfromreusablecomponents
• Separatesdatafromlogic(REUSABILITY)
• Providesastandardizedtest“language”andreportingstructureforanapplicationundertest
1. Define 2. Decompose 3. Decide
1
A Quick Introduction
I am a veteran of software development covering a wide range
of disciplines. While studying Computer Information Systems
and later Computer Science at Missouri State University I got an
early start to my career by being QA Analyst for Black Lantern
Studios, testing video games for various platforms. My 8-year
tenure at BLS saw my growth into a QA Lead, and eventually a
project manager. During this time my day-to-day role varied
between, test management, project management, design, and
level scripting. In March 2015, I started at O'Reilly as a QA
Automation developer. The automation department was very
young at the time and I was an integral part of selecting an
automation tool (Test Complete) and developing a set of
standards and frameworks which lead to my current position of
QA Automation Supervisor. Connect with me on LinkedIn:
https://www.linkedin.com/in/cunderw/
Getting Started
• Setupandorganizesourcecontrol
• Keepdifferentassetsseparate
• CreateTemplateprojecttoclonefrom
• Createthefilesyouknowyouwillneed
• Makesureyourassetsaresetupinawaytobereferencednotlocal(allowsupdatestokeepALLtestsstable)
• Familiarizeyourselfwiththeapplication
• Dosomeexploratorytesting
• Startcreatingsomebasicnamemapping(orobjectidentificationviascript)
• Takenotesforwhatyourrequirementswillbeandhowyourautomationassetswilltestthis
Environment / Data
• Environment
• Thisincludessettingconfigsforbeingabletoruninmorethanoneenvironment(dev,test,prod,etc…)
• Eventhandlersareyourfriend
• Setup/teardown
• Opening/Closingyourapplication
• Otherusefulutilities,documentandincludeyourloggingfromthestart
• Getasmoketestprojectsetup–thatisgoingtobethemostusedandimportantsetoftestsyouwilluseand
we’llusethissoontovalidateourutilitiesandmakeatestatthesametime
• Data
• Keepyourpropertiesandreferencesgenericandnotindividualtestspecific
• Allowyourdataobjectstobesharedbetweenscripts
On Screen Actions
• Lettheframeworkhandlethese
• Createsharedutilitiesforcommonactionslikemenunavigation,textinput,etc…
• Thesearethebuildingblockofatest,onceyouhavethemsetupcreatingatestisassimpleasdraggingthings
intoakeywordtest
• Allowsyouframeworktodriveyourflowandverificationssothatminimalmaintenancetoindividualtestsis
required
• Useclassestodriveyouronscreenobjectstokeeploggingconsistent
• Supportyouralreadycreateddataobjects
• Inpututilitiesshouldbeabletoacceptdataobjectscreatedinthepreviousstepssoyoucanchainthings
together
• Thisiswhyit’simportanttoabstractyourframeworkdatafromtheactualtestdata
Verifications
• BuildUIverificationsintoyouractions
• Allowyouractionsutilitiestoverifytheiractions
• Allowsyoutoupdateinonceplaceandkeepyourtestsstable
• Makethemoptional,ifwehaveatestthatverifiesafieldacceptsinputproperlyanditpassesthereisnoneed
toverifythateverysingletime
• Buildyourdataverificationsaroundyourdataobjects
• Againwe’retryingtoreducemaintenanceandmaketestcreationeasier
• Thesamedataobjectswecreatedbefore,usethoseforallofyourverifications
Logging & Reporting
• Customizeyourlogging
• Logmessagesbeforeverificationsstatingwhatyou’reverifyingandit’sexpectedresult
• Makeithumanreadable,non-technicalusersshouldbeabletolookatyourlogandknowexactlywhereand
whyafailureoccurredandnothavetoguesswhythereisanerrorthatiscriptic
• Ifyouhavefollowedeverythingelseyouractions,dataprep,environmentandverificationsshouldallbeusing
aconsistentloggingstructurealready
• Logfoldersareyourfriend
• Reporting
• Automateyourreportingtoo
• Sendemails
• Exportthelogstoacommonplace
More: https://github.com/cunderw/smartbear-november
Join us at 8:00 AM | 2:00 PM EDT
November 14th
https://smartbear.com/resources/webinars/open-source-testing-frameworks/
Dan Giordano
SmartBear Testing Framework Webinar Series
Part 3: The Growing Trend of Open Source Testing Frameworks
Thank you!
https://github.com/cunderw/smartbear-november

Building an Automation Framework

  • 1.
    Laying the Foundation forEnduring Success Part2: Building An Automation Framework
  • 2.
    Software Development LifeCycle Requirements Tests Defects What do we make and how should it behave Make sure it works as stated in the Requirement Definition Sets Environments Actual Results do not equal Expected Results
  • 3.
    What is aTest Framework? ATestFramework: • LinksteststootherSDLCitems • IsNOTaTestAutomationFrameworkbutoftencontainsone • Allowsforrapidcreationoftestsfromreusablecomponents • Separatesdatafromlogic(REUSABILITY) • Providesastandardizedtest“language”andreportingstructureforanapplicationundertest 1. Define 2. Decompose 3. Decide
  • 4.
    1 A Quick Introduction Iam a veteran of software development covering a wide range of disciplines. While studying Computer Information Systems and later Computer Science at Missouri State University I got an early start to my career by being QA Analyst for Black Lantern Studios, testing video games for various platforms. My 8-year tenure at BLS saw my growth into a QA Lead, and eventually a project manager. During this time my day-to-day role varied between, test management, project management, design, and level scripting. In March 2015, I started at O'Reilly as a QA Automation developer. The automation department was very young at the time and I was an integral part of selecting an automation tool (Test Complete) and developing a set of standards and frameworks which lead to my current position of QA Automation Supervisor. Connect with me on LinkedIn: https://www.linkedin.com/in/cunderw/
  • 5.
    Getting Started • Setupandorganizesourcecontrol •Keepdifferentassetsseparate • CreateTemplateprojecttoclonefrom • Createthefilesyouknowyouwillneed • Makesureyourassetsaresetupinawaytobereferencednotlocal(allowsupdatestokeepALLtestsstable) • Familiarizeyourselfwiththeapplication • Dosomeexploratorytesting • Startcreatingsomebasicnamemapping(orobjectidentificationviascript) • Takenotesforwhatyourrequirementswillbeandhowyourautomationassetswilltestthis
  • 6.
    Environment / Data •Environment • Thisincludessettingconfigsforbeingabletoruninmorethanoneenvironment(dev,test,prod,etc…) • Eventhandlersareyourfriend • Setup/teardown • Opening/Closingyourapplication • Otherusefulutilities,documentandincludeyourloggingfromthestart • Getasmoketestprojectsetup–thatisgoingtobethemostusedandimportantsetoftestsyouwilluseand we’llusethissoontovalidateourutilitiesandmakeatestatthesametime • Data • Keepyourpropertiesandreferencesgenericandnotindividualtestspecific • Allowyourdataobjectstobesharedbetweenscripts
  • 7.
    On Screen Actions •Lettheframeworkhandlethese • Createsharedutilitiesforcommonactionslikemenunavigation,textinput,etc… • Thesearethebuildingblockofatest,onceyouhavethemsetupcreatingatestisassimpleasdraggingthings intoakeywordtest • Allowsyouframeworktodriveyourflowandverificationssothatminimalmaintenancetoindividualtestsis required • Useclassestodriveyouronscreenobjectstokeeploggingconsistent • Supportyouralreadycreateddataobjects • Inpututilitiesshouldbeabletoacceptdataobjectscreatedinthepreviousstepssoyoucanchainthings together • Thisiswhyit’simportanttoabstractyourframeworkdatafromtheactualtestdata
  • 8.
    Verifications • BuildUIverificationsintoyouractions • Allowyouractionsutilitiestoverifytheiractions •Allowsyoutoupdateinonceplaceandkeepyourtestsstable • Makethemoptional,ifwehaveatestthatverifiesafieldacceptsinputproperlyanditpassesthereisnoneed toverifythateverysingletime • Buildyourdataverificationsaroundyourdataobjects • Againwe’retryingtoreducemaintenanceandmaketestcreationeasier • Thesamedataobjectswecreatedbefore,usethoseforallofyourverifications
  • 9.
    Logging & Reporting •Customizeyourlogging • Logmessagesbeforeverificationsstatingwhatyou’reverifyingandit’sexpectedresult • Makeithumanreadable,non-technicalusersshouldbeabletolookatyourlogandknowexactlywhereand whyafailureoccurredandnothavetoguesswhythereisanerrorthatiscriptic • Ifyouhavefollowedeverythingelseyouractions,dataprep,environmentandverificationsshouldallbeusing aconsistentloggingstructurealready • Logfoldersareyourfriend • Reporting • Automateyourreportingtoo • Sendemails • Exportthelogstoacommonplace More: https://github.com/cunderw/smartbear-november
  • 10.
    Join us at8:00 AM | 2:00 PM EDT November 14th https://smartbear.com/resources/webinars/open-source-testing-frameworks/ Dan Giordano SmartBear Testing Framework Webinar Series Part 3: The Growing Trend of Open Source Testing Frameworks
  • 11.