• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
'Let Test Automation Play the Role it Deserves' by Ruud Teunissen
 

'Let Test Automation Play the Role it Deserves' by Ruud Teunissen

on

  • 532 views

Test automation used to be a luxury. Since nowadays improving product quality, reducing costs, shortening time-to-market combined with more frequent releases are an irreversible trend, test automation ...

Test automation used to be a luxury. Since nowadays improving product quality, reducing costs, shortening time-to-market combined with more frequent releases are an irreversible trend, test automation has become a necessity. "Have we been successful at test automation thus far?" Asking the question is answering it. We need to change our approach to test automation. Like Jerry Weinberg said: "When something is not working, use more of it" is wrong. Core of the approach is to act "Goal driven", not "Tool driven"; look beyond technical selection criteria. Approach test automation step by step. Define realistic objectives and develop the business case for test automation. Analyze the context and define the required test automation solution. Experience has learned that the first level of test automation is best described as "enabling testing to achieve its goals". Level 2? Mitigate more risks than without automation! The optimal level? "Fit for purpose" and "Fit for context", scalable and future proof. Ruud describes a practical approach to implement and improve your test automation. Take back valuable references and examples that enable you to implement test automation effectively and efficiently, tuned to your specific situation. Learn how test automation can play the role it deserves: an indispensable and tireless part of your testing. And by the way. Don't be afraid, the human part of testing cannot be automated (yet). Test automation, if done well, enables the tester to focus on his essential role.

Key Takeaways

1. Goal driven, not tool driven
2. Automate what is necessary, not what is possible
3. Test automation levels:
a. enabling testing to achieve its goals
b. mitigate more risks than without automation
c. scalable and future proof
4. Translate "Fit for purpose" and "Fit for context" into concrete and tangible goals and actions
5. Let test automation finally play the role it deserves: an indispensable and tireless part of your testing.

Statistics

Views

Total Views
532
Views on SlideShare
306
Embed Views
226

Actions

Likes
1
Downloads
0
Comments
0

1 Embed 226

http://www.eurostarconferences.com 226

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    'Let Test Automation Play the Role it Deserves' by Ruud Teunissen 'Let Test Automation Play the Role it Deserves' by Ruud Teunissen Presentation Transcript

    • Today’s webinar is presented by Ruud Teunissen and he willdiscuss Let Test Automation Play the Role it DeservesWelcome to the EuroSTARMayWebinar Serieswww.eurostarconferences.comThis webinar is due to start at 2pm. Make sure you stick around at the end for theQ&A session and continue the conversation with the speaker on Twitter after theshow!@esconfs#esconfs
    • Let Test Automation Playthe Role it DeservesRuud Teunissen, Polteq Test Services BV, NetherlandsRuud Teunissen is best described as a passionate software tester. Throughout his career he hasplayed almost any possible role in testing (tester, test manager, testtrainer, coach, sales, manager, test consultant, ...) in a variety of environments and companies. Ruudis coauthor of several books on structured testing, including Software Testing: A Guide to the TMap®Approach. Currently he is senior test consultant with Polteq Test Services BV and focuses on testimprovement and management in any context. Ruud is frequently invited to speak at conferences.Within Polteq, Ruud is responsible for TI4Automation, the approach for successful implementationand improvement of test automation based on hands-on experiences and good practices.www.eurostarconferences.com@esconfs#esconfs
    • Join the conversation on Twitter#esconfs@esconfs@RuudTeunissenwww.eurostarconferences.comContinue the conversation with the speaker on Twitter after the show!@esconfs#esconfs
    • Let Test Automation Playthe Role it DeservesRuud Teunissen, Polteq Test Services BV, Netherlandswww.eurostarconferences.com@esconfs#esconfs
    • 5Test automation - InitiallyWe’re going to automateour testing! This might bethe best news since thediscovery of testing itself…
    • 6Recurring test automation issuesManagement expects immediate R.O.I.“Automate everything”Unable to measure impact of test automationCurrent test process: ad-hoc & undocumentedCommunicationTool-driven in stead of goal-drivenFocus on testing components“One tool to rule them all”We do not need an automation plan
    • 7• Fit for purpose– Realistic objectives• Fit for context– Achievable in the current contextTest Automation = Automation of testing
    • 8Roadmap to success TI4Automation®[1]AwarenessPlan• Define objectives• Set scopeAssess• Analyze• Define approachImplement• Automate• OrganizeOptimize• Innovate• RenovateTI4Automation is a registered trademark of Polteq Test Services BV
    • 9Roadmap to success TI4AutomationAwareness
    • 10Awareness triggers
    • 11Management commitment and support• Set realistic goals• Monitor the “business case” for test automation• Recognize the long term added value• Provide sufficient and appropriate resources• Grant the necessary budget• Deliver the committed resources• Enable necessary changes in software developmentand/or test process to better suit test automationWithout involvement, there is no commitment.Mark it down, asterisk it, circle it, underline itStephen Covey
    • 12Roadmap to success TI4AutomationAwarenessPlan• Define objectives• Set scopeAssess• Analyze• Define approachImplement• Automate• OrganizeOptimize• Innovate• Renovate
    • 13Plan• Define test automation objectives– When is test automation successful?– What are the restrictions?– What are the timelines? TimeQualityCostREALISTIC AND ACHIEVABLEDIFFERENT FROM TEST OBJECTIVESSHOULD SUPPORT TESTING ACTIVITIESSHORT AND LONG TERMREGULARLY RE-VISITED AND REVISED
    • 14Plan• Define realistic and achievable objectives• Develop the business case– Cost and benefits– Now and in the future
    • 15Business case including additional benefits• Costs and Benefits• Less time spent in regression test execution• Continuous testing through CI• Quality of regression test execution is a constant• Opportunity to– perform additional tests & implement more releasesTasks Current Future DeltaInvest in automation (once) 358,0 -358,0Total per test cycle (twice a year) 154,0 37,5 +116,5Cost/Benefit after 1 yr / 2 test cycles 308,0 433,0 -125,0Cost/Benefit after 2 yr / 4 test cycles 616,0 508,0 +108,0
    • 16Plan• Define realistic and achievable objectives• Develop the business case• Set scope– Project? Projects? Organization?– What test types/levels/objects/…?
    • 17Set scope• Supplier– Focus on “works as designed”– Components– Services– Bottom-up
    • 18Set scope• Business– Focus on “fit for purpose”– End-to-End– Business processes– Top-down
    • 19A real life example: Objectives & ScopeObjective• To support the Agile way of working the regression test approachneeds to be improved, and complimented with automated testing.• Aim is to reduce the duration (time), reduce the effort (cost) andimprove the quality of regression testing within an agile sprint.Scope• Set up the automated test environment• Automate at least 175 functional test cases of which– 20% is considered “simple”– 45% considered “medium”– 35% considered “complex”Out of scope• Development testing• Interface testingTimeline• Delivery within 3 – 4 months (or sooner)
    • 20Plan• Define realistic and achievable objectives• Develop the business case• Set scope• Define measurements– In line with objectives– Test automation realted
    • 21Roadmap to success TI4AutomationAwarenessPlan• Define objectives• Set scopeAssess• Analyze• Define approachImplement• Automate• OrganizeOptimize• Innovate• Renovate
    • 22AssessQuality PolicySoftware ProcessOrganization PolicyTest AutomationTestProcess• Test automation context
    • 23Assess• Test automation context– Testing, Development, Maintenance– Projects, Changes, Fixes– Patches, Updates– Management (Configuration, Defect, Requirements, …)– Reporting (Project, Test, System, …)
    • 24Assess• Test automation context• Analyze– Where are we?– Where do we need to be?
    • 25Automation architecture• High level structure of automation including thediscipline of creating, maintaining and documenting
    • 26Automation scripts Automation standards• An executable sequenceof automated actionsthat execute a test• Test automation issoftware. Treat it assuch.
    • 27Tooling Tool integration• The platform on whichthe test automationsolution willrun, including allnecessary software andtools• Test automation toolingis able to integrate– with reporting and defectmanagement tooling to bemore efficient– with change andconfiguration managementtooling to be moreeffective
    • 28Test environment Test data• Hardware, simulators, software tools, etc. neededto conduct automatedtesting• Test data is data whichhas been specificallyidentified for use in tests
    • 29Automation strategy Planning & Estimation• Ensures that the goalsare efficiently andtransparently achievable• All activities need to beplanned and estimated;monitoring progress andregular evaluation arekey in setting/achievingrealistic objectivesAutomationStrategy
    • 30Test automation team• Includes the required skills for testing, design anddevelopment• Strives for continuous improvement• Participates proactively in the software developmentlife cycle
    • 31Keys to success TI4Automation• Automation architecture• Automation scripts• Automation standards• Tooling• Test environment• Test data• Tool integration• Test automation team• Automation strategy• Planning and estimation
    • 32Assess – scope & objectives• Test automation context• AnalyzeContributing Adding value OptimizingTestAutomationLevelsEnable testingto achieve itsgoalsMitigate morerisksScalable testautomationFit for purposeand contextTimeQualityCost
    • 33Roadmap to success TI4Automation“Checklist”Key area Contributing Adding value Optimizing1 Automation architecture 1 2 3 1 2 3 4 1 2 32 Automation scripts 1 2 3 4 1 2 3 4 1 2 33 Automation standards 1 2 3 1 2 3 4 1 2 34 Tooling 1 2 3 4 1 2 3 4 1 2 3 45 Test environment 1 2 3 4 1 2 3 4 1 2 3 46 Test data 1 2 3 1 2 3 4 1 2 3 47 Tool integration 1 2 3 1 2 3 4 1 2 38 Test automation team 1 2 3 1 2 3 4 1 2 39 Automation strategy 1 2 3 4 1 2 3 4 1 2 310 Planning & Estimation 1 2 3 4 1 2 3 4 1 2 3Enable testingto achieve itsgoalsMitigate morerisksScalable, fit forpurpose andcontext
    • 34Roadmap to success TI4AutomationWhere are we and where do we want to be?Key area Contributing Adding value Optimizing1 Automation architecture 1 2 3 1 2 3 4 1 2 32 Automation scripts 1 2 3 4 1 2 3 4 1 2 33 Automation standards 1 2 3 1 2 3 4 1 2 34 Tooling 1 2 3 4 1 2 3 4 1 2 3 45 Test environment 1 2 3 4 1 2 3 4 1 2 3 46 Test data 1 2 3 1 2 3 4 1 2 3 47 Tool integration 1 2 3 1 2 3 4 1 2 38 Test automation team 1 2 3 1 2 3 4 1 2 39 Automation strategy 1 2 3 4 1 2 3 4 1 2 310 Planning & Estimation 1 2 3 4 1 2 3 4 1 2 3Enable testingto achieve itsgoalsMitigate morerisksScalable, fit forpurpose andcontextCurrent situationTarget situation
    • 35Test Automation Plan• Goal & Scope, including business case• Approach• Input, conditions• Deliverables• Measurements• Resources– Budget– People– Material• Schedule• Potential/essential Test Process Improvement (TPI)• Potential/essential Software Process Improvement (SPI)
    • 36Architecture requirements• Automation level:– Business Process level, Application level, API level, …• Test level:– E2E, UAT, FAT, SIT, ST, UIT, UT, …• Intended use – CRUD of test cases– (End)users, testers, development, test automationteam, operations, …• Maintenance of the architecture– Test team, Development, In-house, Near/Off-shore, …• Available / selectable tooling– “Every tool has it’s goal”
    • 37Tool selection• Objectives• Architecture• Available budget• Integration with existing tooling– Management (Configuration, Defect, Requirements, …)– Reporting (Project, Test, System, …)• Fit in with current test, development and maintenance approach• Alignment with system landscapeSystem requirementsSupported environmentsInstallationEase of useIntegration with other toolsSupportManualsTool customizationCommoncriteriaScripting language facilitiesObject recognitionObject mappingData driven testingData base testingCheckpointsRecovery scenarioTest results analysisSpecificcriteria
    • 38Roadmap to success TI4AutomationAwarenessPlan• Define objectives• Set scopeAssess• Analyze• Define approachImplement• Automate• OrganizeOptimize• Innovate• Renovate
    • 39• Automate and organize– Maintainable, transferable– Acquire the required skills• Sanity check– Fit for context & Fit for purposeImplement
    • 40Roadmap to success TI4AutomationAwarenessPlan• Define objectives• Set scopeAssess• Analyze• Define approachImplement• Automate• OrganizeOptimize• Innovate• Renovate
    • 41Optimize• Innovate and renovate– Changes, Updates, Patches, Fixes– Redesign, Refactor, Rewrite• Sanity check– Stay “Up-and-running”– Growing maturity within the context– Continuously contribute, add value
    • 42Roadmap to success TI4AutomationAwarenessPlan• Define objectives• Set scopeAssess• Analyze• Define approachImplement• Automate• OrganizeOptimize• Innovate• Renovate
    • EuroSTAR Webinar ArchiveAccess almost 50 on-demand software testing webinars by the worlds leadingtesting experts. Topics include….www.eurostarconferences.comAgile, Automation, Test Management, Process Improvements, Risk-based Testing, PerformanceTesting, Test Documentation, Cloud Testing, Session-based Testing, Weekend TestersVisit the archive!http://www.eurostarconferences.com/community/member/webinar-archive@esconfs#esconfs
    • The EuroSTAR Community.Are you a member?Did you know we have an online resource area with almost …www.eurostarconferences.com600 software testing presentations, 50 topicalwebinars, over 40 videos and podcasts and 20 experteBooksDo you know that membership is completely free and you can access thesepremium testing resources anytime?Join today!http://www.eurostarconferences.com/community/member@esconfs#esconfs
    • www.eurostarconferences.comAre you coming to Gothenburg?Join us in the Swedish Exhibition& CongressCentre, Gothenburg, Swedenfrom the 04-07 November forthe 21st annual EuroSTARConference.Visitwww.eurostarconferences.comfor more information on whatyou can expect to see at Europe’slargest software testing event!@esconfs#esconfs