Test Automation - Moving Beyond the Test Tool


Published on

When implementing automation, many organizations begin their process with the tool and limit themselves by what that tool is capable of. In many cases they do not include automation features the tool does not natively perform out of the box.

Software Test Automation is a software engineering project. Just as an experienced software architect would not pick the development tool before knowing whether the application is for a mobile device or web applications, so too should an experienced automation architect pick a tool before understanding the automation problem.

In this webinar, we will present some of the dangers in focusing on the automation tool and some of the advantages of using an engineered framework where the testing tool itself is a commodity.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Test Automation - Moving Beyond the Test Tool

  1. 1. Test Automation: Moving Beyond the Tool Aligning your automation to you 26 February 2014
  2. 2. Today’s Presenter As Director of Solution Engineering, Joshua Berry has 20 years of industry experience, with more than a decade in leadership and innovation. He is a leader within Alliance’s Solution Engineering organization in developing cutting edge engineered solutions. His focus includes test automation and tooling, problem solving, product testing, and agile techniques in software development. Today’s Host As Director of Marketing, Sharon Lee heads the marketing strategy and brand messaging for Alliance. With over 12 years of experience in both digital and traditional marketing, she is focused on the effective use of media for compelling brand messaging and creating successful marketing programs with measurable results that impact revenue. Sharon holds a B.A. from the University of Pennsylvania. ©Alliance Global Services 2014 2
  3. 3. Overview • Introduction • Webinar Origins • Topic deep dives • Agenda • The Tooling Trap • Problems with current Automation Training • Engineering a Solution • Selecting a Tool(s) • Customizing the Tool(s) ©Alliance Global Services 2014 3
  4. 4. The “Textbook” Definition What is Software Test Automation? Software Test Automation is the application of computer software and electro-mechanical equipment to: • • • • Dynamically Control the execution of tests, Compare actual outcomes to predicted outcomes, Set up test preconditions and cleanup test modifications, Perform other test control and test reporting functions. In other words, Test Automation is using software to test software. One part test engineering + one part software engineering. ©Alliance Global Services 2014 4
  6. 6. A Common “New to Automation” Tooling Cycle “Phil” ©Alliance Global Services 2014 6
  7. 7. How did that happen? • Test Automation is a software engineering project Omitted basic engineering practices – An experienced architect – Seasoned engineers – Requirements / Design – A framework – Coding standards, version control, etc… • Phil’s Automation Engineer Transformation – “The Solution” was the tool without investing in Phil – A three day course on a tool is not enough – Automation Engineer career progression • Test Engineering + Software Engineering + Time – Phil had no frame of reference to judge tools • Would you reassign your best Business Analysts by giving them Visual Studio and expecting them to code? – A tool is only as good as the craftsman using it ©Alliance Global Services 2014 7
  8. 8. A Tooling Allegory – Limiting to a single tool • Craftsman Bolt-On Kit • Good tool but with flaws: – Pulling a nail? – Cutting a thick piece of wood? Or metal? • Screws and wood could be your web app, nails your mobile app, metal your web service ©Alliance Global Services 2014 8
  9. 9. The Dangers of Current Tool Training • Tool specific training is designed to teach you how to use the tool not generic cross tool automation. • What if the Craftsman Bolt-On was only tool and training to build a house? • Tradesmen learn the trade first, then how to use specific tools • Why has the IT industry gotten this backwards? – No need to train until tools existed – To lock you into their process and thus their tool • Expecting a Rookie to build good automation because of a tool is like expecting your accountant to build you a good house because you buy him a good hammer and saw. ©Alliance Global Services 2014 9
  10. 10. AN ENGINEER’S APPROACH: THE TOOL BECOMES A COMMODITY ©Alliance Global Services 2014 10
  11. 11. Step 1: Requirements • With your architect in place… • Understand the application(s) to be tested – – – – – – GUI / Web service / API PC / Web / Mobile Technology stack / Flash / Applets Custom Controls Code Quality / Coding Consistency What interfaces will you be automating? • Understand your environment – – – – – – Engineering skillset Upfront and ongoing budget for tooling and customizations Enterprise wide or project automation? How many people to create framework and script Regulatory issues for logging and reporting Corporate politics ©Alliance Global Services 2014 11
  12. 12. Step 2: Design • Framework Design Points – – – – – – – Scripting paradigm Function Modularity and Reusability Reporting and logging Error and Exception Handling Object versus Image-based testing Data Management Development vs Execution stations • Integrate into the SDLC Design Points and Process – – – – – – Waterfall vs Agile Metrics Continuous Integration Source code control Test Case Management integration Departmental structure • Design the framework based on your needs! – Switch tools and keep the framework design ©Alliance Global Services 2014 12
  13. 13. Step 3: Selecting the Tool (Finally!!) • Toolkit Mentality – Most tools will not do everything out of the box – Can a secondary tool do the few things the primary tool can’t? • Research all tools available + internal build option – Clearly communicate requirements and design to tool vendors – Consider “consultant” tool re-seller • Eliminate non-Feasible tools – Application(s) – Framework • Final Tools selection – Do you need all of the tools features? – Scorecard comparison – Vendor POC / Bake-off ©Alliance Global Services 2014 13
  14. 14. Customize Tool Training to you • Seek generic automation training • Skip unneeded components – e.g. recovery scenarios • Address your specific test requirements – Teacher should be generic expert on the tool • Labs can be done using your application ©Alliance Global Services 2014 14
  15. 15. Customize your tools • Search internet for solutions • Think of the tool as a development platform – Use the tools native scripting language – VB, Java, C/C++, C#, Perl / Python, etc… • Don’t need to use tools native widgets • Integrate with COM DLL’s • Create your own framework – Scriptless / Codeless Scripting engines • Build what doesn’t exist • Iterative framework implementation ©Alliance Global Services 2014 15
  16. 16. Applied Automation • Using Automation for other purposes other than Regression Testing • Application-level production monitor – Be your own canary! • Configure applications for manual tester • Validate environments • Environment troubleshooting • Automate manual production process ©Alliance Global Services 2014 16
  17. 17. Questions? Thank You Joshua Berry Director of Solution Engineering jberry@allianceglobalservices.com www.allianceglobalservices.com ©Alliance Global Services 2014 17
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.