Test Automation as a Service

5,966 views

Published on

TaaaS - Next Generation Testing Conference

Practically every area of the commercial software industry is being continuously revolutionised by new concepts and technologies.
The diversity and customer reach of web technologies is a fabulous opportunity for many organisations, but it is also a minefield of complexity requiring new skills and new thinking. If automated testing has any future it must work proactively with business to address these opportunities.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,966
On SlideShare
0
From Embeds
0
Number of Embeds
2,030
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Test Automation as a Service

  1. 1. Introduction<br />Test Automation as a Service (TaaaS)<br />Next Generation Testing: Innovations and Challenges<br />Jonathon Lee WrightDirector of Testingjlwright@iee.org@jonathon_wright www.TaaaS.net (blog)www.automation.org.ukautomation.sharepoint.comwww.linkedin.com/in/automation <br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 1<br />
  2. 2. Introduction<br />Agenda<br />Why the cloud?<br />Automation in the cloud;<br />New opportunities and benefits;<br />New risks;<br />What we are doing now;<br />The future;<br />Live demo.<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 2<br />
  3. 3. Cloud Services<br />Why the cloud?<br />The cloud provides no up-front investment,featuring instant scalability, flexibility and availability;<br />Pay as you use (PAYU), no more expensive vendor licencing costs or un-utilised maintenance agreements;<br />Complex test environments on demand:“built to expect & withstand failure”;<br />Established technology avoiding support issues with non-commercial products;<br />Avoid the minefield of complexity requiring internal specialist staff skills following industry best practice;<br />100% web delivered providing collaboration and mobility.<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 3<br />
  4. 4. New Opportunities<br />New Testing Opportunities<br />Cross Platform and Compatibility - browser, operating system, mobile devices, script runtimes, languages, HTML versions;<br />Market Configuration Testing – 10+ Applications (Shop / ShelfReview / Nominations), 10+ Clients (Tesco / Macy’s), 10+ Workstreams (Tesco UK / Tesco France), 10+ Languages (en-GB / fr-CA), 10+ Platforms (XP-SP1, SP2 & SP3),10+ Browsers (I.E 7, 8, 9), 10+ Runtimes (Flash 9.0.44 ~ 10.3.181).<br />Accessibility/Usability/UXD/WAI - due diligence in guaranteeing accessibility to people with disabilities;<br />88% of the delegate organisations here today failed W3C compliance tests; <br />Globalisation – realistic performance & load testing plus compliance with the commercial and civil law of the country;<br />Search Engine Optimisation – periodic ranking checks;<br />Penetration, Performance & Load Testing.<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 4<br />
  5. 5. New Risks<br />New Risks (in the cloud)<br />What happens when these Services FAIL?<br />Systemic Failure? (e.g. DNS or Network link) <br />Alternative service solution provider?<br />Unique Services? (such as GPS)<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 5<br />
  6. 6. Cloud Automation<br />Taking Automation to the Cloud<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 6<br />
  7. 7. Cloud Automation<br />Why this approach?<br />○<br />- Limited Support<br />●<br />- Full Support<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 7<br />
  8. 8. Cloud Automation<br />What is different with this approach?<br />Design tests before release ‘first day’ automation testing<br />Create complex test scenarios before the application has even been written;<br />Generic Reusable Libraries <br />Actions<br />Object<br />Handler<br />Event Handler<br />BPM<br />The test language is self-validating and human readable<br />The verbs and nouns in the domain specific language are written in natural language which uses context sensitive validation;<br />Example Business Level Keyword<br />Press | Login | Button<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 8<br />
  9. 9. Cloud Automation<br />What is different with this approach?<br />The test definition is not implementation specific <br />Test execution is performed by cloud based technology adapters which act as interpreters for the vendor agnostic and implementation agnostic language;<br /><ul><li>Application Server
  10. 10. Database Server
  11. 11. End-Client Machine
  12. 12. Windows 2008 RC1
  13. 13. Windows 2003 RC2
  14. 14. Windows 7 SP1
  15. 15. Load Testing
  16. 16. Performance
  17. 17. Functional
  18. 18. Coded-UI (Win32)
  19. 19. Selenium (ASP.net)
  20. 20. Ranorex (Flex 4.0)</li></ul>The definition of a test artefacts is self-maintaining<br />Asset loader uses fuzzy logic matching to identify UI objects; <br /><br />XPATH Check <br />Regular Expressions<br />Descriptive Programming<br />Fuzzy Logic<br />Artificial Intelligence <br /><br /><br /><br />Please enter your password to logon<br />Please enter your password to login<br />Enter password here ……<br />Enter password here ……<br />n/a<br />logon<br />logoff<br />login<br />logoff<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 9<br />
  21. 21. Cloud Automation<br />Event Based / BDD Testing<br />Automated testing based on trigger events (passive & active):<br />Third party system events such as user story or defect status workflow change;<br />Passive<br /><ul><li>Continuous build and integration or source code gated check-in.</li></ul>Active<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 10<br />
  22. 22. Cloud Automation<br />Where are we now?<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 11<br />
  23. 23. Cloud Automation<br />How it works (in the cloud)<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 12<br />
  24. 24. Cloud Automation<br />Benefits to the Customer<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 13<br />
  25. 25. Cloud Automation<br />‘Keep it Simple’ Automation<br />User stories represented by a number of Business Level Keywords<br />Start Test (As per the custom list of business level keywords)<br />Save Test (Selectable in the future from the Your Saved Tests List Box)<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 14<br />
  26. 26. Cloud Automation<br />TaaaS vs. DIY<br />TaaaS<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 15<br />
  27. 27. The Future<br />Future forecast "Cloudy" <br />Avoid the Hype:<br />Ensure that the critical business flow can be automated followed by complex custom components "avoid automating the simple components first“ e.g. simple login pages;<br />Don't be tempted:<br />By third generation automation vendors offering so-called "script-less" tools. Be realistic, you need to be able support custom classes and the latest technologies without having to wait on the possibility that it may be supported in the next release;<br />Integration into testing process:<br />Automation is no longer an "us and them" scenario between manual and automated tests;<br />Make the necessary plans:<br />How easy is it to start using the service and how easy is it to move away from the service.<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 16<br />
  28. 28. The Future<br />Something to think about<br />“The best aspects of these proven test approaches demonstrate how they have evolved over the past decade, and this echoes some of the changes towards more lean and agile business methodologies. They are in a constant state of evolution – just as the underpinning technology evolves over time.”<br />my section in ‘Experiences in Test Automation’ bookDorothy Graham & Mark Fewster 2011<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 17<br />
  29. 29. Cloud Automation<br />Automation - Demo<br />18th May 2011<br />Test Automation as a Service (TaaaS) - Slide 18<br />

×