Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Why Test automation guys should have hacker mindset?!


Published on

Presentation from meetup occurred in Chicago, May 24

Published in: Technology
  • Be the first to comment

Why Test automation guys should have hacker mindset?!

  1. 1. Why Test automation guys should have hacker mindset?!<br />a talk about being test automation engineer<br /> By Peter Kartashov @2011<br />
  2. 2. A few silly observations about Goodtest automation engineer:<br /><ul><li> lazy one who got tired doing testing manually
  3. 3. crazy about automating everything
  4. 4. tester who wants to know and manipulate apps inside of that interfaces
  5. 5. looks for new opportunities to break apps
  6. 6. relies on testability</li></ul> By Peter Kartashov @2011<br />
  7. 7. Greattest automation engineer is same as Good one<br /><ul><li> lazy one who got tired doing testing manually
  8. 8. crazy about automating everything
  9. 9. tester who wants to know and manipulate apps inside of that interfaces
  10. 10. looks for new opportunities to break apps
  11. 11. relies on testability</li></ul>+ plus +<br /><ul><li> experience,
  12. 12. effectiveness
  13. 13. flexibility </li></ul> By Peter Kartashov @2011<br />
  14. 14. Challenges (technical)<br />You will never have stable app and interfaces<br />You will have different ways to automate (interfaces, approaches, frameworks)<br />You will have to either reuse or grow new own framework<br />You will never have smooth path<br />Be ready to use these words in your vocabulary: “workaround”, “hack”, “backdoor”, “crutch”<br /> …<br />If you have neither these symptoms – There is a good chance you are doing either wrong or trivial test automation<br /> By Peter Kartashov @2011<br />
  15. 15. Challenges (technical)<br />Technically making enterprise wide automation implies thinking over solid architectural solution that involves:<br /><ul><li> Infrastructure
  16. 16. Integration with other systems and practices (CI, test management, version control, etc)
  17. 17. Automation framework
  18. 18. Persistent storage test data, logs and historical information </li></ul> By Peter Kartashov @2011<br />
  19. 19. Challenges (organizational)<br />Struggle to find out right scope, resources and solution (notorious ROI)<br />You will have to prove that you are useful (not money wasters)<br />Some people will consider you as neither tester nor developer<br />If a group of automation engineers – a right person should manage and drive this team (test automation lead with background). If you looked after by test manager or manual test engineer only – personally I don’t envy you.<br />If you have neither these symptoms – There is a good chance you are doing either wrong automation or you are lucky <br /> By Peter Kartashov @2011<br />
  20. 20. Testability<br />Testability is a property allowing you to manipulate with your software project by calling its interfaces. <br />Some most popular Issues:<br /><ul><li> Some interfaces will be open, some will be closed.
  21. 21. Some interfaces will be stable, some will be freaky to any code commit
  22. 22. It is normal to have a choice on which level to automate your app. As functional and E2E testing is pretty complex, you will likely utilize different levels of testing.
  23. 23. Another challenge related to testability is cross-platform (cross-browser) implementation</li></ul> By Peter Kartashov @2011<br />
  24. 24. Testability: examples of mixed implementation<br /><ul><li>API level testing is utilized to manipulate system components internally, transmit data and trigger events. Verification of results is running on API-level, then verification on GUI as double check
  25. 25. Integration between 2 components is verifying in module integration level (API). Integration with GUI layer is checking in GUI as checkpoints.
  26. 26. Change data in database (SQL scripting); checking results in API as checkpoint.
  27. 27. Vice versa: change data using UI, validate on database side (SQL querying). Additional validation can be incorporated on API level
  28. 28. Check Web service I/O, check queue object, check POST/GET Header and Body</li></ul> By Peter Kartashov @2011<br />
  29. 29. Hacks, workarounds and tools<br />You will never have one tool or/and language covering all your needs.<br />So that you have to be creative and all-round to resolve blockers quickly<br /> By Peter Kartashov @2011<br />
  30. 30. Examples: Reflection<br />Reflection allows youto work with native code of running application in runtime, i.e. you can intercept and evaluate currently running code on machine (like attaching to currently running COM object)<br />Java and .Net platforms allows to enable reflection so that testing framework “understands” running code of compiled application. <br />import java.lang.reflect.*;<br /> public class DumpMethods {<br /> public static void main(String args[])<br /> {<br /> try {<br /> Class c = Class.forName(args[0]);<br /> Method m[] = c.getDeclaredMethods();<br /> for (inti = 0; i < m.length; i++)<br />System.out.println(m[i].toString());<br /> }<br /> catch (Throwable e) {<br />System.err.println(e);<br /> }<br /> }<br />}<br />*Don’t use reflection in production code – main concerns are performance degradation and security<br /> By Peter Kartashov @2011<br />
  31. 31. Examples: OS manipulation<br />Use internal Scripting capabilities: WMI, PowerShell, bash, Win Registry<br />E.g. VB Script utilizing WMI object to monitor CPU usage on hosts:<br /> By Peter Kartashov @2011<br />
  32. 32. Examples: using tools and libs<br />Firebug to watch DOM rendering, scripting errors and memory leaking<br />… and evaluate scripts. I recommend using Java script to traverse, manipulate and hack over Web<br /> By Peter Kartashov @2011<br />
  33. 33. Examples: using tools and libs<br />To simplify your Java Script injection – I recommend utilizing jQuery<br /> By Peter Kartashov @2011<br />
  34. 34. Examples: using tools and libs<br />WinDbg to get memory dump, call critical section, emulate stack overflaw…<br /> By Peter Kartashov @2011<br />
  35. 35. Core skills of good test automation engineer<br />+Knowledge of internal mechanisms of interacted objects (e.g. DOM model, COM object, MFC, Pure Java). Basically you have to be able to invent efficient object recognition model with high robustness on intermediate changes and proper control identification (avoiding concurrent identification) +Knowledge of test automation tool(s) itself + Familiarity with programming language and scripting +Knowledge of development practices, principles (like single responsibility Open/Closed, reusability, OOP, procedural approach…) and common design patterns (singleton, decorator, factory, wrapper, etc)+Regular expressions. That’s a very powerful tool for us. Must to know concepts and how to use. Ideally you are ready to write any expression at fly. +Knowledge of performance testing, modeling and identification of bottlenecks (for load testing)+Knowledge of theory and practices on relational databases.+Good knowledge of one on-demand RDBMS, querying and performance analysis+XML – based technologies and service model (XML, XSLT, XSD, DTD, WSDL)+ Experience in version control systems<br />+ Experience in tuning application/web/DB servers+Broad knowledge of operation systems (server and consumers solutions) as advanced user or as administrator.<br /> By Peter Kartashov @2011<br />
  36. 36. In general, Test automation engineer should quickly solve technical challenges utilizing right tools and libraries.<br />Finally, the effort on building automation has to be empowered by testing aptitude and passion to break software with keen tests.<br />Aren’t all these slides about Hackers?<br /> By Peter Kartashov @2011<br />
  37. 37. Thank you! Questions?<br /> By Peter Kartashov @2011<br />