Scripting Recipes for Testers


Published on

Here are the slides from my tutorial on Scripting Recipes for Testers. In it I share a number of reusable scripts and some tips I learned writing them to help testers do their job better.

The scripts themselves can be found on my site ( under the category 'GLSEC2008'

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Scripting Recipes for Testers

  1. 1. Script Recipes for Testers Adam Goucher Manager of Quality, Zerofootprint Software
  2. 2. About Me • Testing for 10 years • Tool-smithing for most of them • Scripting has been and important part of both my career and testing success
  3. 3. About You • Want to be a better tester • Who to find better bugs • More efficiently
  4. 4. About This Tutorial • Reusable scripts explained • Tips for your own scripts
  5. 5. So Why Script? • Provide new insight into your product • Generate test data • Automate repetitive / common / complex tasks • Replace expensive tools with ones tailored to your environment
  6. 6. Language Choice • Is far too often a religious choice • Use what is right for the task
  7. 7. Language Choice • Do you know the language? • Could you learn it? • Is it supported on all your (current) platforms? • Are those platforms all considered first-class citizens? • Can it interact with your database?
  8. 8. Language Choice • Does it control your other tools? (Selenium, Watir, etc.) • Can you recycle code (Java jars or .NET assemblies) from elsewhere in your application? • Does it support Unicode as a core data type? • Is there seamless interaction with the operating system? • Is there an active member community (to whom you can turn to for help)
  9. 9. Language Choice
  10. 10. Developer Notes • hack, broken, kludge are not things I like seeing in code I am testing • fixme and todo in comments imply hidden knowledge • style
  11. 11. A better grep • grep is a unix tool for searching for things inside files • The ‘developer notes’ script is like a cross- platform grep
  12. 12. How to run • Set the source_base to point to the root of your project source code • Set the interesting value to the types of file you want to check • Add items you want to check to the notes list; all lowercase
  13. 13. Future Changes • Faster • Use regex
  14. 14. Tip: Complexity • If you don’t have to do something in a complicated way, don’t.
  15. 15. Tip: Common Case • Converting both sides of a comparison to a common case means you don’t have to worry about case
  16. 16. Tip: Performance • Unless performance really matters, don’t worry about it • Optimizations add complexity
  17. 17. LOUD • i18n and l10n are hard things to get right • But is pretty easy to test • Since we (the testers) are not responsible for content; the translators are
  18. 18. LOUD • The ‘loud’ screen alters the text which is shown on the screen - ^LOUD$ • Everything that should be translated should be UPPERCASED • Everything that shouldn’t be is not • Blocks of text should not be split into different LOUDed strings
  19. 19. How to run • This script needs to be customized for your organization’s method of creating resource bundles
  20. 20. Tip: Use fake ‘files’ • Rather than using your real resource bundle to build the script, use a variable that looks like a file • It is faster, and you cannot accidently delete your master copy while on the train without an internet connection
  21. 21. Audible Log Monitoring • Log files provide a wealth of information • If your eyes are on a log while doing a test they cannot be somewhere else • Use your ears instead
  22. 22. How to run • in watchlist.txt, create a list of strings or regexes which it needs to watch for • and the wav file it will play when it finds it
  23. 23. Tip: Package for Convenience • Some people are scared of anything that looks like ‘code’ • More often than not these are ones who are not ‘unix people’ • A tool is only useful if it is used • All scripting languages have the ability to compile themselves into .exe
  24. 24. Tail’ing the Event Log • Another tool I miss on windows is tail • When on remote unix machines I am constantly tailing logs • ‘Proper’ windows applications however log to the Event Viewer which cannot be tailed (even if you have a windows version installed)
  25. 25. How to run • Once you have installed the Python Win32 Extensions, just run the script
  26. 26. Future Changes • Merge with Audible Log Monitoring
  27. 27. Tip: Platform neutrality • Platform neutrality is a noble goal • But the goal should be abandoned if the situation makes sense
  28. 28. Browser Percents • When testing web applications, one of the key decisions to make is how much to test with what browser • Aggregate 3rd party information, or • Information from your actual webserver?
  29. 29. How to run • I only run this script every so often so the filename is in the script • The regex in the script is written for the Apache access log format • If you are using a different webserver you just have to change the regex
  30. 30. Tip: Reinvent the wheel • There are a tonne of libraries and tools which parse log files • Reinvent the wheel if • you can learn something • you want specific information and don’t need anything else
  31. 31. Tip: Filter the data • Your access log is going to be littered with your own office; filter it out to get a real information
  32. 32. Algorithmic Oracles • Oracles can be people, specifications (like an RFC) or a comparable product • Or if something is an algorithm it can be an alternative implication • In something like a script
  33. 33. How to Use • Implement your algorithm as a verification task • Run what your developers produce through the script • Warning: If you implement it wrong, then you will could be ignoring bugs or creating red herring ones
  34. 34. Test Case Generation • This is the opposite of the Oracle script • One of the main reasons I create test data ahead of time is to not have to think under pressure • Script that thinking
  35. 35. How to Use • Just implement the algorithm • Warning: If you implement it wrong and you will be using bad data
  36. 36. Tip: Random • Real world data is random • Create your test data the same way • The infamous Pesticide Paradox
  37. 37. Tip: Dual Purpose • I put the oracle and generation scripts for the same algorithm in the same script
  38. 38. Excel manipulation • Excel is a really convenient tool for data creation • It is also really powerful and comes with its own scripting language
  39. 39. Tip: Layers • If something has a native scripting layer, use it
  40. 40. GUI Automation • Commercial GUI automation tools are budget killers • For a number of application types the open- source alternatives are just as effective • You just need to fall back on manual scripting sooner
  41. 41. GUI Automation • Here are the 4 steps I use when automating a UI 1. Record the test 2. Add the testing conditions 3. Traditional data driving 4. Give the script smarts • 2 - 4 tend to require scripting
  42. 42. Metaframeworks • Metaframeworks are frameworks which run different test frameworks and aggregates the results
  43. 43. Meta why? • Imagine from one command getting the results of your JUnit, Selenium, homebrew and Watir at once? • This is a big bit of scripting...
  44. 44. Tip: Language Variants • A lot of the major scripting languages have been ported to other interpreters • Java: Jython, JRuby • .NET: IronPython, IronRuby
  45. 45. Go Forth and Script!