Script Recipes for
Testers
Adam Goucher
Manager of Quality, Zerofootprint Software
adam@goucher.ca
http://adam.goucher.ca

About Me
• Testing for 10 years
• Tool-smithing for most of them
• Scripting has been and important part of
both my career and testing success

About You
• Want to be a better tester
• Who to find better bugs
• More efficiently

About This Tutorial
• Reusable scripts explained
• Tips for your own scripts

So Why Script?
• Provide new insight into your product
• Generate test data
• Automate repetitive / common / complex
tasks
• Replace expensive tools with ones tailored
to your environment

Language Choice
• Is far too often a religious choice
• Use what is right for the task

Language Choice
• Do you know the language?
• Could you learn it?
• Is it supported on all your (current)
platforms?
• Are those platforms all considered first-class
citizens?
• Can it interact with your database?

Language Choice
• Does it control your other tools? (Selenium, Watir,
etc.)
• Can you recycle code (Java jars or .NET assemblies)
from elsewhere in your application?
• Does it support Unicode as a core data type?
• Is there seamless interaction with the operating
system?
• Is there an active member community (to whom you
can turn to for help)

Language Choice

Developer Notes
• hack, broken, kludge are not things I like
seeing in code I am testing
• fixme and todo in comments imply hidden
knowledge
• style

A better grep
• grep is a unix tool for searching for things
inside files
• The ‘developer notes’ script is like a cross-
platform grep

How to run
• Set the source_base to point to the root of
your project source code
• Set the interesting value to the types of file
you want to check
• Add items you want to check to the notes
list; all lowercase

Future Changes
• Faster
• Use regex

Tip: Complexity
• If you don’t have to do something in a
complicated way, don’t.

Tip: Common Case
• Converting both sides of a comparison to a
common case means you don’t have to
worry about case

Tip: Performance
• Unless performance really matters, don’t
worry about it
• Optimizations add complexity

LOUD
• i18n and l10n are hard things to get right
• But is pretty easy to test
• Since we (the testers) are not responsible
for content; the translators are

LOUD
• The ‘loud’ screen alters the text which is
shown on the screen - ^LOUD$
• Everything that should be translated
should be UPPERCASED
• Everything that shouldn’t be is not
• Blocks of text should not be split into
different LOUDed strings

How to run
• This script needs to be customized for your
organization’s method of creating resource
bundles

Tip: Use fake ‘files’
• Rather than using your real resource bundle
to build the script, use a variable that looks
like a file
• It is faster, and you cannot accidently delete
your master copy while on the train without
an internet connection

Audible Log Monitoring
• Log files provide a wealth of information
• If your eyes are on a log while doing a test
they cannot be somewhere else
• Use your ears instead

How to run
• in watchlist.txt, create a list of strings or
regexes which it needs to watch for
• and the wav file it will play when it finds it

Tip: Package for
Convenience
• Some people are scared of anything that
looks like ‘code’
• More often than not these are ones who are
not ‘unix people’
• A tool is only useful if it is used
• All scripting languages have the ability to
compile themselves into .exe

Tail’ing the Event Log
• Another tool I miss on windows is tail
• When on remote unix machines I am
constantly tailing logs
• ‘Proper’ windows applications however log
to the Event Viewer which cannot be tailed
(even if you have a windows version
installed)

How to run
• Once you have installed the Python Win32
Extensions, just run the script

Future Changes
• Merge with Audible Log Monitoring

Tip: Platform neutrality
• Platform neutrality is a noble goal
• But the goal should be abandoned if the
situation makes sense

Browser Percents
• When testing web applications, one of the
key decisions to make is how much to test
with what browser
• Aggregate 3rd party information, or
• Information from your actual webserver?

How to run
• I only run this script every so often so the
filename is in the script
• The regex in the script is written for the
Apache access log format
• If you are using a different webserver you
just have to change the regex

Tip: Reinvent the wheel
• There are a tonne of libraries and tools
which parse log files
• Reinvent the wheel if
• you can learn something
• you want specific information and don’t
need anything else

Tip: Filter the data
• Your access log is going to be littered with
your own office; filter it out to get a real
information

Algorithmic Oracles
• Oracles can be people, specifications (like an
RFC) or a comparable product
• Or if something is an algorithm it can be an
alternative implication
• In something like a script

How to Use
• Implement your algorithm as a verification
task
• Run what your developers produce through
the script
• Warning: If you implement it wrong, then
you will could be ignoring bugs or creating
red herring ones

Test Case Generation
• This is the opposite of the Oracle script
• One of the main reasons I create test data
ahead of time is to not have to think under
pressure
• Script that thinking

How to Use
• Just implement the algorithm
• Warning: If you implement it wrong and you
will be using bad data

Tip: Random
• Real world data is random
• Create your test data the same way
• The infamous Pesticide Paradox

Tip: Dual Purpose
• I put the oracle and generation scripts for
the same algorithm in the same script

Excel manipulation
• Excel is a really convenient tool for data
creation
• It is also really powerful and comes with its
own scripting language

Tip: Layers
• If something has a native scripting layer, use
it

GUI Automation
• Commercial GUI automation tools are
budget killers
• For a number of application types the open-
source alternatives are just as effective
• You just need to fall back on manual
scripting sooner

GUI Automation
• Here are the 4 steps I use when automating
a UI
1. Record the test
2. Add the testing conditions
3. Traditional data driving
4. Give the script smarts
• 2 - 4 tend to require scripting

Metaframeworks
• Metaframeworks are frameworks which run
different test frameworks and aggregates the
results

Meta why?
• Imagine from one command getting the
results of your JUnit, Selenium, homebrew
and Watir at once?
• This is a big bit of scripting...

Tip: Language Variants
• A lot of the major scripting languages have
been ported to other interpreters
• Java: Jython, JRuby
• .NET: IronPython, IronRuby

Go Forth and Script!