Software Test Conference in Anaheim, LA

1. Alexander Andelkovic, Spotify
STARWEST 2011

2. Introduction
Background
Challenges
Success factors
Pitfalls
Tool
Examples
Summary
Questions

3. Lives in Stockholm, Sweden
15 years testing experience
Test engineer at Spotify
Passionate about advanced test techniques

4. Our reality:
- Testing in an agile developing cycle
- Commited to feature testing (Stories)
Problem:
Less time for story testing as stories accumulates
Solution:
Automate regression test, manual testers can focus on story testing
and rely on automation testing take the bulk of regression tests

5. Model-Based Testing (MBT) is the automatic generation of test procedures/vectors using models
of system requirements
Specific activities of the practice are:
1. Design models
2. Code test automation
3. Run tests
+ Easier to maintain test automation
+ Find bugs
+ Hire developers to work with testing
- Learning curve to get started
- Cost to setup test infrastructure

6. - MBT needs a different mindset
- Introducing test automation as a criteria for definition as done
- Test design (modelling) is done by testers
- Programmers do the automation
- Integrate test automation with nightly builds
- Tool chain
- Start with too complex modelling

7. GraphWalker is an Open Source Model-Based Testing Tool written in Java for
generating offline and online test sequences from Finite State Machines and
Extended Finite State Machines
The main features of GraphWalker are:
No UML: Uses GraphML, easier to get started with than UML
No exit/stop points:
- Long, unpredictable test sequences
- Does not want to walk the same path every time we execute a test
- Variation spiced with randomness. better 'test coverage'
- Stop generating test sequences are done by Stop Criterias
Online: Path walked through the model is decided at runtime (Web service)
Event-driven: Possibility to switch model caused by an event
ModelHandler: Re-use of models

8. yEd is a powerful diagram editor that can be used
to generate high-quality drawings of diagrams
yEd is free and runs on all major platforms:
Windows, Unix/Linux, and Mac OS
Supports .graphml format readable by Graphwalker
GraphML is a XML-based file format for graphs

11. Sikuli is a visual technology to automate and test graphical
user interfaces (GUI) using images (screenshots)
Sikuli includes Sikuli Script and Sikuli IDE, for writing visual scripts with
screenshots easily
Sikuli Script automates anything you see on the screen
without internal API's support. Uses OCR and image recognition
You can programmatically control a web page, a desktop application, or
even a smartphone application running in a simulator or via VNC
Access through Java API supported

12. Graphical editor for classification trees
Top features of CTE XL:
- Systematic and common classification tree method
- Automated test case generation
- Methodical linking with test automation tools

13. Problem: Many combinations
Generate:
Testsequence file (embedded code)
Execute:
CTEXL testcase model

14. Finite State Machine: Extended Finite State Machine:

16. Path walked through the model is decided runtime
An example of a script in QTP, executing a test using graphwalker
1. $> java -jar graphwalker.jar soap -f xml/login.xml
Now running as a SOAP server. For the WSDL file, see: http://myComputerName:9090/graphwalker-services?WSDL
2. WebService("SoapServicesService").SetTOProperty "WSDL", http://myComputerName:9090/graphwalker-services?WSDL
3. Do until not WebService("SoapServicesService").HasNextStep()
4. action = WebService("SoapServicesService").GetNextStep()
5. Loop
6. Reporter.ReportEvent micDone, "MBT Statistics", WebService("SoapServicesService").GetStatistics()

17. Mother model Sub model
Merged model (uses same data domain)

18. Purpose to reuse code and models (uses separate data domains)
3
9
11 15
1
13 7
10 16 4
WhatsNew
17 5
2 8
12 18 6 14
SendSongs CheckInbox CleanInbox FreeUser PremiumUser

19. Switch model caused by an event

20. - Mindset
- Verifies functionality and finds bugs
- Test automation as a criteria of definition of done
- Testers to model
- Developers for test automation
- Integrate with nightly builds

21. E-mail: alex@spotify.com
Graphwalker: http://graphwalker.org/
yED: http://www.yworks.com/en/products_yed_about.html
Sikuli: http://sikuli.org/
CTE XL: http://www.berner-mattner.com/en/
berner-mattner-home/products/cte-xl/index.html