Starwest 2011


Published on

Software Test Conference in Anaheim, LA

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Starwest 2011

  1. 1. Alexander Andelkovic, Spotify STARWEST 2011
  2. 2. IntroductionBackgroundChallengesSuccess factorsPitfallsToolExamplesSummaryQuestions
  3. 3. Lives in Stockholm, Sweden15 years testing experienceTest engineer at SpotifyPassionate about advanced test techniques
  4. 4. Our reality:- Testing in an agile developing cycle- Commited to feature testing (Stories)Problem:Less time for story testing as stories accumulatesSolution:Automate regression test, manual testers can focus on story testingand rely on automation testing take the bulk of regression tests
  5. 5. Model-Based Testing (MBT) is the automatic generation of test procedures/vectors using modelsof system requirementsSpecific activities of the practice are: 1. Design models 2. Code test automation 3. Run tests+ Easier to maintain test automation+ Find bugs+ Hire developers to work with testing- Learning curve to get started- Cost to setup test infrastructure
  6. 6. - MBT needs a different mindset- Introducing test automation as a criteria for definition as done- Test design (modelling) is done by testers- Programmers do the automation- Integrate test automation with nightly builds- Tool chain- Start with too complex modelling
  7. 7. GraphWalker is an Open Source Model-Based Testing Tool written in Java for generating offline and online test sequences from Finite State Machines and Extended Finite State MachinesThe main features of GraphWalker are: No UML: Uses GraphML, easier to get started with than UML No exit/stop points: - Long, unpredictable test sequences - Does not want to walk the same path every time we execute a test - Variation spiced with randomness. better test coverage - Stop generating test sequences are done by Stop Criterias Online: Path walked through the model is decided at runtime (Web service) Event-driven: Possibility to switch model caused by an event ModelHandler: Re-use of models
  8. 8. yEd is a powerful diagram editor that can be usedto generate high-quality drawings of diagramsyEd is free and runs on all major platforms:Windows, Unix/Linux, and Mac OSSupports .graphml format readable by GraphwalkerGraphML is a XML-based file format for graphs
  9. 9. Sikuli is a visual technology to automate and test graphicaluser interfaces (GUI) using images (screenshots)Sikuli includes Sikuli Script and Sikuli IDE, for writing visual scripts withscreenshots easilySikuli Script automates anything you see on the screenwithout internal APIs support. Uses OCR and image recognitionYou can programmatically control a web page, a desktop application, oreven a smartphone application running in a simulator or via VNCAccess through Java API supported
  10. 10. Graphical editor for classification treesTop features of CTE XL:- Systematic and common classification tree method- Automated test case generation- Methodical linking with test automation tools
  11. 11. Problem: Many combinations Generate: Testsequence file (embedded code) Execute: CTEXL testcase model
  12. 12. Finite State Machine: Extended Finite State Machine:
  13. 13. Path walked through the model is decided runtimeAn example of a script in QTP, executing a test using graphwalker1. $> java -jar graphwalker.jar soap -f xml/login.xmlNow running as a SOAP server. For the WSDL file, see: http://myComputerName:9090/graphwalker-services?WSDL2. WebService("SoapServicesService").SetTOProperty "WSDL", http://myComputerName:9090/graphwalker-services?WSDL3. Do until not WebService("SoapServicesService").HasNextStep()4. action = WebService("SoapServicesService").GetNextStep()5. Loop6. Reporter.ReportEvent micDone, "MBT Statistics", WebService("SoapServicesService").GetStatistics()
  14. 14. Mother model Sub model Merged model (uses same data domain)
  15. 15. Purpose to reuse code and models (uses separate data domains) 3 9 11 15 1 13 7 10 16 4 WhatsNew 17 5 2 8 12 18 6 14 SendSongs CheckInbox CleanInbox FreeUser PremiumUser
  16. 16. Switch model caused by an event
  17. 17. - Mindset- Verifies functionality and finds bugs- Test automation as a criteria of definition of done- Testers to model- Developers for test automation- Integrate with nightly builds
  18. 18. E-mail: alex@spotify.comGraphwalker: XL: berner-mattner-home/products/cte-xl/index.html