Downloaded 22 times
Uploaded byJoris Meerts
Moving from Ad Hoc Testing to Continuous Test Data with FitNesse
The document discusses moving from ad hoc testing to continuous test data using FitNesse. It describes the context of a legacy e-commerce system and outlines reasons for automating testing. FitNesse is presented as a tool that allows separating test data from tests and modularizing test code. Some lessons learned include balancing test automation with exploratory testing and ensuring test code is maintainable. In conclusion, test automation helps observe system behavior and catch bugs with data running through the system continuously.
More Related Content
Similar to Moving from Ad Hoc Testing to Continuous Test Data with FitNesse
![[Paul Holland] Trends in Software Testing](https://cdn.slidesharecdn.com/ss_thumbnails/paulhollandtrendsinswtesting-140717230533-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Moving from Ad Hoc Testing to Continuous Test Data with FitNesse
- 1. Moving from AdHoc Testing to Continuous Test Data with Moving from Ad Hoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 FitNesse Agile Testing Days Joris Meerts, 12 November 2014
- 2. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 about me • Joris Meerts • Software tester since 2007 • Capgemini • www.testingreferences.com • Context-driven school • Member of the Dutch Exploratory Workshop on Testing (DEWT) • @testingref
- 3. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 outline • The context • The starting point • FitNesse • Java • Some lessons learned
- 4. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 The context
- 5. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014
- 6. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014
- 7. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 the context: the system • Legacy e-commerce database • Many (190+) complex tables • Many stored procedures, triggers • Complex scheduling • Hardly any functional documentation • Processing may take days or weeks
- 8. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 the context: the team • Agile team • 1 - 2 senior testers • 4 - 6 senior database developers • 1 product owner • 1 scrum master
- 9. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 The starting point
- 10. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 Why automate?
- 11. • Ad hoctesting • No scheduled tests • ‘Technical tests’ in FitNesse • No use of development tools for test code Moving from Ad Hoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 the starting point
- 12. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 a first time for everything • FitNesse, Java, JUnit, Jenkins, Eclipse • Advanced SQL, PL/SQL
- 13. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 FitNesse
- 14. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 test architecture
- 15. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 why I would recommend FitNesse • All purpose tool • Modularization • Separating data from tests • Symbolic links
- 16. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 all purpose tool • Database • REST interface • SSH / Telnet • FTP • No GUI automation
- 17. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 modularization • Use of building blocks • Parameterized includes
- 18. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014
- 19. • Include testdata in the scripts • Restrict the use of test data • Easy to find out where the data is used • Use scripts to check configuration Moving from Ad Hoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 separate data from tests
- 20. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 other useful things in FitNesse • Symbolic links • Templates • Tags
- 21. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 Java
- 22. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 or… how to learn programming in a nutshell
- 23. • JUnit fortesting • Very easy build and deploy • Eclipse for development • OpenJPA replacing scattered SQL statements Moving from Ad Hoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 some wonderful things about Java
- 24. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 Some lessons learned
- 25. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 some lessons learned • It is not easy to strike a balance between developing test automation and (exploratory) testing.
- 26. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 some lessons learned • Code that is hard to test can really drain the energy out of the test automation effort.
- 27. Moving from AdHoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 some lessons learned • Test automation does not become a team effort by placing it in the definition of done. • Who should write the test code?
- 28. Determination is animportant success factor. Moving from Ad Hoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 some lessons learned
- 29. • Data isrunning through the system. • We are able to observe system behavior. • We are able to test some features better. • Test automation catches bugs. Moving from Ad Hoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 conclusion
- 30. • Gojko Adzic- Test Driven .NET Development with FitNesse (2009) • Stack Overflow • Bruce Eckel - Thinking in Java (2006) Moving from Ad Hoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 some useful resources
- 31. thank you forattending my presentation Moving from Ad Hoc Testing to Continuous Test Data with FitNesse Joris Meerts, 12 November 2014 joris.meerts@capgemini.com @testingref www.testingreferences.com patternsofproof.wordpress.com
Editor's Notes
- #2 This is an experience story about automating the creation of test data by making use of FitNesse. It was done in a project for the e-commerce operation of the largest supermarket chain in the Netherlands; Albert Heijn Online.
- #6 This is the current website. However, from my perspective it is not much more than a collection of buttons, text and images.
- #7 This is the distribution centre in Almere that was built in 2012 to serve, among other places, the city of Amsterdam. It measures about 13.000 m2. Most of the things that are physically in this building have their representation in our database: the batch pickers, the products, the shelves, the loading bays, the carts, the crates etc…. All of the processes that are used in this building have their representation in our database.
- #8 E-commerce back-end: handling everything: placement of an order allocating delivery slots scheduling delivery runs maintaining stock generating financial data. track employee productivity replenishment operational data Business logic is scattered all over the code. It is like a huge glass vase that your drop on a tile floor and it breaks into a million pieces. Oracle databases, Oracle forms order influences reports production forecast replenishment payment
- #11 Wen I started in the project I compared the system to a desert. No data was flowing through the system. As a result it was impossible to observe system behavior, since system behavior can only be observed when the system is processing data. I am not writing tests. I am writing scripts that generate data and push data through the system.
- #12 What do I mean by ad hoc testing? A test that is not scheduled is not a test! Code was scattered all across FitNesse scripts. When you need to do refactoring FitNesse is not the best place to do it. Development tools: Eclipse, version control (subversion)
- #13 I introduced the use of Eclipse, the use of JUnit, the use of version control, automatic build and deployment of the code.
- #17 Much of what FitNesse is able to do depends on what is done in the Java layer. I investigated GUI automation using OpenScript, but it seems that this solution would pose serious diffficulties.
- #18 Gradually a building block evolved to which I supplied a functional description and a list of input and output parameters. This made it easier to connect with this building block and use it in a script. The purpose of making building blocks is to keep refactoring in mind. So that if the Java code changes there is a very limited (and easily traceable number of places) where the scripts have to change.
- #19 This is an example of a building block. As you can see, I added a functional description.
- #20 Make it impossible to use data that is not configured. Search for the use of data with ‘Where used’ in FitNesse. Data configuration is a major part in the testing effort. If the data is incorrect, somewhere along the line the results that you expect may not be there.
- #21 Symbolic links allow you to run the same group of tests against different environments. Tags are very handy because you can build large test suites and run only selection of those based on tags.
- #22 The flexibility of FitNesse depends on what happens in the Java layer. The fixtures are a sort of API against the back end system.
- #23 The team consisted of PL / SQL developers, so developing the Java fixtures was up to the testers. Programming can only be learned by doing it. There is no course you can take to make you a better programmer faster. There are books that are helpful. After almost a year I wrote the first piece of what I believe to be efficient code.
- #24 JUnit allows you to explore what you built. It takes about 30 seconds to build and deploy the code for the fixtures. It runs automatically. Build & deploy is something we do not have to think about, it just happens. Eclipse makes life easier in so many ways. It can generate much of the code, gives suggestions, has useful plugins etc… Starting with OpenJPA (object-relational mapping software) was a risk. It requires an investment. In return it gives you (much) better control over database access.
- #26 Time spent on test automation cannot be spent on anything else. There are user stories in a sprint that need to be tested. On the other hand there is the apparent need for proper test data that can only be generated using test automation. I had to make choices in this area more than once and each time it is a difficult decision, especially when your choices impact the quality of the code that goes into production.
- #27 Test automation doesn’t do the root cause analysis for you. If the root cause analysis takes a couple of hours and there is a failure every day then automated checks consume quite a lot of time. This may negatively impact the value of automated checks. It is hard to write tests that give you effective and reliable information about the cause of failure.
- #28 Test automation should be a team effort; automation code should be maintained by the whole team. But placing it in the definition of done does not make it a team effort. People on the team have different thoughts about test automation; who should do it, how much effort should be put into it, in which language it should be written. Test automation can only become a team effort once you start discussing it with your team members.
- #30 These are some of the outcomes of the test automation effort.