Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Bro, manage test data like a pro! [QA Fest 2018]

317 views

Published on

Almost any application or software system manages data. It is hard to imagine test automation that is not affected by this fact. There are many differenct approaches how to prepare system under test, providing predefined test data: use application UI, invoke API methods, run business logic directly, access DB from test scenarios, etc. In this talk we will review most of existing approaches, starting from the easiest and the most popular ones and finishing with really tricky ways to manage your test data for large distributed systems. There is no ideal solution for every case or silver bullet, but I hope your toolset will become wider after visiting this talk.

Published in: Technology
  • Be the first to comment

Bro, manage test data like a pro! [QA Fest 2018]

  1. 1. Bro, manage your test data like a pro! Mikalai Alimenkou @xpinjection http://xpinjection.com
  2. 2. Telegram channel https://t.me/xpinjection_channel
  3. 3. Test Data Application Driver Test Logic Test components
  4. 4. Try to avoid dependent tests to improve stability and speed
  5. 5. Data independency as performance driver • Most dependencies are data related • Dependent tests = no parallel execution • Data should be test specific with no reuse • Use small focused datasets
  6. 6. Regular system example SYSTEM DB QUEUE UI API EXTERNAL SERVICES API CONTRACTS
  7. 7. If you have user interface, then you can use it for testing
  8. 8. #1. Prepare test data using UI SYSTEM DB QUEUE UI API EXTERNAL SERVICES API CONTRACTS #1. UI
  9. 9. Data Registry
  10. 10. Easiest approach to start with Blackbox solution Clear and simple Unstable Too long Failures chaining
  11. 11. If UI is unreliable, you can switch to API level
  12. 12. #2. Data insertion via API SYSTEM DB QUEUE UI API EXTERNAL SERVICES API CONTRACTS #2. API
  13. 13. Reuse existing API for tests Data oriented Much quicker Require clear API Greybox Coding skills Existing tools
  14. 14. If you have data, put it in the database
  15. 15. #3. SQL for direct data insertion to DB SYSTEM DB QUEUE UI API EXTERNAL SERVICES API CONTRACTS #3. SQL
  16. 16. Unit/integration tests
  17. 17. Use internal system knowledge Flexibility Quickest way Whitebox Support and sync Too verbose Direct data access Caching
  18. 18. #4. Datasets with optional commit SYSTEM DB QUEUE UI API EXTERNAL SERVICES API CONTRACTS #4. DBUnit
  19. 19. Unit/integration tests
  20. 20. API/System level tests
  21. 21. The same approach for NoSQL
  22. 22. Managed or embedded NoSQL storage
  23. 23. Additional level of data abstraction Great tools around DBUnit Suitable format Whitebox Support and sync Small datasets Caching Flexible constraints
  24. 24. #5. Versioned DUMPs SYSTEM DB QUEUE UI API EXTERNAL SERVICES API CONTRACTS #5. DUMPs
  25. 25. Support DUMPs for different suites /serviceA/dump/smoke/v50/data.tar.gz /serviceA/dump/smoke/v51/data.tar.gz /serviceA/schema/v51/db.jar /serviceA/dump/regression/v50/data.tar.gz /serviceA/dump/regression/v51/data.tar.gz
  26. 26. Wrap DUMPs in Docker images
  27. 27. DB Object Pools • DB instance • Heavy domain objects • Available data roots
  28. 28. Version your test data with schema Direct data access Quick Unclear Required skills DUMP preparation via UI Infrastructure support Artifact repository
  29. 29. If your live system produces data, it can be used for testing
  30. 30. #6. Production data DUMP SYSTEM DB QUEUE UI API EXTERNAL SERVICES API CONTRACTS #6. LIVE DATA
  31. 31. Real data is always better for testing Live data No preparation Large Permissions More realistic cases Security
  32. 32. Summary • Use UI for test data if the system is black box for you • Speed up you test data management with API calls • Put data directly in DB if system is white box for you • Actively use Docker to prepare test DB images • Reuse live data if possible for complex systems • Mix approaches to fill test DB and store dumps to build Docker images • Be happy unicorn in the rainbow world ;)
  33. 33. It is not an easy task to create reliable tests, try hard and know your tools
  34. 34. @xpinjection http://xpinjection.com mikalai.alimenkou@xpinjection.com

×