When considering how automation can be useful in testing, many people think only about identifying difference from some expected behaviour, for example with unit tests. Much of my testing is about exploration, about learning, about finding things that might matter that we haven't already thought about. In this talk, I'll discuss how I use automation to help me to do that. In particular I'll demo some ways in which I ask questions of the product and use automation to find expedient ways to get answers.

1. Exploring with Automation
James Thomas. 2022-01-14. EC Utbildning
@qahiccupps

2. www.associationforsoftwaretesting.org
www.ada.com
@qahiccupps

3. Automation
Using a tool that, largely unattended,
replaces or extends the human ability
to do a job
Exploratory Testing
@qahiccupps

4. The Information Space
Known
Unknown
Knowns Unknowns
Data
generation
Data analysis
Data
processing
Repeated
tasks
Boring tasks
Parallel tasks
Documenting
@qahiccupps

5. Today’s Scenario
We’ve joined the Best Buy web team
and want to get familiar with their API
Today’s Mission
Explore the Best Buy API
using any tool that’s helpful
to bootstrap our knowledge
Today’s Caveats
This is not a tutorial in tools or testing
@qahiccupps

6. Testing an API
Products, categories, and delivery costs.
What do you know about?
USB speakers?
Here’s a list: …
HTTP + JSON
GET /actions/
{ actions: [ product, categories, delivery ] }
GET /product?type=USB_speaker
[ {id:5, name: Akai}, {id:7, name: Sony},...]
@qahiccupps

7. @qahiccupps

8. @qahiccupps

9. What Does a Product Look Like?
@qahiccupps

10. What Do Product Descriptions Look Like?
@qahiccupps

11. Can we do Better? Can we Automate?
@qahiccupps

12. Is This Really Automation?
@qahiccupps

13. Can we See More Results?
@qahiccupps

14. Can we Get All of the Results?
@qahiccupps

15. What are the Categories?
@qahiccupps

16. What are the Categories in Use?
@qahiccupps

17. Why so many Prices?
@qahiccupps

18. @qahiccupps

19. What Did We Do?
Explored and learned
● Observing
● Asking questions
● Finding expedient methods
Found potential issues
● Product (server errors)
● Data (typos)
● Metadata (collision in category names)
Built little by little
● Postman - curl
● Single command - pipes
● One-shot - scripts
@qahiccupps

20. @qahiccupps

21. What Could We Do Next?
Revisit interesting places
Explore areas we didn’t look at
Find another lens, e.g. database, unit tests
Document some of the learning as regression tests
@qahiccupps

22. Summary
Automation
● isn’t only for the known-knowns
● can be exploited in exploration
● can give us new lenses to view a system through
● can amplify our capacity to explore
● can be used once and discarded
● we really just want it to help us to answer our questions
@qahiccupps

23. Questions?
@qahiccupps

24. Related Articles From my Blog
Not Sure About Uncertainty
Tools, Take Your Pick
Stuck in the Middle
Test Automation is not Automated Testing
Don’t Just Check Please
Use the Force Multiplier
Exploratory Tooling
Exploring to the Choir
You Can Tidy the Data
Bog-standard Testing
@qahiccupps

25. Selected References
TestPyramid, Martin Fowler
Known Knowns, Wikipedia
Best Buy API Playground (and others at Elizabeth Zagroba, Automation Panda)
Exploratory Testing Elisabeth Hendrickson
Exploratory Testing on Computer Interfaces Maaret Pyhäjärvi
Image Credits
Robot
Clip art
Map of London
@qahiccupps

26. The Tools
Postman
Shell (pipe, redirect, variables, script)
jq
curl
Linux utilities (grep, sort, uniq, wc, diff)
Google Sheets (pivot table)
@qahiccupps