Agile2016: Exploratory Testing an API

Maaret Pyhäjärvi
Maaret PyhäjärviPrincipal Test Engineer at Vaisala Oyj
@maaretp http://maaretp.com
Exploratory Testing
an API
by Maaret Pyhäjärvi
@maaretp http://maaretp.com
Testers don’t break your
code, they break your
illusions about the code.
-- adapted from James Bach
@maaretp http://maaretp.com
Exploratory testing is a
systematic approach for
discovering risks using
rigorous analysis techniques
coupled with testing heuristics.
@maaretp http://maaretp.com
What Testing gives Us
UnitTesting
ExploratoryTesting
SPEC
FEEDBACK
REGRESSION
GRANULARITY
GUIDANCE
UNDERSTANDING
MODELS
SERENDIPITYTesting as
artifact
creation
Testing as
performance
@maaretp http://maaretp.com
Before
implementing
While
implementing
Before
production
While in
production
Testing as
artifact creation
Testing as
performance
(exploration)
Ways to think of testing peak at different moments with regards to adding a capability
TOTALS
Testing is Everywhere!
@maaretp http://maaretp.com
An application programming
interface (API) is a set of
routines, protocols, and tools
for building software and
applications.
@maaretp http://maaretp.com
Product is my external
imagination
I am my developer’s external imagination
@maaretp http://maaretp.com
Teaching Exploratory Testing
in a Mob
@maaretp http://maaretp.com
Let’s Test!
A Unit Testing Library
@maaretp http://maaretp.com
The Test Target
@maaretp http://maaretp.com
Mob Testing – Rules for the
Exercise
No thinking as the driver
Yes, and…
Kindness, consideration and respect
@maaretp http://maaretp.com
From Michael Sahota
STICKYNOTESDONERIGHT
@maaretp http://maaretp.com
Takeaways
Exploratory Testing an API
@maaretp http://maaretp.com
@maaretp http://maaretp.com
@maaretp http://maaretp.com
@maaretp http://maaretp.com
How to Explore
•  Do something with it
•  Find out what is the common thing to do with it
•  Find out what you could do with it
•  Reflect after anything and everything – make
notes
–  What do we know from other connections?
–  What do we know from empirical evidence?
–  How do we turn it all into empirical evidence?
@maaretp http://maaretp.com
Structure
Function
Data
Platform
Operations
Time
SFDPOT heuristics from James Bach / Michael Bolton, Rapid SoftwareTesting
@maaretp http://maaretp.com
Approvers do
•  Formatting
•  Sorting
•  File Extensions
•  Scrubbing (removing common inconsistencies)
•  Serialization(saving to a file)
•  Mocking
•  Proxying
•  Rendering
•  Execution (e.g. retrieve the URL)
•  Aggregating test cases
•  File naming
•  PRINCIPLE: ”Every time you handle this type of object, you do
these things to it.”
I LEARNED ABOUT FUNCTIONS
@maaretp http://maaretp.com
Reporters do
•  Waiting
•  Scrubbing (removing common inconsistencies)
•  Execution
•  Launching
•  Serialization
•  Decompilation
•  Chain of responsibility
•  Creating Approved file
•  Environmental awareness
I LEARNED ABOUT FUNCTIONS
@maaretp http://maaretp.comI LEARNED ABOUTTHE ENVIRONMENT
@maaretp http://maaretp.com
@maaretp http://maaretp.com
A Few Ideas of Exploratory Testing
•  It’s not just about GUI and finalized features
–  You can explore an API
–  You can adapt to known limitations
•  It’s not just for functional
–  You should do exploratory performance/security …
testing
•  It’s not without automation
–  Sometimes you need to do things humans can’t do! –
exploratory test automation
•  Repeating is seldom an issue
–  Vary the data, the environment, the story around your
testing – exploratory regression testing
@maaretp http://maaretp.com
Patterns of Exploration for
APIs
Lessons Distilled
@maaretp http://maaretp.com
Working with limited
understanding: Focus!
@maaretp http://maaretp.com
Calls and Operations.
Inputs and Outputs.
Exceptions.
@maaretp http://maaretp.com
Target of your testing
vs. the environment
it depends on
@maaretp http://maaretp.com
Specific user with
a specific purpose
@maaretp http://maaretp.com
Why would anyone
use this?
@maaretp http://maaretp.com
Think lifecycle
Versioning and Deprecation
@maaretp http://maaretp.com
Google for Concepts
like “Conservative Overloading Strategy”
@maaretp http://maaretp.com
Fast-track your
understanding:
collaborate
@maaretp http://maaretp.com
Documentation Matters
a Lot for APIs
@maaretp http://maaretp.com
Explore to
create documentation
@maaretp http://maaretp.com
Some patterns become
visible with repetition
@maaretp http://maaretp.com
Disposable
test automation
@maaretp http://maaretp.com
#MobProgrammingGuidebook
@maaretp http://maaretp.com
Maaret Pyhäjärvi
Email: maaret@iki.fi
Twitter: @maaretp
Web: maaretp.com
Blog: visible-quality.blogspot.fi
(please connect with me through
Twitter or LinkedIn)
1 of 38

More Related Content

What's hot(20)

AATC2016: Exploratory testing an APIAATC2016: Exploratory testing an API
AATC2016: Exploratory testing an API
Maaret Pyhäjärvi769 views
AgileTD: Mob testing Training DayAgileTD: Mob testing Training Day
AgileTD: Mob testing Training Day
Maaret Pyhäjärvi962 views
AgileTD: Learning through osmosisAgileTD: Learning through osmosis
AgileTD: Learning through osmosis
Maaret Pyhäjärvi1.1K views
Geekle QA: Automationist's GambitGeekle QA: Automationist's Gambit
Geekle QA: Automationist's Gambit
Maaret Pyhäjärvi615 views
SLASSCOM: Breaking illusions with TestingSLASSCOM: Breaking illusions with Testing
SLASSCOM: Breaking illusions with Testing
Maaret Pyhäjärvi142 views
Testing javascriptTesting javascript
Testing javascript
Robert Greiner1.3K views
Lessons Learned When AutomatingLessons Learned When Automating
Lessons Learned When Automating
Alan Richardson1.4K views
Andrew rusling 21 experiments to increase velocityAndrew rusling 21 experiments to increase velocity
Andrew rusling 21 experiments to increase velocity
Scrum Australia Pty Ltd774 views
WooCommerce snippetsWooCommerce snippets
WooCommerce snippets
Rodolfo Melogli110 views
Technical Testing WebinarTechnical Testing Webinar
Technical Testing Webinar
Alan Richardson6.9K views
Technology Based TestingTechnology Based Testing
Technology Based Testing
Alan Richardson2K views

Similar to Agile2016: Exploratory Testing an API

Rails TestingRails Testing
Rails Testingmikeblake
622 views17 slides
Test Driven DevelopmentTest Driven Development
Test Driven DevelopmentZendCon
2.1K views34 slides

Similar to Agile2016: Exploratory Testing an API(20)

Test driven developmentTest driven development
Test driven development
Dennis Ahaus585 views
Rails TestingRails Testing
Rails Testing
mikeblake622 views
App Assessments ReloadedApp Assessments Reloaded
App Assessments Reloaded
Ernest Mueller611 views
Test Driven DevelopmentTest Driven Development
Test Driven Development
ZendCon2.1K views
Completely Test-DrivenCompletely Test-Driven
Completely Test-Driven
Ian Truslove358 views
TestTest
Test
Eddie Kao790 views
Best practices for JavaScript RIAsBest practices for JavaScript RIAs
Best practices for JavaScript RIAs
Carlos Ble2.6K views
Measuring Coverage From E2E TestsMeasuring Coverage From E2E Tests
Measuring Coverage From E2E Tests
Anand Bagmar757 views
Rtt presoRtt preso
Rtt preso
fdschoeneman776 views
You and your code.pdfYou and your code.pdf
You and your code.pdf
Tony Khánh5 views
Bsides Knoxville - APT2Bsides Knoxville - APT2
Bsides Knoxville - APT2
Adam Compton41 views
Exploratory Testing on Computer interfacesExploratory Testing on Computer interfaces
Exploratory Testing on Computer interfaces
Maaret Pyhäjärvi278 views
Ui Testing with Ghost InspectorUi Testing with Ghost Inspector
Ui Testing with Ghost Inspector
Harvard Web Working Group3.6K views

More from Maaret Pyhäjärvi(20)

ATAGTR '20 Keynote: Building statusATAGTR '20 Keynote: Building status
ATAGTR '20 Keynote: Building status
Maaret Pyhäjärvi46 views
SauceCon: It's More Complicated than THATSauceCon: It's More Complicated than THAT
SauceCon: It's More Complicated than THAT
Maaret Pyhäjärvi7 views
TSQA: Something In The Way We TestTSQA: Something In The Way We Test
TSQA: Something In The Way We Test
Maaret Pyhäjärvi7 views
ATAGTR: Targeting Errors Of OmissionATAGTR: Targeting Errors Of Omission
ATAGTR: Targeting Errors Of Omission
Maaret Pyhäjärvi27 views
Pivotal Moments, Pivotal ExperiencesPivotal Moments, Pivotal Experiences
Pivotal Moments, Pivotal Experiences
Maaret Pyhäjärvi25 views
Let's Do a Thing and Call it FooLet's Do a Thing and Call it Foo
Let's Do a Thing and Call it Foo
Maaret Pyhäjärvi48 views
Programming Assignments in Tester InterviewsProgramming Assignments in Tester Interviews
Programming Assignments in Tester Interviews
Maaret Pyhäjärvi346 views
EuroSTAR '21 Keynote: Automationist's GambitEuroSTAR '21 Keynote: Automationist's Gambit
EuroSTAR '21 Keynote: Automationist's Gambit
Maaret Pyhäjärvi332 views

Agile2016: Exploratory Testing an API