1. Welcome to the 2013
www.eurostarconferences.com
This webinar is presented by Bj Rollison and he will discuss:
‘API Testing: The heart of functional testing’
Join us on Twitter! @esconfs #esconfs

2. Overview
• What is API Testing?
• Why should we do API testing?
• What’s the difference between Unit and API Tests?
• How to we approach API testing?
• What types of bugs will we find?
www.eurostarconferences.com
@esconfs
#esconfs

3. What Is API Testing?
• API – Application Programming Interface
• Implementation hidden from calling program
• Component & integration levels
• Individual API
• End-to-End scenarios
• API Testing – testing below the UI
• Programmatically simulate data or
control flow scenarios
• Focus on functionality;
not on behavior or customer experience
www.eurostarconferences.com
@esconfs
#esconfs
http://www.faqs.org/faqs/
software-eng/testing-
faq/section-14.html

4. What Is API Testing?
• Functional testing where it lives!
• Separation of business logic and “look and feel”
www.eurostarconferences.com
@esconfs
#esconfs
Cloud Services
Core business logic
functionality is here!
Customer experience
and behavior is here!

5. Why Should We Do API Testing
• Agile software development
• Team approach – testing engaged early
• Drive testing upstream
• Testing in parallel with development, not after the fact
• Automated test cost reduction
• Stable API interfaces == automation stability
• Reduced business costs
• Reduced build breaks (pre-check-in suite)
• Refocus testing objectives at system level or through GUI
• Software behavior and customer experience
www.eurostarconferences.com
@esconfs
#esconfs

6. When Is API Testing A Good Investment
• Business logic is encapsulated in classes
rather than within forms
• No graphical user interface –
headless testing
• External APIs used by 3rd party developers
• Internal APIs used by 1st party application
developers
www.eurostarconferences.com
@esconfs
#esconfs

7. Unit Testing vs. API Testing
Unit Testing
• Owned by development
• Limited in scope
• Mocked dependencies
• “Does it work by itself”
• Usually ran before check-in
API Testing
• Owned by test
• Broader in scope
• Mocked and real dependencies
• “Does it play well with others”
• Pre-check-in suites
• Ran after build is created
www.eurostarconferences.com
@esconfs
#esconfs

8. Approaches To API Testing
It’s not either black box or white box;
it’s both black and white!
www.eurostarconferences.com
@esconfs
#esconfs

9. Testing an API as a “Black Box”
www.eurostarconferences.com
@esconfs
#esconfs
INPUT PARAMETERS
• lpLocaleName [in, optional]
Pointer to a locale name, or
one of the predefined
variants
• LCType [in]
The locale information to
retrieve.
• lpLCData [out, optional]
Pointer to a buffer in which
this function retrieves the
requested locale
information.
• cchData [in]
Size, in characters, of the
data buffer indicated by
lpLCData.
OUTPUT VALUE
• Returns the number of
characters retrieved in the
locale data buffer if successful
and cchData is a nonzero
value.
• The function returns 0 if it
does not succeed.
• Error codes
ERROR_INSUFFICIENT_BUFFER
ERROR_INVALID_FLAGS
ERROR_INVALID_PARAMETER
int GetLocaleInfoEx(
_In_opt_ LPCWSTR lpLocaleName,
_In_ LCTYPE LCType,
_Out_opt_ LPWSTR lpLCData,
_In_ int cchData
);
API testing starts as soon as the API
interface is spec’d out!

10. Testing an API as a “Black Box”
• Input argument values
• Boundary values, equivalent set samples
• Parameter combinations
• Combinatorial (n-wise) testing
• Error Validation
• Exception handling, and output data w/error codes
• Properties
• Getters and setters
• Scenario testing
• Sequence of API calls, E-2-E scenario testing
www.eurostarconferences.com
@esconfs
#esconfs

11. Testing an API as a “White Box”
BOOL KillProcessByName(char *szProcessToKill)
{
HANDLE hProcessSnap;
HANDLE hProcess;
PROCESSENTRY32 pe32;
DWORD dwPriorityClass;
hProcessSnap = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
if(hProcessSnap == INVALID_HANDLE_VALUE){
return( FALSE );
}
pe32.dwSize = sizeof(PROCESSENTRY32);
if (!Process32First(hProcessSnap, &pe32)) {
CloseHandle(hProcessSnap);
return( FALSE );
}
do{
if(!strcmp(pe32.szExeFile,szProcessToKill)){
hProcess = OpenProcess(PROCESS_TERMINATE,0, pe32.th32ProcessID);
TerminateProcess(hProcess,0);
CloseHandle(hProcess);
}
}while(Process32Next(hProcessSnap,&pe32));
…..
www.eurostarconferences.com
@esconfs
#esconfs

12. Testing an API as a “White Box”
• Code Review
• Early and effective bug detection
• Code coverage analysis
• Structural testing, identify holes
• Fault injection
• Break points, resource constraints, system failures
• Traces
• Trace statements useful for debugging
www.eurostarconferences.com
@esconfs
#esconfs

13. • Robust automated test cases 5 stages of API test design
• Setup
• Initialize data, create objects, start services, etc.
• Execution
• Steps to exercise API or scenario, also logging
• Verification
• Oracles to evaluate execution outcome
• Reporting
• Pass, Fail, or Blocked
• Clean up
• Pre-test state
API Test Design Principles
www.eurostarconferences.com
@esconfs
#esconfs

14. What Types Of Bugs Will API Testing Find
• Fails to handle error conditions gracefully
• Missing or duplicate functionality
• Unused flags
• Not implemented errors
• Stress
• Reliability
• Security
• Multi-threading issues
• Inconsistent error handling
• Performance
• Improper errors/warnings to caller
• Incorrect handling of valid argument values
www.eurostarconferences.com
@esconfs
#esconfs

15. Key Points
• API testing benefits both the development and testing organization.
• API testing is both a black box and white box activity
• Finds some types of bugs more
efficiently; but…can’t find all bugs!
• Doesn’t evaluate “look and feel”
• Can change focus of GUI automation
• Requires testers with additional skills
• Not for everyone!
www.eurostarconferences.com
@esconfs
#esconfs

16. Questions or Comments…
• For more thoughts on API testing see…
• www.testingmentor.com/imtesty
• If you have questions or comments…
• Email – Bj.Rollison@TestingMentor.com
• Twitter - @TestingMentor
www.eurostarconferences.com
@esconfs
#esconfs

17. Thanks for listening!
@esconfs
#esconfs
www.eurostarconferences.com
Go to www.eurostarconferences.com for more information