View webinar: http://www.eurostarconferences.com/community/member/webinar-archive/webinar-81-api-testing-the-heart-of-functional-testing An API, or Application Programming Interface, is a collection of functions that provide much of the functional capabilities in complex software systems. Most customers are accustomed to interacting with a graphical user interface on the computer. But, many customers do not realize the much of the functionality of a program comes from APIs in the operating system or program's dynamic-link libraries (DLL). So, if the business logic or core functionality is exposed via an API call then and if we want to find functional bugs sooner than API testing may be an approach that provides additional value in your overall test strategy. Additionally, API testing can start even before the user interface is complete so functional capabilities can be tested while designers are hashing out the "look and feel." API testing will not replace testing through the user interface, but it can augment your test strategy and provide a solid foundation of automated tests that increase your confidence in the functional quality of your product.

1. Welcome to the 2013
www.eurostarconferences.com
This webinar is presented by Bj Rollison and he will discuss:
‘API Testing: The heart of functional testing’
Join us on Twitter! @esconfs #esconfs

2. Overview
• What is API Testing?
• Why should we do API testing?
• What’s the difference between Unit and API Tests?
• How to we approach API testing?
• What types of bugs will we find?
www.eurostarconferences.com
@esconfs
#esconfs

3. What Is API Testing?
• API – Application Programming Interface
• Implementation hidden from calling program
• Component & integration levels
• Individual API
• End-to-End scenarios
• API Testing – testing below the UI
• Programmatically simulate data or
control flow scenarios
• Focus on functionality;
not on behavior or customer experience
www.eurostarconferences.com
@esconfs
#esconfs
http://www.faqs.org/faqs/
software-eng/testing-
faq/section-14.html

4. What Is API Testing?
• Functional testing where it lives!
• Separation of business logic and “look and feel”
www.eurostarconferences.com
@esconfs
#esconfs
Cloud Services
Core business logic
functionality is here!
Customer experience
and behavior is here!

5. Why Should We Do API Testing
• Agile software development
• Team approach – testing engaged early
• Drive testing upstream
• Testing in parallel with development, not after the fact
• Automated test cost reduction
• Stable API interfaces == automation stability
• Reduced business costs
• Reduced build breaks (pre-check-in suite)
• Refocus testing objectives at system level or through GUI
• Software behavior and customer experience
www.eurostarconferences.com
@esconfs
#esconfs

6. When Is API Testing A Good Investment
• Business logic is encapsulated in classes
rather than within forms
• No graphical user interface –
headless testing
• External APIs used by 3rd party developers
• Internal APIs used by 1st party application
developers
www.eurostarconferences.com
@esconfs
#esconfs

7. Unit Testing vs. API Testing
Unit Testing
• Owned by development
• Limited in scope
• Mocked dependencies
• “Does it work by itself”
• Usually ran before check-in
API Testing
• Owned by test
• Broader in scope
• Mocked and real dependencies
• “Does it play well with others”
• Pre-check-in suites
• Ran after build is created
www.eurostarconferences.com
@esconfs
#esconfs

8. Approaches To API Testing
It’s not either black box or white box;
it’s both black and white!
www.eurostarconferences.com
@esconfs
#esconfs

9. Testing an API as a “Black Box”
www.eurostarconferences.com
@esconfs
#esconfs
INPUT PARAMETERS
• lpLocaleName [in, optional]
Pointer to a locale name, or
one of the predefined
variants
• LCType [in]
The locale information to
retrieve.
• lpLCData [out, optional]
Pointer to a buffer in which
this function retrieves the
requested locale
information.
• cchData [in]
Size, in characters, of the
data buffer indicated by
lpLCData.
OUTPUT VALUE
• Returns the number of
characters retrieved in the
locale data buffer if successful
and cchData is a nonzero
value.
• The function returns 0 if it
does not succeed.
• Error codes
ERROR_INSUFFICIENT_BUFFER
ERROR_INVALID_FLAGS
ERROR_INVALID_PARAMETER
int GetLocaleInfoEx(
_In_opt_ LPCWSTR lpLocaleName,
_In_ LCTYPE LCType,
_Out_opt_ LPWSTR lpLCData,
_In_ int cchData
);
API testing starts as soon as the API
interface is spec’d out!

10. Testing an API as a “Black Box”
• Input argument values
• Boundary values, equivalent set samples
• Parameter combinations
• Combinatorial (n-wise) testing
• Error Validation
• Exception handling, and output data w/error codes
• Properties
• Getters and setters
• Scenario testing
• Sequence of API calls, E-2-E scenario testing
www.eurostarconferences.com
@esconfs
#esconfs

11. Testing an API as a “White Box”
BOOL KillProcessByName(char *szProcessToKill)
{
HANDLE hProcessSnap;
HANDLE hProcess;
PROCESSENTRY32 pe32;
DWORD dwPriorityClass;
hProcessSnap = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
if(hProcessSnap == INVALID_HANDLE_VALUE){
return( FALSE );
}
pe32.dwSize = sizeof(PROCESSENTRY32);
if (!Process32First(hProcessSnap, &pe32)) {
CloseHandle(hProcessSnap);
return( FALSE );
}
do{
if(!strcmp(pe32.szExeFile,szProcessToKill)){
hProcess = OpenProcess(PROCESS_TERMINATE,0, pe32.th32ProcessID);
TerminateProcess(hProcess,0);
CloseHandle(hProcess);
}
}while(Process32Next(hProcessSnap,&pe32));
…..
www.eurostarconferences.com
@esconfs
#esconfs

12. Testing an API as a “White Box”
• Code Review
• Early and effective bug detection
• Code coverage analysis
• Structural testing, identify holes
• Fault injection
• Break points, resource constraints, system failures
• Traces
• Trace statements useful for debugging
www.eurostarconferences.com
@esconfs
#esconfs

13. • Robust automated test cases 5 stages of API test design
• Setup
• Initialize data, create objects, start services, etc.
• Execution
• Steps to exercise API or scenario, also logging
• Verification
• Oracles to evaluate execution outcome
• Reporting
• Pass, Fail, or Blocked
• Clean up
• Pre-test state
API Test Design Principles
www.eurostarconferences.com
@esconfs
#esconfs

14. What Types Of Bugs Will API Testing Find
• Fails to handle error conditions gracefully
• Missing or duplicate functionality
• Unused flags
• Not implemented errors
• Stress
• Reliability
• Security
• Multi-threading issues
• Inconsistent error handling
• Performance
• Improper errors/warnings to caller
• Incorrect handling of valid argument values
www.eurostarconferences.com
@esconfs
#esconfs

15. Key Points
• API testing benefits both the development and testing organization.
• API testing is both a black box and white box activity
• Finds some types of bugs more
efficiently; but…can’t find all bugs!
• Doesn’t evaluate “look and feel”
• Can change focus of GUI automation
• Requires testers with additional skills
• Not for everyone!
www.eurostarconferences.com
@esconfs
#esconfs

16. Questions or Comments…
• For more thoughts on API testing see…
• www.testingmentor.com/imtesty
• If you have questions or comments…
• Email – Bj.Rollison@TestingMentor.com
• Twitter - @TestingMentor
www.eurostarconferences.com
@esconfs
#esconfs

17. Thanks for listening!
@esconfs
#esconfs
www.eurostarconferences.com
Go to www.eurostarconferences.com for more information