API Testing: The heart of functional testing" with Bj Rollison
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


API Testing: The heart of functional testing" with Bj Rollison



View webinar: http://www.eurostarconferences.com/community/member/webinar-archive/webinar-81-api-testing-the-heart-of-functional-testing ...

View webinar: http://www.eurostarconferences.com/community/member/webinar-archive/webinar-81-api-testing-the-heart-of-functional-testing

An API, or Application Programming Interface, is a collection of functions that provide much of the functional capabilities in complex software systems. Most customers are accustomed to interacting with a graphical user interface on the computer. But, many customers do not realize the much of the functionality of a program comes from APIs in the operating system or program's dynamic-link libraries (DLL). So, if the business logic or core functionality is exposed via an API call then and if we want to find functional bugs sooner than API testing may be an approach that provides additional value in your overall test strategy. Additionally, API testing can start even before the user interface is complete so functional capabilities can be tested while designers are hashing out the "look and feel." API testing will not replace testing through the user interface, but it can augment your test strategy and provide a solid foundation of automated tests that increase your confidence in the functional quality of your product.



Total Views
Views on SlideShare
Embed Views



2 Embeds 177

http://www.eurostarconferences.com 176
http://webmarshal.home 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

API Testing: The heart of functional testing" with Bj Rollison Presentation Transcript

  • 1. Welcome to the 2013 www.eurostarconferences.com This webinar is presented by Bj Rollison and he will discuss: ‘API Testing: The heart of functional testing’ Join us on Twitter! @esconfs #esconfs
  • 2. Overview • What is API Testing? • Why should we do API testing? • What’s the difference between Unit and API Tests? • How to we approach API testing? • What types of bugs will we find? www.eurostarconferences.com @esconfs #esconfs
  • 3. What Is API Testing? • API – Application Programming Interface • Implementation hidden from calling program • Component & integration levels • Individual API • End-to-End scenarios • API Testing – testing below the UI • Programmatically simulate data or control flow scenarios • Focus on functionality; not on behavior or customer experience www.eurostarconferences.com @esconfs #esconfs http://www.faqs.org/faqs/ software-eng/testing- faq/section-14.html
  • 4. What Is API Testing? • Functional testing where it lives! • Separation of business logic and “look and feel” www.eurostarconferences.com @esconfs #esconfs Cloud Services Core business logic functionality is here! Customer experience and behavior is here!
  • 5. Why Should We Do API Testing • Agile software development • Team approach – testing engaged early • Drive testing upstream • Testing in parallel with development, not after the fact • Automated test cost reduction • Stable API interfaces == automation stability • Reduced business costs • Reduced build breaks (pre-check-in suite) • Refocus testing objectives at system level or through GUI • Software behavior and customer experience www.eurostarconferences.com @esconfs #esconfs
  • 6. When Is API Testing A Good Investment • Business logic is encapsulated in classes rather than within forms • No graphical user interface – headless testing • External APIs used by 3rd party developers • Internal APIs used by 1st party application developers www.eurostarconferences.com @esconfs #esconfs
  • 7. Unit Testing vs. API Testing Unit Testing • Owned by development • Limited in scope • Mocked dependencies • “Does it work by itself” • Usually ran before check-in API Testing • Owned by test • Broader in scope • Mocked and real dependencies • “Does it play well with others” • Pre-check-in suites • Ran after build is created www.eurostarconferences.com @esconfs #esconfs
  • 8. Approaches To API Testing It’s not either black box or white box; it’s both black and white! www.eurostarconferences.com @esconfs #esconfs
  • 9. Testing an API as a “Black Box” www.eurostarconferences.com @esconfs #esconfs INPUT PARAMETERS • lpLocaleName [in, optional] Pointer to a locale name, or one of the predefined variants • LCType [in] The locale information to retrieve. • lpLCData [out, optional] Pointer to a buffer in which this function retrieves the requested locale information. • cchData [in] Size, in characters, of the data buffer indicated by lpLCData. OUTPUT VALUE • Returns the number of characters retrieved in the locale data buffer if successful and cchData is a nonzero value. • The function returns 0 if it does not succeed. • Error codes ERROR_INSUFFICIENT_BUFFER ERROR_INVALID_FLAGS ERROR_INVALID_PARAMETER int GetLocaleInfoEx( _In_opt_ LPCWSTR lpLocaleName, _In_ LCTYPE LCType, _Out_opt_ LPWSTR lpLCData, _In_ int cchData ); API testing starts as soon as the API interface is spec’d out!
  • 10. Testing an API as a “Black Box” • Input argument values • Boundary values, equivalent set samples • Parameter combinations • Combinatorial (n-wise) testing • Error Validation • Exception handling, and output data w/error codes • Properties • Getters and setters • Scenario testing • Sequence of API calls, E-2-E scenario testing www.eurostarconferences.com @esconfs #esconfs
  • 11. Testing an API as a “White Box” BOOL KillProcessByName(char *szProcessToKill) { HANDLE hProcessSnap; HANDLE hProcess; PROCESSENTRY32 pe32; DWORD dwPriorityClass; hProcessSnap = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0); if(hProcessSnap == INVALID_HANDLE_VALUE){ return( FALSE ); } pe32.dwSize = sizeof(PROCESSENTRY32); if (!Process32First(hProcessSnap, &pe32)) { CloseHandle(hProcessSnap); return( FALSE ); } do{ if(!strcmp(pe32.szExeFile,szProcessToKill)){ hProcess = OpenProcess(PROCESS_TERMINATE,0, pe32.th32ProcessID); TerminateProcess(hProcess,0); CloseHandle(hProcess); } }while(Process32Next(hProcessSnap,&pe32)); ….. www.eurostarconferences.com @esconfs #esconfs
  • 12. Testing an API as a “White Box” • Code Review • Early and effective bug detection • Code coverage analysis • Structural testing, identify holes • Fault injection • Break points, resource constraints, system failures • Traces • Trace statements useful for debugging www.eurostarconferences.com @esconfs #esconfs
  • 13. • Robust automated test cases 5 stages of API test design • Setup • Initialize data, create objects, start services, etc. • Execution • Steps to exercise API or scenario, also logging • Verification • Oracles to evaluate execution outcome • Reporting • Pass, Fail, or Blocked • Clean up • Pre-test state API Test Design Principles www.eurostarconferences.com @esconfs #esconfs
  • 14. What Types Of Bugs Will API Testing Find • Fails to handle error conditions gracefully • Missing or duplicate functionality • Unused flags • Not implemented errors • Stress • Reliability • Security • Multi-threading issues • Inconsistent error handling • Performance • Improper errors/warnings to caller • Incorrect handling of valid argument values www.eurostarconferences.com @esconfs #esconfs
  • 15. Key Points • API testing benefits both the development and testing organization. • API testing is both a black box and white box activity • Finds some types of bugs more efficiently; but…can’t find all bugs! • Doesn’t evaluate “look and feel” • Can change focus of GUI automation • Requires testers with additional skills • Not for everyone! www.eurostarconferences.com @esconfs #esconfs
  • 16. Questions or Comments… • For more thoughts on API testing see… • www.testingmentor.com/imtesty • If you have questions or comments… • Email – Bj.Rollison@TestingMentor.com • Twitter - @TestingMentor www.eurostarconferences.com @esconfs #esconfs
  • 17. Thanks for listening! @esconfs #esconfs www.eurostarconferences.com Go to www.eurostarconferences.com for more information