Testing API's: Tools & Tips & Tricks (Oh My!)
•Download as PPTX, PDF•
1 like•158 views
A super enjoyable and entertaining walk through API memory lane, then a primer on how to test API's from a unit, integration, and monitoring standpoint. Then a demo on a CI/CD implementation I created at Snagajob.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Testing API's: Tools & Tips & Tricks (Oh My!)
Similar to Testing API's: Tools & Tips & Tricks (Oh My!) (20)
More from Ford Prior
More from Ford Prior (12)
Recently uploaded
Recently uploaded (20)
Testing API's: Tools & Tips & Tricks (Oh My!)
- 1. Testing API’s Strategy & Tools & Tricks (Oh My!)
- 2. ABOUT ME CONTACT • Multi-disciplinary background • QA practitioner for 5 years • Corporate, start-up, mid-size • Agile & waterfall • Stoked about: • Agile • CI/CD • AWS • Open-source • Wife/kid/jiu-jitsu/eating Email (fordprior@gmail.com) GitHub (fordprior) LinkedIn (Ford Prior)
- 5. 5
- 9. 9
- 12. 1 2
- 16. WDDX + XML-RPC = SOAP
- 21. rest exposes database resources to handle CRUD operations soap exposes application functions (not data) to implement some business logic
- 22. database rest
- 28. so… You mentioned API testing?
- 29. What • Why • When • Who • Where • How ● API testing…What’s the point, man? 29 ● Quick feedback loop…like 200 ms quick ● Easy to create/run/maintain/extend
- 30. What • Why • When • Who • Where • How ● Where in the SDLC should we start validating API’s? 3 0 Before • During • Interval •After • Trigger • Manually
- 31. What • Why • When • Who • Where • How ● Where in our tech stack should we test? 3 1 Local • Sandbox • Staging • Prod
- 32. What • Why • When • Who • Where • How ● Who should own testing? 3 2 ● Define “testing”, change culture ● PO, QA, Dev, robots
- 33. What • Why • When • Who • Where • How ● What approaches and tools should we use? 3 3 exploratory • regression • performance
- 34. Popular tools (free) Name Summary Pros Cons* Swagger UI Dynamically-generated documentation for API’s • Free. Beautiful, easy, & everyone likes it (dev, QA, business). • “Try it out!” button. • Not built for testing. Postman (free) SaaS-based desktop solution for REST & SOAP • Awesome UI • Super easy to test single calls or huge collections. • Environment & global variables + JavaScript libraries • Open-sourced CLI package that can run exported collections + Active community • No team collaboration SoapUI (free) • Free • Best solution for SOAP (I think) • Less easy to use Browser console Capture incoming HTTP traffic from any web UI • Free Fiddler Capture incoming HTTP traffic from any web UI • Free • Allows filtering and in-depth analysis of calls • Sometimes doesn’t work on certain browsers or certain sites.
- 35. Popular tools (not free) Name Summary Pros Cons* Postman Pro Free Postman + more… • Postman Pro API (grab collections w/ a token from any environment) • Team collaboration • Prod monitoring • Mock servers • $8/mo per user Runscope SaaS web solution for REST • Awesome UI • Does pretty much everything (smoke, functional, performance). • Environment & global variables. • Reporting & dashboard • Great support • Not free ($90/mo for 5 users). • Gets slow when you start editing large collections of tests. • Must install agent to trigger. Custom framework • Infinite possibilities • More secure (in theory) • Huge upfront commitment of time and expertise. Not free. • Enough rope to hang yourself
- 36. gger
- 37. browser console
- 38. fiddler
- 39. postman
- 40. runscope
- 41. soap ui
- 42. custom
- 44. The problem 1. Greenfield development. Developing lots of new stuff. 2. Regression issues. New features are fine, but we’re breaking older stuff with each deploy. 3. UI tests suck. They’re (1) inconsistent; (2) difficult to maintain; and (3) time-consuming to create.
- 45. Additional hurdles 1. Aggressive timeline. We’re trying to stay ahead of the competition. 2. Evolving product. Ongoing user testing means changing front-end experience, sometimes even underlying architecture. 3. Dev mentality. Developers do not write/maintain automated tests or vet releases. Spotify model (embedded QA)
- 46. Our solution 1. Automated testing on API’s against any integration environment. Bonus: Developers can maintain Extra bonus: PO’s can, too.
- 47. The steps 1. Explore API testing. Make a list of options & rank them. Prototype top 3 & score them. Build consensus as a QA chapter. 2. Implement it on 1 product team. Find a willing collaborator and start using it. 3. Roll out org-wide. Bring in Engineering leadership, including app dev & devops. <???>. Profit.
- 48. Proof-of-concept demo The proof of concept we ended up trashing, but that works! GitHub push (QA to UAT) 1 CircleCI build begins (remote servers) 2 Trigger Runscope tests in QA environment 3 App deployed in full to UAT environment 4 Slack + Results display5
- 49. v2 This is what’s cooking at the moment…but doesn’t work quite yet ;) CircleCI build begins (remote servers) 2 Postman rests run in QA environment 3 Deploy triggered (SNS + Lambda) 4 Results indexed with ElasticSearch 6 Slack + Results display 7 GitHub push (QA to UAT) 1 App deployed in full to UAT 5
- 52. Challenges 1. Environment. Where exactly do we test the code? Intermediary environment problem. 2. Triggering. How can we make CircleCI (hosted) talk directly to our custom-built internal services? 3. Extensibility. How can we build this in a way that (a) acts as a true service; (b) does not require regular maintenance; and (c) does not require hand-holding?
- 54. Questions?
Editor's Notes
- API testing 101 About Me About Snagajob HTTP Is how the web communicates. Super quick over the wire, 234 ms. API's An API isn’t the same as the remote server — rather it is the part of the server that receives requests and sends responses. The code that takes the HTTP requests and turns it into business vaue. Stores in database so another app can access it, Applies some logic to transform it into a number someone really cares about Feeds it into some proprietary algo to drive insight All of the above Public & Private (amazon button story) Microservices are new hottness HTTP basics this is a protocol, just like a language. 1) headers (includes auth, tells service what type of content it's delivering, and what to accept in terms of a response) 2) querystrings (optional) which are on the actual route and specify what to ask the application for (e.g. `http://gateway.api.snagQA.corp/customers?isActive=true` is asking for customers where isActive=true) 3) body (also optional) which is giving some detailed data to the service so, those 3 things the BODY is usually part of anything related to creation, but it can also be passed to an existing entity to overwrite some existing data. so new customers will have large bodies
- Apple introduces the iMac in August of 1998. iBook comes in 1999. XML 1.0 becomes a W3C recommendation on February 8, 1998. IE just starting to eclipse NetScape
- Jeff Bezos of Amazon.com is named Time Person of the Year December 27, 1999.
- So, now the internet is like super OPEN, all thanks to HTTP among other things Before, the internet was this hard-wired rigid proprietary place. Now anyone can sell stuff there. All of the sudden, to do business on a flattened global marketplace, business apps had to get their shit together so they could talk to all of the services required to do business. Before, you’d have to call a sales person who sat a desktop behind a firewall in a building somewhere, and they’d key in your order into some crusty old legacy app. Now, with all of these people able to submit online orders, and all this demand to talk to other services quickly to make sure those orders got through and were fulfilled, CEO’s were facing a dilemma: Rewrite our apps so they’re client-server ready (instead of being mainframe) - $10 million! Figure out some way to open up those old closed systems so they could talk to eachother and the internet in general. Solution? The message bus. SOA. Just make sure mainframe can throw stuff onto a message bus. So far, this is all internal. All of the sudden, people are having the clever idea of opening up their API’s to the e-commerce world (other systems or business partners). This is where SOAP comes in.
- 2000 - SOA & the word “loosely-coupled” is now a thing Time to market I snow like ZERO Salesforce talks to your apps for you so they don’t have to talk to each other (selling their API) Salesforce.com launched its enterprise-class, web-based, sales force automation as a "Internet as a service". XML APIs were part of Salesforce.com from day one. Salesforce.com identified that customers needed to share data across their different business applications, and APIs were the way to do this.
- Dave winer invents XML-RPC, the idea of wrapping HTTP requests in XML to standardize them MEANWHILE, SOAP gets W3C endorsement thanks to push from IBM & Microsoft, and Roy Fielding goes off to create REST and publishes a huge treatise as to why it’s better. Soon, all the big guys are using it (IBM, Microsoft, Sun, Oracle, HP). Fielding wanted to make more of the fact that _____ had built-in HTTP methods (GET, POST, DELETE, PUT). Another important difference between REST and SOAP is that it's resource-based. So the API accesses nouns (aka URIs), instead of verbs. Then, HTTP verbs are used to access those resources.
- At the time, this is how SOAP worked. XML was the new hotness, Before JSON became the go-to hypermedia. JSON is way more succint, compact, easy-to-read, and is uniform so everyone knows what it does after 3 seconds of looking at it This simplicity, while it has limitations, is good and lightweight, when you start dealing in complex systems. It forces simple code. FIRST ATTEMPT AT STANDARDIZING web services CORBA, DCOM, to share data or objects across network boundaries. SOAP exposes operations that implement some business logic The goal: designed to loosely couple systems for integration of diverse systems STRONG typing = stability and governance
- Had already been helping Tim Berners-Lee in forming the World Wide Web Consortium (to start defining standards), specifically in writing the first standards list for HTTP.
- Architectural Styles and the Design of Network-based Software Architectures Instead of SOAP (operations that represent the implementation of some business logic), REST is all about resources which represent data. Instead of storing server-side
- http://spf13.com/post/soap-vs-rest/
- 2000- The launch of the eBay API was a response to the growing number of applications that were already relying on its site either legitimately or illegitimately. The API aimed to standardize how applications integrated with eBay, and make it easier for partners and developers to build a business around the eBay ecosystem. eBay is considered the leading pioneer in the current era of web-based APIs and web services and still leads with one of the most successful developer ecosystem today. 2002- they actually start offering an accessible API for anyone to use. And it’s in REST, so it’s easy to use, too.
- 2004 Flickr launched their own REST API in August of 2004, just in time for the rise of social networking and blogging. They quickly became the go-to platform for images, which bloggers were finally able to easily embed on their sites and social media feeds. They really led the way.
- Local – you can run them locally and stub out services Sandbox– you can run them in an integration environment to see if their “shippable” code passes Integration – you can run them on a staging environment to see if a final PR is ready for release Prod – you can run them on an ongoing basis as a monitor
- Strategies Functional (works as designed) Regression (nothing else breaks) Performance & Load (avg. response times)
- No trigger