4. Gojko Adzic - Quality Criteria
Don't reinvent a wheel
5. Mike Cohn (2009)– Automated regression testing
Don't reinvent a wheel
6. API testing in 2001
• Kaner, Bach, Pettichord: Lessons Learned in Software Testing – Lesson 132:
Automate tests using programming interfaces
“…When we look at many product testing effort we’ve observed, we conclude
that there is a strong correlation between the availability of programming
interfaces for testing and the development of powerful automated test
suites. Programming interfaces include APIs, command-line interfaces, COM
interfaces, HTTP and more. …If you’re serious about automation, you’ll learn
or get someone who knows this to help..
…Focus your attention on where automation can help the most. Sometimes,
you can effectively automate the GUI; other times you can’t. Don’t let
people’s preconceptions of what automation has to look like limit you.”
Don't reinvent a wheel
8. Reasons for antipattern(traditional approach)
• Automate everything!
• Just QA is responsible for quality
• Few testing rounds – no Continuous Integration
• WHAT TO TEST(Business Functionality)=HOW TO TEST (GUI)?
• “This small hotfix won’t break anything”
• Production delivery 1 per year
• DIDN’T FAIL YET
• Trying to find a lot bugs
Don't reinvent a wheel
9. Test find bugs, not automation
• Value of the regression test depend on value of the covered
functionality
• Count of found bugs say nothig about valueeffectivity of automated
test
• Most of regression bugs is caused by unexpected impact of backend
integration not GUI
• Automation is not medicine for wrong communication
Don't reinvent a wheel
10. Conclusions
• Unit testing is the core of regression testing
• API testing is not the FUTURE but it is really just a reinvented wheel
• Future of automated GUI testing is (Exploratory tests, …)???
• GUI testing vs API testing
Don't reinvent a wheel
11. Something about REST API
How to test API through HTTP (REST)
Don't reinvent a wheel
12. REST – Representational State Transfer
• REST or RESTful web services are one way of providing
interoperability between computer systems on the internet. REST-
compliant web services allow requesting systems to access and
manipulate textual representations of web resources using a uniform
and predefined set of stateless operations.
Don't reinvent a wheel
14. RESTful URLs
• URLs intended to improve the usability and accessibility of
a website or web service by being immediately and intuitively
meaningful to non-expert users.
Don't reinvent a wheel
Non-semantic URL Semantic URL
http://example.com/index.php?page=name http://example.com/name
http://example.com/index.php?page=consulting/mar
keting
http://example.com/consulting/marketing
http://example.com/products?category=2&pid=25 http://example.com/products/2/25
http://example.com/cgi-
bin/feed.cgi?feed=news&frm=rss
http://example.com/news.rss
http://example.com/services/index.jsp?category=lega
l&id=patents
http://example.com/services/legal/patents
15. Relationships between URL and HTTP methods
Don't reinvent a wheel
Uniform Resource Locator (URL) GET POST
Collection, such as
http://mgs.com/resources/
List the URIs and perhaps other
details of the collection's
members.
Create a new entry in the
collection. The new entry's URI is
assigned automatically and is
usually returned by the operation.
Element, such as
http://mgs.com/resources/item17
Retrieve a representation of the
addressed member of the
collection, expressed in an
appropriate Internet media type.
Not generally used. Treat the
addressed member as a collection
in its own right and create a new
entry within it.
17. Important HTTP Request Headers
Don't reinvent a wheel
Header field
name
Description Example
Cache-Control
Used to specify directives that must be obeyed by all
caching mechanisms along the request-response chain.
Cache-Control: no-cache
Content-Type
The MIME type of the body of the request (used with
POST and PUT requests).
Content-Type: application/x-www-form-urlencoded
Content-Type: application/json
Content-Type: multipart/form-data
Cookie
An HTTP cookie previously sent by the server with Set-
Cookie.
Cookie: $Version=1; Skin=new;
Host
The domain name of the server (for virtual hosting), and
the TCP port number on which the server is listening. The
port number may be omitted if the port is the standard
port for the service requested.
Host: qa16-ats.mgspdtesting.com
18. Important HTTP Status Codes
Don't reinvent a wheel
Status Code Description
200
Ok
Standard response for successful HTTP requests. The actual response will depend on the request method
used. In a GET request, the response will contain an entity corresponding to the requested resource. In a
POST request, the response will contain an entity describing or containing the result of the action.
302 Found Common way of performing URL redirection.
400 Bad Request The server cannot or will not process the request due to an apparent client error
403 Forbidden
The request was a valid request, but the server is refusing to respond to it. The user might be logged in but
does not have the necessary permissions for the resource.
404
Not Found
The request was a valid request, but the server is refusing to respond to it. The user might be logged in but
does not have the necessary permissions for the resource. Usually means that the resource already didn’t
exist.
501 Internal Server
Error
Indicates that the request could not be processed because of conflict in the request, such as an edit
conflict between multiple simultaneous updates.
19. Open Source Tools for API Testing
• Performance testing tools
• SoapUI (SoapUI NG Pro), Jmeter(SmartMeter, Blazemeter)
• Web Service testing tool
• Postman, DHC, HTTPRequester, Wizdler, WebInject, Http Master, …
• Automation Frameworks
• FitNesse, Cucumber, Robot Framework, …
Don't reinvent a wheel
20. SoapUI
• Open-source web service testing application:
• for service-oriented architectures (SOA)
• for representational state transfers (REST).
• Covers:
• web service inspection, invoking, development, simulation and mocking,
• functional testing,
• load and compliance testing.
• A commercial version, SoapUI Pro (Api Ready), which mainly focuses
on features designed to enhance productivity.
Don't reinvent a wheel
21. SoapUI Open Source– pros and cons
• Pros
• support HTTP(S), REST, SOAP, JDBC, AMF
• IDE -Groovy -> command line, visible SoapUI API
• User forums, tradition
• Easy to learn, easy to use
• Cons
• Team work support
• No recording
• Weak user documentation
• Test history, test debugging, compare results
• Buggy
Don't reinvent a wheel
22. Conclusions
• Web application automated testing is possible through HTTP(S)
• Web application automated testing is effective through REST
• REST API popularity is growing
Don't reinvent a wheel
24. Fiddler
• Captures HTTP and HTTPS traffic and logs it for the user to
review (the latter by implementing man-in-the-middle interception
using self-signed certificates).
• https://www.telerik.com/download/fiddler
Don't reinvent a wheel
25. HTTPS Traffic settings
• Capture - F12 or Click on left bottom corner
• Select captured process by
• Preview - Tab Inspector
• Request – Top part
• Response – Bottom part
Don't reinvent a wheel
27. Fiddler - Inspector
• Request Inspectors
• Headers—Shows request headers and status.
• Webforms – Shows parameters visualized as table from QueryString and Body
• Raw—Shows the “real” request header and body
• Response Inspectors
• Headers—Shows response headers and status.
• SyntaxView—Shows the response body with HTML syntax
• WebView – Shows the browser output
• Raw – Shows “real” response header and body
Don't reinvent a wheel
32. What you get for 500$ in SoapUI NG Pro
• a
• FileImport project (FileCreate Empty Project)
Don't reinvent a wheel
33. Useful links
REST 101: The Beginner's Guide to Using and Testing RESTful APIs
SoapUI - Getting Started with REST Testing
SoapUI - Functional Testing
SoapUI – Scripting & Properties
SoapUI – API Docs
Don't reinvent a wheel
34. Reference List
• Gojko Adzic, David Evans - Fifty Quick Ideas To Improve Your Tests
• Mike Cohn - Succeeding with Agile: Software Development Using
Scrum
• Lisa Crispin, Janet Gregory - Agile Testing: A Practical Guide for Testers
and Agile Teams
• Dorothy Graham, Mark Fewster - Experiences of Test Automation:
Case Studies of Software Test Automation
• Cem Kaner, James Bach, Bret Pettichord - Lessons Learned in Software
Testing: A Context-Driven Approach
Don't reinvent a wheel