Introduction to AWS Testing

1. Testing of Serverless Applications
on Amazon Web Services Cloud
Rustam Zeinalov
November 2020

2. ABOUT.ME
QA tech lead with 10+ years of experience in various testing domains
Expertise in test automation of back-end services and web applications
Avid open-source supporter and contributor

3. A Little Bit of History
Let’s look how application architecture paradigms evolved in the past
1989–1991 —World Wide Web invented by Sir Tim Berners-Lee
1991–1995 — The bare metal age
1995 — The invention of web hosting
1999 — Software as a Service (SaaS) concept was introduced by
Salesforce
2001 — VMWare releases ESXi, “server virtualization” becomes a thing
2002–2006 — AWS is born (IaaS), people started to talk about “Cloud
computing”
2009 — Heroku and the invention of the “Platform as a Service”
(PaaS)
2011 — Envolve/Firebase, real-time database as a service
2012 — Parse.com and the first Backend as a Service (BaaS)
2013 — Docker, “containers are better than virtual machines”
2013–2015 — Kubernetes / Swarm / Nomad / CoreOs (containers at
scale)
2014 — Launch of AWS Lambda (FaaS/Serverless)
(Reference — http://www.computerweekly.com/feature/A-history-of-cloud-computing)

4. Comparison of historical paradigms
On-premise - software that’s installed on servers
managed by organizations themselves
IaaS - cloud-based services, pay-as-you-go for services
such as storage, networking, and virtualization
(DigitalOcean, Linode, Amazon Web Services (AWS),
Microsoft Azure, Google Compute Engine (GCE)
PaaS - hardware and software tools available over the
internet (Heroku, Force.com, Google App Engine, Apache
Stratos, OpenShift)
SaaS - software that’s available via a third-party over the
internet (e.g Dropbox, Salesforce, Cisco WebEx, Concur,
Office365)

5. What Is Serverless
As per MartinFowler.com, Serverless architectures refer
to applications that significantly depend on third-party
services (knows as Backend as a Service or “BaaS”) or on
custom code that’s run-in ephemeral containers (Function
as a Service or “FaaS”)
In simple terms -if you are not maintaining or managing
your own infrastructure to run your application and you
pay as per your usage (never pay for idle), while getting
required level of high availability, scalability and fault
tolerance automatically from your vendor, from your point
of view you are running a serverless application.
As an owner of an application which runs on a serverless
environment, you can put your 100% focus on your
application business logic without worrying about the
infrastructure that it runs on, and other non-functional
requirements around your application such as high
availability, fault tolerance, and scalability.

6. AWS Services
Amazon Web Service is a platform that offers
flexible, reliable, scalable, easy-to-use and cost-
effective cloud computing solutions

7. Key Components for Serverless Applications
AWS Lambda is an event-driven, serverless computing platform provided by Amazon as a part
of Amazon Web Services. It is a computing service that runs code in response to events and
automatically manages the computing resources required by that code.
AWS Fargate is a compute engine for Amazon ECS and helps in running containers without
managing servers or clusters.
AWS S3 helps in storing almost any type of data object or flat files in the cloud
AWS Step Functions is a serverless orchestration service that allows combination of various
AWS services/components into executable workflows(state machine)
Amazon CloudWatch is a monitoring and management service that provides data and actionable
insights for AWS applications and infrastructure resources
Amazon EventBridge is a serverless event bus.

8. Testing Levels
Unit tests[1]:
In case of serverless applications role of unit test diminishes, as the bulk of
complexities is usually clustered around how a function interacts with
external services. However, if you have a piece of complex business logic,
you should encapsulate it into its own module so that you can test it as a
unit. You can use the same testing frameworks that you are familiar with
already, such as Junit. For mocking AWS services you may use Localstack
Integration tests:
These tests help with catching errors in the way your code interacts with
external services. The test should be run against real AWS services, not
mocks!
System/E2E tests:
These tests are run for verification of the whole system’s e2e flow and
help to catch issues like permissions and other configuration errors that
will likely be missed by unit and integration tests.
Acceptance tests
They run against a deployed serverless application and test the happy and
failure paths that the code is expected to produce. The acceptance tests
provide immediate feedback on the state of the deployed application.
They are a must-have if you’re aiming for Continuous Deployment or
Delivery.
[1] https://epsagon.com/development/how-to-test-serverless-apps/

9. AWS Console
The AWS Management Console is a browser-
based GUI for Amazon Web Services (AWS).
Primary goal of console – help with finding
new AWS services, configure them, view service
usage and etc.
AWS Management Console provides an inbuilt
user interface to perform AWS tasks like working
with Amazon S3 buckets, launching and
connecting to Amazon EC2 instances, setting
Amazon CloudWatch alarms, etc.
At the same time console could be used to
test components such as lambda functions, API
gateways, step Functions by specifying custom
input data and running component with this test
data

10. AWS SDK
AWS SDK simplifies use of AWS Services by
providing a set of libraries that are consistent
and familiar for developers.
Following languages supported - Java, .NET,
JavaScript & NodeJS, Python, C++, Ruby, Go,
PHP
It’s possible to do CRUD operations on S3
buckets, call Lambda functions and execute
step Functions with parametrized inputs with
AWS SDK
It’s possible to integrate AWS SDK into test
automation frameworks and create wrappers
util classes for various service clients (i.e.
S3Manager, LambdaManager,
StepFunctionManager and etc.)

11. AWS Toolkit plugin for JetBrains IDEs
The AWS Toolkit for JetBrains is an open
source plugin for the integrated development
environments (IDEs) from JetBrains - IntelliJ
IDEA, WebStorm, PyCharm etc.
The toolkit makes it easier for developers to
develop, debug, and deploy serverless
applications that use Amazon Web Services
(AWS), but in could also be used for testing
purposes

12. Testing Challenges
 Lack of established and mature test
frameworks and solutions for automated
integration and acceptance testing
 Limited capabilities of AWS Console testing
tools (for manual testing)
 Limited possibilities of mocking (Localstack
solution for simulation AWS has certain
limitations)
 Access to containers running on
cluster(Fargate/ECS)
 Defects root cause establishment – big
number of loosely connected components
complicates finding of root cause and
debugging
 AWS security model (IAM roles, Key
Management Service)
 AWS services execution history and logs
retention policies

13. LIVE DEMO

14. Q&A