Errors happen in every application. In serverless, additional failures exist - timeouts, out of memory, and more. In many cases, errors are handled by the cloud vendor using a retry mechanism. Since the code is stateless, how can you be sure that the application actually works?

1. devopsdays Tel Aviv
Error Handling in
Stateless Environments
Nitzan Shapira, Epsagon

2. Nitzan Shapira (@nitzanshapira)
Software engineer > 12 years
Co-Founder, CEO at Epsagon
Tel Aviv
2
> whoami

3. What is serverless? How is it different?
What is stateless? Why is it relevant today?
How to handle errors in such environments?
How will it help my job?
3
Things to discuss

4. 4
[Compute-as-a-Service]
FaaS: Function-as-a-Service
CaaS: Container-as-a-Service
+
Managed services (APIs)
=
Don’t manage infrastructure
Focus on business logic
What is serverless?

5. 5
Why serverless?
Pay-per-use: reduces cloud compute cost by 90%
Out-of-the-box auto-scaling
DevOps à LowOps
++Developer velocity
Focus on business logic – iterate faster
Server Utilization

6. 6
The limitations of FaaS
Limited memory Limited running time
Cold startsStateless
+ concurrency limit
+ some others…

7. 7
The properties of serverless applications
Serverless is micro-services
Serverless applications are
- Highly distributed
- Highly event-driven
Utilizing managed services via APIs is key

8. A real example – HSBC
8
Source: re:Invent 2018

9. 9
The challenge in serverless
SIMPLE COMPLEX
Yan Cui

10. 10
What the community thinks
2018 Serverless Community Survey, serverless.com, July 2018
2017 results

11. 11
Stateless system properties
Event-drive design
No server to connect to – difficult to troubleshoot
No current state – difficult to determine system health

12. 12
Error handling instateless systems

13. 13
Types of failures in serverless
Unhandled exception
Timeout
Out-of-memory

14. 14
Retries behavior
Synchronous events
• The invoking application is in charge of the error
Asynchronous events
• A retry mechanism is triggered for a certain period of time
Stream-based events
• A retry mechanism is triggered until the data is expired

15. 15
Retry behavior consequences
Retry might change the logical flow of the application
In order for retries to succeed, the code must be idempotent:
“Idempotence is the property of certain operations in
mathematics and computer science that they can be
applied multiple times without changing the result
beyond the initial application” (Wikipedia).

16. 16
Practical methods for retries
Write idempotent code
Tough!
Use a proper service
Example: AWS Step Functions

17. 17
AWS Step Functions

18. 18
Deploying using Serverless Framework

19. 19
Observability – why do we need it?
Track system health Troubleshoot and fix Optimize performance and cost

20. 20
Observability in serverless and stateless systems
Let’s go one by one

21. 21
Track system health
System == Functions ?

22. 22
Functions are important
- Errors
- Timeout
- Out-of-memory
- Cold start

23. 23
Track system health
System > Functions !Serverless != Functions

24. 24
Track system health
System > Functions !
Functions
APIs
Transactions

25. 25
Troubleshoot and fix
Functions are not enough
Need: track asynchronous events
e

26. 26
Transactions

27. 27
Tracing asynchronous invocations

28. 28
Tracing asynchronous invocations

29. 29
Tracing asynchronous invocations

30. 30
Distributed tracing
…a trace tells the story of a transaction or
workflow as it propagates through a
(potentially distributed) system. Distributed
tracing is a method used to profile and
monitor applications.

31. 31
Distributed tracing
Jaeger

32. 32
Implementing distributed tracing
Manual tracing/instrumentation
Before/after calls
At the end of each micro-service
High maintenance
High potential of errors

33. 33
Serverless apps are very distributed
Complex systems have thousands of functions
What about the developer velocity?

34. 34
Can it be done differently in serverless?

35. 35
Automation can help to keep up with the
development speed of serverless

36. 36
Example

37. 37
Example

38. 38
Monitoring serverless
Limited memory Limited running time
Cold startsStateless

39. 46
Business flows
Subscribe
Transfer Payment

40. 47
What should I optimize first?

41. nitzan@epsagon.com
@nitzanshapira
www.epsagon.com
Thank you!