BuildStuff 2019 Kiev, Ukraine

1. Lessons learned while going Serverless
in production
Mohamed Labouardy - Foxintelligence

2. Mohamed Labouardy
Lead DevOps Engineer/Software Engineer
Founder of Komiser - https://komiser.io
Open Source contributor: Docker, Jenkins, Telegraf, etc
DevSecOps Evangelist
@mlabouardy
https://labouardy.com

3. Detect Potential Cost Savings on Public Cloud
+2K stars +185K downloads
“Stay under budget by uncovering hidden
costs, monitoring increases in spend, and
making impactful changes based on custom
recommendations.”
https://komiser.io

4. How we do it ?
Millions of real online
transaction emails
anonymized and
processed in real time
from a panel of 2.5M
consumers
DATA ACQUISITION &
COLLECTION
Billions of datapoints
processed, cleaned and
enriched through
machine learning,
scrapping, and blending
DATA PROCESSING
& ENRICHMENT
Business acumen and
analytical skills combined
to provide
game-changing insights
to our clients on an
intuitive platform
DATA ANALYSIS
& VISUALIZATION

6. Platform as Service - PaaS

8. Foxintelligence Platform Statistics

9. Foxintelligence Platform Statistics

10. CI/CD for Dockerized Microservices

11. Serverless Hidden Costs

12. Serverless Functions & Dockerized Microservices
10%
Tiny part of our architecture

14. Foxintelligence Platform Statistics
AWS Lambda AWS EKS API Gateway ElasticSearch
+2B Invocations +5K Containers +80M Requests +5TB of Logs

16. Single Page Applications

17. Monoliths vs Nanoservice
GET
POST
DELETE
GET
POST
DELETE

18. Testing
-
● SAM Local for testing locally
● Sandbox account for post-integrations tests
○ Best way to test is on the AWS infrastructure itself
● A/B testing with alias and weighted versions
● Performance testing for memory allocation
○ Apache benchmark
● Quality, unit & security tests included on CI/CD
AWS Lambda Power Tuning

19. Deployment Framework
● CloudFormation extension optimized for Serverless
● Supports anything CloudFormation supports
● Infrastructure as Code

20. CI/CD for Lambda based Applications

21. Gitflow Branching
master
preprod
develop
merge
feature/A
feature/B
merge
merge
merge
v1.0.0
hotfix
merge
v1.0.1
merge
Production
=
Staging
=
Sandbox
=

22. CI/CD for Lambda based Applications

23. CI/CD for Lambda based Applications
Jenkinsfile
Dockerfile
Audit dependencies to keep bundle sizes small

24. CI/CD for Lambda based Applications

25. Debugging & Troubleshooting

26. Near Real Time Monitoring

27. Security
● Enable traceability
● Use most-restrictive permissions when setting IAM policies
● Protect secrets in transit and at rest
○ SSM Parameter Store, KMS
○ Vault
● Apply security at all layers
○ Code reviews, versioning
● Inject security and compliance into CI/CD
○ Dependency checks CVE/CVSE
○ Static/Dynamic analysis

28. Cold Start vs Warm Start
● Factors which increase the cold-start time:
○ Runtime environment
○ Memory size
○ Bundle size
○ VPC
● Solutions:
○ Programming Language
○ Warm-up
○ AWS Hyperplane

29. Cold Start vs Warm Start

30. Caching and Reuse
Best Practices Connecting from AWS Lambda - MongoDB Atlas
context.callbackWaitsForEmptyEventLoop=false

31. Serverless Hidden Costs
“Memory allocation can make a big difference in your
Lambda function cost. Too much allocated memory
and you’ll overpay. Too little and your function will be
at risk of failing. Therefore, you want to keep a healthy
balance when it comes to memory allocation.”

32. Cloudwatch Logs eating up money

33. Key Takeaways
● Serverless or Containers? Why not both
● Be aware of Serverless hidden costs
● Start simple and iterate
● Adopt agile mindset
● Follow AWS best practices
● Share what you learn

34. @mlabouardy
Follow me on Medium - @mlabouardy