Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DevOps Fest 2020. Андрей Шишенко. CI/CD for AWS Lambdas with Serverless framework and Gitlab-CI

31 views

Published on

Доклад рассказывает, как оптимизировать и автоматизировать процесс активной разработки и деплоймента serverless приложений/микросервисов. Сравниваются разные подходы в разворачивании serverless стека на AWS (API GW + Lambda), В процессе презентации я поясню, почему мы выбрали Gitlab-Ci, а не Jenkins ,как CI/CD движок, почему мы выбрали Serverless Framework, а не AWS SAM, как способ разворачивания Serverless приложений. В презентации я покажу, как легко с помощью данного технического решения описывать комплексные задачи, их мониторинг, алертинг, обеспечивать безопасность. Расскажу о проблемах, с которыми мы столкнулись (у нас больше 130 API GWы и 600 Lamdas) за последние 2 года, и какие планы на модернизацию текущего решения.

Published in: Education
  • Be the first to comment

DevOps Fest 2020. Андрей Шишенко. CI/CD for AWS Lambdas with Serverless framework and Gitlab-CI

  1. 1. Continuous Delivery. Continuous DevOps. KYIV, 2020 CONTINUOUS DELIVERY. CONTINUOUS DEVOPS. 5-6,JUNE 2020 KYIV, UKRAINE Efficient CI/CD for AWS Lambda (FaaS) with Serverless framework and Gitlab-CI
  2. 2. Continuous Delivery. Continuous DevOps. KYIV, 2020 About author My name is Andrey Shyshenko. I am DevOps Manager / Team Lead in CreatorIQ (US Product company). I am more than 10 years in IT. I have 4 AWS Certificates (Solution Architect Professional, SysAdmin, Developer, Solution Architect Associate)
  3. 3. Continuous Delivery. Continuous DevOps. KYIV, 2020 Agenda - What is serverless and AWS Lambda (FaaS)? Pros. and Cons. - Hi-level architecture of serverless application - Motivation: full cycle approach for development and delivery - Solutions available on market - Final solution - Effort vs outcome - Responsibilities in a project - Serverless framework, plugins and benefits - Advices from 2 years of hands-on experience - Links - Q&A
  4. 4. Continuous Delivery. Continuous DevOps. KYIV, 2020 FaaS (Function as a Service) pros and cons Pros • No infrastructure • No maintenance and support • Low costs • Easy to start to use • Easy to scale • Availability and fault tolerance built in Cons • Doesn’t suit for long-running operations • Hard to debug • Hard to run and test locally • Hard to trace • Hard to monitor • Hard to control performance and latency • Hard to manage dependencies AWS Lambda, Google Cloud Function, Azure Function
  5. 5. Continuous Delivery. Continuous DevOps. KYIV, 2020 Hi-level architecture of serverless application in AWS Input Processing business logic Output Serverless event-driven application 1. Trigger (input) 2. Processing (Lambda Function) 3. Destination (output)
  6. 6. Continuous Delivery. Continuous DevOps. KYIV, 2020 Full cycle approach for development and delivery Serverless technology (AWS Lambda) is very cool, but: • How to develop on local environment? • How to test? • How to deploy? • How to support git flow (dev, staging, production)? • How to monitor and alert? • How to extend? • How to be compliant with security?
  7. 7. Continuous Delivery. Continuous DevOps. KYIV, 2020 Git Flow
  8. 8. Continuous Delivery. Continuous DevOps. KYIV, 2020 Deployment type We've chosen multi-stack model of deployment. This means that each branch has its own dedicated stack (lambda, api gateway, logs, limits, sqs, etc.)
  9. 9. Continuous Delivery. Continuous DevOps. KYIV, 2020 WorkFlow for CI/CD process (schema)
  10. 10. Continuous Delivery. Continuous DevOps. KYIV, 2020 Solutions available on market Local development: - AWS SAM - Serverless framework - Localstack CI: - Jenkins - Gitlab-CI (Github Actions) - AWS Code Pipeline Deploy: - AWS SAM - Serverless framework - Cloudformation - Terraform
  11. 11. Continuous Delivery. Continuous DevOps. KYIV, 2020 Solutions available on market (Local development) 1. https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-comman d-reference.html 2. https://www.serverless.com/framework/docs/providers/aws/cli-reference/ 3. https://www.serverless.com/plugins/ AWS SAM [1] Serverless framework [2, 3] 1. sam init 2. sam build 3. sam local invoke 4. sam local start-api 5. sam local generate-event 6. sam deploy 7. sam logs 1. sls create 2. sls install 3. sls invoke local 4. sls deploy 5. sls logs 6. sls metrics 7. sls plugin install via plugins [3]: emulate API GateWay, SNS, SQS, DynamoDB, S3, SMM and others
  12. 12. Continuous Delivery. Continuous DevOps. KYIV, 2020 Solutions available on market (CI) Add CI/CD to the Project Jenkins/CodePipeline Flow Add CI/CD to the Project GitLab-CI Flow 1. add jenkins.pipeline / CodeCommit.pipeline to project 2. create job in web-interface 3. set up web-hook for push/tag event in Gitlab 4. set up custom git-hook for ‘delete branch’ in GitLab 5. support the Job if something was changed (GitLab, Jenkins, Plugin, CodeCommit) 1. add .gitlab-ci.yml to project
  13. 13. Continuous Delivery. Continuous DevOps. KYIV, 2020 Solutions available on market (Deploy) Terraform Serverless
  14. 14. Continuous Delivery. Continuous DevOps. KYIV, 2020 Terraform Serverless Summary > 100 lines
  15. 15. Continuous Delivery. Continuous DevOps. KYIV, 2020 Terraform Serverless Summary > 100 lines
  16. 16. Continuous Delivery. Continuous DevOps. KYIV, 2020 Git variables to Environment
  17. 17. Continuous Delivery. Continuous DevOps. KYIV, 2020 GitLab-CI.yaml
  18. 18. Continuous Delivery. Continuous DevOps. KYIV, 2020 Final Solution
  19. 19. Continuous Delivery. Continuous DevOps. KYIV, 2020 Serverless framework, plugins and benefits - Easy create and clean up of all resources - Auto deletion of old versions of Lambdas - Sensitive information can be stored in AWS SMM or Secrets Manager - Auto warm up functionality - Auto-canary release and rollback - Git parameters in environment variables - Write your business logic so that it is separate from your FaaS provider https://www.npmjs.com/package/serverless-lambda-version https://www.npmjs.com/package/serverless-offline https://www.npmjs.com/package/serverless-plugin-git-variables
  20. 20. Continuous Delivery. Continuous DevOps. KYIV, 2020
  21. 21. Continuous Delivery. Continuous DevOps. KYIV, 2020 Responsibilities in a project Developers responsibilities: - Initial version of serverless.yml (runtime, memory, events, plugins, etc.) - Initial CI job .gitlab-ci.yml - Tests (lint, unit, events) DevOps responsibilities: - Verification of CI job .gitlab-ci.yml - Verification of final version of serverless.yml (all plugins, monitoring thresholds, proper IAM roles, additional AWS resources, etc.) - Set of prepared images for Gitlab-runners QA responsibilities: - Tests (functional, integration)
  22. 22. Continuous Delivery. Continuous DevOps. KYIV, 2020 Effort vs outcome We have: - ~ 300 repositories - > 600 Lambdas - > 130 API GW - > 150 SQS queues - only 3 DevOpses and they don’t do Lambdas and CI/CDs for them No blockers from another teams: - Developers can test/deploy without DevOps help - Developer better understand operation of their applications - Developers are responsible for production - Shorter time to market - Real DevOps!
  23. 23. Continuous Delivery. Continuous DevOps. KYIV, 2020 Tips and tricks from 2 years of hands-on experience We have more than 600 Lambdas and 130 API GW - Monitor and Alert: What: • Concurrency • Duration • Throttling • Out of memory • Time out • Errors in Logs • DeadLetter Queue • Costs How: AWS CloudWatch + Alarms / ELK / any other solution Advice: Use different recipient lists for different serverless apps
  24. 24. Continuous Delivery. Continuous DevOps. KYIV, 2020 Tips and tricks from 2 years of hands-on experience - Do not deploy EDGE API GW, use REGIONAL - Use concrete S3 deployment buckets in different regions: to avoid creation of S3 bucket for each deployment use one special bucket in each region - Use tracing (AWS X-Rays or Jaeger) - Don’t store credentials in code or environments - Each Lambda should use only own IAM role with only necessary permissions (only least priveledges) - Use Lambda best practices (CPU/Memory optimization, Effective Duration time, Warming) - Use Serverless Framework Plugins - Run tests and security checks in CI/CD pipelines - Lambda Orchestration with AWS StepFunction
  25. 25. Continuous Delivery. Continuous DevOps. KYIV, 2020 Tips and tricks from 2 years of hands-on experience - Do NOT do MONOREPO (real evil) We tried… 1 repo 9 lambdas 9 Serverless.yml files 370 lines of CI/CD pipeline Hard to publish only one lambda Hard to maintain and extend Hard to find real relations Hard to find root cause of failed job Too long time to build
  26. 26. Continuous Delivery. Continuous DevOps. KYIV, 2020 If you will use Serverless (forecast)
  27. 27. Continuous Delivery. Continuous DevOps. KYIV, 2020 Serverless Infrastructure Providers
  28. 28. Continuous Delivery. Continuous DevOps. KYIV, 2020 Links 1. AWS SAM https://aws.amazon.com/serverless/sam/ 2. Serverless framework https://serverless.com/ 3. Serverless plugins https://github.com/serverless/plugins 4. Gitlab-CI https://docs.gitlab.com/ee/ci/yaml/ 5. Application Tracing AWS X-Rays https://docs.aws.amazon.com/lambda/latest/dg/lambda-x-ray.htm 6. AWS StepFunction https://docs.aws.amazon.com/step-functions/latest/dg/welcome.html 7. Local Stack https://localstack.cloud/ 8. Secrets Management for AWS https://www.serverless.com/blog/aws-secrets-management/
  29. 29. Continuous Delivery. Continuous DevOps. KYIV, 2020 Thank you Q&A

×