Successfully reported this slideshow.
Your SlideShare is downloading. ×

Lessons learnt in CI/CD with AWS serverless architecture

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 17 Ad

Lessons learnt in CI/CD with AWS serverless architecture

Download to read offline

Talk by Sriguru V, Engineer, Altran Engineering Solutions on the topic "Lessons learnt in CI/CD with AWS serverless architecture" at AWS Community Day, Bangalore 2018

Talk by Sriguru V, Engineer, Altran Engineering Solutions on the topic "Lessons learnt in CI/CD with AWS serverless architecture" at AWS Community Day, Bangalore 2018

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to Lessons learnt in CI/CD with AWS serverless architecture (20)

Advertisement

More from AWS User Group Bengaluru (20)

Advertisement

Recently uploaded (20)

Lessons learnt in CI/CD with AWS serverless architecture

  1. 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. BENGALURU
  2. 2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lessons Learnt in CI/CD with AWS Serverless Architecture Sriguru V | 06-Oct-2018
  3. 3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda  Background  Needs  How we started ?  Challenges  Lessons Learnt  Key Takeaways
  4. 4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Background AngularJS Front-End Application AWS Serverless Architecture Connected Products
  5. 5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Needs identified Developer • Speed in deployments • Integrated testing of Unit Tests, E2E Tests, API Tests • Minimal time of Build, Test and deployment process • Replicable infrastructure for pipelines • Different environments
  6. 6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Developers How we started ? Developer 1 2 3 Push/Pull/Commit Jenkins Server Build triggered through Web Hooks Publish to S3 on Successful Build
  7. 7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Challenges with jenkins adoption Developer • Cost • Operation issues • Dependencies on the Plug - ins • Unnecessary heaving lifting • Maintenance of security of the instance
  8. 8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Different Environments Developer Development QA Pre Production Production Build Test Release Build Test Release Build Test Release Build Test Release Our Customer Verified / Validated by Dev. Team Verified / Validated by QA. Team Verified / Validated by Customer Plan Monitor Development Team
  9. 9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Developers Jenkins replaced with AWS Services Developer 1 2 Push/Pull/Commit Build triggered through Web Hooks CodeBuild Deployment Cache Invalidation 3
  10. 10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Tools adopted Developer SCM Unit Test Build E2E Deploy& Invalidation
  11. 11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Services – Post adoption challenges Developer • Cross account issues in the pipelines • Adoption appropriate build tools • Customized docker container in Code Build for manage End to End Tests • Integration with Agile tools • Email and Slack Notifications • Bloated containers to be skimmed for only necessary libs
  12. 12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Cross Account Handling, Pipeline Steps Developer buildspec_ut.yml buildspec_e2e.yml Unit Testing E2E Testing S3 Bucket Invalidate Halt Operation QMetry JUnit Reporter CI CD Jira Tickets Developer Account Production Account Personal Access Token 1 2 3 4 5
  13. 13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.  SQS and SES Integration  CloudWatch Rule  Cloud Formation Template – Infrastructure as a Code Managing the pipeline
  14. 14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Risks and Mitigation Plan Version Issues in npm Bower to yarn migration E2E Integration
  15. 15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Best Practices Checklists adoption Unit Testing of scripts Keep track of Domains and Route53 Continuous monitoring of Security Advisor Update the CLI commands on Version upgrades in Code Build Frequently review budget through cost explorer
  16. 16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  17. 17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Editor's Notes

  • This is the overview of the solution context we are going to discuss on today
    Solution has been developed for a North American OEM who develops High end products for Education, Government and Corporate customers
    We have a front end application which helps the end users of multiple tenants and OEM to work with connected products
    We have adopted serverless approach and this session is going to talk about the various lessons learnt in the implementation of this solution
    Development was done using agile practices
  • Speed in deployments
    Integrated testing of Unit Tests, E2E Tests, API Tests
    Minimal time of Build, Test and deployment process
    Ability to recreate the pipeline infrastructure
  • We first setup of dev environment with Dev Branch
    Adopted Jenkins with plugins for managing the Build and deploy jobs
    Basically using the Web hooks to trigger the unit tests and then build process and subsequently
    Upon successful build deploy to S3 Bucket
  • We had some security threats and cost impact of the running instances
    Customer is interested in going more serverless to understand granularity of cost by leveraging AWS services
    Security of EC2 instances
    Some of the team members has to maintain the EC2 instance for the Jenkins jobs which also to be monitored and managed
    Upgrade plug-ins, etc.,
  • After identify some of the challenges as outlined in the previous slide on Jenkins we restored to use AWS services
    We were getting into the need for different environments for different stakeholders
    Dev - Dev Team
    QA – testing Team
    Pre Production – UAT/Business Team
    Production – Customers
    This also defined our approach on code deploy and manual handoffs required for approval

  • We have replaced Jenkins with AWS services
    The docker containers will be running in the code pipeline
    Separate credentials with IAM polices for dealing with Code pipeline execution
    Code Build Services would be adopted for Build & Tests, Deployment and Cache Invalidation to complete the deployment
    This would get replicated for different environments

  • Serverless code, API Test Scripts, Unit Test Scripts, E2E Test Scripts, Database migration scripts are stored in the GitHub repository
    Test libraries and its dependencies (Example: npm) are managed to be more lightweight through custom built docker containers
    Used tools like Gulp/Yarn for managing dependencies and deploy:
    Obfuscation
    Cache Invalidation (CloudFront)
    Deployment Caching



  • Code build process should handle obfuscation, minification of the front end code
    Headless chrome adoption for running the End to Tests
    Integration of test results for Slack Notifications
    Integrate the Qmetry test results with AWS Pipeline through S3 Buckets
    Managing the pipelines for managing cross account issues between dev and production

×