Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Containers and mission-critical applications - SEP309-R - AWS re:Inforce 2019

207 views

Published on

Vanguard is running mission-critical applications on AWS Fargate that require enhanced security controls. In this session, we show you how Vanguard is using Amazon ECS, AWS Fargate, and Application Load Balancer to run its Docker-based microservices.

  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Containers and mission-critical applications - SEP309-R - AWS re:Inforce 2019

  1. 1. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers and mission-critical applications Yonatan Ryabinski Chief Enterprise Architect Vanguard S E P 3 0 9 - R 1
  2. 2. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Vanguard background Began operations on May 1, 1975, in Valley Forge, PA One of the world's largest investment companies Wall St.
  3. 3. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda Containers journey at Vanguard Reference architecture Data protection Identity and access management Infrastructure protection Reliability Cost optimization
  4. 4. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Design principles • Fully distributed architecture • Enabled DevSecOps patterns • Backward-compatiblility with legacy container orchestration platform • Easy migration path from legacy container orchestration platform • Full automation • Security by default • Cost-effectiveness
  5. 5. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon ElasticContainer Service (Amazon ECS) — AWS Fargate stack Amazon Elastic Container Registry (Amazon ECR)Amazon ECS AWS Fargate Amazon DynamoDB Application Load Balancer (ALB) AWS Certificate Manager (ACM) VPC Availability Zone 1 Amazon ECS (task) Amazon ECS (service) Availability Zone 2 Amazon ECS (task) Endpoint Endpoint AWS Auto Scaling
  6. 6. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Fargate security • Taking advantage of the shared responsibility model • Full tenant isolation at the hypervisor level via Firecracker • An ability to use a task role for native AWS integration • No operational overhead, no hosts to patch or maintain • Compliance with PCI DSS Level 1, ISO 9001, ISO 27001, ISO 27017, ISO 27018, SOC 1, SOC 2, SOC 3, and HIPAA
  7. 7. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
  8. 8. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Data protection • Sensitive environment variables encryption • End-to-end encryption in motion • Certificates that are issued and rotated with ACM • Data access via VPC endpoints only
  9. 9. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Sensitive environment variables encryption Pipeline agents Amazon ECS (task) AWS KMS "Action": [ "kms:Decrypt", "kms:Encrypt" ], "Effect": "Allow", "Resource": [ "Key1", "Key2" ] "Action": [ "kms:Decrypt" ], "Effect": "Allow", "Resource": [ "Key2" ]
  10. 10. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. env_variable1_key_value = template.add_parameter(Parameter( "EnvVar1KeyVal", # regex that allows | separated key value as a single string AllowedPattern = "(([A-Za-z0-9+.-/_]+|[A-Za-z0-9+.-/_:,]+)|)", # Description = "Env Var 1 Key Value", Type = "String", NoEcho = True, )) Passing the environment variables via CFT
  11. 11. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. decrypt_env_protected_variable1_key_value = template.add_resource(vg_encrypt_string_py_cr( "DecryptEnvProtectedVar1KeyVal", # using cloudformation Split function to extract the encrypted value EncryptedText=Select(1, Split("|", Ref(env_protected_variable1_key_value)) # ServiceToken=Join("", ["arn:aws:lambda:", Ref("AWS::Region"), ":", Ref("AWS::AccountId"), ":function:vgEncryptStringPyCR"]), )) Securing the environment variables: Decrypt
  12. 12. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. encrypt_env_protected_variable1_key_value = template.add_resource(vg_encrypt_string_py_cr( "EncryptEnvProtectedVar1KeyVal", # get the decrypted value ClearText=GetAtt("DecryptEnvProtectedVar1KeyVal", "Decrypted") # ServiceToken=Join("", ["arn:aws:lambda:", Ref("AWS::Region"), ":", Ref("AWS::AccountId"), ":function:vgEncryptStringPyCR"]), KeyId=Ref(ecKmsKey) )) Securing the environment variables: Re-encrypt
  13. 13. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. If("HasEnvProtectedVar1KeyVal", ecs.Environment( # parsing the | separated key value and pass to ECS via environment variable Name = Select(0, Split("|", Ref(env_protected_variable1_key_value))), Value = GetAtt(encrypt_env_protected_variable1_key_value, "Encrypted") # ), Ref("AWS::NoValue")), Parsing the environment variables in CFT
  14. 14. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. acm_custom = template.add_resource(AcmDomainValidator( "AcmCertValidCustom", # pass all the SANs to the custom resource that issues and validates the certificate SANs=If("HasApplicationName", Split(",", Join(",", [ Join("", [ Ref(application_name), "-pre-", Ref(user_location), ".ecs.", If("HasRegionalDnsName", Join("", [Ref("AWS::Region"), "."]), Ref("AWS::NoValue")), Ref(hosted_zone_domain_name) ]), # ServiceToken = Join(":", [ "arn:aws:lambda", Ref("AWS::Region"), Ref("AWS::AccountId"), "function:AcmDomainValidator" ] ), HostedZoneId=GetAtt("Route53ZoneInfoPub", "ZoneId"), DomainName=Join(".", [ Join("-", [Ref("AWS::Region"),Ref(hosted_zone_domain_name) ]))) Requesting and validating a certificate with ACM
  15. 15. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. application_lb_listener = template.add_resource(elasticloadbalancingv2.Listener( "ApplicationLbListener", # attaching the certificate ARN to the listener of ALB Certificates = [ elasticloadbalancingv2.Certificate( CertificateArn = If("HasNoCertificateId", GetAtt("AcmCertValidCustom", "CertificateArn"), Join("", ["arn:aws:acm:", Ref("AWS::Region"), ":", Ref("AWS::AccountId"), ":certificate/", Ref(certificate_id)]) ) ], # SslPolicy = If("IsAlb", Ref(alb_ssl_policy), Ref("AWS::NoValue")), LoadBalancerArn = Ref(application_lb), ... Issuing a certificate with ACM
  16. 16. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Task roles • Similar to an instance profile for Amazon EC2 • Narrow the scope of the role to an individual task • Tighten the application-specific access
  17. 17. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. application_ecs_task = template.add_resource(ecs.TaskDefinition( "ApplicationEcsTask", # task role parameter in task definition to attach to a task TaskRoleArn = If("HasTaskRole", Join("", [ "arn:aws:iam::", Ref("AWS::AccountId"), ":role/", Ref(task_role_name) ]), GetAtt('EcsTaskRole', 'Arn')), # ContainerDefinitions = [ecs.ContainerDefinition( ... )], ExecutionRoleArn = Join("", [ "arn:aws:iam::", Ref("AWS::AccountId"), ":role/ecsTaskExecutionRole" ]), )) Attaching a role to a task
  18. 18. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
  19. 19. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Infrastructure protection • Scan the Docker images • Protect the application at runtime • Make sure that only tasks with runtime protection can run • Alert when anomalies are detected • Prevent access in compromised scenario
  20. 20. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Aqua stack in Amazon ECS – AWS Fargate Amazon ECS AWS Fargate VPC Application zone 1 ... N Aqua gatewayAmazon ECS (service) Aqua console AWS Auto Scaling Aqua scanners Postgre Amazon ECR Endpoint Network Load Balancer (NLB) ALB SSH Microenforcer
  21. 21. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Scan the Docker images Scan the images in Amazon ECR 1. Aqua scanner is using role-based access to scan Docker images in Amazon ECR 2. Scanned images are tagged approved or unapproved based on policies 3. Images are periodically rescanned for the latest signatures
  22. 22. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. FROM vdocker.artifactory.opst.c1.vanguard.com/ecs-python:python ... # Run in microenforcer # microenforcer only allows to start when span from an Approved image ENTRYPOINT [ "/microenforcer", "/start.sh" ] Protect application at runtime
  23. 23. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Alert and block when anomalies are detected Block a malicious action 1. The microenforcer runs inside a container 2. It blocks all malicious or unauthorized code invocations 3. SOC is alerted when a malicious invocation attempt is made 4. Even compromised containers are protected
  24. 24. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
  25. 25. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Reliability • Amazon ECS keeps the needed number of tasks running • A target group of an ALB performs a health check against each task • An automatic scaling service increases the number of wanted tasks based on Amazon CloudWatch metrics
  26. 26. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Wanted number of tasks running Perform self-healing 1. Amazon ECS service evaluates the number of tasks running against the number of tasks wanted 2. If the number of tasks running is less than the number of tasks wanted, the service launches a new task
  27. 27. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Mapping between Amazon ECS and target group Amazon ECS service and load balancer 1. A target group is defined in Amazon ECS service 2. Load balancer service informs Amazon ECS service when health checks fail 3. Amazon ECS service restarts the failed task
  28. 28. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Health check is performed by the target group Amazon ECS service and load balancer 1. Load balancer service performs health checks based on the parameters specified 2. If health checks have failed, load balancer informs Amazon ECS service that a task failed
  29. 29. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon ECS automatic scaling Amazon ECS automatic scaling Using Amazon CloudWatch alarms, Amazon ECS increases or decreases the number of wanted tasks in a given service We are using the CPUUtilization metric; other metrics can be used also
  30. 30. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
  31. 31. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Cost optimization and next steps • Finding an alternative to ALB will reduce the cost • Sizing the task’s memory and CPU will reduce the cost • Fine-tuning automatic scaling will reduce the cost • Smaller Docker images will reduce the cost • Experiment with AWS Cloud Map to see whether ALB can be retired • Experiment with AWS App Mesh for a better service-to-service call security model • Experiment with Amazon ECS health check using Docker API v1.35 for more aggressive self-healing and automatic scaling
  32. 32. Thank you! © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.

×