Download as PDF, PPTX
Uploaded byCarlos Santana
Amazon EKS multi-cluster gitops-bridge
The document discusses Amazon EKS multi-cluster topologies using a GitOps approach. It describes key challenges with multi-cluster management and proposes using GitOps with the GitOps Bridge pattern to define infrastructure and Kubernetes resources declaratively. This allows enforcing standards across clusters and automating deployments. It provides examples of cluster topologies like standalone, centralized hub-spoke, and discusses managing teams, workloads, addons and configurations through this approach.
In this document
Powered by AI
Presentation by AWS experts on Amazon EKS Multi-Cluster with focus on GitOps.
Outline of key topics including customer builds, challenges, and solutions.
Customers are developing AI/ML technologies, IoT, data analytics, and various applications.
Challenges include automation of deployments, cluster management, team boundaries, and configuration.
Proposes GitOps as a solution to enforce desired state management, simplifying deployments across clusters.
Defines how GitOps states are managed, emphasizing immutability, versioning, and software agents.
Presentation of concepts related to cluster management in Kubernetes.
Describes challenges of enforcement, lifecycle management, and onboarding in multi-cluster setups.
Illustrates the setup of multiple Kubernetes clusters across different environments.
Introduction to EKS Blueprints for managing cluster configurations.
Details on using EKS Blueprints for cluster management and configurations.
Highlights Terraform and CDK for deploying EKS clusters according to AWS best practices.
Discusses challenges associated with deploying clusters in various scenarios.
Outlines issues faced when using IaC tools like Terraform with GitOps principles.
Explains how IaC elements are transformed into a GitOps model for managing Kubernetes.
Further examples of IaC components integrated into GitOps workflows.
How cluster configurations are managed within the GitOps framework.
List of tools and frameworks that help in GitOps management.
Presentation of various cluster topologies relevant to Kubernetes architectures.
Describes a standalone and distributed model for managing clusters across tenants.
Explains centralized and hub-spoke models for cluster management.
Presents a shared resource model as a hub-spoke configuration for managing applications.
Details on a centralized hub-spoke model incorporating agents for enhanced management.
Focuses on team roles in managing applications and standardizing deployment processes.
Explores the need for standardized deployment processes while allowing team autonomy.
Addresses IAM policies and management of namespaces in a multi-team context.
Overview of team structures and repository management for managing application workloads.
Introduction to managing add-ons within the EKS environment.
Details various add-ons focusing on security, observability, reliability, and delivery.
Describes the configuration management of add-ons within a GitOps framework.
Introduction to managing configurations in a multi-cluster architecture.
Discusses strategies for managing configurations between staging and production environments.
Further details on strategies for deploying and scaling applications across stages.
Explores integration of AWS services within a GitOps management framework.
Discusses versioning and management of applications within GitOps using ApplicationSet.
How configuration management is addressed through Git and overrides.
Lists resources and links for further learning about GitOps and EKS implementations.
Conclusion and thanks for attending the presentation.
More Related Content
![[오픈소스컨설팅]인프라 자동화 도구 Chef](https://cdn.slidesharecdn.com/ss_thumbnails/chef-171123014826-thumbnail.jpg?width=640&height=640&fit=bounds)
![[AWS Builders] AWS 네트워크 서비스 소개 및 사용 방법 - 김기현, AWS 솔루션즈 아키텍트](https://cdn.slidesharecdn.com/ss_thumbnails/1network-191112092246-thumbnail.jpg?width=640&height=640&fit=bounds)
![[20220126] JAWS-UG 2022初頭までに葬ったAWSアンチパターン大紹介](https://cdn.slidesharecdn.com/ss_thumbnails/20220126-anti-220126190603-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Amazon EKS multi-cluster gitops-bridge
![Getting Started with DevOps on AWS [Mar 2020]](https://cdn.slidesharecdn.com/ss_thumbnails/gettingstartedwithdevopsonawsmar2020-240904112851-e692c900-thumbnail.jpg?width=640&height=640&fit=bounds)
Amazon EKS multi-cluster gitops-bridge
- 1. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. A M A Z O N E K S M U L T I - C L U S T E R T O P O L O G I E S Amazon EKS Multi-Cluster The Gitops Bridge Pattern Carlos Santana Sr. Kubernetes Solutions Architect AWS CNCF Ambassador Isaac Mosquera Principal Containers Specialist AWS Christina Andonov Sr. Specialist Solutions Architect AWS
- 2. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 1. What are customers building 2. Key challenges 3. Proposed solution 4. How we solved the problem 5. How to use Agenda
- 3. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. What are customers building AI/ML Autonomous Vehicles Robotics Modeling, Training, and Inference Legacy Apps .NET Apps Homegrown Apps Monolith Analytics Streaming MapReduce Batch Web Static Dynamic Backend DB IoT Mobile
- 4. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Enforce security standards and best practices across clusters to automate deployments Define boundaries between multiple teams Provision multiple workloads at scale Cluster management Team management Workload management Install add-ons and their dependencies Add-on management Configuration management Automate configuration and upgrade lifecycle from a single source of truth Key challenges
- 5. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Proposed solution G I T O P S Build Test Scan Operate/Fix Deploy/Verify Observe/Alert Immutability Firewall Git becomes the single source of truth for the system’s desired state, enabling reproducible automated deployments, cluster management, and monitoring.
- 6. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. What is GitOps P R I N C I P L E S A system managed by GitOps must have its desired state expressed declaratively Desired state is stored in a way that enforces immutability, versioning and retains a complete version history Software agents continuously observe actual system state and attempt to apply the desired state Software agents automatically pull the desired state declarations from the source Enforces Consistency Reduces Business Risk Enhances Auditability Boosts Security
- 7. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cluster Management
- 8. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Multi-cluster challenges Enforcing best practices across clusters Consistent cluster lifecycle management Supporting multiple teams with guardrails in place Easily onboarding new applications
- 9. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Multi-Cluster Kubernetes 9 Cloud (prod) Cloud (pre-prod) 1.22 1.23 1.24 1.25 1.26
- 10. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. EKS Blueprints
- 11. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Cluster management EKS Blueprints AWS Cloud Virtual private cloud Internet gateway Public subnet Private subnet Amazon EKS NAT gateway AWS account Instances Instances Instances Role Permissions
- 12. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Infrastructure as Code with Terraform and CDK Based on AWS best practices and recommendations Integrated with popular K8s tools and services Fully extensible and customizable Amazon EKS Blueprints An open-source framework that allows you to configure and deploy complete Amazon EKS clusters across accounts and Regions
- 13. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cluster Deployment
- 14. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Challenges with IaC and GitOps • Terraform § Great for setting up underlying infrastructure § Not design to work with Kubernetes § Imperative, can lead to drift configuration • GitOps § Designed for Kubernetes resources § Declarative and convergence, minimizing drift § Limited scope, focused on app deployment rather than infrastructure
- 15. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. GitOps Bridge: IaC -> GitOps IaC Internet gateway Virtual private cloud (VPC) NAT gateway Private subnet Public subnet AWS KMS key Role Amazon EKS Security group Kubernetes Gatekeeper Karpenter Cert-manager Instances GitOps cluster.yaml
- 16. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. GitOps Bridge: IaC -> GitOps IaC Internet gateway Virtual private cloud (VPC) NAT gateway Private subnet Public subnet AWS KMS key Role Amazon EKS Security group Instances GitOps cluster.yaml Kubernetes Cert-manager Gatekeeper Karpenter cluster config Addons Apps
- 17. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. GitOps Bridge: IaC -> GitOps IaC Internet gateway Virtual private cloud (VPC) NAT gateway Private subnet Public subnet AWS KMS key Role Amazon EKS Security group GitOps cluster.yaml Kubernetes Cert-manager Gatekeeper Karpenter cluster config Addons Apps Instances
- 18. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. GitOps Bridge: IaC -> GitOps GitOps cluster.yaml Kubernetes Cert-manager Gatekeeper Karpenter cluster config Addons Apps Instances ack terraform kops ansible pulumi cdk capa crossplane
- 19. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cluster Topologies
- 20. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Standalone/distributed Namespace Tenant AWS account Amazon EKS Tenant AWS account Amazon EKS Namespace On premises Kubernetes Namespace On premises Kubernetes Namespace Full ArgoCD UI/CLI API Server Redis Server Repo Controllers Addons + Apps
- 21. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Centralized/Hub-Spoke (Push) Central Amazon EKS cluster Central AWS account Namespace Tenant AWS account On premises Amazon EKS Tenant AWS account Kubernetes On premises Kubernetes Amazon EKS Addons + Apps Full ArgoCD UI/CLI API Server Redis Server Repo Controllers
- 22. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Centralized/Hub-Spoke (Shared) Central Amazon EKS cluster Central AWS account Namespace Tenant AWS account On premises Amazon EKS Tenant AWS account Kubernetes On premises Kubernetes Amazon EKS App-2 repo App-1 repo App-4 repo Addons App-3 repo Full ArgoCD UI/CLI API Server Redis Server Repo Controllers
- 23. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Centralized/Hub-Spoke (Agent) Central Amazon EKS cluster Central AWS account Namespace Tenant AWS account On premises Amazon EKS Tenant AWS account Kubernetes On premises Kubernetes Amazon EKS App-2 repo App-1 repo App-4 repo App-3 repo Core ArgoCD UI/CLI API Server Redis Server Repo Controllers https://argo-cd.readthedocs.io/en/stable/operator-manual/core https://akuity.io/blog/reducing-argocd-operational-burden https://github.com/open-cluster-management-io/argocd-pull-integration Full ArgoCD UI/CLI API Server Redis Server Repo Controllers Addons
- 24. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Teams
- 25. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Separation of concerns I would like to standardize the deployment process for application teams while enforcing organizational standards Platform team Development teams I would like to have full control of my application and its dependencies deployment lifecycle
- 26. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Identity & Access Management Policy Management Namespace as a Service Multi-Team Considerations
- 27. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Amazon EKS (Prod) Teams management Frontend Team Backend Team Platform Team Frontend Repository Platform Repository Frontend ArgoCD Projects Platform ArgoCD Projects Backend Repository Backend ArgoCD Projects Amazon EKS (Staging) workloads addons
- 28. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Add-Ons
- 29. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Add-Ons Security Cilium Gatekeeper Kyverno Observability Prometheus Fluent Bit OTEL Reliability Karpenter Autoscaler Keda Delivery ArgoCD Flux Crossplane Others
- 30. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Add-ons management G I T O P S B R I D G E C O N F I G U R A T I O N Control Plane Addons App Of ApplicationSet Addon-1 charts/ environments/ Addon-2 App Of ApplicationSets Addon-1 ApplicationSet Addon-2 ApplicationSet clusters/ Addon-1 Application Addon-2 Application Platform repo Platform team Amazon EKS AWS account OTEL
- 31. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Configuration Management
- 32. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Amazon EKS (Central) AWS account (Central) AWS account (Tenant A) Amazon EKS (Staging) Amazon EKS (Prod) ui v1.0 dynamodb v1.0 ui v1.0 dynamodb v1.0 frontend backend frontend backend apps ui base prod dynamodb/ Apps repo Apps teams staging clusters/ base prod staging Configuration management E N V I R O N M E N T S ( S T A G I N G V S . P R O D )
- 33. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Configuration management D E P L O Y A N D S C A L E Amazon EKS (Central) AWS account (Central) AWS account (Tenant A) Amazon EKS (Staging) ui v2.0 dynamodb v1.0 frontend backend apps ui base prod dynamodb/ Apps repo Apps teams staging clusters/ base prod staging Amazon EKS (Prod) ui v1.0 dynamodb v1.0 frontend backend
- 34. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Configuration management A W S S E R V I C E S W I T H G I T O P S Amazon EKS (Central) AWS account (Central) AWS account (Tenant A) Amazon EKS (Staging) Amazon EKS (Prod) ui v2.0 ui v2.0 frontend frontend Amazon DynamoDB (Staging) apps ui base prod clusters/ Apps repo staging Apps teams Amazon DynamoDB (Production) Control Plane addons aws ack clusters/ Platforms repo environment/ Platform team
- 35. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. GitOps Bridge: ApplicationSet (Addon versions) version in dev is 1.6.0 version in staging is 1.5.5 version in production is 1.5.4 Cluster opt-in for the addon Chart name and repo in a single place Merge generator Prevent Outages
- 36. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. GitOps Bridge: ApplicationSet (overrides) Metadata based on IaC Namespace based on IaC Override values files Value files in git
- 37. © 2023, AmazonWeb Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Resources https://github.com/gitops-bridge-dev https://aws-ia.github.io/terraform-aws-eks-blueprints/patterns/gitops-getting-started-argocd/ https://aws-ia.github.io/terraform-aws-eks-blueprints/patterns/gitops-multi-cluster-hub-spoke-argocd/
- 38. Thank you! © 2023,Amazon Web Services, Inc. or its affiliates. All rights reserved.