Presented at All Things Open 2023 Presented by Carlos Santana - AWS Title: Securing Cloud Resources Deployed with Control Planes on Kubernetes using Governance and Policy as Code Abstract: Are you concerned about the security of your cloud resources deployed on Kubernetes? Are you struggling to ensure compliance with regulatory requirements while managing your cloud infrastructure? If yes, then this talk is for you! We will discuss how to secure cloud resources deployed with Crossplane on Kubernetes using Governance and Policy as Code. We will explore how to leverage Governance and Policy as Code tools like Rego, Kyverno, and OPA to ensure security and compliance. By the end of this talk, you will have a better understanding of the challenges associated with securing cloud resources deployed with Crossplane or ACK on Kubernetes, the importance of Governance and Policy as Code in ensuring security and compliance, and why it is critical to use open source and open standards in these technologies. Find more info about All Things Open: On the web: https://www.allthingsopen.org/ Twitter: https://twitter.com/AllThingsOpen LinkedIn: https://www.linkedin.com/company/all-things-open/ Instagram: https://www.instagram.com/allthingsopen/ Facebook: https://www.facebook.com/AllThingsOpen Mastodon: https://mastodon.social/@allthingsopen Threads: https://www.threads.net/@allthingsopen 2023 conference: https://2023.allthingsopen.org/

1. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Control Planes on Kubernetes and
Policy Validation
Carlos Santana
Sr. Specialist Solutions Architect, Kubernetes
@csantanapr
csantanapr

2. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EKS is the most trusted and secure way to run Kubernetes
Amazon EKS
AWS is the best place to run Kubernetes. 65% of organizations
choose AWS to run their containers.
EKS makes Kubernetes operations, administration, and
management simple and boring.
EKS provides a managed Kubernetes experience for performant,
reliable, and secure Kubernetes.
EKS supports 5 versions of Kubernetes, giving customers time to
test and roll out upgrades.
EKS runs vanilla Kubernetes. EKS is upstream and certified
conformant version of Kubernetes (with backported security fixes)
CNCF State of Cloud Native Development

3. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deploy Kubernetes Applications
Developer
CI/CD
Git
Kubernetes
Application

4. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deploy Kubernetes Applications
Developer
CI/CD
Git
Kubernetes
Application
Amazon S3
Amazon SQS
Amazon DynamoDB

5. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deploy Kubernetes Applications
Developer
CI/CD
Git
Kubernetes
Application
Amazon S3
Amazon SQS
Amazon DynamoDB
Platform
Team

6. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Separation of Concerns
I would like to standardize the
deployment process for application
teams while enforcing
organizational standards for cloud.
Platform Team Application Teams
I would like to have full
autonomy of applications
and its cloud dependencies
deployment lifecycle.

7. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Autonomy vs Standardization
A
U
T
O
N
O
M
Y
LEVEL OF STANDARDIZATION

8. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Disjointed Workflows Adds Developer Overhead
How do you get the state of
the entire system?
How do you debug and
remediate issues?
How do you pass secrets
from one pipeline passed to
the other?
How do you tear-down all
unused resources?
Developers Request
Database
Repo
Ticketing
System
Central
Team
Code Review
Develop
Develop
RDS
Kinesis
Load Balancer
EBS
AWS CDK
Platform
Team
AWS
CloudFormation

9. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
9
Kubernetes to Deploy Cloud Services
Developers
Repo
Code Review
RDS
Kinesis
Load
Balancer
EBS

10. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Leverage existing workflows for deploying into
Kubernetes without maintaining several pipelines
Centralized cost, security, and auditing for any
cloud resource with OPA or Kyverno, utilize
existing RBAC and IRSA
Self healing using GitOps for all cloud resources
(containers, databases, queues, and streams)

11. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubernetes Infrastructure Controller
Developer
CI/CD
Git
EKS Cluster
Application
Ticketing
System
Platform
Team
AWS CDK
Amazon S3
Amazon SQS
Amazon DynamoDB

12. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubernetes Infrastructure Controller
Git
Developer
CI/CD
Kubernetes
Application
AWS
Resource
Claim
ACK
Crossplane
Amazon S3
Amazon SQS
Amazon DynamoDB

13. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Infrastructure controllers
Manage cloud services using Kubernetes API
Create your own platform API
A single API for Kubernetes and cloud services
Declarative infrastructure configuration
ACK
Crossplane

14. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Where to check?
Developer
Amazon S3
Amazon SQS
Amazon DynamoDB
Git
Pre
Commit
CI
(Pull Request)
CRD
Validation
Validating
Admission
Policy
Kube
API
Mutating
Webhook
Validating
Webhook
Dynamic
Admission
Control
ACK
Crossplane
OPA
Gatekeeper
Kyverno ETCD
CD
(GitOps)
Since 1.26,
1.28 (beta)
Gator cli
Conftest cli
Kyverno cli
KubeConform

15. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Local Testing
• Rego
§ opa
§ gator
§ conftest
• Kyverno
• Kubeconform
Developer Git
Pre
Commit
CI
(Pull Request)

16. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubernetes CRD
• CRD Validation Expression
• Common Expression Language (CEL)
• 1.25 Beta
• 1.29 GA (~12/23) KEP 2876
CD
(GitOps)
CRD
Validation
Kube
API

17. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubernetes CRD

18. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Validating Admission
• Validating Admission Policy
• Common Expression Language (CEL)
• 1.26 Beta
• 1.29 GA (~12/23) KEP 3488
Validating
Admission
Policy
Kube
API

19. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Validating Admission

20. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Policy Engines
• OPA/Gatekeeper
§ Webhooks, Controller
§ AWS Config custom
rules
• Kyverno
§ Webhooks, Controller
§ Adapter for EKS
Validating
Admission
Policy
Mutating
Webhook
Validating
Webhook
Dynamic
Admission
Control
Kube
API
ETCD
OPA
Gatekeeper
Kyverno
AWS Config
Amazon EKS

21. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubernetes Infrastructure Controller
AWS Account
EKS Cluster
Developer
Platform
Team
code
Deploy
CI CD
Helm Chart
Prometheus Grafana
OPA
Policy Cost Control
OTEL FluentBit
Observability
Auth
Abstraction
DB CR
Amazon DynamoDB Amazon DynamoDB
Crossplane
values.yaml
dynamodb:
enable: true
ACK
Kyverno

22. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank You
carrlos@amazon.com
Carlos Santana
@csantanapr
csantanapr
GitHub Repo
Demo and Resources