slides for kubernetes london meetup 20180412 on crd vs AA

1. Extending Kubernetes API
Aggregated API vs CRD
Andrea Wong
Kubernetes Meetup London
2018-04-12

2. @
• Our team
• Our platform
• Why do we need custom resources?

3. Thank you, Yohan!

4. Azure/acs-engine
Magic
Custom resources

5. Extending Kubernetes API:
The resources
• What is a resource?
• What is a custom resource?
• How do I create a custom resource?

6. Extending Kubernetes API:
controllers
• What is a controller?
• Control loop
• Watches event for a particular resource and
react
• constantly trying to converge the existing state
into the desired state

7. Get current State
Diff with desired stateConverge
Extending Kubernetes API:
Controller

8. Common Terminologies
• Version conversion
• Defaulting
• Admission
• Validation

9. Validation & Admission
V1beta1 Internal
Validation
Admission
V1 JSON/YAML Etcd

10. Third Party Resources
• No version conversion
• no defaulting
• No validation
• No sub resources
• Deprecated in v1.7 (the only version that supports both
TPR and CRD)

11. What is a CRD?
• apiextension.k8s.io/v1beta1
• Replaced TPR in 1.7+
• Integrates with kubectl
• Integrates with controllers

12. Users of CRDs
• Tensorflow - Machine learning framework
• Istio - Service Mesh
• Calico - Overlay network
• Etcd Operator - etcd clusters
• Rook Operator - Distributed storage systems
• Prometheus Operator - Monitoring

13. CRD pros and cons
• Pros
• Lightweight
• Building the extension is “technically” easier compared to building AA
• Lower net memory requirement
• No need to run a separated etcd
• Cons
• Single version per resource, no version conversion - 1.11
• Only supports json encoding
• (old) known limitations
• Validation with JSON schema - 1.8
• Defaulting with JSON schema - 1.8
• Subresource i.e. scale, status - scale and status should on 1.10?
• Admissions now available via webhook

14. What is Aggregated API?
• k8s.io/apiserver since 1.6
• Each custom api server is its own process
• Delegates authentication/authorisation to kube-apiserver
• Uses etcd storage (possibly shared with k8s)

16. Users of Aggregated APIs
• kube-aggregator
• kube-apiserver
• Service catalog
• kubernetes-incubator/metrics-server

17. AA pros and cons
• Pros
• Provides a deeper integration with k8s
• Supports protobuf
• In-process admission
• Can use cluster etcd or its own for security purposes
• Cons
• Complicated
• Steeper learning curve compared to CRD
• More moving parts to deal with

18. Thank you