Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
J On The Beach, profile picture
Uploaded byJ On The Beach
2,919 views

An intro to Kubernetes operators

The document provides an introduction to Kubernetes operators, which are controllers that manage application deployment and operations on Kubernetes. It explains key concepts such as custom resource definitions (CRDs), the controller pattern, and the use of tools like kubebuilder and operator-sdk for building operators. Additionally, it discusses scenarios for when to create an operator versus using simpler configurations, along with methods for deploying and debugging operators.

Software◦
Related topics:
Red Hat Technologies•
Downloaded 204 times
An intro to Kubernetes operators Lili Cosic Software Engineer at Red Hat Twitter: @LiliCosic Github: lilic
An Intro to Kubernetes Operators - Lili Cosic2 Intro to Kubernetes
An Intro to Kubernetes Operators - Lili Cosic3 Kubernetes What is Kubernetes? ● Open source platform for managing containerized workloads and services ● Containers, containers, containers ● Name originates from Greek - helmsman or pilot ● Google open-sourced in 2014 ● Based on Borg - Google’ internal project ● K_ _ _ _ _ _ _ _S -> k8s
An Intro to Kubernetes Operators - Lili Cosic4 Kubernetes What is great about Kubernetes? ● Scalability of workloads ● Separation of workloads ● Native stable resources (Deployments, Pods) ● API / Custom Resources
An Intro to Kubernetes Operators - Lili Cosic5 Kubernetes What is Kubernetes not? ● Not a PaaS ● Not limited to the types of apps supported ● Not opinionated on: ○ Deploying ○ Run CI/CD ○ Logging, monitoring or alerting
An Intro to Kubernetes Operators - Lili Cosic6 Kubernetes controllers
An Intro to Kubernetes Operators - Lili Cosic7 What is a controller? ● Reconciles given state ● Controller pattern ○ Non terminating loop that regulates the state of the system ○ A control loop that watches the shared state of the cluster via the API server and makes changes to move from current to desired state
An Intro to Kubernetes Operators - Lili Cosic8 ReplicaSet controller Example controller ● ReplicaSet ○ Created by a Deployments ○ Creates Pods ● Reconciles to the correct (specified) number of pods running the cluster
An Intro to Kubernetes Operators - Lili Cosic9 Controller Credit: github.com/kubernetes/sample-controller
An Intro to Kubernetes Operators - Lili Cosic10 Informer pattern More important API functions ● ListWatcher ○ Helps you filter resources you want to watch ● ResourceEventHandler ○ Add, Update and Delete event trigger functions
An Intro to Kubernetes Operators - Lili Cosic11 Workqueue? ● Resource Event Handler puts an item to the workqueue ● Workqueue consists of keys: ○ <resource_namespace>/<resource_name> ● Only one worker works on one item at a time ● First In First Out
An Intro to Kubernetes Operators - Lili Cosic12 CRD - Custom Resource Definition
An Intro to Kubernetes Operators - Lili Cosic13 CRD - custom resource definition apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: # name must match the spec fields below, and be in the form: <plural>.<group> name: crontabs.stable.example.com spec: # group name to use for REST API: /apis/<group>/<version> group: stable.example.com # list of versions supported by this CustomResourceDefinition versions: - name: v1 # Each version can be enabled/disabled by Served flag. served: true # One and only one version must be marked as the storage version. storage: true # either Namespaced or Cluster scope: Namespaced names: # plural name to be used in the URL: /apis/<group>/<version>/<plural> plural: crontabs # singular name to be used as an alias on the CLI and for display singular: crontab # kind is normally the CamelCased singular type. Your resource manifests use this. kind: CronTab # shortNames allow shorter string to match your resource on the CLI shortNames: - ct
An Intro to Kubernetes Operators - Lili Cosic14 Operators, operators, operators
An Intro to Kubernetes Operators - Lili Cosic15
An Intro to Kubernetes Operators - Lili Cosic16 What is an “operator”? Operator is a K8s controller - specific to operating an application
An Intro to Kubernetes Operators - Lili Cosic17 What is an “operator”? ● Non core Kubernetes controller ● Makes use of CRDs - custom resource definition ● Holds the knowledge of how an application needs to be deployed, managed and packaged ● Reconcile loop ● Extends the Kubernetes API to make the application a custom resource - part of the Kubernetes ecosystem ● Concept introduced by CoreOS
An Intro to Kubernetes Operators - Lili Cosic18 Good examples of operators Awesome operators! ● github.com/coreos/prometheus-operator ● github.com/zalando-incubator/postgres-operator ● github.com/coreos/etcd-operator
An Intro to Kubernetes Operators - Lili Cosic19 When to choose creating an operator? ● Application uses declarative API ● Resources are scoped to a namespace or a cluster ● Encapsulate business logic ● Build automation that watches for updates of Kubernetes objects ● Create or update resources via the Kubernetes native API ● Top level support from kubectl
An Intro to Kubernetes Operators - Lili Cosic20 When to just use a ConfigMap or Secret? Sometimes creating an operator is not needed ● Existing well known config file format (e.g. mysql.cnf ) ● Config file is used to just configure an application running in a Pod ● No need to reconcile to the state
An Intro to Kubernetes Operators - Lili Cosic21 101 ways to build an operator ● go language: ○ client-go & co. ○ operator-sdk ○ kubebuilder ● Other languages: ○ Kubernetes python and java clients and others ● Other tools: ○ helm ○ ansible
An Intro to Kubernetes Operators - Lili Cosic22 Building operators using Kubernetes native clients
An Intro to Kubernetes Operators - Lili Cosic23 Kubernetes native clients ● k8s.io/client-go ○ rest ○ discovery ○ dynamic ● k8s.io/api ● ks8.io/apimachinery
An Intro to Kubernetes Operators - Lili Cosic24 Kubernetes native clients Pros ● Same as upstream controllers use ● Stability of K8s code ● Versioning based on Kubernetes releases ● Ability to finetune Cons ● Large ecosystem ● No abstractions/helpers ● A lot of inside knowledge to optimize correctly ● New major version on every Kubernetes minor version release
An Intro to Kubernetes Operators - Lili Cosic25 Using Kubernetes native go clients Example of an “operator” github.com/kubernetes/sample-controller
An Intro to Kubernetes Operators - Lili Cosic26 kubebuilder
An Intro to Kubernetes Operators - Lili Cosic27 kubebuilder ● github.com/kubernetes-sigs/kubebuilder ● SDK for building Kubernetes APIs using CRDs ● Part of kubernetes-sigs repo ● Uses controller-runtime under the hood ● Doesn’t strictly advertise itself as operator builder tool
An Intro to Kubernetes Operators - Lili Cosic28 Using kubebuilder Example of a go operator $ mkdir kubebuilder-operator && cd kubebuilder-operator $ kubebuilder init --domain k8s.io --license apache2 --owner "The JOnTheBeach Audience" $ kubebuilder create api --group ships --version v1beta1 --kind Sloop $ # Edit the logic code $ pkg/controller/sloop/sloop_controller.go
An Intro to Kubernetes Operators - Lili Cosic29 operator-sdk
An Intro to Kubernetes Operators - Lili Cosic30 operator-sdk ● github.com/operator-framework/operator-sdk ● Aimed at creating operators ● Part of operator-framework ● Operator types you can create: ○ go ○ helm ○ ansible ● Testing framework
An Intro to Kubernetes Operators - Lili Cosic31 Using operator-sdk Example of a go operator $ operator-sdk new app-operator $ cd app-operator $ # Add a new API for the custom resource AppService $ operator-sdk add api --api-version=app.example.com/v1alpha1 --kind=AppService $ # Add a new controller that watches for AppService $ operator-sdk add controller --api-version=app.example.com/v1alpha1 --kind=AppService $ # Build operator $ operator-sdk build quay.io/example/app-operator
An Intro to Kubernetes Operators - Lili Cosic32 operator-sdk - helm
An Intro to Kubernetes Operators - Lili Cosic33 Using operator-sdk Example of a helm operator $ # Create a new helm operator project $ operator-sdk new nginx-operator --api-version=example.com/v1alpha1 --kind=Nginx --type=helm $ # Edit watches.yaml file to customize the operator logic
An Intro to Kubernetes Operators - Lili Cosic34 operator-sdk - ansible
An Intro to Kubernetes Operators - Lili Cosic35 Using operator-sdk Example of an ansible operator $ # Create a new ansible based operator project: $ operator-sdk new memcached-operator --api-version=cache.example.com/v1alpha1 --kind=Memcached --type=ansible $ # Edit watches file to customize the logic
An Intro to Kubernetes Operators - Lili Cosic36 Deploying an operator
An Intro to Kubernetes Operators - Lili Cosic37 Deploying (operator) on Kubernetes ● YAML ● Kustomize ● Jsonnet ● OLM ● Helm
An Intro to Kubernetes Operators - Lili Cosic38 YAML $ # Register the CRD $ kubectl apply -f crd.yaml $ # Create any needed Role Based Access Control $ kubectl apply -f rbac.yaml $ # Deploy the operator $ kubectl apply -f deploy.yaml $ # Create an instance $ kubectl apply -f cr.yaml
An Intro to Kubernetes Operators - Lili Cosic39 YAML - CRD manifest file apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: memcacheds.cache.example.com spec: group: cache.example.com names: kind: Memcached listKind: MemcachedList plural: memcacheds singular: memcached scope: Namespaced subresources: status: {} validation: openAPIV3Schema: properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds' type: string metadata: type: object spec: properties: size: format: int32 type: integer required: - size type: object status: properties: nodes: items: type: string type: array required: - nodes type: object version: v1alpha1 versions: - name: v1alpha1 served: true storage: true
An Intro to Kubernetes Operators - Lili Cosic40 YAML - deployment manifest file apiVersion: apps/v1 kind: Deployment metadata: name: memcached-operator spec: replicas: 1 selector: matchLabels: name: memcached-operator template: metadata: labels: name: memcached-operator spec: serviceAccountName: memcached-operator containers: - name: memcached-operator image: memached:123 command: - memcached-operator imagePullPolicy: Always env: - name: WATCH_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: OPERATOR_NAME value: "memcached-operator"
An Intro to Kubernetes Operators - Lili Cosic41 YAML - CR instance manifest file apiVersion: cache.example.com/v1alpha1 kind: Memcached metadata: name: example-memcached spec: size: 3
An Intro to Kubernetes Operators - Lili Cosic42 OLM ● Upgrades ● Descriptions and metadata ● Dependency resolution ● Multiple versions of operator to install Key features
An Intro to Kubernetes Operators - Lili Cosic43 Debugging an operator
An Intro to Kubernetes Operators - Lili Cosic44 Debugging an operator ● `operator-sdk up local` - run ● Log all the things! ● Get logs of an operator ○ `kubectl logs <pod-name>` ● Use CR name to label any pods the operator creates ● `kubectl events -n NAMESPACE` ● `Spec.Paused`
An Intro to Kubernetes Operators - Lili Cosic45 Kubernetes cluster stack for big data
An Intro to Kubernetes Operators - Lili Cosic46 More information ● Kubernetes Special Interest Group Big Data ● User group meeting: Wednesdays at 18:00 UTC (biweekly) ● Slack channel - #ug-big-data
An Intro to Kubernetes Operators - Lili Cosic47 Spark operator Spark on Kubernetes github.com/GoogleCloudPlatform/spark-on-k8s-operator
An Intro to Kubernetes Operators - Lili Cosic48 Future Whats next for operators? ● OLM ● cluster-addons ● CRDs ○ GA in 1.16
QUESTIONS Lili Cosic Twitter: @LiliCosic Github: lilic