Operator pattern is pioneered by CoreOS, and it is embraced by the Kubernetes community. The Operator framework from CoreOS/Red Hat is also getting adoption. Operator Lifecycle Management (OLM) is part of the Operator framework. This talk walks you through testing Kubernetes operator with Operator Lifecycle Management (OLM).

1. Test your Kubernetes operator with OLM
Baiju Muthukadan Avni Sharma
May 15, 2019
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 1 / 36

2. About Us
Baiju is a Senior Software Engineer
Avni is an Associate Software Engineer
Both of us work on DevConsole Operator
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 2 / 36

3. Problem Statement
As a development team, we need to run end-to-end test for our operator. To
run end-to-end test, we need a mechanism to setup the infrastructure which
includes the operator deployment and other dependent resources (CRDs,
service account, role, and role binding).
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 3 / 36

4. Agenda
Introduction to Operator Framework
Setting up resources through Operator Lifecycle Manager (OLM) for e2e
testing
Running e2e test through OpenShift CI
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 4 / 36

5. DevConsole Operator
Provides a developer-focused view in OpenShift 4
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 5 / 36

6. Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 6 / 36

7. Operator Framework
A toolkit to manage Kubernetes native applications, called Operators.
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 7 / 36

8. Example Operators
https://github.com/redhat-developer/devconsole-operator
https://github.com/openshift/tektoncd-pipeline-operator
https://github.com/operator-framework/operator-marketplace
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 8 / 36

9. Operator Framework Parts
https://github.com/operator-framework
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 9 / 36

10. Operator Lifecycle Manager (OLM)
Oversees operator:
Installation
Updates
Lifecycle
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 10 / 36

11. OLM CRDs
catalogsources.operators.coreos.com
clusterserviceversions.operators.coreos.com
installplans.operators.coreos.com
operatorgroups.operators.coreos.com
subscriptions.operators.coreos.com
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 11 / 36

12. Catalog Source
Repository of Cluster Service Version (CSV) files and CRDs
Two types
gRPC
Internal (based on ConfigMap)
Public repository: https://www.operatorhub.io
To create a custom gRPC use this base image:
quay.io/openshift/origin-operator-registry:latest
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 12 / 36

13. File: manifests/devconsole/0.1.0/devconsole-
operator.v0.1.0.clusterserviceversion.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
annotations:
capabilities: Full Lifecycle
description: The operator that enables a developer-focused perspective
categories: "Developer Tools"
name: devconsole-operator.v0.1.0
namespace: placeholder
spec:
apiservicedefinitions: {}
customresourcedefinitions:
...
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 13 / 36

14. Manifests directory structure
manifests/
└── devconsole
├── 0.1.0
│ ├── devconsole-operator.v0.1.0.clusterserviceversion.yaml
│ ├── devconsole_v1alpha1_component_crd.yaml
│ ├── devconsole_v1alpha1_gitsourceanalysis_crd.yaml
│ └── devconsole_v1alpha1_gitsource_crd.yaml
└── devconsole.package.yaml
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 14 / 36

15. Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 15 / 36

16. Dockerfile to create registry
FROM quay.io/openshift/origin-operator-registry:latest
ARG image=quay.io/redhat-developer/devconsole-operator
ARG version=0.1.0
COPY manifests manifests
COPY deploy/crds/*.yaml manifests/devconsole/${version}/
USER root
RUN sed -e "s,REPLACE_IMAGE,${image}," -i
manifests/devconsole/${version}/devconsole-operator.v${version}.
clusterserviceversion.yaml
USER 1001
RUN initializer
CMD "registry-server", "--termination-log=log.txt"
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 16 / 36

17. Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 17 / 36

18. Manifests directory structure
manifests/
└── devconsole
├── 0.1.0
│ ├── devconsole-operator.v0.1.0.clusterserviceversion.yaml
│ ├── devconsole_v1alpha1_component_crd.yaml
│ ├── devconsole_v1alpha1_gitsourceanalysis_crd.yaml
│ └── devconsole_v1alpha1_gitsource_crd.yaml
└── devconsole.package.yaml
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 18 / 36

19. Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 19 / 36

20. CatalogSource
# Ref. https://github.com/operator-framework/operator-lifecycle-manager
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
name: my-catalog
namespace: openshift-operator-lifecycle-manager
spec:
sourceType: grpc
image: REPLACE_IMAGE
displayName: Community Operators
publisher: Red Hat
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 20 / 36

21. Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 21 / 36

22. Subscription
Keep CSVs up to date by tracking a channel in a package.
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 22 / 36

23. File: manifests/devconsole/devconsole.package.yaml
packageName: devconsole
channels:
- name: alpha
currentCSV: devconsole-operator.v0.1.0
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 23 / 36

24. Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 24 / 36

25. Subscription
# Ref. https://github.com/operator-framework/operator-lifecycle-manager
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: my-devconsole
namespace: openshift-operators
spec:
channel: alpha
name: devconsole
source: my-catalog
sourceNamespace: openshift-operator-lifecycle-manager
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 25 / 36

26. Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 26 / 36

27. OpenShift CI
OpenShift CI automates and simplifies the process of building and testing
OpenShift component images
OpenShift CI is run using ci-operator (This is not a Kubernetes operator)
ci-operator is built using Kubernetes prow
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 27 / 36

28. Setup operator resources without OLM
Create a temporary namespace/project
Create all the CRDs
Create service account
Create role
Create role binding
Run the test
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 28 / 36

29. Setup operator resources without OLM
.PHONY: test-e2e-ci
test-e2e-ci: get-test-namespace ./vendor
$(Q)oc new-project $(TEST_NAMESPACE)
$(Q)-oc apply -f ./deploy/crds/devconsole_v1alpha1_component_crd.yaml
$(Q)-oc apply -f ./deploy/crds/devconsole_v1alpha1_gitsource_crd.yaml
$(Q)-oc apply -f ./deploy/crds/devconsole_v1alpha1_gitsourceanalysis_crd
$(Q)-oc apply -f ./deploy/service_account.yaml --namespace $(TEST_NAMESP
$(Q)-oc apply -f ./deploy/role.yaml --namespace $(TEST_NAMESPACE)
$(Q)sed -e 's|REPLACE_NAMESPACE|$(TEST_NAMESPACE)|g' ./deploy/test/role_
$(Q)sed -e 's|REPLACE_IMAGE|registry.svc.ci.openshift.org/$
{OPENSHIFT_BUILD_NAMESPACE}/stable:devconsole-operator|g'
./deploy/test/operator_test.yaml | oc apply -f - --namespace $(TEST_NA
$(eval DEPLOYED_NAMESPACE := $(TEST_NAMESPACE))
$(Q)operator-sdk test local ./test/e2e --namespace
$(TEST_NAMESPACE) --no-setup --go-test-flags "-v -timeout=15m"
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 29 / 36

30. Setup operator resources with OLM
Create catalog source
Create subscription
Run the test
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 30 / 36

31. Setup operator resources with OLM
.PHONY: test-e2e-olm-ci
test-e2e-olm-ci: ./vendor
$(Q)sed -e "s,REPLACE_IMAGE,registry.svc.ci.openshift.org/
${OPENSHIFT_BUILD_NAMESPACE}/stable:devconsole-operator-registry,"
./test/e2e/catalog_source_OS4.yaml | oc apply -f -
$(Q)oc apply -f ./test/e2e/subscription_OS4.yaml
$(Q)./hack/check-crds.sh
$(Q)operator-sdk test local ./test/e2e --no-setup
--go-test-flags "-v -timeout=15m"
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 31 / 36

32. OpenShift CI Configuration
base_images:
operator-registry:
name: "4.0"
namespace: ocp
tag: operator-registry
images:
- from: operator-registry
dockerfile_path: openshift-ci/Dockerfile.registry.intermediate
to: operator-registry-base
- from: operator-registry-base
dockerfile_path: openshift-ci/Dockerfile.registry.build
to: devconsole-operator-registry
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 32 / 36

33. File: openshift-ci/Dockerfile.registry.intermediate
FROM quay.io/openshift/origin-operator-registry:latest
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 33 / 36

34. File: openshift-ci/Dockerfile.registry.build
FROM quay.io/openshift/origin-operator-registry:latest
ARG version=0.1.0
COPY manifests manifests
COPY deploy/crds/*.yaml manifests/devconsole/${version}/
RUN sed -e "s,REPLACE_IMAGE,registry.svc.ci.openshift.org/
${OPENSHIFT_BUILD_NAMESPACE}/stable:devconsole-operator,"
-i manifests/devconsole/${version}/devconsole-operator.v${version}
.clusterserviceversion.yaml
RUN initializer
USER 1001
EXPOSE 50051
CMD "registry-server", "--termination-log=log.txt"
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 34 / 36

35. https://github.com/redhat-developer/devconsole-operator
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 35 / 36

36. Thank You!
bmuthuka@redhat.com
avsharma@redhat.com
Baiju Muthukadan, Avni Sharma Test your Kubernetes operator with OLM May 15, 2019 36 / 36