Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Architecting the Future: Abstractions and Metadata - STL SilverLinings

101 views

Published on

Kubernetes and Docker are two of the top open source projects, and they’re built around abstractions and metadata. These two concepts are the key to architecting in the future. Come with me as I dig a little deeper into these concepts within k8s and Docker and provide some examples from my own work on Deployment Pipelines.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Architecting the Future: Abstractions and Metadata - STL SilverLinings

  1. 1. announcements
  2. 2. sponsors platinum gold silver
  3. 3. Architecting the Future Abstractions and Metadata Dan Barker @barkerd427 danbarker.codes
  4. 4. The current data center is...challenging... RHEL 6.9 Dev RHEL 6.8 Test RHEL 6.6 Prod Dev Test Prod RHEL 6.7 Prod Admin Admin Admin Admin Admin Admin Dev RHEL 6.7 Dev RHEL 6.4 Dev RHEL 6.8 Dev Ubuntu Trusty RHEL 6.9 Dev RHEL 6.6 Dev Ubuntu Trusty RHEL 6.7 Dev RHEL 6.4 Dev RHEL 6.8 Dev Ubuntu Trusty RHEL 6.9 Dev RHEL 6.6 Dev RHEL 6.7 Dev RHEL 6.4 Dev RHEL 6.8 Dev Ubuntu Trusty RHEL 6.9 Dev RHEL 6.6 Dev
  5. 5. The new data center is understandable and usable. Developer Access Production Controlled Network Storage Compute Platform Deployment Pipeline RHEL 6.9 App1 RHEL 6.9 App1 RHEL 6.9 App1 RHEL 6.9 App1 RHEL 6.9 App1 RHEL 6.9 App1 RHEL 6.9 App2 RHEL 6.9 App2 RHEL 6.9 App2 RHEL 6.9 App2 RHEL 6.9 App2 RHEL 6.9 App2
  6. 6. Docker - the early ● Docker is an abstraction ○ cgroups ○ Namespaces ● Not Included ○ Metadata ○ Volumes ○ Secrets ○ Services ○ Network ○ Plugins
  7. 7. Docker - the latter ● Volumes ● Secrets ● Networks ● Plugins ● Services ● Labels
  8. 8. Kubernetes (k8s) ● Secrets ● PersistentVolumes ● Services ● Pods ● Ingress ● DaemonSets ● ReplicaSets ● Deployments
  9. 9. OpenShift ● Routes ○ > Ingress ● DeploymentConfig ○ > Deployments ● ImageStream ● BuildConfig ● RBAC
  10. 10. Services will change your life
  11. 11. Services ● Identifies a set of pods using label selectors ○ Can be any label ○ Should be specific to avoid picking up disparate applications unintentionally ● Passes requests to pods internally to a Kubernetes cluster ○ Routes and Services are different ● Provides an abstraction for a Route to pass traffic from outside the cluster to desired endpoints
  12. 12. One Route, One Service, One Application Route Service Pod
  13. 13. The Route directs to the Service application0 ➜ ~ oc export routes application0 apiVersion: v1 kind: Route [...] spec: host: application0-presentation... to: kind: Service name: application0 weight: 100 [...]
  14. 14. The Service matches on the label “deploymentconfig” with the value “application0”. ➜ ~ oc export svc application0 apiVersion: v1 kind: Service spec: selector: deploymentconfig: application0
  15. 15. The Pod has many labels. ➜ ~ oc export -o yaml po/application0-1-ao16l apiVersion: v1 kind: Pod metadata: labels: app: application0 deploymentconfig: application0 environment: dev partition: customerA release: stable tier: frontend [...]
  16. 16. The Service now matches on the label “tier” with the value “frontend”. ➜ ~ oc export svc application0 apiVersion: v1 kind: Service spec: selector: tier: frontend
  17. 17. One Route, One Service, Two Applications Route Application Application Service Service
  18. 18. Curling the same Route results in two different applications responding.
  19. 19. The Pod has many labels. ➜ ~ oc export -o yaml po/application0-beta-1-ao16l apiVersion: v1 kind: Pod metadata: labels: app: application0-beta deploymentconfig: application0-beta environment: dev partition: customerA release: stable tier: frontend [...]
  20. 20. We’ve deleted application1 and added application0-beta.
  21. 21. ImageStreams are an image abstraction
  22. 22. ImageStreams ● May contain images from: ○ Image repository in OpenShift’s integrated registry ○ Other image streams ○ Image repositories from external registries ● Automatically trigger an action when new images are created ● Convenient abstraction ○ Related images ○ Image organization ○ Image history
  23. 23. ImageStreams ● Metadata ○ Commands ○ Entrypoint ○ Environment variables ○ Image Layers ○ Labels ○ Ports ● Could also reference other container kinds like rkt
  24. 24. CoreOS Operators are magical (not really)
  25. 25. Not this
  26. 26. This
  27. 27. Operators ● Represents human operational knowledge in software to reliably manage an application ○ AI? ● Uses the Kubernetes concept of 3rd-party resources ○ Operates as a controller of controllers and resources ● Identical model to current Kubernetes controllers ○ Observe, Analyze, Act ○ Deployments, DaemonSets, ReplicationControllers ● Not supported in OpenShift
  28. 28. Operators ● Deployed into a k8s cluster ● Interactions occur through the new controller ○ kubectl get prometheuses --all-namespaces ○ kubectl get alertmanagers --all-namespaces ● Abstraction around k8s primitives ○ Users just want to use a MySQL cluster. ● Complex tasks that can be performed ○ Rotating credentials, certs, versions ○ Perform backups
  29. 29. Deployment Pipeline’s have fallen behind
  30. 30. An Application includes a Pipeline, which is based on an opinionated PipelineTemplate. These combine to make a PipelineConfig. apiVersion: v1 kind: Application name: app1 cap: template: name: approvedTemplates/Tomcat8.yaml pipeline: notifications: mattermost: team: cloud channel: general on_success: never on_failure: always dependencies: - name: authn dnsName: authn - name: key-management username: reference_to_username password: reference_to_password stages: - name: build steps: - action: build baseImage: version: 8.0.41 - name: dev approvers: - role: app1-dev steps: - action: deploy params: environment: dev apiVersion: v1 kind: PipelineTemplate name: Tomcat8 labels: type: application build: manager: maven version: latest builderImage: java8-builder version: latest baseImage: tomcat8 version: latest deploy: deploymentType: canary maxUnavailable: 10% maxSurge: 20% apiVersion: v1 kind: PipelineConfig name: app1-pipeline labels: type: application pipeline: notifications: mattermost: team: cloud channel: general on_success: never on_failure: always dependencies: - name: authn dnsName: authn - name: key-management username: reference_to_username password: reference_to_password stages: - name: build steps: - action: build manager: maven builderImage: java8-builder baseImage: tomcat8 version: 8.0.41 - name: dev approvers: - role: app1-dev steps: - action: deploy params: environment: dev
  31. 31. Pipelines ● Stages ● Steps ● Application ● PipelineTemplate ● PipelineConfig
  32. 32. An Application and PipelineTemplate also combine to create a DeploymentConfig. apiVersion: v1 kind: Application name: app1 cap: template: name: approvedTemplates/Tomcat8.yaml pipeline: notifications: mattermost: team: cloud channel: general on_success: never on_failure: always dependencies: - name: authn dnsName: authn - name: key-management username: reference_to_username password: reference_to_password stages: - name: build steps: - action: build baseImage: version: 8.0.41 - name: dev approvers: - role: app1-dev steps: - action: deploy params: environment: dev apiVersion: v1 kind: PipelineTemplate name: Tomcat8 labels: type: application build: manager: maven version: latest builderImage: java8-builder version: latest baseImage: tomcat8 version: latest deploy: deploymentType: canary maxUnavailable: 10% maxSurge: 20% apiVersion: v1 kind: DeploymentConfig metadata: name: app1-pipeline type: application spec: replicas: 2 selector: name: frontend template: { ... } triggers: - type: ConfigChange - imageChangeParams: automatic: true containerNames: - helloworld from: kind: ImageStreamTag name: hello-openshift:latest type: ImageChange strategy: type: Rolling
  33. 33. The value of Pipelines ● Abstract the details of audit and compliance ○ Approvals are added dynamically and automatically ● Trivialities eliminated ○ Tabs vs. spaces ○ Curly braces placement ○ Semicolons or not ● Security checks occur early and often with helpful feedback ○ When a violation of policy or vulnerability is detected a direction for remediating it should be provided with additional resources or contacts available
  34. 34. The value of Pipelines ● Inject security testing across all applications easily ● Update security tooling and configuration centrally ● Utilize common artifact repositories ○ Restrict undesirable dependencies ○ Notify dependent applications when a vulnerability is discovered ● Standardized/Centralized approval system for Audit/Compliance ● Applications will become secure by default as the pipeline enforces additional policies
  35. 35. announcements

×