Downloaded 109 times
Uploaded byVMware Tanzu
Building Developer Pipelines with PKS, Harbor, Clair, and Concourse
The document provides an overview of building developer pipelines using PKS, Harbor, Clair, and Concourse, highlighting their components and functions. It emphasizes the importance of security features like image vulnerability scanning and signing, and outlines the operational processes involved in managing Kubernetes applications. Additionally, the presentation includes a demo and discusses lessons learned for enhancing the development workflow and security.
More Related Content
Similar to Building Developer Pipelines with PKS, Harbor, Clair, and Concourse
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
Building Developer Pipelines with PKS, Harbor, Clair, and Concourse
- 1. Building Developer Pipelines withPKS, Harbor, Claire, and Concourse Merlin Glynn Thomas Kraus Cloud Native Apps Business Unit
- 2. • This presentationmay contain product features that are currently under development. • This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. • Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. • Technical feasibility and market demand will affect final delivery. • Pricing and packaging for any new technologies or features discussed or presented have not been determined. Disclaimer 2
- 3. Agenda 3 1 Context 2 PKSand its components 3 Developer Solution based on PKS 4 Demo !
- 4. 4 Analytics Automation SecurityOperations MonitoringLogging Physical Infrastructure Container Registry vSpherevSAN Kubernetes on BOSH (Kubo/CFCR) NSX-T BOSH GCP Service Broker masteretcd workermasteretcd worker Pivotal Container Service (PKS) PKS Control Plane
- 5. Agenda 5 1 Context 2 PKSand its components 3 Developer Solution based on PKS 4 Demo !
- 6. PKS BOSH K8S-1 Work er Worker K8S-2 BOSH Agent BOSH Agent K8s-api Team A K8s-api KUBO BOSH Release (tgz) DAY 2Ops - Auto/Manual Rebuild - Auto/Manual Repair - Manual Scale - Patch & Upgrade - Control & Audit OPS Events NAMESPACE_1: TEAM A NAMESPACE_2: TEAM B Team C Team B NAMESPACE_1: DEFAULT DAY 1 Ops DEPLOY OperateK8s+RunApps/Containers UI & API Worker Application Dev/Ops Owner Application Dev/Ops Owner Application Dev/Ops Owner Work er MASTER WorkerMASTER ETCD WorkerMASTER ETCD MASTER MASTER ETCD Platform Reliability Engineer Self Service K8s PKS Technical Overview 1.7 -> 1.8 1.7 -> 1.8 PKS Controller 6 Ops Manager (OVA)
- 7. 7 Harbor – EnterpriseGrade Private Registry CONFIDENTIAL Development Team Image Registry Clair Notary Build Image Push Image Scan Image For CVEs Sign Trusted Image Harbor Project(s) RBAC UAA AUTH REPL ‘kubectl run’ Image PullImage Push
- 8. 8CONFIDENTIAL Harbor – ImageVulnerability Scanning Details (Clair) CVE Datasources Clair Harbor Notifications Postgres Debian Sec Bug Tracker Ubuntu CVE Tracker Red Hat Security Data Oracle Linux Sec Data Alpine SecDB NIST NVD ubuntu:trusty nginx:latest layer1 03ac7b930e4b layer2 fc4540b46019 layer3 225948c9a8b1
- 9. 9 Harbor – ImageSigning (Content Trust) When Enabled Un-signed Images Can’t Be Pulled CONFIDENTIAL
- 10. Agenda 10 1 Context 2 PKSand its components 3 Developer Solution based on PKS 4 Demo !
- 11. Dockerfile Dev Host (Laptop/VM) Putting itall together 11 K8s Cluster Worker kubectl apply –f myapp.yml Worker POD POD Developer CONFIDENTIAL $ docker build $ docker tag $ docker push clair notary PKS scan sign
- 12. Agenda 12 1 Context 2 PKSand its components 3 Developer Solution based on PKS 4 Demo !
- 13.
- 14. Reference CONFIDENTIAL 14 • Hugo– website framework template and instructions • https://github.com/tkrausjr/my-conference • Example Conference Go Application • https://github.com/tkrausjr/go-http • Concourse Pipeline • https://github.com/virtmerlin/spring1-goapp-ci • Kubernetes Resource Type for Concourse • https://github.com/virtmerlin/concourse-kubernetes-resource
- 15. Lessons Learned /Enhancements CONFIDENTIAL 15 • Enhance Security - Notary Image Signing right now has signing keys in the pipeline for updating an existing image which has initially been signed and pushed. • Create a Harbor and Notary Resource Type for Concourse instead of using clairctl and Docker Cli to increase reusability. • Consider using separate Docker Images for Build and Run and add in artifact repository like Nexus or Artifactory to store build artifacts. Right now the same Base Docker Image is used for both the Application Build and the Application run via Kubernetes. • Concourse or any other CI / Automation tool is almost required for a solutions like this and removes the human element and creates a version controlled and tracked location for everything from your Application and
- 16. @cloudnativeapps #vmwcna #vmwpks vmware.github.io Thank You! VMware PivotalContainer Services (PKS) blogs.vmware.com/cloudnative 16
- 17. BACKUP SLIDES CONFIDENTIAL 17 •BACKUP SLIDES
- 18. 18 Need Harbor screenshot •Authentication & Authorization • AD/LDAP integration • Security vulnerability scanning (Clair) • content trust - image signing • policy based image replication • audit and logs • Restful API • open-source under Apache 2 license Harbor – Enterprise Grade Private Registry (WIP) CONFIDENTIAL
- 19. 19CONFIDENTIAL Harbor – ImageVulnerability Scanning Details (Clair)
Editor's Notes
- #5 Goals of PKS GOAL – Bring the release engineering, deployment, and lifecycle management qualities of BOSH to Kubernetes (K8s) clusters. Provide Day 1 and Day 2 Deployment, and Operations of Enterprise Grade Prod ready K8s Fully Supported Kubernetes Better Operational experience running K8s in Production especially Day 2(upgrades, HA, deployment, Scaling, self healing, life-cycle management) Deep Integration with NSX Hardened, Production-grade Runs on vSphere and VMC HA, Security, Multi-tenancy, Tools Bosh – Open Source software that delivers the capability to deploy complex distributed software (Day1) and keep it healthy (Day2) 1 – CPI Is how BOSH can deploy software across many clouds. PKS will support GCP & vSphere at GA 2 – BOSH Stemcells allow BOSH to abstract the OS image from the software being deployed. Stemcells for PKS 1.0 will be Ubuntu based VMDKs with a minimal OS and a BOSH agent 3 – BOSH Releases are collections of software and the metadata require to tell BOSH how to deploy that software. KUBO (Now Called CFCR) is literally a BOSH release to deploy Kubernetes 4 – BOSH Manifest is a set of YAML (which also uses another set of global YAML called a cloud-config, which is not pictured above). The Manifest will define which Stemcellls, releases, configuration options for the releases, and number of instances should be deployed. The manifest is then given to BOSH 5 – BOSH deployments is a given instantiation of a BOSH Deployment manifest PKS 3 Key components of PKS PKS Controller : The API that will allow self service KUBO : CFCR BOSH Releases BOSH HARBOR role-based access control w/ AD/LDAP integration Security vulnerability scanning (Clair) content trust - image signing policy based image replication audit and logs open-source under Apache 2 license
- #7 This image is the first ‘introduction’ to to what PKS actually provides. At it heart PKS provides a self service API for development teams to consume container services, while still allowing the PRE (or Operator) the ability to control the environment for the enterprise. The animations will play thru to demonstrate key Day 1 & Day 2 capabilities; 3 Key components of PKS PKS Controller : The API that will allow self service KUBO : CFCR BOSH Releases BOSH *** Manual scale is intended to denote that BOSH scale ops of each K8s cluster is not automated. While various K8s mechanisms can be used to auto scale pod workloads, K8s itself is not auto scaled in 1.0
- #8 I have had a lot of experience w/ large scale deployment of Container Orchestration systems like Mesos or Kubernetes and on of the often overlooked aspects of these projects is the Container Registry and the Enterprise Control over both Container Images and SW artifacts used in the build process. Harbor addresses this problem. Harbor – Open Source Docker v2 Registry fronted by a Reverse Proxy and automatically included Clair & Notary. *** Harbor provides Authentication & Authorization with AD/LDAP integration Security vulnerability scanning (Clair) and content trust - image signing policy based image replication audit and logs Restful API open-source under Apache 2 license
- #9 Clair CVE scanning. Clair is a French word which translates to clear, bright, transparent. Clair API, Scanning Engine for scanning Docker Images Layer by Layer for CVE vulnerabilities. Returns a list of all files and packages in use by a particular LAYER and compares those files against the files in the CVE database that is stored in Postgres. Clair API Allows for Creation, Put and Deletes of existing vulnerabilities BUT if you are using a FETCHER to obtain vulnerabilities (CVE) the vulnerabilities may be re-inserted when the next scheduled fetch occurs.
- #10 Image signing Curated Images is kind of a big thing. Picture this - - Dev on laptop pulls git repo modifies a Dockerfile and copies binaries or packages (vulnerable) into the Docker container and builds an image. - Dev has permissions to push to your registry REPO or REPO is PUBLIC. - You now have vulnerable images in your registry which can be deployed 100's of times and exploited. UNLESS 1) You enable Notary (DTR) , so you cant pull or push without correctly signing the image first. 2) You si
- #12 From the Developer point of view: I check my code in just like if I were pushing to PCF But in addition to application artifacts, the pipeline is going to build an image for me … In this visual we have a K8S cluster already running docker as the backend container engine, so our CI/CD pipeline will build a docker image for us and post it to a registry, in this case VMware Harbor Afterwhich, the pipeline will instantiate a K8S deployment to run our docker image based application as a set of pods in a replica set in case a worker note goes offline. The developer can than create a ‘service’ that gives worker nodes (or any external node) running the kube-proxy service the ability to route to where those pods are and access the apps/microservices running in them. Ingress routing from external is similar to that of CF with an external DNS map being required to forward requests to 1 or more worker nodes running kube-proxy One of the key differences is that Kubernetes isn’t opinionated on how the container image should be built, this give more flex to the developers but in some cases can make things more difficult for operators as we’ll see later on in the presentation Agility is why developers want it
- #19 Harbor Core Features avail to PKS deployed clusters *** Harbor will ship (as in will be packaged with and entitled with PKS)
- #20 Clair CVE scanning