Software Supply Chain Observability
with Grafeas and Kritis
Aysylu Greenberg June 4 2019
Photo via https://www.goodfreephotos.com/
Aysylu
Greenberg
- Sr Software Engineer
@Google
Aysylu
Greenberg
- Sr Software Engineer
@Google
- Eng Lead of
open-source Grafeas
and Kritis
Aysylu
Greenberg
- Sr Software Engineer
@Google
- Eng Lead of
open-source Grafeas
and Kritis
- @aysylu22
In This Talk
Software
Supply Chain
Management
Kritis Grafeas Kritis &
Grafeas 0.1.0
1 2 3 4
In This Talk
Software
Supply Chain
Management
Kritis Grafeas Kritis &
Grafeas 0.1.0
1 2 3 4
Software Supply
Chain Management
Code Checkin
Test &
Verification
Write code
Build Image
Deploy to
Production
QA
Software Supply
Chain Management
Code Checkin
Test &
Verification
Write code
Build Image
Deploy to
Production
QA
Software Supply
Chain Management
Code Checkin
Test &
Verification
Write code
Build Image
Deploy to
Production
QA
Software Supply
Chain Management
Code Checkin
Test &
Verification
Write code
Build Image
Deploy to
Production
QA
Software Supply
Chain Management
Code Checkin
Test &
Verification
Write code
Build Image
Deploy to
Production
QA
CI pipelines
Software Supply
Chain Management
Code Checkin
Test &
Verification
Write code
Build Image
Deploy to
Production
QA
Software Supply
Chain Management
Code Checkin
Test &
Verification
Write code
Build Image
Deploy to
Production
QA
CD pipelines
Software Supply
Chain Management
Code Checkin
Test &
Verification
Write code
Build Image
Deploy to
Production
QA
Software Supply
Chain Management
what happens to
code from source to
deployment?
Code Checkin
Test &
Verification
Write code
Build Image
Deploy to
Production
QA
Software Supply
Chain Management
what happens to
code from source to
deployment?
CI/CD pipelines,
observability tools
Code Checkin
Test &
Verification
Write code
Build Image
Deploy to
Production
QA
Software Supply Chain with Grafeas & Kritis
CI/CD pipelines
Build &
Deploy
Software Supply Chain with Grafeas & Kritis
CI/CD pipelines
Build &
Deploy
Secure
build
process
Automated
test, scan,
analysis
Deploy
checks
Software Supply Chain with Grafeas & Kritis
CI/CD pipelines
Build &
Deploy
Secure
build
process
Automated
test, scan,
analysis
Deploy
checks
Centralized metadata
knowledge base
Grafeas backed storage
vulnerabilities, build info, etc.
Software Supply Chain with Grafeas & Kritis
CI/CD pipelines
Build &
Deploy
Secure
build
process
Automated
test, scan,
analysis
Deploy
checks
Centralized metadata
knowledge base
Kritis
Admission
controller
Grafeas backed storage
vulnerabilities, build info, etc.
Software Supply Chain with Grafeas & Kritis
CI/CD pipelines
Build &
Deploy
Secure
build
process
Automated
test, scan,
analysis
Deploy
checks
Centralized metadata
knowledge base
Kritis
Admission
controller
Deploy time policy chokepoint
Enforce policies for
severity of vulnerabilities, image location, etc.
Grafeas backed storage
vulnerabilities, build info, etc.
Software Supply Chain with Grafeas & Kritis
CI/CD pipelines
Build &
Deploy
Secure
build
process
Automated
test, scan,
analysis
Deploy
checks
Centralized metadata
knowledge base
Kritis
Admission
controller
Deploy time policy chokepoint
Enforce policies for
severity of vulnerabilities, image location, etc.
Production
Grafeas backed storage
vulnerabilities, build info, etc.
Software Supply Chain with Grafeas & Kritis
CI/CD pipelines
Build &
Deploy
Secure
build
process
Automated
test, scan,
analysis
Deploy
checks
Grafeas backed storage
vulnerabilities, build info, etc.
Centralized metadata
knowledge base
Kritis
Admission
controller
Deploy time policy chokepoint
Enforce policies for
severity of vulnerabilities, image location, etc.
Production
Software Supply Chain Observability
CI/CD pipelines
Build &
Deploy
Secure
build
process
Automated
test, scan,
analysis
Deploy
checks
Grafeas backed storage
vulnerabilities, build info, etc.
Centralized metadata
knowledge base
Kritis
Admission
controller
Deploy time policy chokepoint
Enforce policies for
severity of vulnerabilities, image location, etc.
Production
Grafeas & Kritis
Binary
Authorization
Container Registry
Vulnerability
Scanning
Είναι όλα ελληνικά για μένα
[It's all Greek to me]
Grafeas & Kritis
JudgeScribe
In This Talk
Software
Supply Chain
Management
Kritis Grafeas Kritis &
Grafeas 0.1.0
1 2 3 4
In This Talk
Software
Supply Chain
Management
Kritis Grafeas Kritis &
Grafeas 0.1.0
1 2 3 4
Kritis
Code Checkin
Test &
Verification
Write code
Build Image
Deploy to
Production
QA
github.com/grafeas/kritis
Kritis:
Deploy-Time Policy Enforcer
Let's deploy our
e-commerce website...
Kritis: Admission Flow
$ kubectl apply site.yaml
Kritis: Admission Flow
kubectl
apply
site.yaml
Kritis: Admission Flow
k8s
kubectl
apply
site.yaml
Kritis: Admission Flow
k8sKritis
kubectl
apply
site.yaml
Kritis: Admission Flow
k8sKritis
kubectl
apply
site.yaml
$ helm install <path>/kritis-charts-0.1.0.tgz
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
Pod
spec
1. Admission
Request
Kritis
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
Oh no! Vulnerability scan
isn't finished...
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
4 a)
denied
4 a)
denied
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
4 a)
denied
4 a)
denied
Pod
Vulnerability scanning is
finished!
CVE-2019-5514 is found...
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
vuln
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
4 a)
denied
Pod
vuln
Whitelist CVE-2019-5514
because it doesn't affect
the website...
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
vuln
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
4 b) admitted4 b) admitted
Pod
vuln
It's time to scale up your site!
$ kubectl scale deployments/site --replicas=4
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
Pod PodPod Pod vuln
A new vulnerability is
found during scale up...
CVE-2019-9919
vuln
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
Pod PodPod Pod
CVE-2019-9919
Kritis attestations to the
rescue...
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
4 b) admitted4 b) admitted
Pod
vuln
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
4 b) admitted4 b) admitted
Pod
Attestor
Attestation
Authority CRD
vuln
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
4 b) admitted4 b) admitted
Pod
Attestor
Attestation
Authority CRD
5. Store attestations for
admitted images
vuln
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
vuln
4 b) admitted4 b) admitted
Pod
Attestor
Attestation
Authority CRD
5. Store attestations for
admitted images
attestation
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
vuln
4 b) admitted4 b) admitted
Pod
Attestor
Attestation
Authority CRD
5. Store attestations for
admitted images
attestation
Pod
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
vuln
4 b) admitted4 b) admitted
Pod
Attestor
Attestation
Authority CRD
5. Store attestations for
admitted images
attestation
Pod
CVE-2019-9919
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
vuln
4 b) admitted4 b) admitted
Pod
Attestor
Attestation
Authority CRD
5. Store attestations for
admitted images
attestation
Pod
CVE-2019-9919
6. Fetch
attestations
for admitted
image
Kritis: Admission Flow
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
vuln
4 b) admitted4 b) admitted
Pod
Attestor
Attestation
Authority CRD
5. Store attestations for
admitted images
attestation
Pod
CVE-2019-9919
6. Fetch
attestations
for admitted
image
Pod Pod
7. admitted
Discovering new
vulnerabilities in admitted
containers ...
Kritis: Background Cron
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
vuln
4 b) admitted4 b) admitted
Pod
Attestor
Attestation
Authority CRD
5. Store attestations for
admitted images
attestation
Pod
6. Fetch
attestations
for admitted
image
Pod Pod
7. admitted
Kritis: Background Cron
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
vuln
4 b) admitted4 b) admitted
Pod
Attestor
Attestation
Authority CRD
5. Store attestations for
admitted images
attestation
Pod
6. Fetch
attestations
for admitted
image
Pod Pod
Background
Cron
7. admitted
Kritis: Background Cron
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
vuln
4 b) admitted4 b) admitted
Pod
Attestor
Attestation
Authority CRD
5. Store attestations for
admitted images
attestation
Pod
6. Fetch
attestations
for admitted
image
Pod Pod
Background
Cron
7. admitted
Kritis: Background Cron
kubectl
apply
site.yaml
k8s
WebHook
Pod
spec
1. Admission
Request
Kritis
2. review
Policies
ns:prod
Image
Security
Policy
CRD
ns:qa
Image
Security
Policy
CRD
ns:prod
Image
Security
Policy
CRD
Image
Security
Validator
3. Fetch
metadata Grafeas
vuln
4 b) admitted4 b) admitted
Pod
Attestor
Attestation
Authority CRD
5. Store attestations for
admitted images
attestation
Pod
6. Fetch
attestations
for admitted
image
Pod Pod
Background
Cron
7. admitted
ImageSecurityPolicy CRD
apiVersion: kritis.grafeas.io/v1beta1
kind: ImageSecurityPolicy
metadata:
name: my-isp
spec:
imageWhitelist:
- gcr.io/kritis-int-test/nginx-digest-whitelist:latest
packageVulnerabilityRequirements:
maximumSeverity: MEDIUM
whitelistCVEs:
- providers/goog-vulnz/notes/CVE-2017-1000082
- providers/goog-vulnz/notes/CVE-2017-1000081
Observability with Kritis
Observability with Kritis
When did the image deploy?
Observability with Kritis
When did the image deploy?
When did the image pass policy checks?
Observability with Kritis
When did the image deploy?
When did the image pass policy checks?
When did the image stop satisfying policy?
Kritis
Open source, built with the community
Plugs into the k8s admission controller
Attest images and verify before deployment
Apply consistent deploy policy across k8s
environments
github.com/grafeas/kritis
kritis-users@googlegroups.com
In This Talk
Software
Supply Chain
Management
Kritis Grafeas Kritis &
Grafeas 0.1.0
1 2 3 4
In This Talk
Software
Supply Chain
Management
Kritis Grafeas Kritis &
Grafeas 0.1.0
1 2 3 4
Grafeas
Code Checkin
Test &
Verification
Write code
Build Image
Deploy to
Production
QA
github.com/grafeas/grafeas
Grafeas:
Artifact Metadata API
Grafeas:
Artifact Metadata API
= images, binaries, packages...
Grafeas:
Artifact Metadata API
= build, deployment, vulnerability, ...
Grafeas:
Artifact Metadata API
= store & retrieve metadata about artifacts
Grafeas: Terminology
● Notes: high-level description of types of metadata
○ e.g. Common Vulnerabilities and Exposures (CVE) as
Vulnerability Note
Grafeas: Terminology
● Notes: high-level description of types of metadata
○ e.g. Common Vulnerabilities and Exposures (CVE) as
Vulnerability Note
● Occurrences: instance of note in an artifact
○ e.g. CVE presence in an image
Grafeas: Terminology
● Notes: high-level description of types of metadata
○ e.g. Common Vulnerabilities and Exposures (CVE) as
Vulnerability Note
● Occurrences: instance of note in an artifact
○ e.g. CVE presence in an image
Grafeas: Terminology (cont'd)
● Resource URL: identifier for artifact in Occurrence
Grafeas: Terminology (cont'd)
● Resource URL: identifier for artifact in Occurrence
Grafeas: Terminology (cont'd)
● Resource URL: identifier for artifact in Occurrence
● Kind specific schemas
Grafeas: Deployment Note
// An artifact that can be deployed in some runtime.
message DeploymentNote {
// Required. Resource URI for the artifact being deployed.
repeated string resource_uri = 1;
}
Grafeas: Deployment Occurrence
// The period during which some deployable was active in a runtime.
message DeploymentOccurrence {
// Identity of the user that triggered this deployment.
string user_email = 1;
// Required. Beginning of the lifetime of this deployment.
google.protobuf.Timestamp deploy_time = 2;
// Output only. Resource URI for the artifact being deployed taken
from the deployable field with the same name.
repeated string resource_uri = 6;
...}
Observability
with Grafeas
Code Checkin
Test &
Verification
Write code
Build Image
Deploy to
Production
QA
Grafeas
Open artifact metadata standard with
contributions from the industry
Audit and govern your software supply chain
Knowledge base for on-premises and cloud
clusters
API with pluggable storage backendsgithub.com/grafeas/grafeas
grafeas-users@googlegroups.com
grafeas-dev@googlegroups.com
@Grafeasio
In This Talk
Software
Supply Chain
Management
Kritis Grafeas Kritis &
Grafeas 0.1.0
1 2 3 4
In This Talk
Software
Supply Chain
Management
Kritis Grafeas Kritis &
Grafeas 0.1.0
1 2 3 4
Coming soon... 0.1.0
Goals
Enable users to start experimenting with Kritis and Grafeas
Move towards hybrid-cloud support
Gather community feedback
0.1.0
0.1.0
Scope
Standalone Kritis on Kubernetes with standalone Grafeas
0.1.0
User Journeys
Allow deployment of a container to Kubernetes cluster
Block deployment of a unadmitted container to the cluster
● Grafeas:
○ Helm chart for Grafeas & published image
○ Standalone Grafeas server with Postgres storage backend
○ Basic support for Go client library
Features
0.1.0
● Grafeas:
○ Helm chart for Grafeas & published image
○ Standalone Grafeas server with Postgres storage backend
○ Basic support for Go client library
● Kritis:
○ GenericAttestationPolicy
○ Default admittance fallback policy is well-defined
○ Configurable
Features
0.1.0
Learn more and follow along!
github.com/grafeas/{grafeas,kritis}
Google Groups: {grafeas,kritis}-users, grafeas-dev
@grafeasio
0.1.0
#talk-aysylu-greenberg

Software Supply Chain Observability with Grafeas and Kritis