Solving Trust
Issues At Scale
Building an Authorization
System that Devs Don’t Hate
@omerlh
@omerlh
@omerlh
Incident Response
@omerlh
Lesson Learned
• Automatic Detection (GitGurdian)
• Harden Authorization System
• Better audit logs
• Who used this token?
• Who created it and why?
@omerlh
I’m a builder
@omerlh
DevSecOps @
@omerlh
@omerlh
Registration
Authentication
Analytics
Messages
Content
Notifications
Users API
@omerlh
The Authorization
Problem
@omerlh
The Fort Approach
@omerlh
Registration
Authentication
Analytics
Messages
Content
Notifications
Users API
@omerlh
One horse can take down our fort...
@omerlh
Let’s talk about blast radius
• One vulnerable service affects all the other
• Ensuring all services are trust worthy is
impossible
• We must narrow the blast radius
@omerlh
A Zero-Trust Approach
@omerlh
Registration
Authentication
Analytics
Messages
Content
Notifications
Users API
@omerlh
The Ideal System
Least Privilege
• Just the permissions required
Secure by Default
• Zero permissions by default
Self Service
• Minimize the impact on development speed
Scale
• Hundred of micro-services to manage
@omerlh
@omerlh
Let’s Build it!
@omerlh
High Level Overview
AppIncoming request
Collect decision logs
Get bundlesPublish bundle
Is Authorized?
@omerlh
Zooming In
App
@omerlh
Envoy Proxy
• A CNCF project
• Authentication and Authorization filters
• Run as side car on the pod
• No code changes required
@omerlh
@omerlh
@omerlh
@omerlh
@omerlh
Meet Open Policy Agent
• An open source policy system
• Author policies using Rego DSL
• Built-in Envoy integration
• A CNCF project
@omerlh
Demo Time!
@omerlh
The Candies Problem
@omerlh
Envoy + OPA = KISS
@omerlh
@omerlh
@omerlh
@omerlh
@omerlh
@omerlh
Done?
Least Privilege
• Just the permissions required
Secure by Default
• Zero permissions by default
Self Service
• Minimize the impact on development speed
Scale
• Hundred of micro-services to manage
@omerlh
Wait, all devs need to learn Rego?
@omerlh
Using data for abstraction
• Policies can load data from JSON/YAML
• Author generic policies
• Devs just needs to fill the input
@omerlh
@omerlh
@omerlh
@omerlh
Building our policies
• A common policy, maintained by the security team
• Including unit tests!
• Each service has it’s own data file
• Permission request is just a PR
• Built in audit and review features
@omerlh
@omerlh
An Example PR
@omerlh
Permissions Request PR
Done?
Least Privilege
• Just the permissions required
Secure by Default
• Zero permissions by default
Self Service
• Minimize the impact on development speed
Scale
• Hundred of micro-services to manage
@omerlh
Loading Policies
• Bundles are policies archive
• Hot bundle loading
• Dynamic bundle discovery
@omerlh
@omerlh
@omerlh
@omerlh
@omerlh
@omerlh
@omerlh
@omerlh
@omerlh
What about logging?
• OPA log each decision into decision logs
• Can be used for debug and audit purposes
• Support redacting for sensitive input
• Collect with fluentd/Loki
@omerlh
@omerlh
Putting it all together
AppIncoming request
Collect decision logs
Get bundlesPublish bundle
Is Authorized?
@omerlh
Done?
Least Privilege
• Just the permissions required
Secure by Default
• Zero permissions by default
Self Service
Minimize the impact on development speed
Scale
• Hundred of micro-services to manage
@omerlh
Everything come with a price…
• OPA/Envoy running as side-car
• Resource usage costs per pod
• A bug in OPA/Envoy has serious implication
@omerlh
How can I build it?
• Fork the GitHub repo - https://github.com/omerlh/opa-demo
• Follow the readme
• Reach out with questions 
@omerlh
Wrapping Up
@omerlh
Wrapping Up
@omerlh
Registration
Authentication
Analytics
Messages
Content
Notifications
Users API
@omerlh
Going Forward
• Improved Visibility
• More Complex Policies
• Expend adoption
@omerlh
Lesson Learned
• Automatic Detection (GitGurdian)
• Harden Authorization System
• Better audit logs
• Who used this token?
• Who created it and why?
@omerlh
Feedback appreciated
@omerlh
Questions?
@omerlh
@omerlh
Enable Devs to Move Faster and Safer
Thank You
@omerlh
Resources
• OPA-Envoy integration
• Envoy JWT authentication
• GitHub demo repo
@omerlh

Solving trust issues at scale - Omer Levi Hevroni