At Viasat, Kafka is a backbone for a multi-tenant streaming platform that transports data for 1000 streams and used by more than 60 teams in a production environment. Role-based access control to the sensitive data is an essential requirement for our customers who must comply with a variety of regulations including GDPR. Kafka ships with a pluggable Authorizer that can control access to resources like cluster, topic or consumer group. However, maintaining ACLs in the large multi-tenant deployment can be support-intensive. At Viasat, we developed a custom Kafka Authorizer and Role Manager application that integrates our Kafka cluster with Viasat’s internal LDAP services. The presentation will cover how we designed and built Kafka LDAP Authorizer, which allows us to control resources within the cluster as well as services built around Kafka. We apply our permissions model to our data forwarders, ETL jobs, and stream processing. We will also share how we achieved a stress free migration to secure infrastructure without interruption to the production data flow. Our secure deployment model accomplishes multiple goals: – Integration into an LDAP central authentication system. – Use of the same authorization service to control permissions to data in Kafka as well as services built around Kafka. – Delegation of permissions control to the security officers on the teams using the service. – Detailed audit and breach notifications based on the metrics produced by the custom authorizer. We plan to open source our custom Kafka Authorizer.

1. Kafka Pluggable Authorizer for
Enterprise Security
Anna Kepler
Data Engineer

4. Data Security at Scale is Hard

5. Databus Streaming Platform

6. Shifting Objective Over Time
Data Democratization
Fast Customer Onboarding
Self-Service
High Volume Stream Processing
2014
4 teams
60 streams

7. Shifting Objective Over Time
Data Security
Data Governance
Accountability2019
50+ teams
1,000+ streams

8. Default Kafka® Authorization
authorizer.class.name = kafka.security.auth.SimpleAclAuthorizer
allow.everyone.if.no.acl.found=true
super.users=User:Bob;User:Alice
bin/kafka-acls
--add --allow-principal User:Bob
--producer --topic test-topic

9. Databus Kafka Authorization
authorizer.class.name = com.viasat.databus.DatabusKafkaAuthorizer
role.manager.url = https://roles.visat.io
permissions.expiration.sec = 60

10. Role Manager Service
Endpoints
/tenancies
/resources
/subjects
/capabilities

11. Role Manager cli
COMMANDS:
version Get version information
token Get a JWT for authentication
tenancy Interact with tenancies
resource Interact with resources
capability Interact with capabilities
subject Interact with subjects
help, h Shows a list of commands
role capability list –r stream:my-stream

12. Working with Role Manager
{
"id": "tenancy:team-awesome",
"groups": [
{
"stripe": ”team-awesome",
"group": ”team-awesome-admins",
"capabilities":
[ "read", "write", "describe", "modify", "delete" ]
}
]
}

13. Working with Role Manager
{
"id": "tenancy:team-awesome",
"groups": [
{
"stripe": "team-awesome",
"group": "team-awesome-readers",
"capabilities": [ ”read", "describe" ]
}
]
}

14. Granular Permissions
{
"fromSubjectId": "Bob”,
"toResourceId": "stream:shared-stream",
"action": "read"
}
# With the cli
role capability create capability.json

15. Why do it
Integration into Central Authentication System
Delegation of controls to team admins
REST API used by various components in the platform
In-depth monitoring

16. Thank you
Anna Kepler
Data Engineer, Viasat
https://www.linkedin.com/in/akepler
https://github.com/Viasat
https://careers.viasat.com/