Submit Search
Upload
Kafka Security
•
Download as PPTX, PDF
•
5 likes
•
2,290 views
S
Sriharsha Chintalapani
Follow
In this talk we cover Kafka Security coming up in 0.9 release. SSL, SASL & Authorizer
Read less
Read more
Software
Report
Share
Report
Share
1 of 45
Download now
Recommended
Apache Kafka Security
Apache Kafka Security
DataWorks Summit/Hadoop Summit
Securing Kafka
Securing Kafka
confluent
Apache Knox setup and hive and hdfs Access using KNOX
Apache Knox setup and hive and hdfs Access using KNOX
Abhishek Mallick
Kafka security ssl
Kafka security ssl
Heng-Xiu Xu
The Easiest Way to Configure Security for Clients AND Servers (Dani Traphagen...
The Easiest Way to Configure Security for Clients AND Servers (Dani Traphagen...
confluent
Hadoop Security Now and Future
Hadoop Security Now and Future
tcloudcomputing-tw
Oracle Enterprise Manager - EM12c R5 Hybrid Cloud Management
Oracle Enterprise Manager - EM12c R5 Hybrid Cloud Management
MarketingArrowECS_CZ
Introduction to apache kafka
Introduction to apache kafka
Samuel Kerrien
Recommended
Apache Kafka Security
Apache Kafka Security
DataWorks Summit/Hadoop Summit
Securing Kafka
Securing Kafka
confluent
Apache Knox setup and hive and hdfs Access using KNOX
Apache Knox setup and hive and hdfs Access using KNOX
Abhishek Mallick
Kafka security ssl
Kafka security ssl
Heng-Xiu Xu
The Easiest Way to Configure Security for Clients AND Servers (Dani Traphagen...
The Easiest Way to Configure Security for Clients AND Servers (Dani Traphagen...
confluent
Hadoop Security Now and Future
Hadoop Security Now and Future
tcloudcomputing-tw
Oracle Enterprise Manager - EM12c R5 Hybrid Cloud Management
Oracle Enterprise Manager - EM12c R5 Hybrid Cloud Management
MarketingArrowECS_CZ
Introduction to apache kafka
Introduction to apache kafka
Samuel Kerrien
Introduction to Apache Kafka
Introduction to Apache Kafka
Jeff Holoman
intro-kafka
intro-kafka
Rahul Shukla
Introduction to Apache Kafka
Introduction to Apache Kafka
Shiao-An Yuan
AWS network services
AWS network services
Nagesh Ramamoorthy
VPC Fundamentals & Connectivity - Pop-up Loft Tel Aviv
VPC Fundamentals & Connectivity - Pop-up Loft Tel Aviv
Amazon Web Services
OpenStack- A ringside view of Services and Architecture
OpenStack- A ringside view of Services and Architecture
Ritesh Somani
Introduction to Apache CloudStack by David Nalley
Introduction to Apache CloudStack by David Nalley
buildacloud
Webinar slides: How to deploy and manage HAProxy, MaxScale or ProxySQL with C...
Webinar slides: How to deploy and manage HAProxy, MaxScale or ProxySQL with C...
Severalnines
Apache Kafka Introduction
Apache Kafka Introduction
Amita Mirajkar
Cloud and OpenStack
Cloud and OpenStack
Seyed Ehsan Beheshtian
Sharing is Caring: Toward Creating Self-tuning Multi-tenant Kafka (Anna Povzn...
Sharing is Caring: Toward Creating Self-tuning Multi-tenant Kafka (Anna Povzn...
HostedbyConfluent
Apache Kafka
Apache Kafka
Joe Stein
Reducing Microservice Complexity with Kafka and Reactive Streams
Reducing Microservice Complexity with Kafka and Reactive Streams
jimriecken
5 steps to take setting up a streamlined container pipeline
5 steps to take setting up a streamlined container pipeline
Michel Schildmeijer
Apache kafka
Apache kafka
NexThoughts Technologies
Intro to Apache Kafka
Intro to Apache Kafka
Jason Hubbard
Building Event-Driven Systems with Apache Kafka
Building Event-Driven Systems with Apache Kafka
Brian Ritchie
Microservices with Node.js and Apache Cassandra
Microservices with Node.js and Apache Cassandra
Jorge Bay Gondra
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
Guozhang Wang
(Re)Indexing Large Repositories in Alfresco
(Re)Indexing Large Repositories in Alfresco
Angel Borroy López
Cassandra and security
Cassandra and security
Ben Bromhead
Sqoop2 refactoring for generic data transfer - Hadoop Strata Sqoop Meetup
Sqoop2 refactoring for generic data transfer - Hadoop Strata Sqoop Meetup
aaamase
More Related Content
What's hot
Introduction to Apache Kafka
Introduction to Apache Kafka
Jeff Holoman
intro-kafka
intro-kafka
Rahul Shukla
Introduction to Apache Kafka
Introduction to Apache Kafka
Shiao-An Yuan
AWS network services
AWS network services
Nagesh Ramamoorthy
VPC Fundamentals & Connectivity - Pop-up Loft Tel Aviv
VPC Fundamentals & Connectivity - Pop-up Loft Tel Aviv
Amazon Web Services
OpenStack- A ringside view of Services and Architecture
OpenStack- A ringside view of Services and Architecture
Ritesh Somani
Introduction to Apache CloudStack by David Nalley
Introduction to Apache CloudStack by David Nalley
buildacloud
Webinar slides: How to deploy and manage HAProxy, MaxScale or ProxySQL with C...
Webinar slides: How to deploy and manage HAProxy, MaxScale or ProxySQL with C...
Severalnines
Apache Kafka Introduction
Apache Kafka Introduction
Amita Mirajkar
Cloud and OpenStack
Cloud and OpenStack
Seyed Ehsan Beheshtian
Sharing is Caring: Toward Creating Self-tuning Multi-tenant Kafka (Anna Povzn...
Sharing is Caring: Toward Creating Self-tuning Multi-tenant Kafka (Anna Povzn...
HostedbyConfluent
Apache Kafka
Apache Kafka
Joe Stein
Reducing Microservice Complexity with Kafka and Reactive Streams
Reducing Microservice Complexity with Kafka and Reactive Streams
jimriecken
5 steps to take setting up a streamlined container pipeline
5 steps to take setting up a streamlined container pipeline
Michel Schildmeijer
Apache kafka
Apache kafka
NexThoughts Technologies
Intro to Apache Kafka
Intro to Apache Kafka
Jason Hubbard
Building Event-Driven Systems with Apache Kafka
Building Event-Driven Systems with Apache Kafka
Brian Ritchie
Microservices with Node.js and Apache Cassandra
Microservices with Node.js and Apache Cassandra
Jorge Bay Gondra
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
Guozhang Wang
(Re)Indexing Large Repositories in Alfresco
(Re)Indexing Large Repositories in Alfresco
Angel Borroy López
What's hot
(20)
Introduction to Apache Kafka
Introduction to Apache Kafka
intro-kafka
intro-kafka
Introduction to Apache Kafka
Introduction to Apache Kafka
AWS network services
AWS network services
VPC Fundamentals & Connectivity - Pop-up Loft Tel Aviv
VPC Fundamentals & Connectivity - Pop-up Loft Tel Aviv
OpenStack- A ringside view of Services and Architecture
OpenStack- A ringside view of Services and Architecture
Introduction to Apache CloudStack by David Nalley
Introduction to Apache CloudStack by David Nalley
Webinar slides: How to deploy and manage HAProxy, MaxScale or ProxySQL with C...
Webinar slides: How to deploy and manage HAProxy, MaxScale or ProxySQL with C...
Apache Kafka Introduction
Apache Kafka Introduction
Cloud and OpenStack
Cloud and OpenStack
Sharing is Caring: Toward Creating Self-tuning Multi-tenant Kafka (Anna Povzn...
Sharing is Caring: Toward Creating Self-tuning Multi-tenant Kafka (Anna Povzn...
Apache Kafka
Apache Kafka
Reducing Microservice Complexity with Kafka and Reactive Streams
Reducing Microservice Complexity with Kafka and Reactive Streams
5 steps to take setting up a streamlined container pipeline
5 steps to take setting up a streamlined container pipeline
Apache kafka
Apache kafka
Intro to Apache Kafka
Intro to Apache Kafka
Building Event-Driven Systems with Apache Kafka
Building Event-Driven Systems with Apache Kafka
Microservices with Node.js and Apache Cassandra
Microservices with Node.js and Apache Cassandra
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
(Re)Indexing Large Repositories in Alfresco
(Re)Indexing Large Repositories in Alfresco
Viewers also liked
Cassandra and security
Cassandra and security
Ben Bromhead
Sqoop2 refactoring for generic data transfer - Hadoop Strata Sqoop Meetup
Sqoop2 refactoring for generic data transfer - Hadoop Strata Sqoop Meetup
aaamase
Securing Hadoop's REST APIs with Apache Knox Gateway Hadoop Summit June 6th, ...
Securing Hadoop's REST APIs with Apache Knox Gateway Hadoop Summit June 6th, ...
Kevin Minder
Apache Ranger
Apache Ranger
Rommel Garcia
Apache Zookeeper 分布式服务框架
Apache Zookeeper 分布式服务框架
Cabin WJ
Securing Hadoop with Apache Ranger
Securing Hadoop with Apache Ranger
DataWorks Summit
Apache Sqoop: A Data Transfer Tool for Hadoop
Apache Sqoop: A Data Transfer Tool for Hadoop
Cloudera, Inc.
Opaque: A Data Analytics Platform with Strong Security: Spark Summit East tal...
Opaque: A Data Analytics Platform with Strong Security: Spark Summit East tal...
Spark Summit
Introduction to Apache Sqoop
Introduction to Apache Sqoop
Avkash Chauhan
Spark Security
Spark Security
Yifeng Jiang
Secure Kafka at Salesforce.com
Secure Kafka at Salesforce.com
Rajasekar Elango
From oracle to hadoop with Sqoop and other tools
From oracle to hadoop with Sqoop and other tools
Guy Harrison
Build Features, Not Apps
Build Features, Not Apps
Natasha Murashev
How to Become a Thought Leader in Your Niche
How to Become a Thought Leader in Your Niche
Leslie Samuel
Viewers also liked
(14)
Cassandra and security
Cassandra and security
Sqoop2 refactoring for generic data transfer - Hadoop Strata Sqoop Meetup
Sqoop2 refactoring for generic data transfer - Hadoop Strata Sqoop Meetup
Securing Hadoop's REST APIs with Apache Knox Gateway Hadoop Summit June 6th, ...
Securing Hadoop's REST APIs with Apache Knox Gateway Hadoop Summit June 6th, ...
Apache Ranger
Apache Ranger
Apache Zookeeper 分布式服务框架
Apache Zookeeper 分布式服务框架
Securing Hadoop with Apache Ranger
Securing Hadoop with Apache Ranger
Apache Sqoop: A Data Transfer Tool for Hadoop
Apache Sqoop: A Data Transfer Tool for Hadoop
Opaque: A Data Analytics Platform with Strong Security: Spark Summit East tal...
Opaque: A Data Analytics Platform with Strong Security: Spark Summit East tal...
Introduction to Apache Sqoop
Introduction to Apache Sqoop
Spark Security
Spark Security
Secure Kafka at Salesforce.com
Secure Kafka at Salesforce.com
From oracle to hadoop with Sqoop and other tools
From oracle to hadoop with Sqoop and other tools
Build Features, Not Apps
Build Features, Not Apps
How to Become a Thought Leader in Your Niche
How to Become a Thought Leader in Your Niche
Similar to Kafka Security
Kafka Security
Kafka Security
DataWorks Summit/Hadoop Summit
Kafka 2018 - Securing Kafka the Right Way
Kafka 2018 - Securing Kafka the Right Way
Saylor Twift
Visualizing Kafka Security
Visualizing Kafka Security
DataWorks Summit
Protecting your data at rest with Apache Kafka by Confluent and Vormetric
Protecting your data at rest with Apache Kafka by Confluent and Vormetric
confluent
How to Lock Down Apache Kafka and Keep Your Streams Safe
How to Lock Down Apache Kafka and Keep Your Streams Safe
confluent
Webservice security considerations and measures
Webservice security considerations and measures
Maarten Smeets
Securing Spark Applications
Securing Spark Applications
DataWorks Summit/Hadoop Summit
Adopting Modern SSL / TLS
Adopting Modern SSL / TLS
Avi Networks
Accumulo Summit 2014: Monitoring Apache Accumulo
Accumulo Summit 2014: Monitoring Apache Accumulo
Accumulo Summit
Hadoop REST API Security with Apache Knox Gateway
Hadoop REST API Security with Apache Knox Gateway
DataWorks Summit
Exploring Advanced Authentication Methods in Novell Access Manager
Exploring Advanced Authentication Methods in Novell Access Manager
Novell
Paris FOD meetup - kafka security 101
Paris FOD meetup - kafka security 101
Abdelkrim Hadjidj
Citrix Day 2014: NetScaler 10.5
Citrix Day 2014: NetScaler 10.5
Digicomp Academy AG
MySQL 5.7 + Java
MySQL 5.7 + Java
Mark Swarbrick
NetScaler 11 Update
NetScaler 11 Update
MarketingArrowECS_CZ
Kafka Security 101 and Real-World Tips
Kafka Security 101 and Real-World Tips
confluent
D@W REST security
D@W REST security
Gaurav Sharma
Deploying Next Generation Firewalling with ASA - CX
Deploying Next Generation Firewalling with ASA - CX
Cisco Canada
DockerCon Live 2020 - Securing Your Containerized Application with NGINX
DockerCon Live 2020 - Securing Your Containerized Application with NGINX
Kevin Jones
Vital Aspects of SSL Support in MySQL
Vital Aspects of SSL Support in MySQL
Lesa Cote
Similar to Kafka Security
(20)
Kafka Security
Kafka Security
Kafka 2018 - Securing Kafka the Right Way
Kafka 2018 - Securing Kafka the Right Way
Visualizing Kafka Security
Visualizing Kafka Security
Protecting your data at rest with Apache Kafka by Confluent and Vormetric
Protecting your data at rest with Apache Kafka by Confluent and Vormetric
How to Lock Down Apache Kafka and Keep Your Streams Safe
How to Lock Down Apache Kafka and Keep Your Streams Safe
Webservice security considerations and measures
Webservice security considerations and measures
Securing Spark Applications
Securing Spark Applications
Adopting Modern SSL / TLS
Adopting Modern SSL / TLS
Accumulo Summit 2014: Monitoring Apache Accumulo
Accumulo Summit 2014: Monitoring Apache Accumulo
Hadoop REST API Security with Apache Knox Gateway
Hadoop REST API Security with Apache Knox Gateway
Exploring Advanced Authentication Methods in Novell Access Manager
Exploring Advanced Authentication Methods in Novell Access Manager
Paris FOD meetup - kafka security 101
Paris FOD meetup - kafka security 101
Citrix Day 2014: NetScaler 10.5
Citrix Day 2014: NetScaler 10.5
MySQL 5.7 + Java
MySQL 5.7 + Java
NetScaler 11 Update
NetScaler 11 Update
Kafka Security 101 and Real-World Tips
Kafka Security 101 and Real-World Tips
D@W REST security
D@W REST security
Deploying Next Generation Firewalling with ASA - CX
Deploying Next Generation Firewalling with ASA - CX
DockerCon Live 2020 - Securing Your Containerized Application with NGINX
DockerCon Live 2020 - Securing Your Containerized Application with NGINX
Vital Aspects of SSL Support in MySQL
Vital Aspects of SSL Support in MySQL
Recently uploaded
Software Quality Assurance Interview Questions
Software Quality Assurance Interview Questions
Arshad QA
Active Directory Penetration Testing, cionsystems.com.pdf
Active Directory Penetration Testing, cionsystems.com.pdf
Cionsystems
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽❤️🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽❤️🧑🏻 89...
gurkirankumar98700
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
MyIntelliSource, Inc.
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
kalichargn70th171
Unlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language Models
aagamshah0812
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
ICS
Professional Resume Template for Software Developers
Professional Resume Template for Software Developers
Vinodh Ram
Test Automation Strategy for Frontend and Backend
Test Automation Strategy for Frontend and Backend
Arshad QA
How To Use Server-Side Rendering with Nuxt.js
How To Use Server-Side Rendering with Nuxt.js
Andolasoft Inc
DNT_Corporate presentation know about us
DNT_Corporate presentation know about us
Dynamic Netsoft
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
soniya singh
Cloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStack
VICTOR MAESTRE RAMIREZ
Optimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTV
shikhaohhpro
5 Signs You Need a Fashion PLM Software.pdf
5 Signs You Need a Fashion PLM Software.pdf
Wave PLM
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
kalichargn70th171
Diamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with Precision
SolGuruz
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdf
joe51371421
The Ultimate Test Automation Guide_ Best Practices and Tips.pdf
The Ultimate Test Automation Guide_ Best Practices and Tips.pdf
kalichargn70th171
Project Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanation
kaushalgiri8080
Recently uploaded
(20)
Software Quality Assurance Interview Questions
Software Quality Assurance Interview Questions
Active Directory Penetration Testing, cionsystems.com.pdf
Active Directory Penetration Testing, cionsystems.com.pdf
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽❤️🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽❤️🧑🏻 89...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
Unlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language Models
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
Professional Resume Template for Software Developers
Professional Resume Template for Software Developers
Test Automation Strategy for Frontend and Backend
Test Automation Strategy for Frontend and Backend
How To Use Server-Side Rendering with Nuxt.js
How To Use Server-Side Rendering with Nuxt.js
DNT_Corporate presentation know about us
DNT_Corporate presentation know about us
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Cloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStack
Optimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTV
5 Signs You Need a Fashion PLM Software.pdf
5 Signs You Need a Fashion PLM Software.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Diamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with Precision
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdf
The Ultimate Test Automation Guide_ Best Practices and Tips.pdf
The Ultimate Test Automation Guide_ Best Practices and Tips.pdf
Project Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanation
Kafka Security
1.
Page1 © Hortonworks
Inc. 2014 Kafka Security SSL, Kerberos & Authorization
2.
Page2 © Hortonworks
Inc. 2014
3.
Page3 © Hortonworks
Inc. 2014 Who Are We? Sriharsha Chintalapani Apache Kafka Committer Apache Storm Committer & PMC Parth Brahmbhatt Apache Kafka Contributor Apache Storm Committer & PMC
4.
Page4 © Hortonworks
Inc. 2014 Kafka Security • SSL ( wire encryption) • SASL ( Kerberos ) • Authorizer (Topic/Host/User level Authorization)
5.
Page5 © Hortonworks
Inc. 2014 SSL
6.
Page6 © Hortonworks
Inc. 2014 Kafka Security – SSL • Kafka networking • A TCP server listening for incoming connections • Uses Non-blocking network I/O • When a client connects to a server it opens a socket channel on server side and hands it over selector. • Selector gets polled in a loop. It will wake up whenever there are connections ready with data to be read or write. • Long – living connections , once established it will be used to read/write data until client closed or an exception occurs.
7.
Page7 © Hortonworks
Inc. 2014 Kafka Security – SSL • Kafka networking
8.
Page8 © Hortonworks
Inc. 2014 Kafka Security – SSL • Kafka SSL / SASL requirements • No User-level API changes to clients • Retain length-encoded Kafka protocols • Client must authenticate before sending/receiving requests • Kafka Channel • Instead of using socket channel, we added KafkaChannel which consists a TransportLayer, Authenticator.
9.
Page9 © Hortonworks
Inc. 2014 Kafka Security – SSL • TransportLayer • Handles network level byte transfers • PlaintextTransportLayer • SSLTransportLayer • Authenticator • A pluggable interface for authentication implementations • SaslAuthenticator – Provides SASL handshake and authenticated user.
10.
Page10 © Hortonworks
Inc. 2014 Kafka Security – SSL KafkaChannel TransportLayer Authenticator Kafka Server handshake authenticate
11.
Page11 © Hortonworks
Inc. 2014 Kafka Security – SSL • SSL - Handshake • Kafka Server configures with Keystore and Truststore • Kafka Client also needs a truststore with Kafka Server certificate added to the truststore. • Keystore configuration on client side is optional unless user wants client side authentication.
12.
Page12 © Hortonworks
Inc. 2014 Kafka Security – SSL • KafkaChannel • Before write or read application data , checks if the channel.ready() • A channel is ready if its established a connection and authenticated. No-OP of PlaintextTransportLayer • If a channel is not ready it goes through channel.prepare() which internally calls transportLayer.handshake()
13.
Page13 © Hortonworks
Inc. 2014 Kafka Security – SSL • SSLTransportLayer • Before sending any application data, both client and server needs to go though SSL handshake • SSLTransportLayer uses SSLEngine to establish a non- blocking handshake. • SSLEngine provides a state machine to go through several steps of SSLhandshake
14.
Page14 © Hortonworks
Inc. 2014 Kafka Security – SSL
15.
Page15 © Hortonworks
Inc. 2014 Kafka Security – SSL • SSLTransportLayer • SocketChannel read • Returns encrypted data • Decrypts the data and returns the length of the data from Kafka protocols • SocketChannel Write • Writes encrypted data onto channel • Regular socketChannel returns length of the data written to socket. • Incase of SSL since we encrypt the data we can’t return exact length written to socket which will be more than actual data • Its important to keep track length of data written to network. This signifies if we successfully written data to the network or not and move on to next request.
16.
Page16 © Hortonworks
Inc. 2014 Kafka Security – SSL • Principal Builder • SSLTransportLayer gives hostname as authenticated user • X509Certificate has lot more information about a client identity. • PrincipalBuilder provides interface to plug in a custom PrincipalBuilder that has access to X509Certificate and can construct a user string out of it. • Authenticator can use this custom principal to add ACLs
17.
Page17 © Hortonworks
Inc. 2014 Kafka Security – SSL • Performance Impact • Decrease in throughput by 20%. • Latency increased by 30 % • KAFKA-2481 (Ben Stopford) has more details
18.
Page18 © Hortonworks
Inc. 2014 Kafka Security – SSL • listeners=SSL://host.name:port • ssl.keystore.location • ssl.keystore.password • ssl.key.password • ssl.truststore.location • ssl.truststore.password • security.inter.broker.protocol (optional)
19.
Page19 © Hortonworks
Inc. 2014 SASL/ Kerberos
20.
Page20 © Hortonworks
Inc. 2014 Kafka Security – SASL • Simple Authentication and Security Layer, or SASL • Provides flexibility in using Login Mechanisms • One can use Kerberos , LDAP or simple passwords to authenticate. • JAAS Login • Before client & server can handshake , they need to authenticate with Kerberos or other Identity Provider. • JAAS provides a pluggable way of providing user credentials. One can easily add LDAP or other mechanism just by changing a config file.
21.
Page21 © Hortonworks
Inc. 2014 Kafka Security – SASL • JAAS Config file KafkaServer { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true serviceName="kafka" keyTab="/vagrant/keytabs/kafka1.keytab" principal="kafka/host@EXAMPLE.COM"; }; KafkaConfig { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true serviceName="kafka" keyTab="/vagrant/keytabs/client1.keytab" principal=”client/host@EXAMPLE.COM"; };
22.
Page22 © Hortonworks
Inc. 2014 Kafka Security – SASL • SASL Authenticator • Uses configured login credentials of JAAS config. • Non-blocking handshake to establish clients identity • Once handshake established , Kerberos principal name will be the authenticated user. • Can be layered with SSL for wire encryption or Plaintext incase of wire encryption not needed. • SASL can provide encryption but it has huge performance penalties
23.
Page23 © Hortonworks
Inc. 2014 Kafka Security – SASL Client Broker Connection Mechanism list Selected Mechanism & sasl data Evaluate and Response Sasl data Client Authenticated
24.
Page24 © Hortonworks
Inc. 2014 Kafka Security – SASL • Pass JAAS config file as jvm parameter • -Djava.security.auth.login.config
25.
Page25 © Hortonworks
Inc. 2014 Kafka Security – Resources • SSL • https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka • SASL • https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61326390 • Vagrant Setup • SASL • https://github.com/harshach/kafka-vagrant/tree/master/
26.
Page26 © Hortonworks
Inc. 2014 Authorization
27.
Page27 © Hortonworks
Inc. 2014 Authorizer • Controls who can do what • Pluggable • Acl based approach
28.
Page28 © Hortonworks
Inc. 2014 Acl • Alice is Allowed to Read from Orders-topic from Host-1 Principal Permission Operation Resource Host Alice Allow Read Orders Host-1
29.
Page29 © Hortonworks
Inc. 2014 Principal • PrincipalType:Name • Supported types: User and Group • Extensible so users can add their own types • Wild Card User:*
30.
Page30 © Hortonworks
Inc. 2014 Operation • Read, Write, Create, Delete, Alter, Describe, ClusterAction, All • Each API as an Operation VS Classification that maps to APIs.
31.
Page31 © Hortonworks
Inc. 2014 Resource • ResourceType:ResourceName • Topic, Cluster and ConsumerGroup • Wild card resource ResourceType:*
32.
Page32 © Hortonworks
Inc. 2014 Permissions • Allow and Deny • Anyone without an explicit Allow ACL is denied • Then why do we have Deny? • Deny works as negation • Deny takes precedence over Allow Acls
33.
Page33 © Hortonworks
Inc. 2014 Hosts • Why provide this granularity? • Allows authorizer to provide firewall type security even in non secure environment. • * as Wild card.
34.
Page34 © Hortonworks
Inc. 2014 Configuration • Authorizer class • Super users • Authorizer properties • Default behavior for resources with no ACLs
35.
Page35 © Hortonworks
Inc. 2014 SimpleAclAuthorizer • Out of box authorizer implementation. • Stores all of its ACLs in zookeeper. • In built ACL cache to avoid performance penalty. • Provides authorizer audit log.
36.
Page38 © Hortonworks
Inc. 2014 CLI • Add, Remove and List acls • Convenience options: --producer and --consumer.
37.
Page39 © Hortonworks
Inc. 2014 Ranger Policy
38.
Page40 © Hortonworks
Inc. 2014 Ranger Auditing
39.
Page41 © Hortonworks
Inc. 2014 Ranger ACL management Audit
40.
Page42 © Hortonworks
Inc. 2014 Unsecure zookeeper
41.
Page43 © Hortonworks
Inc. 2014 Zookeeper • Kafka’s metadata store • Has its own security mechanism that supports SASL and MD5-DIGEST for establishing identity and ACL based authorization • Create , Delete directly interacts with zookeeper
42.
Page44 © Hortonworks
Inc. 2014 Securing zookeeper • Acl on zk nodes: user:cdrwa • Zookeeper.set.acl • ZkSecurityMigrator script • Credit where its due: Flavio Junqueira
43.
Page45 © Hortonworks
Inc. 2014 Client JAAS Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true serviceName="zookeeper" keyTab="/vagrant/keytabs/kafka.keytab" principal="kafka/kafka@WITZEND.COM"; };
44.
Page46 © Hortonworks
Inc. 2014 Future • KIP-4: Move everything to server side, no direct interactions with zookeeper • Group Support (PR already available) • Pluggable Auditor
45.
Page47 © Hortonworks
Inc. 2014 Summary • SSL for wire encryption • Sasl for authentication • Authorization • Secure Zookeeper Thanks to the community for participation.
Download now