關於 Kafka security 設置的記錄檔案，裡面包含了，安全機制 one-way + two-way 的觀念提供，以及在 kafka properties 中應該要有的安全設定，最後使用 kafka-producer + kafka-consumer 去驗證看看 SSL 機制是否成功，裡面需要注意的是，保存好自己的 jks 。

1. KAFKA-SECURITY
微光國際
1

2. KAFKA-SECURITY
投影⽚進程
▸ SSL 說明
▸ Kafka 實作 SSL (Two-way)
▸ 測試 SSL
2

3. KAFKA-SECURITY
SSL 說明
▸ One-Way
▸ Two-Way
▸ Tips
3

4. SSL 說明
ONE-WAY
4

5. ⽂字
ONE-WAY DIAGRAM
5

6. SSL 說明
TWO-WAY
6

7. SSL 說明
TWO-WAY DIAGRAM
7

8. SSL 說明
TIPS
▸ 兩兩者在 Kafka 設定上只差在 Client 有沒有加上 KeyStore ⽽而已
▸ 要做到 Authentication 就在設定中加入
8

9. KAFKA-SECURITY
KAFKA 實作 SSL(TWO-WAY)
▸ 1. 建置 CA, TrustStore, KeyStore
▸ 2. Sign trustStore 使⽤用 Shell
▸ 3. Sign KeyStore 使⽤用 Shell
▸ 4. Kafka Broker 設定
▸ 5. Kafka Clients 設定
9

10. KAFKA 實作 SSL(TWO-WAY)
▸ 1. 建置 CA, TrustStore, KeyStore
10

11. 1. 建置 CA, TRUSTSTORE, KEYSTORE
CREATE TRUST STORE
#!/bin/bash
### ENV SETTING ###
BASE_DIR=$(pwd)
CERT_OUTPUT_PATH="$BASE_DIR/certificates"
KEY_STORE="$CERT_OUTPUT_PATH/kafka.keystore.jks" # Kafka keystore⽂文件路路径
PASSWORD=kafka1234567 # 密码
CLUSTER_NAME=test-cluster # 指定别名
DAYS_VALID=365 # key有效期
STORE_PASSWORD=$PASSWORD # keystore的store密码
KEY_PASSWORD=$PASSWORD # keystore的key密码
DNAME="CN=Xiuxiu, OU=Develop, O=Mycena, L=Kaohsiung, ST=Kaohsiung, C=TW" # distinguished name
###################
# Ref: https://docs.confluent.io/current/tutorials/security_tutorial.html#creating-ssl-keys-and-certificates
mkdir -p $CERT_OUTPUT_PATH
echo "1. Create the Cluster certificate into keyStore..."
keytool -keystore $KEY_STORE -alias $CLUSTER_NAME -validity $DAYS_VALID -genkey -keyalg RSA
-storepass $STORE_PASSWORD -keypass $KEY_PASSWORD -dname "$DNAME"
11

12. 1. 建置 CA, TRUSTSTORE, KEYSTORE
CREATE OWN CA
#!/bin/bash
### ENV SETTING ###
BASE_DIR=$(pwd)
CERT_OUTPUT_PATH="$BASE_DIR/certificates"
KEY_STORE="$CERT_OUTPUT_PATH/kafka.keystore.jks" # Kafka keystore⽂文件路路径
PASSWORD=kafka1234567 # 密码
CLUSTER_NAME=test-cluster # 指定别名
DAYS_VALID=365 # key有效期
STORE_PASSWORD=$PASSWORD # keystore的store密码
KEY_PASSWORD=$PASSWORD # keystore的key密码
CERT_AUTH_FILE="$CERT_OUTPUT_PATH/ca-cert" # CA证书⽂文件路路径
DNAME="CN=Xiuxiu, OU=Develop, O=Mycena, L=Kaohsiung, ST=Kaohsiung, C=TW" # distinguished name
CA_SUBJ="/C=TW/ST=Kaohsiung/L=Kaohsiung/O=Mycenaa/CN=Xiuxiu"
###################
# Ref: https://docs.confluent.io/current/tutorials/security_tutorial.html#creating-ssl-keys-and-certificates
echo "2. 建立 Certificate Authority"
openssl req -new -x509 -keyout $CERT_OUTPUT_PATH/ca-key -out "$CERT_AUTH_FILE" -days "$DAYS_VALID"
-passin pass:"$PASSWORD" -passout pass:"$PASSWORD"
-subj "$CA_SUBJ"
12

13. KAFKA 實作 SSL(TWO-WAY)
▸ 2. Sign trustStore 使⽤用 Shell
13

14. KAFKA 實作 SSL(TWO-WAY)
▸ 3. Sign KeyStore 使⽤用 Shell
14

15. 1. 建置 CA, TRUSTSTORE, KEYSTORE
CREATE KEYSTORE && SIGN TRUSTSTORE AND KEYSTONE
#!/bin/bash
### ENV SETTING ###
BASE_DIR=$(pwd)
CERT_OUTPUT_PATH="$BASE_DIR/certificates"
PASSWORD=kafka1234567 # 密码
KEY_STORE="$CERT_OUTPUT_PATH/kafka.keystore.jks" # Kafka keystore⽂文件路路径
STORE_PASSWORD=$PASSWORD # keystore的store密码
KEY_PASSWORD=$PASSWORD # keystore的key密码
TRUST_STORE="$CERT_OUTPUT_PATH/kafka.truststore.jks" # Kafka truststore⽂文件路路径
TRUST_KEY_PASSWORD=$PASSWORD # truststore的key密码
TRUST_STORE_PASSWORD=$PASSWORD # truststore的store密码
DAYS_VALID=365 # key有效期
CLUSTER_NAME=test-cluster # 指定别名
CLUSTER_CERT_FILE="$CERT_OUTPUT_PATH/${CLUSTER_NAME}-cert" # 集群证书⽂文件路路径
CERT_AUTH_FILE="$CERT_OUTPUT_PATH/ca-cert" # CA证书⽂文件路路径
DNAME="CN=Xiuxiu, OU=Develop, O=Mycena, L=Kaohsiung, ST=Kaohsiung, C=TW" # distinguished name
CA_SUBJ="/C=TW/ST=Kaohsiung/L=Kaohsiung/O=Mycenaa/CN=Xiuxiu"
###################
# Ref: https://docs.confluent.io/current/tutorials/security_tutorial.html#creating-ssl-keys-and-certificates
echo "3. Import CA file into truststore(Add the generated CA to the someone's truststore so that someone can trust this CA.)"
keytool -keystore "$TRUST_STORE" -alias CARoot
-import -file "$CERT_AUTH_FILE" -storepass "$TRUST_STORE_PASSWORD" -keypass "$TRUST_KEY_PASS" -noprompt
echo "4. Export Cluster cert form the key store(Export the certificate from the keystore)"
keytool -keystore "$KEY_STORE" -alias "$CLUSTER_NAME" -certreq -file "$CLUSTER_CERT_FILE" -storepass "$STORE_PASSWORD" -keypass "$KEY_PASSWORD" -noprompt
echo "5. Sign the cluster certificate with the CA"
openssl x509 -req -CA "$CERT_AUTH_FILE" -CAkey $CERT_OUTPUT_PATH/ca-key -in "$CLUSTER_CERT_FILE"
-out "${CLUSTER_CERT_FILE}-signed"
-days "$DAYS_VALID" -CAcreateserial -passin pass:"$PASSWORD"
echo "6. Import the CA file ino the keystore(Sign it with the CA)"
keytool -keystore "$KEY_STORE" -alias CARoot -import -file "$CERT_AUTH_FILE" -storepass "$STORE_PASSWORD"
-keypass "$KEY_PASSWORD" -noprompt
echo "7. Impoer the Signed Cluster Cert into key store(Import both the certificate of the CA and the signed certificate into the broker keystore)"
keytool -keystore "$KEY_STORE" -alias "${CLUSTER_NAME}" -import -file "${CLUSTER_CERT_FILE}-signed"
-storepass "$STORE_PASSWORD" -keypass "$KEY_PASSWORD" -noprompt
15

16. KAFKA 實作 SSL(TWO-WAY)
▸ 4. Kafka Broker 設定
▸ listeners=PLAINTEXT://0.0.0.0:29092,SSL://0.0.0.0:29093
▸ advertised.listeners=PLAINTEXT://localhost:29092,SSL://localhost:29093
▸ ssl.keystore.location=/etc/kafka/secrets/kafka.broker2.keystore.jks
▸ ssl.keystore.filename=kafka.broker2.keystore.jks
▸ ssl.keystore.password=kafka1234567
▸ ssl.key.password=kafka1234567
▸ ssl.truststore.location=/etc/kafka/secrets/kafka.broker2.truststore.jks
▸ ssl.truststore.filename=kafka.broker2.truststore.jks
▸ ssl.truststore.password=kafka1234567
▸ security.inter.broker.protocol=SSL
16

17. ⽂字
▸ 5. Kafka Clients 設定(Producer)
▸ ssl.truststore.location=/etc/kafka/secrets/
kafka.producer.truststore.jks
▸ ssl.truststore.password=kafka1234567
▸ ssl.keystore.location=/etc/kafka/secrets/
kafka.producer.keystore.jks
▸ ssl.keystore.password=kafka1234567
▸ ssl.key.password=kafka1234567
▸ security.protocol=SSL
17

18. KAFKA 實作 SSL(TWO-WAY)
▸ 5. Kafka Clients 設定(Consumer)
▸ group.id=ssl-host
▸ ssl.truststore.location=/etc/kafka/secrets/
kafka.consumer.truststore.jks
▸ ssl.truststore.password=kafka1234567
▸ ssl.keystore.location=/etc/kafka/secrets/kafka.consumer.keystore.jks
▸ ssl.keystore.password=kafka1234567
▸ ssl.key.password=kafka1234567
▸ security.protocol=SSL
18

19. ⽂字
使⽤ DOCKER-COMPOSE 進⾏設定
▸ Advertise:SSL://hostname:9093,
▸ KeyStore Location
▸ KeyStore fileName
▸ KeyStore password(Credential)
▸ Inter_Broker_protocal: SSL
▸ Src_Link: https://docs.confluent.io/current/kafka/
authentication_ssl.html
19

20. ⽂字
測試 SSL 是否成功
▸ Command Line
▸ Program
20

21. 測試 SSL 是否成功
COMMAND LINE
▸ Poducer
▸ Consumer
21

22. ⽂字
PRODUCER
▸ ``` kafka-console-producer --broker-list localhost:29093 --
topic bar -producer.config /etc/kafka/secrets/
producer.ssl.config ```
22

23. ⽂字
CONSUMER
▸ ```kafka-console-consumer --bootstrap-server localhost:
29093 --topic bar --new-consumer --from-beginning --
consumer.config /etc/kafka/secrets/consumer.ssl.config ```
23

24. 測試 SSL 是否成功
PROGRAM
▸ 尚未經過檢驗 Producer
24

25. 測試 SSL 是否成功
PROGRAM
▸ 尚未經過檢驗 Consumer
25