Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Securing your Pulsar Cluster with Vault_Chris Kellogg

93 views

Published on

Learn how to secure a Pulsar cluster with Hashicorp Vault and deploy it on Kubernetes. Vault provides a secure way to generate tokens and store sensitive data and Pulsar has a pluggable architecture for authentication, authorization and secret management. This talk will walk through how to create custom plugins for Vault, integrate them with Pulsar and then deploy a Pulsar cluster on Kubernetes.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Securing your Pulsar Cluster with Vault_Chris Kellogg

  1. 1. @cckellogg #PulsarSummit Securing Your Pulsar Cluster with Vault &
  2. 2. Chris Kellogg Software Engineer at Splunk Contributor to Apache Pulsar and Apache Heron committer cckellogg You can find me on: cckellogg
  3. 3. Agenda • Vault Overview • Why Pulsar and Vault • Pulsar Authentication/Authorization Model • Creating Custom Plugins • Packaging Custom Plugins • Kubernetes Integration • Demo
  4. 4. https://www.vaultproject.io “Vault is a tool for securely accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, or certificates. Vault provides a unified interface to any secret, while providing tight access control and recording a detailed audit log.” What is Vault?
  5. 5. Vault Features Secret Management Authentication and Identity Data Encryption
  6. 6. Why Vault • Single source to manage secrets and tokens • Dynamic and Revokable tokens and secrets • Audit tracking for secrets and token • Merges identities across providers - LDAP, Okta, Kubernetes, AWS, GCP • Cloud friendly
  7. 7. Why Pulsar and Vault • No more forever tokens • Revokable tokens • Secure secret management for functions and connectors • Supports authenticating against many trusted sources of identity - LDAP, Okta, Kubernetes, AWS, GCP, GitHub • Central location for all security
  8. 8. Pulsar Security
  9. 9. Default is No Security • Produce and consume from any topic • Modify any tenant, namespace, topic or function • Function/Connector secrets stored as plain text in configs • No auditing of actions
  10. 10. Pulsar Security Features • TLS Encryption for traffic • Authentication - validate identity • Authorization - can user perform an action • Data encryption between producers and consumers
  11. 11. Pulsar Authentication • Responsible for determining identity of clients • Plugin System • Built-in Plugins - TLS - JWT - Authenz - Kerberos
  12. 12. Pulsar Authorization • Determines if a client has permission to perform an action • Plugin System • Built-in Plugin - Role based system backed by Zookeeper - SuperUsers - Tenant Admins - Actions: produce/consume/functions
  13. 13. Developing Auth Plugin
  14. 14. Building Plugins Best Practices • Minimize third party dependencies • Use your own executor and threads for remote requests • Cache responses
  15. 15. public class VaultAuthenticationProvider implements AuthenticationProvider { void initialize(ServiceConfiguration config) throws IOException {}; String getAuthMethodName() { return "token" }; boolean authenticateHttpRequest(HttpServletRequest req, HttpServletResponse resp) throws Exception { throw new AuthenticationException("Not supported"); } String authenticate(AuthenticationDataSource authData) throws AuthenticationException { // Implement code to authenticate the client with vault } AuthenticationState newAuthState(AuthData authData, SocketAddress remoteAddress, SSLSession sslSession) throws AuthenticationException { // Implement code to authenticate the client with vault - // Used in binary connections for challenges } } Vault Authentication Plugin
  16. 16. ### --- Authentication --- ### # Enable authentication authenticationEnabled=true # Autentication provider name list, which is comma separated list of class names authenticationProviders=org.apache.pulsar.vault.authentication.VaultAuthentictionProvider # Interval of time for checking for expired authentication credentials authenticationRefreshCheckSeconds=60 Configuring Auth Plugin broker.conf
  17. 17. Client BROKER Vault Authentication Provider Authentication Service 1 5 2 4 6 3 1. Client Request with Vault Token 2. Authenticate Client 3. Token pass to Vault for Authentication 4. Vault token info returned 5. Return user identity 6. Return result to client Pulsar Vault Authentication
  18. 18. Developing Function Plugins
  19. 19. Pulsar Secret Plugins Secrets Provider • Run in the instance • Provides secrets through the function context api Secrets Configurator • Runs on the server (Broker or Function Worker) • Determines the Secret Provider the instance should use
  20. 20. public interface SecretsProvider { // Initialize the SecretsProvider. default void init(Map<String, String> config) {} // Fetches a secret String provideSecret(String secretName, Object pathToSecret); } public class MySecretFunction implements Function<String, Void> { @Override public Void process(String input, Context context) throws Exception { final String password = context.getSecret("password"); context.getLogger().info("read secret password=" + password); return null; } } Example code SecretsProvider - Client Side Plugin
  21. 21. public interface SecretsProviderConfigurator { default void init(Map<String, String> config) {} void configureKubernetesRuntimeSecretsProvider(V1PodSpec ps, String container, Function.FunctionDetails details; void configureProcessRuntimeSecretsProvider(ProcessBuilder pb, Function.FunctionDetails detailsetails); Type getSecretObjectType(); default void doAdmissionChecks(AppsV1Api appsV1Api, CoreV1Api coreV1Api, String ns, Function.FunctionDetails details) {} String getSecretsProviderClassName(Function.FunctionDetails details); Map<String, String> getSecretsProviderConfig(Function.FunctionDetails details); } SecretsProviderConfigurator - Server Side Plugin Highlighted methods are used to setup secrets plugins on the instances
  22. 22. ######################## # Secrets ######################## secretsProviderConfiguratorClassName: org.apache.pulsar.vault.secrets.VaultSecretsProviderConfigurator secretsProviderConfiguratorConfig: vaultAddress: http://localhost:8200 tokenPath: /etc/auth/token Configuring Secret Plugins Secrets Configurator functions_worker.yml
  23. 23. Secrets Provider public class VaultSecretsProviderConfigurator implements SecretsProviderConfigurator { @Override public String getSecretsProviderClassName(Function.FunctionDetails details) { if (!isEmpty(functionDetails.getSecretsMap())) { if (Function.FunctionDetails.Runtime.JAVA == details.getRuntime()) { return "org.apache.pulsar.vault.secrets.VaultSecretsProvider"; } else if (Function.FunctionDetails.Runtime.PYTHON == details.getRuntime()) { return "python_secret_provider"; } } return null; } @Override public Map<String, String> getSecretsProviderConfig(Function.FunctionDetails details) { final Map<String, String> secrets = new HashMap<>(); secrets.put("vaultAddress", "http://localhost:8200"); secrets.put("tokenPath", "/var/auth/token"); return secrets; } Configuring Secret Plugins
  24. 24. Java Function Instance User Code final String password = context.getSecret("password"); Vault Secret Provider 1 2 3 4 1. Request secret from code 2. Secret request with token 3. Secret returned to plugin 4. Return secret value Vault Secret Provider
  25. 25. Pulsar Kubernetes Plugins Kubernetes Manifest Customizer • Runs on the server (Broker or Function Worker) • Enables customization to the K8s function specs Kubernetes Function Auth Provider • Runs on the server (Broker or Function Worker) • Determines the auth params passed to the instances
  26. 26. public interface KubernetesManifestCustomizer extends RuntimeCustomizer { default V1StatefulSet customizeStatefulSet(Function.FunctionDetails funcDetails, V1StatefulSet statefulSet) { return statefulSet; } default V1Service customizeService(Function.FunctionDetails funcDetails, V1Service service) { return service; } default String customizeNamespace(Function.FunctionDetails funcDetails, String currentNamespace) { return currentNamespace; } } KubernetesManifestCustomizer - Server Side Plugin
  27. 27. public interface KubernetesFunctionAuthProvider extends FunctionAuthProvider { public void configureAuthDataStatefulSet(V1StatefulSet sts, Optional<FunctionAuthData> o) {} public void configureAuthenticationConfig(AuthenticationConfig config, Optional<FunctionAuthData> o) { ** configures the client auth for the function instances } public Optional<FunctionAuthData> cacheAuthData(Function.FunctionDetails details, AuthenticationDataSource s) throws Exception { ** Optional<FunctionAuthData> returned is used in configureAuthenticationConfig } public Optional<FunctionAuthData> updateAuthData(Function.FunctionDetails details, Optional<FunctionAuthData> o, AuthenticationDataSource s) throws Exception { ** Optional<FunctionAuthData> returned is used in configureAuthenticationConfig } public void cleanUpAuthData(Function.FunctionDetails details, Optional<FunctionAuthData> o) throws Exception {} } KubernetesFunctionAuthProvider - Server Side Plugin
  28. 28. Java Function Instance User Code final String password = context.getSecret("password"); Vault Secret Provider 1 2 3 1. Request secret from code 2. Read secret from file 3. Return secret value Vault Secret Provider with Vault Agent
  29. 29. Packaging Plugins Where do my plugins go? pulsar/ instances/ lib/ authentication.jar secret-configurator.jar secret-provider.jar deps/ kubernetes-plugins.jar
  30. 30. Kubernetes Pulsar Vault
  31. 31. pulsar functions vault zookeeper brokers bookies proxy
  32. 32. Pulsar Kubernetes Pod Pulsar Process Vault Agent Kubernetes JWT 4 3 2 1 1. Service Account JWT passed to Vault for Authentication 2. Vault Token auth returned 3. Write token to file 4. Pulsar process reads token from file Pulsar Vault Kubernetes Integration
  33. 33. Function Secret Configuration tenant: "public" namespace: "default" name: “secrets-printer" className: “secrets_printer” inputs: ["public/default/secrets-trigger"] autoAck: true parallelism: 1 resources: cpu: 0.5 ram: 536870912 disk: 536870912 secrets: username: path: "internal/data/database/config" key: username password: path: "internal/data/database/config" key: password customRuntimeOptions: >- { "serviceAccountName": "pf-secrets-printer" } Used by the VaultKubernetesCustomizer to add annotations for vault token and secret injection
  34. 34. Function Vault Annotations vault.hashicorp.com/role: pf-secrets-printer vault.hashicorp.com/agent-inject: 'true' vault.hashicorp.com/agent-inject-token: 'true' vault.hashicorp.com/agent-inject-secret-password: secret-path vault.hashicorp.com/agent-inject-template-password: | '{{- with secret "secret-path" }}{{ .Data.data.password }}{{ end }}'
  35. 35. Demo
  36. 36. Future Enhancements • Vault for certificate management • Pulsar Vault authorization plugin • Vault for data encryption
  37. 37. Resources • https://pulsar.apache.org/docs/en/security-overview/ • https://pulsar.apache.org/docs/en/security-authorization/ • https://pulsar.apache.org/docs/en/security-extending/ • https://github.com/hashicorp/vault-k8s • https://www.vaultproject.io/docs/platform/k8s/helm • https://www.vaultproject.io/docs/platform/k8s/injector • https://learn.hashicorp.com/vault/kubernetes/k8s-reference-architecture Pulsar Vault
  38. 38. Questions
  39. 39. Thank You cckellogg cckellogg #PulsarSummit Code: https://github.com/cckellogg/pulsar-vault

×