This document discusses securing Kafka at PayPal, which processes 500 billion messages per day. It covers:
1. Enabling TLS on Kafka brokers using self-signed certificates to encrypt communication.
2. Implementing mutual TLS authentication between brokers and clients by generating and distributing keystores and truststores.
3. Authenticating clients using SASL and OAuthBearer, with custom callback handlers to retrieve credentials from an internal key management service.
Amazon Managed Blockchain is a fully managed blockchain service that makes it easy for customers to create and manage scalable blockchain-based transaction networks (blockchain networks) using the popular open source blockchain frameworks Hyperledger Fabric and Ethereum. Blockchain technologies enable groups of organizations, oftentimes in financial services and manufacturing, to securely transact, run application code, and share data without a trusted central authority. We will explore the components of blockchain technology, discuss use cases, and do a deep dive into capabilities, performance, and key innovations in Amazon Managed Blockchain.
Speaker: Bill Baldwin - Database Technical Evangelist, AWS
Secure and Integrated - Using IAM with Amazon MSK | Mitchell Henderson, AWSHostedbyConfluent
The document discusses using Amazon Managed Streaming for Kafka (Amazon MSK) with AWS Identity and Access Management (IAM) for authentication. Amazon MSK manages Apache Kafka to make it easier to use on AWS. IAM can be used to securely authenticate users and control access. Using IAM with MSK provides centralized credential management, cross-account access without sharing secrets, and integration with other AWS services like CloudTrail for auditing. An example setup demonstrates configuring a client to connect to MSK using IAM credentials.
AWS Certificate Manager (ACM) Private Certificate Authority (CA) is a managed private CA service that helps you easily and securely manage the lifecycle of your private certificates. In this session, learn how ACM Private CA extends ACM’s certificate management capabilities to private certificates and enables you to centrally manage public and private certificates. We also demonstrate how ACM Private CA enables you to create a Private CA and use it to create and deploy private certificates for your AWS resource and internal resources. We also discuss case studies demonstrating how customers use ACM Private CA to automate security and certificate management.
The document discusses crafting consumable APIs using the WSO2 API Manager. It covers best practices for API design, documentation, versioning, community building, authentication, access tokens, extensibility, collecting statistics, metering, billing/monetization models, deployment, scaling, and maintenance of consumable APIs. The goal is to provide guidance on developing APIs that can be subscribed to and generate revenue through various billing approaches.
Best Practices for Implementing Your Encryption Strategy Using AWS Key Manage...Amazon Web Services
AWS Key Management Service (KMS) is a managed service that makes it easy for you to create and manage the encryption keys used to encrypt your data. In this session, we will dive deep into best practices learned by implementing AWS KMS at AWS’ largest enterprise clients. We will review the different capabilities described in the AWS Cloud Adoption Framework (CAF) Security Perspective and how to implement these recommendations using AWS KMS. In addition to sharing recommendations, we will also provide examples that will help you protect sensitive information on the AWS Cloud.
Modern application architectures are embracing public clouds, microservices, and container schedulers like Kubernetes and Nomad. These bring complex service-to-service communication patterns, increased scale, dynamic IP addresses, ephemeral infrastructure, and higher failure rates. These changes require a new approach for service discovery, configuration, and segmentation. Service discovery enables services to find and communicate with each other. Service configuration allows us to dynamically configure applications at runtime. Service segmentations lets us secure our microservices architectures by limiting access. In this talk, we cover these challenges and how to solve them with Consul providing as a service mesh.
The session covers how Cisco SD-WAN can be used to extend the WAN connectivity to AWS. We show how the Viptela-based SD-WAN solution accelerates the path to cloud migration while maintaining the application SLA using the policy-based app fabric model. We cover Viptela's cloud-first network management, orchestration, and overlay technologies with industry-leading routing platforms, services, and SD-WAN capabilities from Cisco. We also cover how a customer deployed Cisco SD-WAN and the benefits they achieved, how a customer extended Cisco SD-WAN fabric to AWS, and the benefits of consistent security and segmentation, policy, network visibility, and connectivity options across branch, campus, data center, and cloud. This session is brought to you by AWS Partner, Cisco.
Running more than one containerized application in production makes teams look for solutions to quickly deploy and orchestrate containers. One of the most popular options is the open-source project Kubernetes. With the release of the Amazon Elastic Container Service for Kubernetes (EKS), engineering teams now have access to a fully managed Kubernetes control plane and time to focus on building applications. This workshop will deliver hands-on labs to support you getting familiar with Amazon's EKS.
Amazon Managed Blockchain is a fully managed blockchain service that makes it easy for customers to create and manage scalable blockchain-based transaction networks (blockchain networks) using the popular open source blockchain frameworks Hyperledger Fabric and Ethereum. Blockchain technologies enable groups of organizations, oftentimes in financial services and manufacturing, to securely transact, run application code, and share data without a trusted central authority. We will explore the components of blockchain technology, discuss use cases, and do a deep dive into capabilities, performance, and key innovations in Amazon Managed Blockchain.
Speaker: Bill Baldwin - Database Technical Evangelist, AWS
Secure and Integrated - Using IAM with Amazon MSK | Mitchell Henderson, AWSHostedbyConfluent
The document discusses using Amazon Managed Streaming for Kafka (Amazon MSK) with AWS Identity and Access Management (IAM) for authentication. Amazon MSK manages Apache Kafka to make it easier to use on AWS. IAM can be used to securely authenticate users and control access. Using IAM with MSK provides centralized credential management, cross-account access without sharing secrets, and integration with other AWS services like CloudTrail for auditing. An example setup demonstrates configuring a client to connect to MSK using IAM credentials.
AWS Certificate Manager (ACM) Private Certificate Authority (CA) is a managed private CA service that helps you easily and securely manage the lifecycle of your private certificates. In this session, learn how ACM Private CA extends ACM’s certificate management capabilities to private certificates and enables you to centrally manage public and private certificates. We also demonstrate how ACM Private CA enables you to create a Private CA and use it to create and deploy private certificates for your AWS resource and internal resources. We also discuss case studies demonstrating how customers use ACM Private CA to automate security and certificate management.
The document discusses crafting consumable APIs using the WSO2 API Manager. It covers best practices for API design, documentation, versioning, community building, authentication, access tokens, extensibility, collecting statistics, metering, billing/monetization models, deployment, scaling, and maintenance of consumable APIs. The goal is to provide guidance on developing APIs that can be subscribed to and generate revenue through various billing approaches.
Best Practices for Implementing Your Encryption Strategy Using AWS Key Manage...Amazon Web Services
AWS Key Management Service (KMS) is a managed service that makes it easy for you to create and manage the encryption keys used to encrypt your data. In this session, we will dive deep into best practices learned by implementing AWS KMS at AWS’ largest enterprise clients. We will review the different capabilities described in the AWS Cloud Adoption Framework (CAF) Security Perspective and how to implement these recommendations using AWS KMS. In addition to sharing recommendations, we will also provide examples that will help you protect sensitive information on the AWS Cloud.
Modern application architectures are embracing public clouds, microservices, and container schedulers like Kubernetes and Nomad. These bring complex service-to-service communication patterns, increased scale, dynamic IP addresses, ephemeral infrastructure, and higher failure rates. These changes require a new approach for service discovery, configuration, and segmentation. Service discovery enables services to find and communicate with each other. Service configuration allows us to dynamically configure applications at runtime. Service segmentations lets us secure our microservices architectures by limiting access. In this talk, we cover these challenges and how to solve them with Consul providing as a service mesh.
The session covers how Cisco SD-WAN can be used to extend the WAN connectivity to AWS. We show how the Viptela-based SD-WAN solution accelerates the path to cloud migration while maintaining the application SLA using the policy-based app fabric model. We cover Viptela's cloud-first network management, orchestration, and overlay technologies with industry-leading routing platforms, services, and SD-WAN capabilities from Cisco. We also cover how a customer deployed Cisco SD-WAN and the benefits they achieved, how a customer extended Cisco SD-WAN fabric to AWS, and the benefits of consistent security and segmentation, policy, network visibility, and connectivity options across branch, campus, data center, and cloud. This session is brought to you by AWS Partner, Cisco.
Running more than one containerized application in production makes teams look for solutions to quickly deploy and orchestrate containers. One of the most popular options is the open-source project Kubernetes. With the release of the Amazon Elastic Container Service for Kubernetes (EKS), engineering teams now have access to a fully managed Kubernetes control plane and time to focus on building applications. This workshop will deliver hands-on labs to support you getting familiar with Amazon's EKS.
Hybrid Infrastructure Integration is an approach to connect on-premises IT resources with AWS and bridge processes, services, and technologies used in common enterprise customer environments. This session addresses connectivity patterns, security controls, account governance, and operations monitoring approaches successfully implemented in enterprise engagements. Infrastructure architects and IT professionals can get an overview of various integration types, approaches, methodologies, and common service patterns, helping them to better understand and overcome typical challenges in hybrid enterprise environments.
The “Twelve-Factor” application model has come to represent twelve best practices for building modern, cloud-native applications. With guidance on things like configuration, deployment, runtime, and multiple service communication, the Twelve-Factor model prescribes best practices that apply to everything from web applications to APIs to data processing applications.
Although serverless computing and AWS Lambda have changed how application development is done, the “Twelve-Factor” best practices remain relevant and applicable in a serverless world. In this talk, Chris will share with you how to apply the “Twelve-Factor” model to serverless application development with AWS Lambda and Amazon API Gateway and show you how these services enable you to build scalable, low cost, and low administration applications.
AWS Certificate Management and Private Certificate Authority Deep Dive (SEC41...Amazon Web Services
This document discusses AWS Certificate Manager (ACM) and ACM Private Certificate Authority (CA). ACM makes it easy to provision, manage, deploy and renew TLS/SSL certificates on AWS. ACM Private CA allows customers to establish a managed private CA to issue private certificates trusted within their organization. Examples are provided of using private certificates with Elastic Load Balancing and for device authentication. The document also covers customizing private certificates, chaining a private CA to an enterprise root CA, and revocation.
CloudHSM: Secure, Scalable Key Storage in AWS - AWS Online Tech TalksAmazon Web Services
AWS CloudHSM allows customers to securely generate, store, and manage cryptographic keys in a Hardware Security Module (HSM) in the AWS cloud. It protects keys with FIPS 140-2 Level 3 validated HSMs and allows customers to have full control over key access management without AWS having access. AWS CloudHSM can be used for use cases like certificate authorities, database encryption, and SSL/TLS offloading. The new second-generation CloudHSM service is fully managed and offers high availability, scalability, and ease of use.
Sessão Avançada: VMware Cloud na AWS - ENT204 - Sao Paulo SummitAmazon Web Services
O VMware Cloud na AWS permite migrar rapidamente as cargas de trabalho existentes para o AWS Cloud usando ferramentas com as quais você já está familiarizado. O VMware Cloud na AWS traz o software de data center definido por software da VMware para a nuvem pública da Amazon, fornecido como uma solução baseada em nuvem sob demanda, elasticamente escalável. Vendida e operada pela VMware, a solução permite que os clientes usem um conjunto comum de softwares e ferramentas para gerenciar consistentemente seus recursos do vSphere com base na AWS e no local. Esta sessão usa exemplos práticos e reais de implantação de clientes para aprofundar a conectividade de rede em nuvem híbrida, as práticas recomendadas de proteção de dados e as integrações de serviços nativas da AWS.
The document discusses Amazon EKS (Elastic Kubernetes Service), which allows users to run Kubernetes on AWS. It highlights that EKS manages the control plane for users and provides native integrations with other AWS services like load balancers, IAM, and container registry. The document also summarizes key capabilities like high availability of the Kubernetes masters, networking options, version upgrades, and how to provision Kubernetes nodes on EKS.
This document provides an overview and update of AWS Elemental Media Services, including MediaConnect, MediaLive, MediaPackage, MediaStore, and MediaConvert. It discusses how live video transport works today and the challenges with existing solutions. It then introduces AWS Elemental MediaConnect as a reliable, secure, and flexible live video transport service in the cloud. Key features of MediaConnect like robustness, security, agility, and transparency are covered. Example customer use cases for MediaConnect like live video cloud processing workflows and global distribution are described. The document discusses MediaConnect pricing and flow. Finally, it provides updates on MediaLive, MediaPackage, MediaStore, and MediaConvert including new features like QVBR encoding and tools to
Executando Kubernetes com Amazon EKS - DEV303 - Sao Paulo SummitAmazon Web Services
O Kubernetes oferece uma poderosa camada de abstração para gerenciar a infraestrutura conteinerizada. O Amazon Elastic Container Service for Kubernetes (Amazon EKS) facilita a execução do Kubernetes na AWS sem ter que gerenciar os nós principais ou o operador do etcd. Nesta sessão, abordamos como o Amazon EKS torna a implementação do Kubernetes na AWS simples e escalável, incluindo rede, segurança, monitoramento e registro. Discutiremos as principais contribuições que estamos dando para que a AWS seja um lugar ainda melhor para executar o Kubernetes e demonstraremos como os clientes da AWS estão começando a usar o Amazon EKS.
Highly secure content delivery at global scale with amazon cloudfrontAmazon Web Services
Our very own Canadian Solution Architects: Matt Nowina and Jonathan Dion, presented to an audience of broadcasters, special effects producers, video editors, during the AWS Media and Entertainment Symposium
Don’t Sacrifice Performance for Security: Best Practices for Content Delivery Amazon Web Services
This document summarizes a presentation about using Amazon CloudFront and AWS WAF for securing and accelerating APIs. It discusses the challenges of delivering APIs, how CloudFront addresses these challenges through application acceleration, security features, and high availability. The presentation also provides a case study of how Slack migrated from using Elastic Load Balancing to using CloudFront to deliver their API, improving performance metrics. It concludes with a demonstration of automating protections using Lambda and discusses future plans for rate limiting and blocking malicious traffic.
by Harrell Stiles, Sr. Consultant, AWS ProServe
AWS Lambda and Amazon API Gateway have changed how developers build and run their applications or services. But what are the best practices for tasks such as deployment, monitoring, and debugging in a serverless world? In this session, we’ll dive into best practices that serverless developers can use for application lifecycle management, CI/CD, monitoring, and diagnostics. We’ll talk about how you can build CI/CD pipelines that automatically build, test, and deploy your serverless applications using AWS CodePipeline, AWS CodeBuild, and AWS CloudFormation. We’ll also cover the built-in capabilities of Lambda and API Gateway for creating multiple versions, stages, and environments of your functions and APIs. Finally, we’ll cover monitoring and diagnostics of your Lambda functions with Amazon CloudWatch and AWS X-Ray.
With a minimum security baseline in place, you’re now ready to host data—which means Data Protection is required. Here we will discuss defining encryption strategy and selecting native AWS (KMS, CloudHSM) or third party tools; defining key rotation and key protection mechanisms; and defining data at rest and data in transit protection requirements.
Learning Objectives:
- Learn how to enable users to access their AWS accounts and business applications using their corporate credentials
- Learn how to manage SSO access to all of your AWS accounts managed in AWS Organizations
- Learn how to centrally manage user permissions to AWS resources when they access the AWS Management Console using AWS SSO
Durante il webinar discuteremo brevemente le varie opzioni disponibili per utilizzare Kubernetes su Amazon Web Services con un forte focus su Amazon Elastic Container Service for Kubernetes. Amazon EKS è il servizio gestito indirizzato ai clienti che usano o vogliono usare Kubernetes ma che preferiscono demandare la gestione del famoso software open-source ad AWS.
Hybrid Infrastructure Integration is an approach to connect on-premises IT resources with AWS and bridge processes, services, and technologies used in common enterprise customer environments. This session addresses connectivity patterns, security controls, account governance, and operations monitoring approaches successfully implemented in enterprise engagements. Infrastructure architects and IT professionals can get an overview of various integration types, approaches, methodologies, and common service patterns, helping them to better understand and overcome typical challenges in hybrid enterprise environments.
After IAM and Detective Controls you’ll turn to Infrastructure Security, which means tuning AWS Service configurations, AMI composition, and hardening other digital assets that will be deployed. We will cover how to define networking architecture (e.g. VPC, subnets, security groups); how to develop hardened AMIs based on your requirements; the importance of defining Internet ingress and egress flows, and how to determine Vulnerability Management and operational maintenance cadence.
SID201 Overview of AWS Identity, Directory, and Access ServicesAmazon Web Services
Every journey to the AWS Cloud is unique. Some customers are migrating existing applications, while others are building new applications using cloud-native services. Along each of these journeys, identity and access management helps customers protect their applications and resources. In this session, you learn how AWS identity services provide you a secure, flexible, and easy solution for managing identities and access on the AWS Cloud. With AWS identity services, you do not have to adapt to AWS. Instead, you have a choice of services designed to meet you anywhere along your journey to the AWS Cloud.
Airbnb가 직접 들려주는 Kubernetes 환경 구축 이야기 - Melanie Cebula 소프트웨어 엔지니어, Airbnb :: A...Amazon Web Services Korea
Airbnb가 직접 들려주는 Kubernetes 환경 구축 이야기
Melanie Cebula 소프트웨어 엔지니어, Airbnb
스타트업 개발자라면 Kubernetes 환경 구현을 한번쯤 생각해보셨을 것입니다. 하지만 많은 개발자들이 Kubernetes를 어렵게 느끼고 있는 것도 사실입니다. 본 세션에서는 아마존웹서비스 상에서 수백개의 Kubernetes 서비스 개발을 거치며 천여명의 개발자들이 활용하고 있는 Airbnb에서 직접 사례를 소개 드립니다. Airbnb 본사 인프라스트럭처 엔지니어 Melanie Cebula가 들려주는 시행착오와 전략, 해결책을 통해 Kubernetes 환경을 보다 쉽게 접근하시는데 도움을 드릴 예정입니다.
Kubernetes Networking in Amazon EKS (CON412) - AWS re:Invent 2018Amazon Web Services
In this talk, we share a real-world experience of how AWS securely implements Kubernetes network abstractions at scale. We also explore the pain points in the current Kubernetes networking design, best practices for troubleshooting, and future improvements.
With Apache Kafka 0.9, the community has introduced a number of features to make data streams secure. In this talk, we’ll explain the motivation for making these changes, discuss the design of Kafka security, and explain how to secure a Kafka cluster. We will cover common pitfalls in securing Kafka, and talk about ongoing security work.
Kafka 2018 - Securing Kafka the Right WaySaylor Twift
How to evaluate, implement and maintain Kafka Message Broker in a high-throughput production environment. Taylor Swift's rectum probably smells like a Creamsicle.
Hybrid Infrastructure Integration is an approach to connect on-premises IT resources with AWS and bridge processes, services, and technologies used in common enterprise customer environments. This session addresses connectivity patterns, security controls, account governance, and operations monitoring approaches successfully implemented in enterprise engagements. Infrastructure architects and IT professionals can get an overview of various integration types, approaches, methodologies, and common service patterns, helping them to better understand and overcome typical challenges in hybrid enterprise environments.
The “Twelve-Factor” application model has come to represent twelve best practices for building modern, cloud-native applications. With guidance on things like configuration, deployment, runtime, and multiple service communication, the Twelve-Factor model prescribes best practices that apply to everything from web applications to APIs to data processing applications.
Although serverless computing and AWS Lambda have changed how application development is done, the “Twelve-Factor” best practices remain relevant and applicable in a serverless world. In this talk, Chris will share with you how to apply the “Twelve-Factor” model to serverless application development with AWS Lambda and Amazon API Gateway and show you how these services enable you to build scalable, low cost, and low administration applications.
AWS Certificate Management and Private Certificate Authority Deep Dive (SEC41...Amazon Web Services
This document discusses AWS Certificate Manager (ACM) and ACM Private Certificate Authority (CA). ACM makes it easy to provision, manage, deploy and renew TLS/SSL certificates on AWS. ACM Private CA allows customers to establish a managed private CA to issue private certificates trusted within their organization. Examples are provided of using private certificates with Elastic Load Balancing and for device authentication. The document also covers customizing private certificates, chaining a private CA to an enterprise root CA, and revocation.
CloudHSM: Secure, Scalable Key Storage in AWS - AWS Online Tech TalksAmazon Web Services
AWS CloudHSM allows customers to securely generate, store, and manage cryptographic keys in a Hardware Security Module (HSM) in the AWS cloud. It protects keys with FIPS 140-2 Level 3 validated HSMs and allows customers to have full control over key access management without AWS having access. AWS CloudHSM can be used for use cases like certificate authorities, database encryption, and SSL/TLS offloading. The new second-generation CloudHSM service is fully managed and offers high availability, scalability, and ease of use.
Sessão Avançada: VMware Cloud na AWS - ENT204 - Sao Paulo SummitAmazon Web Services
O VMware Cloud na AWS permite migrar rapidamente as cargas de trabalho existentes para o AWS Cloud usando ferramentas com as quais você já está familiarizado. O VMware Cloud na AWS traz o software de data center definido por software da VMware para a nuvem pública da Amazon, fornecido como uma solução baseada em nuvem sob demanda, elasticamente escalável. Vendida e operada pela VMware, a solução permite que os clientes usem um conjunto comum de softwares e ferramentas para gerenciar consistentemente seus recursos do vSphere com base na AWS e no local. Esta sessão usa exemplos práticos e reais de implantação de clientes para aprofundar a conectividade de rede em nuvem híbrida, as práticas recomendadas de proteção de dados e as integrações de serviços nativas da AWS.
The document discusses Amazon EKS (Elastic Kubernetes Service), which allows users to run Kubernetes on AWS. It highlights that EKS manages the control plane for users and provides native integrations with other AWS services like load balancers, IAM, and container registry. The document also summarizes key capabilities like high availability of the Kubernetes masters, networking options, version upgrades, and how to provision Kubernetes nodes on EKS.
This document provides an overview and update of AWS Elemental Media Services, including MediaConnect, MediaLive, MediaPackage, MediaStore, and MediaConvert. It discusses how live video transport works today and the challenges with existing solutions. It then introduces AWS Elemental MediaConnect as a reliable, secure, and flexible live video transport service in the cloud. Key features of MediaConnect like robustness, security, agility, and transparency are covered. Example customer use cases for MediaConnect like live video cloud processing workflows and global distribution are described. The document discusses MediaConnect pricing and flow. Finally, it provides updates on MediaLive, MediaPackage, MediaStore, and MediaConvert including new features like QVBR encoding and tools to
Executando Kubernetes com Amazon EKS - DEV303 - Sao Paulo SummitAmazon Web Services
O Kubernetes oferece uma poderosa camada de abstração para gerenciar a infraestrutura conteinerizada. O Amazon Elastic Container Service for Kubernetes (Amazon EKS) facilita a execução do Kubernetes na AWS sem ter que gerenciar os nós principais ou o operador do etcd. Nesta sessão, abordamos como o Amazon EKS torna a implementação do Kubernetes na AWS simples e escalável, incluindo rede, segurança, monitoramento e registro. Discutiremos as principais contribuições que estamos dando para que a AWS seja um lugar ainda melhor para executar o Kubernetes e demonstraremos como os clientes da AWS estão começando a usar o Amazon EKS.
Highly secure content delivery at global scale with amazon cloudfrontAmazon Web Services
Our very own Canadian Solution Architects: Matt Nowina and Jonathan Dion, presented to an audience of broadcasters, special effects producers, video editors, during the AWS Media and Entertainment Symposium
Don’t Sacrifice Performance for Security: Best Practices for Content Delivery Amazon Web Services
This document summarizes a presentation about using Amazon CloudFront and AWS WAF for securing and accelerating APIs. It discusses the challenges of delivering APIs, how CloudFront addresses these challenges through application acceleration, security features, and high availability. The presentation also provides a case study of how Slack migrated from using Elastic Load Balancing to using CloudFront to deliver their API, improving performance metrics. It concludes with a demonstration of automating protections using Lambda and discusses future plans for rate limiting and blocking malicious traffic.
by Harrell Stiles, Sr. Consultant, AWS ProServe
AWS Lambda and Amazon API Gateway have changed how developers build and run their applications or services. But what are the best practices for tasks such as deployment, monitoring, and debugging in a serverless world? In this session, we’ll dive into best practices that serverless developers can use for application lifecycle management, CI/CD, monitoring, and diagnostics. We’ll talk about how you can build CI/CD pipelines that automatically build, test, and deploy your serverless applications using AWS CodePipeline, AWS CodeBuild, and AWS CloudFormation. We’ll also cover the built-in capabilities of Lambda and API Gateway for creating multiple versions, stages, and environments of your functions and APIs. Finally, we’ll cover monitoring and diagnostics of your Lambda functions with Amazon CloudWatch and AWS X-Ray.
With a minimum security baseline in place, you’re now ready to host data—which means Data Protection is required. Here we will discuss defining encryption strategy and selecting native AWS (KMS, CloudHSM) or third party tools; defining key rotation and key protection mechanisms; and defining data at rest and data in transit protection requirements.
Learning Objectives:
- Learn how to enable users to access their AWS accounts and business applications using their corporate credentials
- Learn how to manage SSO access to all of your AWS accounts managed in AWS Organizations
- Learn how to centrally manage user permissions to AWS resources when they access the AWS Management Console using AWS SSO
Durante il webinar discuteremo brevemente le varie opzioni disponibili per utilizzare Kubernetes su Amazon Web Services con un forte focus su Amazon Elastic Container Service for Kubernetes. Amazon EKS è il servizio gestito indirizzato ai clienti che usano o vogliono usare Kubernetes ma che preferiscono demandare la gestione del famoso software open-source ad AWS.
Hybrid Infrastructure Integration is an approach to connect on-premises IT resources with AWS and bridge processes, services, and technologies used in common enterprise customer environments. This session addresses connectivity patterns, security controls, account governance, and operations monitoring approaches successfully implemented in enterprise engagements. Infrastructure architects and IT professionals can get an overview of various integration types, approaches, methodologies, and common service patterns, helping them to better understand and overcome typical challenges in hybrid enterprise environments.
After IAM and Detective Controls you’ll turn to Infrastructure Security, which means tuning AWS Service configurations, AMI composition, and hardening other digital assets that will be deployed. We will cover how to define networking architecture (e.g. VPC, subnets, security groups); how to develop hardened AMIs based on your requirements; the importance of defining Internet ingress and egress flows, and how to determine Vulnerability Management and operational maintenance cadence.
SID201 Overview of AWS Identity, Directory, and Access ServicesAmazon Web Services
Every journey to the AWS Cloud is unique. Some customers are migrating existing applications, while others are building new applications using cloud-native services. Along each of these journeys, identity and access management helps customers protect their applications and resources. In this session, you learn how AWS identity services provide you a secure, flexible, and easy solution for managing identities and access on the AWS Cloud. With AWS identity services, you do not have to adapt to AWS. Instead, you have a choice of services designed to meet you anywhere along your journey to the AWS Cloud.
Airbnb가 직접 들려주는 Kubernetes 환경 구축 이야기 - Melanie Cebula 소프트웨어 엔지니어, Airbnb :: A...Amazon Web Services Korea
Airbnb가 직접 들려주는 Kubernetes 환경 구축 이야기
Melanie Cebula 소프트웨어 엔지니어, Airbnb
스타트업 개발자라면 Kubernetes 환경 구현을 한번쯤 생각해보셨을 것입니다. 하지만 많은 개발자들이 Kubernetes를 어렵게 느끼고 있는 것도 사실입니다. 본 세션에서는 아마존웹서비스 상에서 수백개의 Kubernetes 서비스 개발을 거치며 천여명의 개발자들이 활용하고 있는 Airbnb에서 직접 사례를 소개 드립니다. Airbnb 본사 인프라스트럭처 엔지니어 Melanie Cebula가 들려주는 시행착오와 전략, 해결책을 통해 Kubernetes 환경을 보다 쉽게 접근하시는데 도움을 드릴 예정입니다.
Kubernetes Networking in Amazon EKS (CON412) - AWS re:Invent 2018Amazon Web Services
In this talk, we share a real-world experience of how AWS securely implements Kubernetes network abstractions at scale. We also explore the pain points in the current Kubernetes networking design, best practices for troubleshooting, and future improvements.
With Apache Kafka 0.9, the community has introduced a number of features to make data streams secure. In this talk, we’ll explain the motivation for making these changes, discuss the design of Kafka security, and explain how to secure a Kafka cluster. We will cover common pitfalls in securing Kafka, and talk about ongoing security work.
Kafka 2018 - Securing Kafka the Right WaySaylor Twift
How to evaluate, implement and maintain Kafka Message Broker in a high-throughput production environment. Taylor Swift's rectum probably smells like a Creamsicle.
Real time dashboards with Kafka and DruidVenu Ryali
This document describes a tracking platform that provides real-time insights by ingesting streaming data from various sources into Druid for analysis and visualization. It addresses challenges around acquiring data at scale from disparate systems, processing the data using Spark Streaming and Kafka, and aggregating and exploring the data in Druid and dashboards. The platform connects these systems together into a cohesive architecture for real-time analytics and model building.
Data encryption concepts in AWS - FND302 - AWS re:Inforce 2019 Amazon Web Services
In this hands-on workshop, we use the AWS Cloud9 IDE to learn about data encryption services, such as AWS Key Management Service (KMS) and AWS Certificate Manager (ACM). We also explore various aspects of AWS KMS and AWS ACM private certificate authority.
This document discusses security features in Apache Kafka including SSL for encryption, SASL/Kerberos for authentication, authorization controls using an authorizer, and securing Zookeeper. It provides details on how these security components work, such as how SSL establishes an encrypted channel and SASL performs authentication. The authorizer implementation stores ACLs in Zookeeper and caches them for performance. Securing Zookeeper involves setting ACLs on Zookeeper nodes and migrating security configurations. Future plans include moving more functionality to the broker side and adding new authorization features.
How to Lock Down Apache Kafka and Keep Your Streams Safeconfluent
The document discusses how to secure Apache Kafka clusters through authentication. It describes several authentication mechanisms including TLS, SASL/GSSAPI using Kerberos, and SASL/PLAIN and SASL/SCRAM for username and password authentication. TLS provides server and client authentication but has performance overhead while SASL mechanisms like GSSAPI and SCRAM integrate with existing authentication systems with lower performance impact. The document provides configuration details and security considerations for each mechanism.
Kafka security includes SSL for wire encryption, SASL (Kerberos) for authentication, and authorization controls. SSL uses certificates for encryption during network communication. SASL performs authentication using Kerberos credentials. Authorization is provided by pluggable authorizers that define access control lists controlling permissions for principals to perform operations on resources and hosts. Securing Zookeeper with ACLs and SASL is also important as Kafka stores metadata there.
How encryption works in AWS: What assurances do you have that unauthorized us...Amazon Web Services
Customers who want their data encrypted on AWS increasingly take advantage of AWS services that allow them to encrypt data and manage access to the encryption keys. This session discusses how your data is encrypted in transit and at rest in AWS services like Amazon EC2, Amazon S3, and Elastic Load Balancing. Learn about the AWS key management options available, such as AWS KMS, CloudHSM, and ACM. The session also covers some of the security controls that AWS uses to minimize risk of compromise by unauthorized users as it works to keep your data safe.
PrivateLink for Partners: Connectivity, Scale, Security (GPSTEC306) - AWS re:...Amazon Web Services
In this session, we review how technology and consulting partners can utilize AWS PrivateLink, a networking service that allows for a service behind a load balancer to be privately placed into other VPCs as well as on-premises. You can use PrivateLink to help scale a SaaS service, simplify microservices, simplify the network connectivity of managed service providers, and create a more secure environment for partner products inside customer VPCs. In this session, we focus on the design and service architecture requirements as well as the business considerations for implementing PrivateLink for your product or service. We also hear from APN Partner, Snowflake, and its customer, ARC, about how they deployed PrivateLink.
The document discusses security models in Apache Kafka. It describes the PLAINTEXT, SSL, SASL_PLAINTEXT and SASL_SSL security models, covering authentication, authorization, and encryption capabilities. It also provides tips on troubleshooting security issues, including enabling debug logs, and common errors seen with Kafka security.
Building PaaS with Amazon EKS for the Large-Scale, Highly Regulated Enterpris...Amazon Web Services
Containers make it easy to build and deploy applications by abstracting away the underlying operating system. But how do you build secure and compliant containerized applications in a distributed environment, and without direct access to the operating system your code is running on? In this session, hear how Amazon Elastic Container Service for Kubernetes (Amazon EKS) is integrated into a large-scale regulated enterprise in the areas of network, security, CI/CD, and monitoring to cater to the needs of various business units. We cover the basics in each of these areas in Amazon EKS, and we hear from Fidelity on how it is driving its cloud strategy with Amazon EKS in the heavily regulated finance sector. We also share best practices and common architectures for building containerized application in highly regulated industries.
Keeping Secrets: Securing Your Data with AWS Cryptography (SEC353-R1) - AWS r...Amazon Web Services
Safeguarding your information assets is critical for maintaining the confidence of your customers as well as protecting your organization’s own intellectual property. AWS offers a variety of cryptographic services that enable you to bring such protection down to the data level. In this session, we cover core AWS services, including AWS Key Management Service (AWS KMS), AWS CloudHSM, and AWS Secrets Manager, discuss use cases for each, and show how these three services can be part of your corporate information security strategy.
Giving credit where credit’s due - myFICO’s cloud transformation - SVC204 - S...Amazon Web Services
Manish Sharma discusses Fair Isaac Corporation's migration of its myFICO consumer website to AWS. Key reasons for migrating included improving reliability by solving for multiple points of failure and improving security. The migration took 7 months and involved "lift and shift" of applications along with upgrades. Lessons included using small expert teams, automating processes, and designing for security and compliance from the start. Steps outlined for effective security and compliance on AWS included access control, network restrictions, auditing S3 buckets, patching, and using a web application firewall.
Root CA hierarchies for AWS Certificate Manager (ACM) Private CA - FND320 - A...Amazon Web Services
AWS recently announced root certificate authority (CA) hierarchies for AWS Certificate Manager (ACM) Private CA. CA administrators can now quickly and easily create a complete CA hierarchy, including root and subordinate CAs, with no need for external CAs. In this presentation, we provide an overview of ACM Private CA and discuss some common use cases, such as issuing private certificates in order to identify devices. You learn how to create a two-level CA hierarchy and use it to issue private certificates. You also learn security best practices for creating and managing a CA hierarchy, and you have a chance to ask questions.
Using the AWS Encryption SDK for multiple master key encryption - SDD402 - AW...Amazon Web Services
"Do you want client-side encryption for your software but don’t know where to start? In this hands-on workshop, we cover the basics of client-side encryption, perform encrypt and decrypt operations using AWS Key Management Service (KMS) and the AWS Encryption SDK, and discuss security and performance considerations when implementing client-side encryption in your software. This workshop covers the basic challenges of this domain; a best practice for protecting data end-to-end with client-side encryption; KMS-style services and their uses, including AWS KMS; the open-source, open-format AWS Encryption SDK; and considerations for advanced integrations, such as performance tradeoffs and high-availability strategies.
The document discusses secure content delivery with AWS. It provides an overview of Amazon CloudFront as a content delivery network (CDN) and how it can accelerate content delivery globally. It also discusses AWS Certificate Manager (ACM) for provisioning SSL/TLS certificates and integrating with CloudFront. The document then delves into how CloudFront enables secure content delivery and advanced SSL/TLS features like session tickets and OCSP stapling. It concludes with an overview of AWS Web Application Firewall (WAF) and how it can protect websites and applications from attacks.
The document discusses secure content delivery with AWS. It provides an overview of Amazon CloudFront as a content delivery network (CDN) and how it can accelerate content delivery globally. It also discusses AWS Certificate Manager (ACM) for provisioning SSL/TLS certificates and integrating with CloudFront. The document then delves into how CloudFront enables secure content delivery and advanced SSL/TLS features like session tickets and OCSP stapling. It concludes with an overview of AWS Web Application Firewall (WAF) and how it can block malicious requests to protect websites and applications.
This document discusses securing Spark applications. It covers encryption to protect data in transit and at rest, authentication using Kerberos to identify users, and authorization for access control through tools like Sentry and a proposed RecordService. While Spark can be secured today by leveraging Hadoop security, continued work is needed for easier encryption, improved Kerberos support for long-running jobs, and row/column-level authorization beyond file permissions.
AWS Encryption SDK: The Busy Engineer's Guide to Client-Side Encryption (SEC3...Amazon Web Services
Do you want client-side encryption for your service, but don’t know exactly where to start? In this hands-on workshop, we review your client-side encryption options and explore implementing client-side encryption using the AWS Encryption SDK in Java, Python, and C. We cover the basics of client-side encryption, perform encrypt and decrypt operations using AWS KMS and the AWS Encryption SDK, and discuss security and performance considerations when implementing client-side encryption in your service. Bring a laptop, and be sure that you have an active AWS account with Administrator privileges before the workshop.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Trusted Execution Environment for Decentralized Process MiningLucaBarbaro3
Presentation of the paper "Trusted Execution Environment for Decentralized Process Mining" given during the CAiSE 2024 Conference in Cyprus on June 7, 2024.
Best 20 SEO Techniques To Improve Website Visibility In SERPPixlogix Infotech
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
Skybuffer SAM4U tool for SAP license adoptionTatiana Kojar
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slackshyamraj55
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
This presentation provides valuable insights into effective cost-saving techniques on AWS. Learn how to optimize your AWS resources by rightsizing, increasing elasticity, picking the right storage class, and choosing the best pricing model. Additionally, discover essential governance mechanisms to ensure continuous cost efficiency. Whether you are new to AWS or an experienced user, this presentation provides clear and practical tips to help you reduce your cloud costs and get the most out of your budget.
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Introduction of Cybersecurity with OSS at Code Europe 2024Hiroshi SHIBATA
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
Before we begin, let us see what everyone’s experience with Kafka is.
Can you please raise your hand if you run Kafka right now at any scale.
Now keep your hand raised if you process 1 million messages a day.
1 billion.
1 trillion.
Okay cool, we’ve got a pretty good mix here. Last question before we move on - How many of you have a ‘Kakfa’ typo in your automation source code for Kafka?
Hello everybody. Welcome to our presentation.
My name is Maulin and I am here with my colleagues Thomas and Sanat and we are from Kafka team at PayPal.
I’m here to talk about challenges we face at PayPal managing multiple geo-distributed Kafka clusters and the solutions we apply.
This is the agenda for this session.
We’ll start with some details about Kafka @ PayPal in its present state.
Then we will talk about how we enabled TLS and ACLs at PayPal and share some performance numbers related to the same.
Finally we will conclude with highlighting some of the cool and challenging things we are working on.
Our Kafka ecosystem processes 400 billion messages a day. We have over 50 clusters, which includes over 5000 topics, and 7 petabytes of total disk space.
We’ve been running Kafka for a while now, starting with version 0.8 and our current version is 1.1.
Our tech stack has grown enormously over the past few years, with clients using languages ranging from Java to Python to Node.js.
We also have many different application frameworks connected to Kafka.
Our clusters are multi-tenant, which means our clusters are generic and there are multiple use-cases in a single cluster.
Our Kafka ecosystem is also distributed across multiple security and availability zones.
Let’s take a look at data pipelines.
At PayPal, some use-cases for Kafka are user behavioral tracking, experimental testing such as A/B testing, merchant SLA monitoring, and risk & compliance analytics.
All these use-cases generate data in the form of business events, or application logs, or application metrics, or any combination of the three.
This data flows through Kafka using batch processing or real-time streaming, and they end up in frameworks & platforms land where they are used for analytics or other processing.
Additionally, it is very common for flows to have multiple hops where data is pumped into Kafka, consumed by a framework, and then additional data is pumped back into Kafka and consumed by yet another framework.
Thank you Maulin. Hi everyone, I am Thomas.
As Maulin mentioned before, Kafka team is maintaining a large Kafka ecosystem at PayPal, over 500 billion messages are processed by Kafka everyday.
As a Fintech company, security has always been our highest priority.
Then how to secure kafka at PayPal became the beggest thing at Kafka Team this year.
Now, I am going to talk about how we enable mutual TLS at PayPal.
Before moving to Kafka TLS, let's quickly go through some terminologies.
SSL and its successor, TLS, are protocols for establishing authenticated and encrypted connections between networked computers.
Although the SSL protocol was deprecated with the release of TLS 1.0 in 1999, It is still common to refer to these related technologies as “SSL” or “SSL/TLS.”
In terms of SSL Keys, basically, we mean private key and public keys
Public key is used to encrypt while private keys are used to decrypt.
Public keys can be made available to anyone, hence the term public. On the other hand, private key could not be shared.
SSL certificates provide a verified link between public keys and the entity that it claims that it belongs to.
Certificate Authority is the third party who signs certificate.
A trusted CA means this is a known third party certificate issuer.
Let’s take a look at how to enable TLS for open source Kafka.
There are 4 main procedures to go through to get this done.
Reflecting to SSL terminologies, the procedures are to get the key and certificates, create CA and sign the certificates.
Then configured related Kafka properties.
The first 3 procedures could be done through command lines
Let’s just take a look at how many commands you need to run to get this done.
As you can see, there are 8 commands you need to run using keytool to get the first 3 procedures done for 1 host
After running all these commands, you will get 2 things
A keystore contains private keys, and the certificates with their corresponding public keys.
A truststore contains certificates from other parties that you expect to communicate with, or from Certificate Authorities that you trust.
Now we got the key and certs as JKS type keystore and truststore file/
Next step is to configure kafka broker and client properties.
There are 7 properties you need to configure on broker side to tell broker where to load keystore and truststore file and the related credentials.
The client side configurations are pretty similar.
However, keystore is not necessary at client side with 1 way TLS.
Now after this step, we got every thing ready for TLS connection.
Let's take a look at how does one way TLS work.
As you can see in the diagram, there is a file-based keystore on kafka broker and truststore on client application.
While kafka client try to connect to kafka broker, public certificates will be exported from keystore in kafka broker and imported into truststore at client application.
Truststore will try to authenticate the certificate and if it is succeeded, the TLS connection will be made.
This is the most common scenario for TLS connection. However, in this scenario Client Application trust the broker that it connects to, kafka broker may not know this application.
That's why we need mutual auth.
This diagram shows the work flow for mutual TLS.
As you can see, compare with the previous slide, now both kafka broker and kafka client app have the file based keystore and truststore.
Then same as one way auth, server authentication will also happen at this time. Furthermore, on top of server auth, certificate will be exported from keystore at client app and import to truststore in kafka broker to do client authentication.
After 2 authentications are done, the TLS connection will be made. Then it is guaranteed that client and broker knows each other.
Is kafka secured now? Yes.. But is it what we want at PayPal?
Maybe not.. with the things that current Open Source Kafka provide, it is not easy for us to achieve Kafka TLS.
Let's take a look at our challenges for enabling TLS for kafka.
Due to InfoSec and AppSec Restriction, file-based security material is not allowed in PayPal.
How to deploy these security materials to thousands of broker hosts and hundreds of client hosts is also a big challenge for us.
Also, key rotations and credential security are some extra work to think.
Before moving to the solution, let's us move one step back to see what we have at PayPal.
Let me introduce you with PayPal's Key Management Service. As you can easily imagine from the name, it is a In-house Key Management Service like HashiCorp Vault and AWS KMS.
Key Management Service is a CA to issue certificates for all internal applications and it will manage key rotations.
Also I want to show to how does clients connect to Kafka brokers at PayPal.
Let me introduce you with another service named Kafka configuration service. Basically, people send request to kafka configuration service with the topic name and it will return all the required properties to reach to that topic.
The reason for developing this service is that we want to abstract kafka cluster away from kafka client. So instead of connecting kafka using hard-coded bootstrap server list, kafka client will get all the configuration from config service and use those configs to connect to kafka.
With config service, kafka clients don't need to worry about the boostrap server list any more, what they need is only the topic name. Kafka team can easily maintain kafka cluster by adding and removing nodes without worrying customer impact.
Based on the challenges and the 2 services that we have, our approach is very clear. We need a way to fetch keystore and trustore from Key Management Service and load on client and broker side.
We changed kafka source code on client side and broker side and introduce 2 interfaces for customized keystore and truststore loading.
With the implementation class for the interface, people could loader keystore and truststore from wherever they want no matter in disk or memory.
In PayPal, kafka team will provide these implementation class for clients.
Now let's take a look at the work flow with TLS at PayPal.
Kafka client will also request config from configuration service and config service will return the config with the keystore loader and truststore loader class.
Loader class will fetch keystore and truststore from Key Management service and load the keystore and truststore to connect to kafka broker.
Client will not even notice the change behind because there is nothing needs to be changed from them.
Let me show you how simple it is to use this interface.
You only need to have 1 configuration and you can have the connection to kafka secured with SSL!
You don't need to worry about the location and credentials, all the things are inside the loader class.
Kafka Client (producer/consumer) initializes the Authentication,
Kafka Server autheni
In Kafka Authentication happens when establishing connection to the broker, where as Authorization verification happens on each request.
Configured through Client jaas config, update credentials to Subject
AuthenticateCallbackHandler.handle(Callback) <- sasl.login.callback.handler.class(org.apache.kafka.common.security.auth.AuthenticateCallbackHandler), load token/credentails and return it through Callback
Configured through Client jaas config, update credentials to Subject
AuthenticateCallbackHandler.handle(Callback) <- sasl.login.callback.handler.class(org.apache.kafka.common.security.auth.AuthenticateCallbackHandler), load token/credentails and return it through Callback
Configured through Client jaas config, update credentials to Subject
AuthenticateCallbackHandler.handle(Callback) <- sasl.login.callback.handler.class(org.apache.kafka.common.security.auth.AuthenticateCallbackHandler), load token/credentails and return it through Callback
Configured through Client jaas config, update credentials to Subject
AuthenticateCallbackHandler.handle(Callback) <- sasl.login.callback.handler.class(org.apache.kafka.common.security.auth.AuthenticateCallbackHandler), load token/credentails and return it through Callback
Configured through Client jaas config, update credentials to Subject
AuthenticateCallbackHandler.handle(Callback) <- sasl.login.callback.handler.class(org.apache.kafka.common.security.auth.AuthenticateCallbackHandler), load token/credentails and return it through Callback
This is the work flow if we want to integrate with in-house key management system.
You can see with the current infrastructure, clients need to call key management system and convert is to file-based security material then connect to kafka cluster
Which we think is unnecessary
KMS call from loader implementation that we provide
Alright! We are almost nearing the end of the presentation and I would like to highlight some of the cool and challenging things we are working on in the Kafka team.
With that I would like to thank you all for listening! Now, we can take questions if you have. Thank you!