Bringing Apache Cassandra closer to Kubernetes

•

0 likes•2 views

What does Kubernetes provide that allows us to reduce the complexity of Apache Cassandra while making it better suited for cloud native deployments? That was the question we started with as we began a mission to bring Cassandra closer to Kubernetes and eliminate the redundancy. Many great open source databases have been adapted to run on Kubernetes, without relying on the deep ecosystem of projects that it takes to run in Kubernetes(there is a difference). This talk will discuss the design and implementation of the Astra Serverless Database which re-architected Apache Cassandra to run only on Kubernetes infrastructure. Built to be optimized for multi-tenancy and auto-scaling, we set out with a design goal to completely separate compute and storage. Decoupling different aspects of Cassandra into scaleable services and relying on the benefits of Kubernetes and it's ecosystem created a simpler more powerful database service than a stand alone, bare-metal Cassandra cluster. The entire system is now built on Apache Cassandra, Stargate, Etcd, Prometheus, and object-storage like Minio or Ceph. In this talk we will discuss the downstream changes coming to several open source projects based on the work we have done. This talk was given by Jake Luciani for DoK Day Europe @ KubeCon 2022.

Technology

Jake Luciani
DoK Day Europe 2022 @ KubeCon
Bringing Apache Cassandra
closer to Kubernetes

Cassandra Architecture
➔ Assumes it’s running on fixed hardware
➔ All nodes perform all actions
➔ Scale all or nothing at once (data, compute, network)
➔ N^2 direct network interconnects used for Reads and Writes (P2P)
➔ Does not do multi-tenant
➔ Does not auto-scale

A more serverless Cassandra
➔ Scales across different axes
◆ Data size
◆ CPU vs Storage
◆ Reads vs Writes
➔ Is Elastic
◆ Scales quickly & cheaply
➔ Kubernetes Only
◆ Can rely on cloud native ecosystem directly
◆ New Operator that manages scale-up/down intelligently

Fixed to Flexible - Smashing the monolith
Separate out parts of C* into different services and replace some bits with cloud-native
standards
• Coordination Service (Stargate)
• Data Service
• Compaction Service
• Metadata Service to replace gossip (ETCD)
• Consistent Schema management
• Consistent Topology Changes
• Data lives in object storage (S3/Minio/etc)
• Kubernetes Operators for Scaling / Upgrades Stargate
Read / Write
Compaction
User
Data
Repair
Cluster
State
Auth
Data
DoK Day Europe 2022 @ KubeCon

DoK Day Europe 2022 @ KubeCon
User App
CQL Driver
Internet / Private Link
NGINX / ingress
Stargate
S
3
K
u
b
e
r
n
e
t
e
s
O
p
e
r
a
t
o
r
Live Data
Background Data
Cleaners
Regional
Prometheus Monitoring Etcd / API server
Region 1
Cassandra on Kubernetes - Service Map

DoK Day Europe 2022 @ KubeCon
Learn More!
https://bit.ly/3MS23lE
Thank you!

Similar to Bringing Apache Cassandra closer to Kubernetes

Kubernetes intro

Pravin Magdum

Robust Containers by Eric Brewer

Docker, Inc.

No sql databases

Ankit Dubey

prodops.io k8s presentation

Prodops.io

Have you ever thought about migrating your Kubernetes clusters to Google Cloud to get your services closer to your customers? Yes? We too! Join us on an interactive journey to discover the main challenges of live migration at scale of etcd's, traffic routing and application workloads from your on-premise platform to GCP. The talk will discuss the current state of the technical concept, known problems and insides of the already proven migration steps for stateless workload. As part of the journey, we'll see the differences between migrating one or one hundred clusters with productive workloads; What parts can be automated? What steps may need to be manual? Let's see how an automated solution could look like in the future and what steps are missing.

Kubermatic How to Migrate 100 Clusters from On-Prem to Google Cloud Without D...

Tobias Schneck

Have you ever thought about migrating your Kubernetes clusters to Google Cloud to get your services closer to your customers? Yes? Us too! Join us on an interactive journey to discover the main challenges of live migration at scale of etcd’s, traffic routing and application workloads from your on-premise platform to GCP. The talk will discuss the current state of the technical concept, known problems and insides of the already proven migration steps for stateless workloads. As part of the journey, we'll see - The differences between migrating one or one hundred clusters with productive workloads - What parts can be automated? - What steps may need to be done manually?

How to Migrate 100 Clusters from On-Prem to Google Cloud Without Downtime

loodse

Slide presentation from Webinar on February 17, 2016. People in analytical roles are demanding more and more compute and storage to get their jobs done. Instead of building out infrastructure for a few employees or a department, systems engineers and IT managers can find value in creating a compute stack in the cloud to meet the fluctuating demand of their clients. In this 45-minute webinar, you’ll learn: - How to identify the right analytical workloads - How to create a scalable compute environment using the cloud for analysts in under 10 minutes - How to best manage costs associated with the cloud compute stack - How to create dedicated client stacks with their own scratch space as well as general access to reference data Health systems departments, research & development departments, and business analyst groups all face silos of these challenging, compute-intensive use cases. By learning how to quickly build this flexible workflow that can be scaled up and down (or off) instantly, you can support business objectives while efficiently managing costs.

Building a Just-in-Time Application Stack for Analysts

Avere Systems

The application landscape inside our data center is changing: Along with the trend of moving toward microservices and containers, there are a number of new distributed data processing frameworks such as Kafka or Cassandra being released on a weekly basis. These changes have implications for the ways we think about infrastructure. With the growing need for computing power and the rise of distributed applications comes the need for a reliable and simple-use cluster manager and programming abstraction. In this presentation, Mesosphere explains how to use DC/OS to manage microservices and fast data systems on a single platform. We will look at how container orchestration, including resource management and service management, can be streamlined to process fast data in a matter of seconds, allowing for predictive user interfaces, product recommendations, and billing charge back, among other modern app components.

Manage Microservices & Fast Data Systems on One Platform w/ DC/OS

Mesosphere Inc.

Citi Tech Talk: Hybrid Cloud

confluent

Supporting bioinformatics applications with hybrid multi-cloud services

Ahmed Abdullah

Openstack For Beginners

cpallares

By simply looking at structured and unstructured data, Data Lakes enable companies to understand correlations between existing and new external data - such as social media - in ways traditional Business Intelligence tools cannot. For this you need to find out the most efficient way to store and access structured or unstructured petabyte-sized data across your entire infrastructure. In this meetup we’ll give answers on the next questions: 1. Why would someone use a Data Lake? 2. Is it hard to build a Data Lake? 3. What are the main features that a Data Lake should bring in? 4. What’s the role of the microservices in the big data world?

Data Lake and the rise of the microservices

Bigstep

In this webinar experts from DataStax - the lead developer of Cassandra - and from MayaData - the lead developer of OpenEBS and LitmusChaos - will discuss and demonstrate ways to ensure the ease of use and resilience of Cassandra on Kubernetes. Topics to be discussed and demonstrated include: Provisioning underlying storage - how to make it consistent irrespective of the underlying hardware or cloud? Are there are ever reasons to have the storage replicate across nodes or is dynamic LocalPV the best choice in all cases? Cass Operator - DataStax Kubernetes Operator for Apache Cassandra Resilience - how to proactively assess the overall environment including the underlying Kubernetes with the help of Litmus

MayaData Datastax webinar - Operating Cassandra on Kubernetes with the help ...

MayaData Inc

Cloudian_Cassandra Summit 2012

CLOUDIAN KK

Moving to the cloud is hard, and moving Postgres databases to the cloud is even harder. Public cloud or private cloud? Infrastructure as a Service (IaaS), or Platform as a Service (PaaS)? Kubernetes for the application, or for the database and the application? This talk will juxtapose self-managed Kubernetes and container-based database solutions, Postgres deployments on IaaS, and Postgres DBaaS solutions of which EDB’s DBaaS BigAnimal is the latest example.

Cloud Migration Paths: Kubernetes, IaaS, or DBaaS

EDB

On this IT sense, we will talk about the Federation. The Federation is a very flexible tool for connecting several clusters and gives us a lot possibilities in K8s control. With the help of the Federation, we can easily: - sync resources across clusters - cross cluster discovery We will also discuss how to connect several Kubernetes clusters into one network, how to reach services in each Kubernetes cluster...

DevOps Fest 2019. Stanislav Kolenkin. Сonnecting pool Kubernetes clusters: Fe...

DevOps_Fest

Kafka streams decoupling with stores

Yoni Farin

NewSQL - Deliverance from BASE and back to SQL and ACID

Tony Rogerson

Slides built to upskill and enable internal team and/or partners on foundational infra skills to work in a containerized world. Topics covered - Container / Containerization - Docker - k8s / container orchestration - Microservices - Service Mesh / Serverless - Cloud Native (apps & infra) - Relationship between Kubernetes and Runtime Fabric Audiences: MuleSoft internal technical team, partners, Runtime Fabric users.

Introduction to containers, k8s, Microservices & Cloud Native

Terry Wang

Managed Cluster Services

Adam Doyle

Similar to Bringing Apache Cassandra closer to Kubernetes (20)

Kubernetes intro

Robust Containers by Eric Brewer

No sql databases

prodops.io k8s presentation

Kubermatic How to Migrate 100 Clusters from On-Prem to Google Cloud Without D...

How to Migrate 100 Clusters from On-Prem to Google Cloud Without Downtime

Building a Just-in-Time Application Stack for Analysts

Manage Microservices & Fast Data Systems on One Platform w/ DC/OS

Citi Tech Talk: Hybrid Cloud

Supporting bioinformatics applications with hybrid multi-cloud services

Openstack For Beginners

Data Lake and the rise of the microservices

MayaData Datastax webinar - Operating Cassandra on Kubernetes with the help ...

Cloudian_Cassandra Summit 2012

Cloud Migration Paths: Kubernetes, IaaS, or DBaaS

DevOps Fest 2019. Stanislav Kolenkin. Сonnecting pool Kubernetes clusters: Fe...

Kafka streams decoupling with stores

NewSQL - Deliverance from BASE and back to SQL and ACID

Introduction to containers, k8s, Microservices & Cloud Native

Managed Cluster Services

Distributed Vector Databases - What, Why, and How

DoKC

Is It Safe? Security Hardening for Databases Using Kubernetes Operators - Robert Hodges, Altinity Thanks to the Operator Pattern, Kubernetes is now an outstanding platform to run databases. But to quote Marathon Man, "is it safe?" This talk is a top-level review of the database security problem in Kubernetes, standard ways that operators can mitigate threats, and a wallet-sized checklist of security features you should look for in any operator you use. Our talk is practical and focused on needs of Kubernetes developers. Join us!

Is It Safe? Security Hardening for Databases Using Kubernetes Operators

DoKC

Stop Worrying and Keep Querying, Using Automated Multi-Region Disaster Recovery - Shivani Gupta, Elotl & Sergey Pronin, Percona Disaster Recovery(DR) is critical for business continuity in the face of widespread outages taking down entire data centers or cloud provider regions. DR relies on deployment to multiple locations, data replication, monitoring for failure and failover. The process is typically manual involving several moving parts, and, even in the best case, involves some downtime for end-users. A multi-cluster K8s control plane presents the opportunity to automate the DR setup as well as the failure detection and failover. Such automation can dramatically reduce RTO and improve availability for end-users. This talk (and demo) describes one such setup using the open source Percona Operator for PostgreSQL and a multi-cluster K8s orchestrator. The orchestrator will use policy driven placement to replicate the entire workload on multiple clusters (in different regions), detect failure using pluggable logic, and do failover processing by promoting the standby as well as redirecting application traffic

Stop Worrying and Keep Querying, Using Automated Multi-Region Disaster Recovery

DoKC

Transforming Data Processing with Kubernetes: Journey Towards a Self-Serve Data Mesh - Rakesh Subramanian Suresh & Jainik Vora, Intuit This presentation explores how Intuit uses Kubernetes with Domain-Driven Design and Data Mesh principles to transform its data processing landscape, crucial for its AI-driven expert platform. We will discuss the importance of clean data in developing robust generative artificial intelligence and how Intuit is addressing this through the creation of paved paths for data platforms running on Kubernetes. We'll examine the challenges and solutions in managing 100,000 data pipelines and 1000+ engineers interacting with data, highlighting the need for scalable solutions. We'll also discuss how Intuit uses Kubernetes to build its batch and stream processing platform, overcoming hurdles in data pipeline deployment, scheduling, orchestration, and dependency management. We'll conclude by emphasizing how this transformation, based on treating data as a product, has improved decision-making speed and accuracy across the organization and fostered a more efficient, collaborative data culture.

Transforming Data Processing with Kubernetes: Journey Towards a Self-Serve Da...

DoKC

The State of Stateful on Kubernetes - Stateful Workloads in Kubernetes: A Deep Dive - Kaslin Fields & Michelle Au, Google As a platform for distributed computing, Kubernetes enables users to run their workloads across machines. However data has gravity, and when workloads in Kubernetes have to share data with other applications, managing the application’s requirements can get more tricky. In this talk, we will explore what "Stateful" means from Kubernetes' perspective. We will discuss the different types of stateful workloads, and the challenges of deploying them on Kubernetes. We will also look at the features that exist in Kubernetes to support stateful workloads, as well as the features that are in the works. Key Takeaways: What is a stateful workload from Kubernetes’ perspective? What are the challenges of deploying stateful workloads on Kubernetes? What features exist in Kubernetes to support stateful workloads? What features are in the works to support stateful workloads better in the future?

The State of Stateful on Kubernetes

DoKC

Colocating Data Workloads and Web Services on Kubernetes to Improve Resource Utilization - He Cao, ByteDance Recently, more and more data workloads are running on top of Kubernetes, such as ETL processes, Spark and Flink jobs, and more. These workloads typically exhibit high resource utilization and remain relatively stable over time. In contrast, web services often exhibit tidal patterns, characterized by significant fluctuations in resource utilization. The resource model of vanilla Kubernetes is static, which can lead to low resource utilization accumulated over 24 hours. In this talk, He will introduce how ByteDance uses Katalyst to colocate data workloads and online services on Kubernetes to improve resource utilization. In addition, He will explain how Katalyst ensures the QoS of these workloads through QoS-aware scheduling, service profiling, multi-dimensional resource isolation, real-time container resource adjustment, and more. In ByteDance, Katalyst has been deployed on 500,000+ nodes with tens of millions of cores, and has improved daily resource utilization from 20% to 60%.

Colocating Data Workloads and Web Services on Kubernetes to Improve Resource ...

DoKC

Make Your Kafka Cluster Production-Ready - Jakub Scholz, Red Hat Kubernetes became the de-facto standard for running cloud-native applications. And more and more users turn to it also to run stateful applications such as Apache Kafka. While there are different tools such as Helm charts or operators which can get you quickly up and running, there is often still a long way to make sure the Kafka cluster is production-ready. This talk will take you through the main aspects you should consider for your Kafka cluster and will cover things such as resource management, storage, scheduling, rolling updates, or reliability. It will show you how to do it using the Strimzi operator, but the lessons learned will apply also to any other Kafka cluster. If you are interested in production-ready Apache Kafka on Kubernetes, this is a talk for you.

Make Your Kafka Cluster Production-Ready

DoKC

Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo Workflows and Argo Events - Ovidiu Valeanu, AWS & Vara Bonthu, Amazon Are you eager to build and manage large-scale Spark clusters on Kubernetes for powerful data processing? Whether you are starting from scratch or considering migrating Spark workloads from existing Hadoop clusters to Kubernetes, the challenges of configuring storage, compute, networking, and optimizing job scheduling can be daunting. Join us as we unveil the best practices to construct a scalable Spark clusters on Kubernetes, with a special emphasis on leveraging Argo Workflows and Argo Events. In this talk, we will guide you through the journey of building highly scalable Spark clusters on Kubernetes, using the most popular open-source tools. We will showcase how to harness the potential of Argo Workflows and Argo Events for event-driven job scheduling, enabling efficient resource utilization and seamless scalability. By integrating these powerful tools, you will gain better control and flexibility for executing Spark jobs on Kubernetes.

Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo W...

DoKC

Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud - Sagy Volkov, Lightbits PostgreSQL as a SQL engine can accommodate a very high-transaction rate, but as your data grows and the number of connections and queries increases, there is a challenge for the storage to keep up with the SQL engine. To the rescue comes NVMe over TCP (or NVMe/TCP). Developed by Lightbits Labs in 2016 and donated to the Linux community, it is the next evaluation of using NVMe based storage over TCP Fabric. NVMe/TCP simplifies how you interact with remote NVMe devices (targets) and allows your PostgreSQL storage to consume fast storage very easily. In this session I will explain the core concept of the NVMe/TCP protocol, current storage providers that can use it, how you can consume it in Kubernetes (super easy), and discuss the possibilities of using NVMe/TCP in the cloud. The session will also include a performance comparison of a few storage that are available in AWS and even a live demo of how PostgreSQL can run super fast - warp speed fast - in AWS.

Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud

DoKC

An explanation of how ING deals with local persistence at scale in secure and compliant manner for Elastic and Prometheus workloads today and other Data Services in the future. In more detail we will elaborate on the following topics How we solve local persistence Type of workloads now and in the future Typical requirements for a banking environment Automation Scale Resilience Security / Compliance Service offering / demarcation About Tor and Luuk: Tor and Luuk are experienced engineers working at ING for over 10 years and working in the Kubernetes area for the last 5 years. They are specialized in and responsible for the Data Services OpenShift clusters in ING and have a strong focus on resilience, automation and security.

ING Data Services hosted on ICHP DoK Amsterdam 2023

DoKC

A small walkthrough of projects within the dutch government running Data(bases) on OpenShift. This talk shares success stories, provides a proven recipe to `get it done` and debunks some of the FUD. About Sebastiaan: I have always been a weird DBA, trying to combine Databases with out-of-the-box thinking and a DevOps mindset. Around 2016 I fell in love with both Postgres and Kubernetes, and I then committed my life to enabling Dutch organisations with running their Database workloads CloudNative. Over the last few years I worked as a private contractor for 2 large government agencies doing exactly that, and I want to share my and others (success stories) hoping to enable and inspire Data on Kubernetes adoption.

Implementing data and databases on K8s within the Dutch government

DoKC

https://go.dok.community/slack https://dok.community/ Link: https://youtu.be/n_thXwyJNSU ABSTRACT OF THE TALK Deploying Stateless applications is easy but this is not the case for Stateful applications. StatefulSets are the K8s API object that helps to manage stateful application. Learn about what Stateful sets are, how to create, How it differs from Deployments. KEY TAKE-AWAYS FROM THE TALK This talk is focused on basics of StatefulSet, how StatefulSet differs from Deployments, How to manage Stateful app using StatefulSet

StatefulSets in K8s - DoK Talks #154

DoKC

Link: https://youtu.be/cegd3Exg05w https://go.dok.community/slack https://dok.community/ Gabriele Bartolini - Vice President/CTO of Cloud Native and Kubernetes, EDB ABSTRACT OF THE TALK Imagine this: you have a virtual infrastructure based on Kubernetes, made up of virtual data centers, possibly spread across multiple Kubernetes clusters and regions. Your infrastructure could even be hosted on premises or on different cloud service providers. Infrastructure as Code is a requirement. You’ve been tasked to run Postgres databases, alongside your applications. The good news is that you can leverage a fully open source stack with Kubernetes, PostgreSQL and the CloudNativePG operator, and deploy your Postgres database in the same way you deploy applications. Join me in this webinar to discover the key role that you have to make this succeed, starting from day 0 through day 2 operations. I’ll share some examples and best practices for running Postgres databases in Kubernetes, before peeking at the new features we are developing for the months to come.

Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...

DoKC

Link: https://youtu.be/Y-1uFVKDfgY https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK This talk concerns performing analytical tasks with Apache Superset with ClickHouse as the data backend. ClickHouse is a super fast database for analytical tasks, and Apache Superset is an Apache Software foundation project meant for data visualization and exploration. Performing analytical tasks using this combo is super fast since both the software are designed to be scalable and capable of handling data of petabyte scale.

Analytics with Apache Superset and ClickHouse - DoK Talks #151

DoKC

Link: https://youtu.be/EFaRyl4HmmE https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK If you are running or planning a multi-cloud or even a multi-cluster environment, there are several considerations in implementing a data protection solution – especially if you plan on an organic home-grown, do-it-yourself option. This talk will highlight challenges and best practices around centralized management of configuration, credentials, compliance across multiple accounts, regions, providers etc. We will also highlight the deviations in CSI driver implementations of various storage vendors and cloud providers. Finally, we will cover the various recovery options available in the market today. Kubernetes cloud services are popular since they mitigate, but do not eliminate, the difficulties of operating a Kubernetes environment. This is especially true for protecting the stateful configuration and data of your Kubernetes applications, where the inherent high-availability and infrastructure as code are not a substitute for have cloud-native backup and disaster recovery capabilities. Further, many companies now have multi-cloud strategies for their cloud-native applications. These challenges can be addressed with backup applications that are both Kubernetes managed service and multi-cloud aware in order to snapshot, copy, restore, and migrate Kubernetes workloads (resources and data) running on AKS, EKS and GKE. Capturing information from cloud accounts and how the cluster and storage resources are configured allows 1) centralized visibility into all cloud accounts and the clusters and resources in the accounts including for compliance; 2) cross-account, cross-cluster, and cross-region data restores; 3) automation of the cluster and data restores including for Dev, Test, and Production recovery use cases. BIO Sebastian Glab is a Cloud Architect for CloudCasa and he resides in Poland. He is responsible for integrating the different cloud providers with the CloudCasa service, and making sure that all clusters in the cloud service get discovered and protected. In his free time, he plays volleyball and develops his own projects. Martin Phan is the Field CTO in North America for CloudCasa by Catalogic Software. With over 20+ years of experience in the software-industry, he takes pride in supporting, developing, implementing, and selling enterprise software and data protection solutions to help customer solve their backup and recovery challenges. KEY TAKE-AWAYS FROM THE TALK 1) Challenges and best practices around centralized management of configuration, credentials, compliance across multiple accounts, regions, providers etc. 2) Advantages of cloud awareness and Kubernetes managed service awareness for application and data recovery and security 3) Examples of overcoming Container Storage Interface (CSI) deviations 4) Various recovery options available in the market today.

Overcoming challenges with protecting and migrating data in multi-cloud K8s e...

DoKC

Link: https://youtu.be/YVXEpcSclwY https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK In a continuation of a talk given at DoK day at KubeCon EU 2022, join Dinesh Majrekar, Civo's CTO as they walk through their evaluation process of the CNCF Storage market. Civo offers managed Kubernetes clusters powered by K3s to customers around the world. We manage thousands of Virtual Machines and stateful customer data within multiple data centres across several continents. In late 2021, Civo had the opportunity to evaluate the CNCF storage landscape to move to a new technology stack. During the migration project, Civo evaluated Mayastor, Ondat, Ceph and Longhorn against the following metrics: Scalability Performance Ease of Support Attendants will see practical examples on how they could carry out their own similar evaluation and see some of the results of the Civo research project. BIO Dinesh is CTO at Civo. Having worked in the hosting industry for many years, Dinesh has a passion for creating solutions that operate at scale. This not only applies to the technology stack, but for nurturing engineers through their career.

Evaluating Cloud Native Storage Vendors - DoK Talks #147

DoKC

Link: https://youtu.be/qUW8LkxYayc https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK How do you make sure your Stateful Workloads remain available when your Kubernetes infrastructure updates? This talk will discuss different strategies of upgrading a Kubernetes cluster, and how you can manage risk for your workload. The talk will showcase demos of each upgrade strategy. BIO Peter is a Senior Software Engineer on GKE at Google. He works on improving Kubernetes for Stateful workloads. His main focus is on enhancing the Kubernetes ecosystem for high availability applications. KEY TAKE-AWAYS FROM THE TALK The mechanics of different upgrade strategies, when to apply a particular upgrade strategy depending on your Stateful workload and how to mitigate risk to your application’s availability.

Kubernetes Cluster Upgrade Strategies and Data: Best Practices for your State...

DoKC

Link: https://youtu.be/AjvwG53yLMY https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Stateful workloads are the heart of any application, yet they remain confusing and complicated even to daily K8s practitioners. That’s why many organizations shy away from migrating their data - their prized possession - to the unfamiliar stateful realm of Kubernetes. After meeting with many organizations in the adoption phase, I discovered what works best, what to avoid, and how critical it is to gain confidence and the right knowledge in order to successfully adopt stateful workloads. In this talk I will demonstrate how to optimally adopt Kubernetes and stateful workloads in a few steps, based on what I’ve learned from observing dozens of different adoption journeys. If you are taking your first steps in data on K8s or contemplating where to start - this talk is for you! BIO - A Developer turned Solution Architect. - Working at Komodor, a startup building the first K8s-native troubleshooting platform. - Love everything in infrastructure: storage, networks & security - from 70’s era mainframes to cloud-native. - All about “plan well, sleep well”. KEY TAKE-AWAYS FROM THE TALK - Understand how critical stateful workloads are for any system, and that the key challenges to migrating it to Kubernetes are knowledge and confidence. - How to build the foundational knowledge required to overcome adoption challenges by creating a learning path for individuals and teams. - How to gain confidence to run stateful workloads on Kubernetes with support from the community (and yourself!)

We will Dok You! - The journey to adopt stateful workloads on k8s

DoKC

Link: https://youtu.be/Pi5ueyl_1jU https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK During my first talk for DoK community I want to walk you through the world of NoSQL database MongoDB and Kubernetes Operators - Community Edition, Enterprise Edition (MongoDB and Ops Manager on K8s), and Atlas operator, highlight the most important capabilities, talk about use cases and challenges, the theory will be mixed with a live demos! BIO I'm a SRE / NoSQL / DevOps professional. I hold CKA, CKAD, CKS, also I’m MongoDB Certified DBA and MongoDB Champion. I have experience with multiple cloud providers, Kubernetes, different types of K8s operators (Strimzi, RabbitMQ Cluster Operator), but especially MongoDB K8s Operator. I also work with KEDA. Since 2017, I have been a speaker at MongoDB conferences all around the world (USA, China, Europe). KEY TAKE-AWAYS FROM THE TALK I would like to share the best practices of running NoSQL database - MongoDB on Kubernetes also I want to show how to manage Atlas (MongoDB cloud) via K8s operator https://www.mongodb.com/developer/community-champions/arkadiusz-borucki/

Mastering MongoDB on Kubernetes, the power of operators

DoKC

Link: https://youtu.be/KUipuM3UJF4 https://go.dok.community/slack https://dok.community/ DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE) Kubernetes comes with a lot of useful features like Volumes and StatefulSets, which make running stateful workloads simple. Interestingly, when combined with the right tools, these features can make Kubernetes very valuable for developers wanting to run massive production databases in development! This is exactly what was seen at "Extendi". The developers at Extendi deal with a large amount of data in their production Kubernetes clusters. But when developing locally, they didn't have an easy way of replicating this data. This replication was needed because it allowed developers to test new features instantaneously without worrying if they would work as expected when pushed to production. But replicating a 100Gb+ production database for development wasn't turning out to be an easy task! This is where leveraging Kubernetes + remote development environments came to the rescue. Running data on Kubernetes turned out to be way faster than any of the traditional approaches because of Kubernetes' ability to handle stateful workloads exceptionally well. And since Extendi already used Kubernetes in production - the setup process was fairly simple. This talk will cover practical steps on how leveraging Kubernetes based development environments allowed dev teams at Extendi to run production data on Kubernetes during development using features like Volume Snapshots, having a huge positive impact on developer productivity. ----- Arsh is a Developer Experience Engineer at Okteto. He is an active contributor to the upstream Kubernetes project and was awarded the Kubernetes Contributor Award for his contributions in 2021. Arsh has written blogs and spoken about different topics in the cloud-native ecosystem at various conferences before, including KubeCon + CloudNativeCon + Open Source Summit China 2021. He has also been on the Kubernetes Release Team since the 1.23 release. He also serves as the New Contributor Ambassador for the Documentation Special Interest Group of the Kubernetes project and continuously mentors new folks in the community. Previously, he worked at VMware and was an active contributor to other CNCF projects, including cert-manager and Kyverno. ----- Lapo is a Software Engineer currently leading the development team of a Social Listening and Audience Intelligence platform. He started coding at the early age of 14 and since he turned his passion into a real job, he has always been looking for boosting his knowledge by constantly researching for newer and newer technologies. Active on Ruby Open Source projects ----- Ramiro Berrelleza is one of the founders of Okteto. He has spent most of his career (and his free time) building cloud services and developer tools. Before starting Okteto, Ramiro was an Architect at Atlassian and a Software Engineer at Microsoft Azure.

Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...

DoKC

More from DoKC (20)

Distributed Vector Databases - What, Why, and How

Is It Safe? Security Hardening for Databases Using Kubernetes Operators

Stop Worrying and Keep Querying, Using Automated Multi-Region Disaster Recovery

Transforming Data Processing with Kubernetes: Journey Towards a Self-Serve Da...

The State of Stateful on Kubernetes

Colocating Data Workloads and Web Services on Kubernetes to Improve Resource ...

Make Your Kafka Cluster Production-Ready

Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo W...

Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud

ING Data Services hosted on ICHP DoK Amsterdam 2023

Implementing data and databases on K8s within the Dutch government

StatefulSets in K8s - DoK Talks #154

Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...

Analytics with Apache Superset and ClickHouse - DoK Talks #151

Overcoming challenges with protecting and migrating data in multi-cloud K8s e...

Evaluating Cloud Native Storage Vendors - DoK Talks #147

Kubernetes Cluster Upgrade Strategies and Data: Best Practices for your State...

We will Dok You! - The journey to adopt stateful workloads on k8s

Mastering MongoDB on Kubernetes, the power of operators

Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...

Recently uploaded

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Juan lago vázquez

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

In the thrilling conclusion to 2023, ransomware groups had a banner year, really outdoing themselves in the "make everyone's life miserable" department. LockBit 3.0 took gold in the hacking olympics, followed by the plucky upstarts Clop and ALPHV/BlackCat. Apparently, 48% of organizations were feeling left out and decided to get in on the cyber attack action. Business services won the "most likely to get digitally mugged" award, with education and retail nipping at their heels. Hackers expanded their repertoire beyond boring old encryption to the much more exciting world of extortion. The US, UK and Canada took top honors in the "countries most likely to pay up" category. Bitcoins were the currency of choice for discerning hackers, because who doesn't love untraceable money?

Ransomware_Q4_2023. The report. [EN].pdf

Overkill Security

💉💊+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHABI}}+971581248768 +971581248768 Mtp-Kit (500MG) Prices » Dubai [(+971581248768**)] Abortion Pills For Sale In Dubai, UAE, Mifepristone and Misoprostol Tablets Available In Dubai, UAE CONTACT DR.Maya Whatsapp +971581248768 We Have Abortion Pills / Cytotec Tablets /Mifegest Kit Available in Dubai, Sharjah, Abudhabi, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE, Buy cytotec in Dubai +971581248768''''Abortion Pills near me DUBAI | ABU DHABI|UAE. Price of Misoprostol, Cytotec” +971581248768' Dr.DEEM ''BUY ABORTION PILLS MIFEGEST KIT, MISOPROTONE, CYTOTEC PILLS IN DUBAI, ABU DHABI,UAE'' Contact me now via What's App…… abortion Pills Cytotec also available Oman Qatar Doha Saudi Arabia Bahrain Above all, Cytotec Abortion Pills are Available In Dubai / UAE, you will be very happy to do abortion in Dubai we are providing cytotec 200mg abortion pill in Dubai, UAE. Medication abortion offers an alternative to Surgical Abortion for women in the early weeks of pregnancy. We only offer abortion pills from 1 week-6 Months. We then advise you to use surgery if its beyond 6 months. Our Abu Dhabi, Ajman, Al Ain, Dubai, Fujairah, Ras Al Khaimah (RAK), Sharjah, Umm Al Quwain (UAQ) United Arab Emirates Abortion Clinic provides the safest and most advanced techniques for providing non-surgical, medical and surgical abortion methods for early through late second trimester, including the Abortion By Pill Procedure (RU 486, Mifeprex, Mifepristone, early options French Abortion Pill), Tamoxifen, Methotrexate and Cytotec (Misoprostol). The Abu Dhabi, United Arab Emirates Abortion Clinic performs Same Day Abortion Procedure using medications that are taken on the first day of the office visit and will cause the abortion to occur generally within 4 to 6 hours (as early as 30 minutes) for patients who are 3 to 12 weeks pregnant. When Mifepristone and Misoprostol are used, 50% of patients complete in 4 to 6 hours; 75% to 80% in 12 hours; and 90% in 24 hours. We use a regimen that allows for completion without the need for surgery 99% of the time. All advanced second trimester and late term pregnancies at our Tampa clinic (17 to 24 weeks or greater) can be completed within 24 hours or less 99% of the time without the need surgery. The procedure is completed with minimal to no complications. Our Women's Health Center located in Abu Dhabi, United Arab Emirates, uses the latest medications for medical abortions (RU-486, Mifeprex, Mifegyne, Mifepristone, early options French abortion pill), Methotrexate and Cytotec (Misoprostol). The safety standards of our Abu Dhabi, United Arab Emirates Abortion Doctors remain unparalleled. They consistently maintain the lowest complication rates throughout the nation. Our Physicians and staff are always available to answer questions and care for women in one of the most difficult times in their lives. The decision to have an abortion at the Abortion Cl

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

sammart93

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

Accelerating FinTech Innovation: Unleashing API Economy and GenAI Vasa Krishnan, Chief Technology Officer - FinResults Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

apidays

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Edi Saputra

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

Corporate and higher education. Two industries that, in the past, have had a clear divide with very little crossover. The difference in goals, learning styles and objectives paved the way for differing learning technologies platforms to evolve. Now, those stark lines are blurring as both sides are discovering they have content that’s relevant to the other. Join Tammy Rutherford as she walks through the pros and cons of corporate and higher ed collaborating. And the challenges of these different technology platforms working together for a brighter future.

Corporate and higher education May webinar.pptx

Rustici Software

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Zilliz

FWD Group - Insurer Innovation Award 2024

The Digital Insurer

GenAI Risks & Security Meetup 01052024.pdf

lior mazor

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

Manulife - Insurer Transformation Award 2024

The Digital Insurer

Effective data discovery is crucial for maintaining compliance and mitigating risks in today's rapidly evolving privacy landscape. However, traditional manual approaches often struggle to keep pace with the growing volume and complexity of data. Join us for an insightful webinar where industry leaders from TrustArc and Privya will share their expertise on leveraging AI-powered solutions to revolutionize data discovery. You'll learn how to: - Effortlessly maintain a comprehensive, up-to-date data inventory - Harness code scanning insights to gain complete visibility into data flows leveraging the advantages of code scanning over DB scanning - Simplify compliance by leveraging Privya's integration with TrustArc - Implement proven strategies to mitigate third-party risks Our panel of experts will discuss real-world case studies and share practical strategies for overcoming common data discovery challenges. They'll also explore the latest trends and innovations in AI-driven data management, and how these technologies can help organizations stay ahead of the curve in an ever-changing privacy landscape.

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc

This presentations targets students or working professionals. You may know Google for search, YouTube, Android, Chrome, and Gmail, but did you know Google has many developer tools, platforms & APIs? This comprehensive yet still high-level overview outlines the most impactful tools for where to run your code, store & analyze your data. It will also inspire you as to what's possible. This talk is 50 minutes in length.

Powerful Google developer tools for immediate impact! (2023-24 C)

wesley chun

Artificial Intelligence Chap.5 : Uncertainty

Khushali Kathiriya

Recently uploaded (20)

Strategies for Landing an Oracle DBA Job as a Fresher

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Why Teams call analytics are critical to your entire business

Ransomware_Q4_2023. The report. [EN].pdf

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Axa Assurance Maroc - Insurer Innovation Award 2024

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

AWS Community Day CPH - Three problems of Terraform

Corporate and higher education May webinar.pptx

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

FWD Group - Insurer Innovation Award 2024

GenAI Risks & Security Meetup 01052024.pdf

Exploring the Future Potential of AI-Enabled Smartphone Processors

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

Manulife - Insurer Transformation Award 2024

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

Powerful Google developer tools for immediate impact! (2023-24 C)

Artificial Intelligence Chap.5 : Uncertainty

Bringing Apache Cassandra closer to Kubernetes

1. Jake Luciani DoK Day Europe 2022 @ KubeCon Bringing Apache Cassandra closer to Kubernetes

2. Cassandra Architecture ➔ Assumes it’s running on fixed hardware ➔ All nodes perform all actions ➔ Scale all or nothing at once (data, compute, network) ➔ N^2 direct network interconnects used for Reads and Writes (P2P) ➔ Does not do multi-tenant ➔ Does not auto-scale

3. A more serverless Cassandra ➔ Scales across different axes ◆ Data size ◆ CPU vs Storage ◆ Reads vs Writes ➔ Is Elastic ◆ Scales quickly & cheaply ➔ Kubernetes Only ◆ Can rely on cloud native ecosystem directly ◆ New Operator that manages scale-up/down intelligently

4. Fixed to Flexible - Smashing the monolith Separate out parts of C* into different services and replace some bits with cloud-native standards • Coordination Service (Stargate) • Data Service • Compaction Service • Metadata Service to replace gossip (ETCD) • Consistent Schema management • Consistent Topology Changes • Data lives in object storage (S3/Minio/etc) • Kubernetes Operators for Scaling / Upgrades Stargate Read / Write Compaction User Data Repair Cluster State Auth Data DoK Day Europe 2022 @ KubeCon

5. DoK Day Europe 2022 @ KubeCon User App CQL Driver Internet / Private Link NGINX / ingress Stargate S 3 K u b e r n e t e s O p e r a t o r Live Data Background Data Cleaners Regional Prometheus Monitoring Etcd / API server Region 1 Cassandra on Kubernetes - Service Map

6. DoK Day Europe 2022 @ KubeCon Learn More! https://bit.ly/3MS23lE Thank you!

Bringing Apache Cassandra closer to Kubernetes

Recommended

Recommended

More Related Content

Similar to Bringing Apache Cassandra closer to Kubernetes

Similar to Bringing Apache Cassandra closer to Kubernetes (20)

More from DoKC

More from DoKC (20)

Recently uploaded

Recently uploaded (20)

Bringing Apache Cassandra closer to Kubernetes