We will Dok You! - The journey to adopt stateful workloads on k8s

•

0 likes•7 views

Link: https://youtu.be/AjvwG53yLMY https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Stateful workloads are the heart of any application, yet they remain confusing and complicated even to daily K8s practitioners. That’s why many organizations shy away from migrating their data - their prized possession - to the unfamiliar stateful realm of Kubernetes. After meeting with many organizations in the adoption phase, I discovered what works best, what to avoid, and how critical it is to gain confidence and the right knowledge in order to successfully adopt stateful workloads. In this talk I will demonstrate how to optimally adopt Kubernetes and stateful workloads in a few steps, based on what I’ve learned from observing dozens of different adoption journeys. If you are taking your first steps in data on K8s or contemplating where to start - this talk is for you! BIO - A Developer turned Solution Architect. - Working at Komodor, a startup building the first K8s-native troubleshooting platform. - Love everything in infrastructure: storage, networks & security - from 70’s era mainframes to cloud-native. - All about “plan well, sleep well”. KEY TAKE-AWAYS FROM THE TALK - Understand how critical stateful workloads are for any system, and that the key challenges to migrating it to Kubernetes are knowledge and confidence. - How to build the foundational knowledge required to overcome adoption challenges by creating a learning path for individuals and teams. - How to gain confidence to run stateful workloads on Kubernetes with support from the community (and yourself!)

Software

Bios
Solution Architect @Komodor DevRel @Komodor
the_good_guym UHofesh

Drum & Bass - it all comes down to stateful
● App DBs & queues are the
most critical infrastructure for
any organization
● Most applications rely on
stateful data

The impact of Stateful
Minor Errors
Unavailable application
Severe Downtime
DATA LOSS
High possibility
Low impact
Low possibility
Extreme impact

The key challenges with Kubernetes
Lack of knowledge/expertise
“Can’t read notes”
Lack of confidence/fear
“The stage fright”
Lack of instruments/tools
Lack of band members

Why do it anyway?
● Cost
○ Managed vs unmanaged
○ Resources footprint
○ Licences
● Multi-Cloud
● K8s standardization
○ No VMs until x
● 3rd party modernization
○ New DB/Queues
○ Automated operations
● Self-hosted / on-prem
● Your product is being deployed on hosted clusters

But how?
Hint: Not by living on a prayer!

The organizational learning path
K8s networking | storage | operators K8s foundations
Migrate Dev | Sandbox | Test env. Test workloads
Day 2 Operations: backup &
Restore | upgrades | Performance, Sizing
| Monitoring & Alerting |
Availability & recovery drills
Get ready
For users
Migrate non
critical apps
Dev | Staging |
Non critical workloads
Migration process
| Prod drills |
Playbooks | Monitoring
Dashboards | Load testing
Get ready
for prod
Add mission
critical apps

K8s networking | storage | operators K8s foundations
Migrate Dev | Sandbox | Test env. Test workloads
Day 2 Operations: backup &
Restore | upgrades | Performance, Sizing
| Monitoring & Alerting |
Availability & recovery drills
Get ready
For users
Migrate non
critical apps
Dev | Staging |
Non critical workloads
Migration process
| Prod drills |
Playbooks | Monitoring
Dashboards | Load testing
Get ready
for prod
Add mission
critical apps

The personal learning path
● K8s basics: pods, replicaset, deploy, service
● K8s infra: statefulsets, networking, ips, storage
● Join a community!
● Adopt a tool & its operator: DBs, queues,
● Milestone 1 - low environments
● Day 2 ops ready: backup, restore, upgrades
● Basic ops/availbility drills, manual chaos
● Adopt supporting tools: metrics, logs, alerting, monitors & observability
● Learn & test performance
● Milestone 2 - mid environment
● GitOps, CI/CD, IaC
● Prod support: dashboard, alerting, troubleshooting
● Performance: testing, metrics, underlying infra (dns, network interfaces)
● Day-to-day operations/automations
● Share your knowledge! Keep Learning! Repeat!

How to learn?
The k8s Journey
Documentation
● Kubernetes Docs
● 100 days of k8s
● Civo academy
● Vendors learning centers
● Hashnode/Medium blogs
Videos
● Kunal Kushwaha
● Saiyam Pathak
● TechWorldWithNana
● DevOps Toolkit (Viktor
Farcic)
● Anais Urlichs

Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud

DoKC

Link: https://www.youtube.com/watch?v=D8kJCvsHD9Q&list=PLHgdNuGxrJt04Fwaip9aDYvXrbRSmc5HZ&index=12 https://go.dok.community/slack https://dok.community/ From DoK Day NA 2022 (https://www.youtube.com/watch?v=YWTa-DiVljY&list=PLHgdNuGxrJt04Fwaip9aDYvXrbRSmc5HZ) In the software industry we’re fond of terms that define major trends, like “cloud native”, “Kubernetes native” and “serverless”. As more and more organizations move stateful workloads to Kubernetes, we’ve started to see these terms applied to data infrastructure, where they can get overtaken by marketing hype unless we work to define them. In this talk, we’ll examine two different databases, TiDB and Apache Cassandra, in order to identify what it means for a database to be Kubernetes native and why it matters. We’ll look at points including: - The differences between cloud native, Kubernetes native, and serverless - How databases become Kubernetes native - Benefits of Kubernetes native databases - How Kubernetes can better support databases ----- Jeff has worked as a software engineer and architect in multiple industries and as a developer advocate helping engineers get up to speed on Apache Cassandra. He's involved in multiple open source projects in the Cassandra and Kubernetes ecosystems including Stargate and K8ssandra. Jeff is the author of the O’Reilly books “Cassandra: The Definitive Guide" and “Managing Cloud Native Data on Kubernetes".

The Kubernetes Native Database

DoKC

An explanation of how ING deals with local persistence at scale in secure and compliant manner for Elastic and Prometheus workloads today and other Data Services in the future. In more detail we will elaborate on the following topics How we solve local persistence Type of workloads now and in the future Typical requirements for a banking environment Automation Scale Resilience Security / Compliance Service offering / demarcation About Tor and Luuk: Tor and Luuk are experienced engineers working at ING for over 10 years and working in the Kubernetes area for the last 5 years. They are specialized in and responsible for the Data Services OpenShift clusters in ING and have a strong focus on resilience, automation and security.

ING Data Services hosted on ICHP DoK Amsterdam 2023

DoKC

A small walkthrough of projects within the dutch government running Data(bases) on OpenShift. This talk shares success stories, provides a proven recipe to `get it done` and debunks some of the FUD. About Sebastiaan: I have always been a weird DBA, trying to combine Databases with out-of-the-box thinking and a DevOps mindset. Around 2016 I fell in love with both Postgres and Kubernetes, and I then committed my life to enabling Dutch organisations with running their Database workloads CloudNative. Over the last few years I worked as a private contractor for 2 large government agencies doing exactly that, and I want to share my and others (success stories) hoping to enable and inspire Data on Kubernetes adoption.

Implementing data and databases on K8s within the Dutch government

DoKC

https://go.dok.community/slack https://dok.community/ Link: https://youtu.be/n_thXwyJNSU ABSTRACT OF THE TALK Deploying Stateless applications is easy but this is not the case for Stateful applications. StatefulSets are the K8s API object that helps to manage stateful application. Learn about what Stateful sets are, how to create, How it differs from Deployments. KEY TAKE-AWAYS FROM THE TALK This talk is focused on basics of StatefulSet, how StatefulSet differs from Deployments, How to manage Stateful app using StatefulSet

StatefulSets in K8s - DoK Talks #154

DoKC

Link: https://youtu.be/cegd3Exg05w https://go.dok.community/slack https://dok.community/ Gabriele Bartolini - Vice President/CTO of Cloud Native and Kubernetes, EDB ABSTRACT OF THE TALK Imagine this: you have a virtual infrastructure based on Kubernetes, made up of virtual data centers, possibly spread across multiple Kubernetes clusters and regions. Your infrastructure could even be hosted on premises or on different cloud service providers. Infrastructure as Code is a requirement. You’ve been tasked to run Postgres databases, alongside your applications. The good news is that you can leverage a fully open source stack with Kubernetes, PostgreSQL and the CloudNativePG operator, and deploy your Postgres database in the same way you deploy applications. Join me in this webinar to discover the key role that you have to make this succeed, starting from day 0 through day 2 operations. I’ll share some examples and best practices for running Postgres databases in Kubernetes, before peeking at the new features we are developing for the months to come.

Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...

DoKC

Link: https://youtu.be/Y-1uFVKDfgY https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK This talk concerns performing analytical tasks with Apache Superset with ClickHouse as the data backend. ClickHouse is a super fast database for analytical tasks, and Apache Superset is an Apache Software foundation project meant for data visualization and exploration. Performing analytical tasks using this combo is super fast since both the software are designed to be scalable and capable of handling data of petabyte scale.

Analytics with Apache Superset and ClickHouse - DoK Talks #151

DoKC

Link: https://youtu.be/EFaRyl4HmmE https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK If you are running or planning a multi-cloud or even a multi-cluster environment, there are several considerations in implementing a data protection solution – especially if you plan on an organic home-grown, do-it-yourself option. This talk will highlight challenges and best practices around centralized management of configuration, credentials, compliance across multiple accounts, regions, providers etc. We will also highlight the deviations in CSI driver implementations of various storage vendors and cloud providers. Finally, we will cover the various recovery options available in the market today. Kubernetes cloud services are popular since they mitigate, but do not eliminate, the difficulties of operating a Kubernetes environment. This is especially true for protecting the stateful configuration and data of your Kubernetes applications, where the inherent high-availability and infrastructure as code are not a substitute for have cloud-native backup and disaster recovery capabilities. Further, many companies now have multi-cloud strategies for their cloud-native applications. These challenges can be addressed with backup applications that are both Kubernetes managed service and multi-cloud aware in order to snapshot, copy, restore, and migrate Kubernetes workloads (resources and data) running on AKS, EKS and GKE. Capturing information from cloud accounts and how the cluster and storage resources are configured allows 1) centralized visibility into all cloud accounts and the clusters and resources in the accounts including for compliance; 2) cross-account, cross-cluster, and cross-region data restores; 3) automation of the cluster and data restores including for Dev, Test, and Production recovery use cases. BIO Sebastian Glab is a Cloud Architect for CloudCasa and he resides in Poland. He is responsible for integrating the different cloud providers with the CloudCasa service, and making sure that all clusters in the cloud service get discovered and protected. In his free time, he plays volleyball and develops his own projects. Martin Phan is the Field CTO in North America for CloudCasa by Catalogic Software. With over 20+ years of experience in the software-industry, he takes pride in supporting, developing, implementing, and selling enterprise software and data protection solutions to help customer solve their backup and recovery challenges. KEY TAKE-AWAYS FROM THE TALK 1) Challenges and best practices around centralized management of configuration, credentials, compliance across multiple accounts, regions, providers etc. 2) Advantages of cloud awareness and Kubernetes managed service awareness for application and data recovery and security 3) Examples of overcoming Container Storage Interface (CSI) deviations 4) Various recovery options available in the market today.

Overcoming challenges with protecting and migrating data in multi-cloud K8s e...

DoKC

Link: https://youtu.be/YVXEpcSclwY https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK In a continuation of a talk given at DoK day at KubeCon EU 2022, join Dinesh Majrekar, Civo's CTO as they walk through their evaluation process of the CNCF Storage market. Civo offers managed Kubernetes clusters powered by K3s to customers around the world. We manage thousands of Virtual Machines and stateful customer data within multiple data centres across several continents. In late 2021, Civo had the opportunity to evaluate the CNCF storage landscape to move to a new technology stack. During the migration project, Civo evaluated Mayastor, Ondat, Ceph and Longhorn against the following metrics: Scalability Performance Ease of Support Attendants will see practical examples on how they could carry out their own similar evaluation and see some of the results of the Civo research project. BIO Dinesh is CTO at Civo. Having worked in the hosting industry for many years, Dinesh has a passion for creating solutions that operate at scale. This not only applies to the technology stack, but for nurturing engineers through their career.

Evaluating Cloud Native Storage Vendors - DoK Talks #147

DoKC

Link: https://youtu.be/qUW8LkxYayc https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK How do you make sure your Stateful Workloads remain available when your Kubernetes infrastructure updates? This talk will discuss different strategies of upgrading a Kubernetes cluster, and how you can manage risk for your workload. The talk will showcase demos of each upgrade strategy. BIO Peter is a Senior Software Engineer on GKE at Google. He works on improving Kubernetes for Stateful workloads. His main focus is on enhancing the Kubernetes ecosystem for high availability applications. KEY TAKE-AWAYS FROM THE TALK The mechanics of different upgrade strategies, when to apply a particular upgrade strategy depending on your Stateful workload and how to mitigate risk to your application’s availability.

Kubernetes Cluster Upgrade Strategies and Data: Best Practices for your State...

DoKC

Link: https://youtu.be/Pi5ueyl_1jU https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK During my first talk for DoK community I want to walk you through the world of NoSQL database MongoDB and Kubernetes Operators - Community Edition, Enterprise Edition (MongoDB and Ops Manager on K8s), and Atlas operator, highlight the most important capabilities, talk about use cases and challenges, the theory will be mixed with a live demos! BIO I'm a SRE / NoSQL / DevOps professional. I hold CKA, CKAD, CKS, also I’m MongoDB Certified DBA and MongoDB Champion. I have experience with multiple cloud providers, Kubernetes, different types of K8s operators (Strimzi, RabbitMQ Cluster Operator), but especially MongoDB K8s Operator. I also work with KEDA. Since 2017, I have been a speaker at MongoDB conferences all around the world (USA, China, Europe). KEY TAKE-AWAYS FROM THE TALK I would like to share the best practices of running NoSQL database - MongoDB on Kubernetes also I want to show how to manage Atlas (MongoDB cloud) via K8s operator https://www.mongodb.com/developer/community-champions/arkadiusz-borucki/

Mastering MongoDB on Kubernetes, the power of operators

DoKC

Link: https://youtu.be/KUipuM3UJF4 https://go.dok.community/slack https://dok.community/ DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE) Kubernetes comes with a lot of useful features like Volumes and StatefulSets, which make running stateful workloads simple. Interestingly, when combined with the right tools, these features can make Kubernetes very valuable for developers wanting to run massive production databases in development! This is exactly what was seen at "Extendi". The developers at Extendi deal with a large amount of data in their production Kubernetes clusters. But when developing locally, they didn't have an easy way of replicating this data. This replication was needed because it allowed developers to test new features instantaneously without worrying if they would work as expected when pushed to production. But replicating a 100Gb+ production database for development wasn't turning out to be an easy task! This is where leveraging Kubernetes + remote development environments came to the rescue. Running data on Kubernetes turned out to be way faster than any of the traditional approaches because of Kubernetes' ability to handle stateful workloads exceptionally well. And since Extendi already used Kubernetes in production - the setup process was fairly simple. This talk will cover practical steps on how leveraging Kubernetes based development environments allowed dev teams at Extendi to run production data on Kubernetes during development using features like Volume Snapshots, having a huge positive impact on developer productivity. ----- Arsh is a Developer Experience Engineer at Okteto. He is an active contributor to the upstream Kubernetes project and was awarded the Kubernetes Contributor Award for his contributions in 2021. Arsh has written blogs and spoken about different topics in the cloud-native ecosystem at various conferences before, including KubeCon + CloudNativeCon + Open Source Summit China 2021. He has also been on the Kubernetes Release Team since the 1.23 release. He also serves as the New Contributor Ambassador for the Documentation Special Interest Group of the Kubernetes project and continuously mentors new folks in the community. Previously, he worked at VMware and was an active contributor to other CNCF projects, including cert-manager and Kyverno. ----- Lapo is a Software Engineer currently leading the development team of a Social Listening and Audience Intelligence platform. He started coding at the early age of 14 and since he turned his passion into a real job, he has always been looking for boosting his knowledge by constantly researching for newer and newer technologies. Active on Ruby Open Source projects ----- Ramiro Berrelleza is one of the founders of Okteto. He has spent most of his career (and his free time) building cloud services and developer tools. Before starting Okteto, Ramiro was an Architect at Atlassian and a Software Engineer at Microsoft Azure.

Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...

DoKC

Link: https://youtu.be/XRD5-V6907A https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE) Postgres should run inside your Kubernetes cluster. Yes, inside, not outside Kubernetes. After all, a database should be seen as an application, a special type of application - for which it is legitimate to require an additional level of care and attention. However, the small price you pay for this is worth the return on investment that your organization receives by running microservice applications, plus database combos, inside your Cloud Native infrastructure. In this session, I will cover why it is important to make such a decision. Discover the challenges and the opportunities that running Postgres inside Kubernetes presents. I will discuss what we all expect from a good Postgres operator, including self-healing, high availability, scalability, backup and recovery, performance, and - last but not least - security. I will then go beyond technical aspects to spark conversations about the holistic improvements that running a Cloud Native database brings to your organization. ----- A long time open source programmer and entrepreneur, Gabriele has a degree in Statistics from the University of Florence. After having consistently contributed to the growth of 2ndQuadrant and its members through nurturing a lean and devops culture, he is now leading the Cloud Native initiative at EDB. Gabriele lives in Prato, a small but vibrant city located in the northern part of Tuscany, Italy - known for having hosted the first European PostgreSQL conferences. His second home is Melbourne, Australia, where he studied at Monash University and worked in the ICT sector. He loves playing the Blues with his Fender Stratocaster, but his major passions are called Elisabeth and Charlotte!

Why run Postgres in Kubernetes?

DoKC

Link: https://youtu.be/1bqCDw999wg https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE) Kubernetes SIG Storage is responsible for ensuring storage is available for containers in a pod when the pod is scheduled on a node. There is the Container Storage Interface (CSI) for block and file storage that allows storage providers to write CSI drivers. There is also a COSI sub-project that is trying to add object storage support in Kubernetes. In this session, Xing will give an update on some of the features that SIG Storage is working on and discuss what might be coming in the future. ----- Xing Yang is a Tech Lead in the Cloud Native Storage team at VMware. She is a co-chair of the CNCF TAG Storage, a co-chair of the Kubernetes SIG Storage, a co-chair of the Kubernetes Data Protection WG, and a maintainer in Kubernetes CSI. Before joining VMware, Xing was the Lead Architect of OpenSDS while working for Futurewei. She also worked at Dell EMC for many years and has developed deep expertise in storage, data protection, disaster recovery, cloud, and virtualization technologies.

What's New in Kubernetes Storage

DoKC

Link: https://youtu.be/k1Es-S85xRE https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE) Kubernetes is an emerging platform of choice for deploying and running PostgresSQL. Deploying 100 Postgres clusters is as easy as deploying one, and there is no need to tinker with tools like Ansible or Puppet. Resource sharing can be applied when it makes sense, allowing to run multiple Postgres databases in isolation on a single instance, each storing the data on a dedicated persistent volume. There are great open-source tools out there to deal with high-availability and backups than support or can be easily integrated into the Kubernetes workflow. Monitoring and alerting is easy to implement. People reported success in running Postgres on Kubernetes before. But there are also rough edges, like memory management or certain Postgres maintenance operations, such as installing extensions, that normally cause unnecessary database downtimes on Kubernetes. They are less of a problem for in-house deployments, but may become a deciding factor when running a managed service, competing with other such services running on bare-metal servers or virtual machines that are free of those issues. In this talk, I will share some of our learnings from running a managed PostgreSQL/TimescaleDB service on Kubernetes on AWS for a little more than a year: I’ll start with the motivation of running managed PostgreSQL on Kubernetes, the benefits and drawbacks. I’ll describe the architecture of the managed PostgreSQL cloud on Kubernetes I’ll zoom in on how we solved some of the Kubernetes-specific issues within our cloud, such as upgrading extensions without downtimes, taming the dreaded OOM killer, and doing regular maintenance and PostgreSQL major upgrades. I’ll share how open-source tools from the PostgreSQL ecosystem helps us to run the service and explain how we use them in a slightly non-trivial way. ----- Oleksii has been working with PostgresSQL for almost 20 years, and has been deploying Postgres on Kubernetes since 2016, when his team at Zalando started the internal managed PostgreSQL service based on the in-house and open-source postgresql-operator. Around 2015, with some other team members, he stared working on a PostgreSQL HA project that later became Patroni. Long before that he was hacking PosgreSQL source code to implement binary replication on PostgreSQL 7.x, authoring some PostgreSQL extensions and contributing to the core PostgreSQL itself. He started PosgreSQL meetups in Berlin in 2015 and hopes to get back to meeting in-person somewhere in 2022. Being Ukrainian, he lives in Berlin for a bit more than 9 years with his wife, two children and numerous plants

What we've learned from running a PostgreSQL managed service on Kubernetes

DoKC

Link: https://youtu.be/IoEJvsl1ZqM https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE) “Zero downtime” and “always-on” are illusions. All systems fail sooner or later, whether it’s a regional e-commerce website or a major cloud region hosting thousands of applications. That’s why, instead of chasing these illusions, it’s worth focusing on the nines of availability. Based on true stories, this session walks you through modern data availability and reliability patterns used by architects whose applications withstood major cloud outages. With the focus on the data storage layer and Kubernetes, you’ll learn: * How to architect the data layer in Kubernetes with the server, zone, and region-level resiliency in mind. * How to find a compromise between latency and availability for multi-region deployments. * How to ensure the data layer remains reliable (i.e., always returns expected data) even during a major incident. ----- Denis Magda has spent half of his career working on distributed systems, applications, and databases. His experience spans from the development of distributed database engines and high-performance applications to training and education on the topic of distributed and cloud computing. Presently, Denis runs the Developer Relations team at Yugabyte and serves a PMC Member for Apache Ignite. He started his professional career at Sun Microsystems and Oracle, where he led one of the Java development groups and worked on technology evangelism efforts.

Weathering The Cloud Storm: Modern Data Management Patterns for Reliability a...

DoKC

Link: https://youtu.be/C4rlepOPk5o https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE) Serverless promises to change the way we consume software. It allows us to potentially pay for only that which we use and can help drive down operational costs to the minimal amount of resources necessary. Architecting for serverless requires a unique look at app logic and the way it is deployed. It takes a combination of the logical and physical worlds. An architectural pattern has emerged where we can scale ephemeral compute separate from services that need to persist. We use Kubernetes to deliver exactly this. A “serverless” experience that is driven and enabled by compute pods and storage pods. We also have used our experience running thousands of database clusters on Kubernetes to automate the operational expertise of managing a distributed database. In this talk, we will take a dive deep into the architecture of our application and share: * A definition and outline of the challenges of serverless * How we reworked our logic for a serverless approach * How we use Kubernetes to gain serverless autoscaling ----- Jim is a recovering developer turned evangelist who loves useful, cool, cutting-edge tech. He loves to translate and distill complex concepts into compelling, more simple explanations that broader communities can consume. He is an advocate of the developer and an active participant in several open source communities.

Using Kubernetes to deliver a “serverless” service

DoKC

Link: https://youtu.be/J3JiwW5FIAI https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE) Multiple clusters exist in most Kubernetes environments today, and number of clusters will increase overtime. The reasons for having multiple Kubernetes clusters are many, for example, overcoming scale limits, reducing complexity, geo separation, redundancy and having separate production, staging, and development environments. Once you have multiple K8S clusters, it can be useful to have the ability to easily move or duplicate workloads across these different clusters. Kubernetes does not have a native method to allow migration or duplication of workloads across clusters. Fortunately, there are tools that provide this functionality. In this presentation we will explore the different uses cases for cross cluster migration, and what is involved, and how these migration tools work. We'll cover some popular uses cases, such as, Disaster Recovery, Test/Dev, and performance testing. Migration could entail moving the entire cluster, or individual workloads. The components that need to be moved would include configuration and resources stored in etcd, and persistent data residing on PVCs. We'll cover the uses cases and challenges for migration, and run through an example of using one of these migration tools.

The many uses of Kubernetes cross cluster migration of persistent data

DoKC

Link: https://youtu.be/2uJvL1J8yz0 https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE) We know from the first Data on Kubernetes Report that 90% of respondents believe Kubernetes is ready for stateful workloads, but significant challenges remain. The DoK Community continues to grow and build a unique space where people share knowledge and have conversations that are shaping the next decade of data on Kubernetes. Melissa Logan and Sylvain Kalache will discuss the growth of DoK, DoKC, and introduce a new project to help end users on their journey to running data-intensive workloads on Kubernetes more easily and with better control.

The Data on Kubernetes Landscape

DoKC

Link: https://youtu.be/Jl0iJqC6GI0 https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE) The state of the CNCF Storage options has exploded in the past few years, but if you had to choose a project to use today, how would you go about comparing each offering and choosing who to partner with for your future growth? Civo offers managed Kubernetes clusters powered by K3s to customers around the world. We manage thousands of Virtual Machines and stateful customer data within multiple data centres across several continents. In late 2021, Civo had the opportunity to evaluate the CNCF storage landscape to move to a new technology stack. Learn about the steps required to evaluate the market, testing vendor claims in the real world and what needs to be considered outside of the purely technological aspects of any product selection. In late 2021, Civo's Director of Innovation, Dinesh, spoke about the intricacies of migrating between two vendors (https://www.youtube.com/watch?v=GlUsPnSZI_Y&list=PLHgdNuGxrJt2-xlW_l2q1BAE3e4TsPnQo). Now join him as he talks about the wider project and some of business implications of hosting Data on Kubernetes ----- Dinesh is Director of Innovation at Civo and has been the lead architect for the new Civo Stack Platform. Having worked in the hosting industry for many years, Dinesh has a passion for creating solutions that operate at scale. This not only applies to the technology stack, but for nurturing engineers through their career.

Testing the Mettle: Evaluating data solutions for large-scale production to c...

DoKC

More from DoKC (20)

Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud

The Kubernetes Native Database

ING Data Services hosted on ICHP DoK Amsterdam 2023

Implementing data and databases on K8s within the Dutch government

StatefulSets in K8s - DoK Talks #154

Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...

Analytics with Apache Superset and ClickHouse - DoK Talks #151

Overcoming challenges with protecting and migrating data in multi-cloud K8s e...

Evaluating Cloud Native Storage Vendors - DoK Talks #147

Kubernetes Cluster Upgrade Strategies and Data: Best Practices for your State...

Mastering MongoDB on Kubernetes, the power of operators

Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...

Why run Postgres in Kubernetes?

What's New in Kubernetes Storage

What we've learned from running a PostgreSQL managed service on Kubernetes

Weathering The Cloud Storm: Modern Data Management Patterns for Reliability a...

Using Kubernetes to deliver a “serverless” service

The many uses of Kubernetes cross cluster migration of persistent data

The Data on Kubernetes Landscape

Testing the Mettle: Evaluating data solutions for large-scale production to c...

We will Dok You! - The journey to adopt stateful workloads on k8s

2. Bios Solution Architect @Komodor DevRel @Komodor the_good_guym UHofesh

3. Setlist - agenda

4. Drum & Bass - it all comes down to stateful ● App DBs & queues are the most critical infrastructure for any organization ● Most applications rely on stateful data

5. The impact of Stateful Minor Errors Unavailable application Severe Downtime DATA LOSS High possibility Low impact Low possibility Extreme impact

6. The impact of Stateful Minor Errors Unavailable application Severe Downtime DATA LOSS High possibility Low impact Low possibility Extreme impact

7. The key challenges with Kubernetes Lack of knowledge/expertise “Can’t read notes” Lack of confidence/fear “The stage fright” Lack of instruments/tools Lack of band members

8. Why do it anyway? ● Cost ○ Managed vs unmanaged ○ Resources footprint ○ Licences ● Multi-Cloud ● K8s standardization ○ No VMs until x ● 3rd party modernization ○ New DB/Queues ○ Automated operations ● Self-hosted / on-prem ● Your product is being deployed on hosted clusters

9. But how? Hint: Not by living on a prayer!

17. The personal learning path ● K8s basics: pods, replicaset, deploy, service ● K8s infra: statefulsets, networking, ips, storage ● Join a community! ● Adopt a tool & its operator: DBs, queues, ● Milestone 1 - low environments ● Day 2 ops ready: backup, restore, upgrades ● Basic ops/availbility drills, manual chaos ● Adopt supporting tools: metrics, logs, alerting, monitors & observability ● Learn & test performance ● Milestone 2 - mid environment ● GitOps, CI/CD, IaC ● Prod support: dashboard, alerting, troubleshooting ● Performance: testing, metrics, underlying infra (dns, network interfaces) ● Day-to-day operations/automations ● Share your knowledge! Keep Learning! Repeat!

18. How to learn? The k8s Journey Documentation ● Kubernetes Docs ● 100 days of k8s ● Civo academy ● Vendors learning centers ● Hashnode/Medium blogs Videos ● Kunal Kushwaha ● Saiyam Pathak ● TechWorldWithNana ● DevOps Toolkit (Viktor Farcic) ● Anais Urlichs

19. Any questions?

We will Dok You! - The journey to adopt stateful workloads on k8s

Recommended

Recommended

More Related Content

More from DoKC

More from DoKC (20)

We will Dok You! - The journey to adopt stateful workloads on k8s