Building Bridges Between Applications and Data

•

0 likes•221 views

Simply connecting to a Kubernetes Service or a load balancer is nice -- if that is even an option. These days, operators have to contend with hybrid and multi-cloud. Namespaces are not enough; it's not uncommon to operate numerous K8s clusters in a single region with traffic being dynamically routed across multiple control planes. This session dives into the process of identifying requirements for a Kubernetes gateway that needs to handle both HTTP based services as well as hyper-optimized TCP workloads. Attendees will walk away with a working knowledge and a process to develop out ingress solutions that fit their needs, including: Deploying a local K8s cluster with K3d Installing the DataStax Kubernetes Operator for Apache Cassandra with Helm Provisioning and running a multi-node Cassandra cluster Deploying Kong Gateway with Helm Implementing multiple ingress approaches through Kong Gateway, with an accompanying discussion on the pros and cons of each Validation of each approach through an external client application

Software

THE CLOUD
CONNECTIVITY COMPANY
Destination:
Automation
Building Bridges,
Connecting Data
San Francisco, CA
July 2020
THE CLOUD CONNECTIVITY
COMPANY
1

THE CLOUD
CONNECTIVITY COMPANY
Christopher Bradford
Product Manager
Astra, DataStax Kubernetes Operator for Apache
Cassandra
2

THE CLOUD
CONNECTIVITY COMPANY
Jim Dickinson
Lead Software Engineer
Astra, DataStax Kubernetes Operator for Apache
Cassandra
3

THE CLOUD
CONNECTIVITY COMPANY
Topics
● Cassandra Architecture
● Cassandra on Kubernetes
● Speaking the Native Protocol - CQL
● Routing with Kong
● Troubleshooting
● Questions
5

THE CLOUD
CONNECTIVITY COMPANY
Cassandra Architecture
6
AP Database, FTW!

THE CLOUD
CONNECTIVITY COMPANY
The Ring
Replicated
Fault Tolerant
Scalable
7
Node E Node C
Node A
Node BNode F
Node D
Cassandra
Cluster

THE CLOUD
CONNECTIVITY COMPANY
The Ring
Replicated
Fault Tolerant
Scalable
8
Node E Node C
Node A
Node BNode F
Node D
Cassandra
Cluster

THE CLOUD
CONNECTIVITY COMPANY
The Ring
Replicated
Fault Tolerant
Scalable
9
Node E Node C
Node A
Node BNode F
Node D
East
DC
West
DC
Replication
Sync & Async

THE CLOUD
CONNECTIVITY COMPANY
Driver Native Protocol - CQL
10
TCP, frames, versions, OH MY

THE CLOUD
CONNECTIVITY COMPANY
CQL
Protocol
Documented in-tree
apache/cassandra
11

THE CLOUD
CONNECTIVITY COMPANY
Driver
Communication
Topology Aware
Smart Routing
12
Node E Node C
Node A
DC
Traditional LB

THE CLOUD
CONNECTIVITY COMPANY
Driver
Communication
Topology Aware
Smart Routing
13
Node E Node C
Node A

THE CLOUD
CONNECTIVITY COMPANY
Deploying Cassandra on
Kubernetes
14
DataStax Kubernetes Operator for Apache Cassandra

THE CLOUD
CONNECTIVITY COMPANY
Cass Operator Features
15
● Datacenter provisioning
● Node conﬁguration
● Node replacement
● Automatic upgrades
● Scale up

THE CLOUD
CONNECTIVITY COMPANY
Describes a logical C* Datacenter
■ Size
■ Racks (and Availability Zones)
■ Type: OSS or DSE
■ C* version
■ Conﬁguration overrides
Submitted to the k8s cluster with kubectl
16
Custom Resource
CassandraDatacenter
apiVersion: datastax.com/v1beta1
kind: CassandraDatacenter
metadata:
name: dc1
spec:
clusterName: cluster1
serverType: cassandra
serverVersion: 3.11.6
size: 3
config:
cassandra-yaml:
num_tokens: 12

THE CLOUD
CONNECTIVITY COMPANY
helm repo add datastax https://datastax.github.io/charts
helm repo update
helm install cass-operator datastax/cass-operator
17
Install with Helm

THE CLOUD
CONNECTIVITY COMPANY
cass-operator
18

THE CLOUD
CONNECTIVITY COMPANY
Cluster
Deployment
19

THE CLOUD
CONNECTIVITY COMPANY
Cluster
Deployment
20

THE CLOUD
CONNECTIVITY COMPANY
Cluster
Deployment
21

THE CLOUD
CONNECTIVITY COMPANY
Cluster
Deployment
22

THE CLOUD
CONNECTIVITY COMPANY
Cluster
Deployment
23

THE CLOUD
CONNECTIVITY COMPANY
Kubernetes
Encapsulated
24
Node E Node C
Node A

THE CLOUD
CONNECTIVITY COMPANY
Node E Node C
Node A
Application
Outside the
Cluster
25

THE CLOUD
CONNECTIVITY COMPANY
Deployments with
Kong
26
Where are the clients, and how do we communicate with them?

THE CLOUD
CONNECTIVITY COMPANY
Internal Traﬃc
27
Node E Node C
Node A
DC

THE CLOUD
CONNECTIVITY COMPANY
External Ingress
28
Node E Node C
Node A
DC

THE CLOUD
CONNECTIVITY COMPANY
External Ingress
with TLS
29
Node E Node C
Node A
DC
🔒 SNI

THE CLOUD
CONNECTIVITY COMPANY
External Ingress
with TLS
30
Node E Node C
Node A
DC
🔒 SNI & mTLS

THE CLOUD
CONNECTIVITY COMPANY
There be dragons / Demo
(time permitting)
31
Pitfalls, watch out!

THE CLOUD
CONNECTIVITY COMPANY 32
Checkout
Questions?
https://bit.ly/CassOperator
cass-operator on GitHub
Managed Cassandra as a Service
https://astra.datastax.com
DataStax Astra

Modern architectures can quickly grow in complexity, which often means introducing new tools to your environment. Building images and provisioning infrastructure and features can all require distinct tooling -- and often even different workflows and programming languages. The dependencies between these can often mean automation failures and difficult rollback scenarios. There are open source tools that allow you to use the same languages and toolset you’re familiar with to build and deploy your applications. In this hands-on session, we’ll show you how to: Provision infrastructure like Storage buckets, managed Kubernetes and Load Balancers using Go Create and deploy applications without leaving the safety of your favorite programming language

The Building Blocks of DX: K8s Evolution from CLI to GitOps

OlyaSurits

Kubernetes has become the default container orchestrator framework, setting the standards for application deployment in a distributed architecture. Wider adaptability of the tool prompted the diversification of the end-user base, and a consistent DX for cluster interaction became essential for Kubernetes. The community channeled herculean efforts towards the enhancement of developer experience, by extending the cluster CLI, building portals and highly-responsive UIs. This talk will focus on the cluster interaction chronicles, showcasing tools and add-ons which contributed to a wider adoption for Kubernetes. An emphasis will be place on kubectl plugins and cluster state managers using mechanisms such as GitOps, ClickOps and even SheetOps.

Security, Automation and the Software Supply Chain

OlyaSurits

Everyone agrees that automation is crucial today, but what about all of the holes in your process it exposes? The unsafe libraries your developers use without thinking about it, the processes that pull in questionable code, the inadvertent configuration mistakes that leave gaping holes. This talk looks at some of the places in the software development process where freedom and automation can cause major problems, and how to solve it without tying your developers' hands. Attendees will learn about some of the steps necessary to secure their software development supply chain, including signed images, secure registries and security scanning.

Implementing a Reliable, Auto-Healing Scalable Platform at VMware

OlyaSurits

An API gateway is the most critical piece of enterprise integration for IT. Therefore, we needed to implement a robust API gateway in a reliable, auto-healing scalable platform. This talk will share how we implemented an open source gateway on a Kubernetes platform with multi-datacenter auto failover to achieve this objective. For this implementation, we developed a customized monitoring script to execute the auto failover between the data centers in case of API gateway failures. The result was 99.95 percent uptime. With this architecture in place, we also succeeded in performing chaos engineering using a combination of open source and in-house chaos tools. Key takeaways will include: • How to implement a robust API gateway in a reliable, auto-healing scalable platform • How we achieved 99.95 percent uptime • How to combine open source and in-house tools to successfully perform chaos engineering

Continuous Deployment for Staging and Production Environments

OlyaSurits

Continuous Deployment allows teams to get immediate feedback, debug, iterate, and fail fast, ultimately getting a better product into the hands of users faster. In this talk, Kendra and Celia will talk about the challenges and benefits of continuously deploying their staging environments and how they applied that to their production environment during the development of CockroachCloud. They will discuss their use of: Iterative development including PR structure and feature decomposition Pushing all PR to staging environment Feature Flags to make the customer magic happen

A Better Way for Applications to Communicate With Your Mesh

OlyaSurits

Ingress gateways are a quick and easy way to enable external clients to communicate with a service mesh. To demonstrate this, we implemented a powerful ingress gateway with a reliable ingress controller in an auto-healing, scalable platform. In this talk, we will demo several use cases, including service-to-service authorization with mTLS, service segmentation, and connectivity of services running on any platform to the service mesh. The solution we will share is built using HashiCorp Consul Ingress Gateway with Kong Ingress Controller on a Kubernetes platform with Consul Envoy proxy.

Connectivity Is the Future

OlyaSurits

As we grow our businesses, we transform our applications to become distributed and decoupled to iterate faster on customer feedback, adapt quicker to market transitions and improve reliability and availability. In this new distributed and decoupled era, we introduce service connectivity at the backbone of our applications at an unprecedented scale, and we need to adopt a new kind of platform to abstract away service connectivity from the application teams so that the business can innovate with minimal fragmentation and maximum speed. We need a service connectivity platform that allows the organization to break down vertical silos like API management and service mesh for a fully integrated connectivity abstraction. In this session, we will explore the market transitions that are defining this new distributed and decoupled era, and we will identify the main foundations for the new service connectivity platform category.

Flexible, hybrid API-led software architectures with Kong

Sven Bernhardt

Kong is a lightweight, cloud-native API solution that makes it easier and faster than ever to connect APIs and microservices in today’s hybrid, multi-cloud environments. With its agnostic, flexible deployment approach, Kong can be used in today’s heterogeneous IT system landscapes to integrate a wide variety of data and systems – even across company boundaries – using APIs. In addition to REST APIs, Kong also offers support for gRPC and GraphQL, which broadens the possibilities to implement modern application architectures. In this presentation, we will discuss deployment patterns and use cases for Kong to demonstrate the flexibility of the platform. Using a practical example, aspects of the API development and deployment process as well as the integration in existing software development processes will be discussed.

The development of APIs always begins with the old -- and meanwhile boring -- battle of API-first or code-first. However, working with APIs involves a process that includes design, development and operation. With this in mind, along with the various ways API gateways and API platforms can support these processes, the question of how to automate these processes arises very quickly. In this session, Daniel Kocot will show how API gateways and API platforms can positively influence the development process of APIs based on the DevOps concept and CAMS model to deliver better and better APIs to consumers.

Building successful business Java apps: How to deliver more, code less, and c...

Red Hat Developers

Building business applications is, in a higher-level perspective, a way to translate business needs into runnable software. Unfortunately, this is easier said than done. The complexity of encoding business requirements requires overcoming several challenges, such as improving communication between stakeholders and increasing agility of delivery. In this presentation, we'll learn how developers can "up" their game, deliver more in the cloud with less effort, and even increase communication skills.

DevOps monitoring: Best Practices using OpenShift combined with Icinga & Big ...

Icinga

Talk by Marco Bizzantino: In my talk I will demonstrate how monitoring works with the DevOps approach. Within an organisation, different people with different roles need to have a complete view over the whole infrastructure. One of the greatest benefits of Icinga is its ability to integrate with various tools and modules to satisfy the company’s requests. An example is OpenShift monitoring. I will point out the most important parts of monitoring OpenShift infrastructure while maintaining it’s complexity. In my approach I will show how we combine Icinga, Prometheus, Grafana and Elasticsearch for an overall OpenShift monitoring solution. Additional examples will show the benefits of using Icinga’s Business Process modelling to add more value to the result.

Kubernetes on AWS @ Zalando Tech

Michael Dürgner

The Power of GitOps with Flux & GitOps Toolkit

Weaveworks

GitOps Days Community Special Watch the video here: https://youtu.be/0v5bjysXTL8 New to GitOps or been a long-time Flux user? We'll walk you through the benefits of GitOps and then demo it in action with a sneak peak into the next gen Flux and GitOps Toolkit! * Automation! * Visibility! * Reconciliation! * Powerful use of Prometheus and Grafana! * GitOps for Helm! For Flux users, Flux v1 is decoupled into Flux v2 and GitOps Toolkit. We'll demo how this decoupling gives you more control over how you can do GitOps and with fewer steps! Join Leigh Capili and Tamao Nakahara as they show you GitOps in action with Flux and GitOps Toolkit. Note to our Flux community that Flux v2 and the GitOps Toolkit is in development and Flux v1 is in maintenance mode. These talks and upcoming guides will give you the most up-to-date info and steps to migrate once we reach feature parity and start the migration process. We are dedicated to the smoothest experience possible for our Flux community, so please join us if you'd like early access and to give us feedback for the migration process. We are really excited by the improvements and want to take this opportunity to show you what the GitOps Toolkit is all about, walk you through the guides and get your feedback! For more info, see https://toolkit.fluxcd.io/. Here's our latest blog post on Flux v2 and GitOps Toolkit updates: https://www.weave.works/blog/the-road-to-flux-v2-october-update

Visual Studio로 Kubernetes 사용하기

충섭 김

Migrating from oracle soa suite to microservices on kubernetes

Konveyor Community

Watch presentation recording: https://youtu.be/cxH6WjDZc2c In this session, we’ll explore how Randoli helped a Postal Technology company migrate their payment gateway applications off Oracle SOA Suite to Camel/Springboot on Kubernetes. The primary drivers for the migration were: move to cloud-native technologies in keeping with the organizational digital transformation mandate; move away from an outdated centralized platform to a decentralized architecture for efficiency, scalability, and manageability; and very high licensing costs of the existing platform. We’ll discuss: - The high-level approach we took during the migration including architecture and design decisions. - How we used Camel/Springboot to implement the services. - Why and how we used Drools for implementing business rules. - The test-driven approach using Camel testing framework and how it helped reduce issues. - CI/CD and build process on Kubernetes. - How we tackled logging, monitoring, and tracing challenges. Presenter: Rajith Attapattu, Managing Partner & CTO @ Randoli Inc.

Serverless integration anatomy

Christina Lin

Cloud Native Testing, 2020 Edition: A Modern Blueprint for Pre-production Tes...

OlyaSurits

Cloud native applications can benefit greatly from end-to-end testing before deployment, but integration testing of microservices is often discouraged because it's costly and difficult. This talk proposes a modern blueprint for cloud native application testing, focusing on pre-production testing and in particular integration testing. Topics discussed include how to handle common challenges with end-to-end and integration testing, such as: Dealing with state How to speed up tests runs for improved developer feedback loops How to test the configuration of a whole system in the era of Infrastructure-as-Code We will also discuss other types of testing (such as testing in production), and pre-production workarounds often used as an alternative to integration testing (such as contract testing and test doubles), evaluating the pros and cons of each approach, and how they can complement each other.

Cicd pixelfederation

Juraj Hantak

London Java Community: Spring Cloud -> Docker -> Kubernetes

Mauricio (Salaboy) Salatino

CNCF Keynote - What is cloud native?

Weaveworks

Migrating a Large Fortune 100 Healthcare Company to Kubernetes in 7 months

Konveyor Community

Watch the presentation: https://youtu.be/E3LeAmH6Ems At the beginning of 2019, Chris Nuland and team were tasked with migrating a large mesosphere DC/OS cluster with hundreds of running containers to Kubernetes for a large fortune 100 healthcare company. One of the key challenges with this migration was the need to finish it within a 7 month timeframe to allow the sunsetting of DC/OS before the cluster’s end of life. In conjunction with this migration, there was also the need to containerize a couple hundred applications and deploy them into the newly built cluster. These tasks were completed in the desired time frame using a variety of migration and onboarding techniques, including the use of a few migration tools, like pathfinder, that would eventually be part of the Konveyor suite of applications. This presentation will go over many of the challenges of that migration, how certain tooling aided in the process, and how the process would look differently now given many of the migration tooling advantages found in the Konveyor suite of applications. Presenter: Christopher Nuland, Architect at Red Hat

Beyond OpenStack | OpenStack in Real Life

Opsta

Free GitOps Workshop + Intro to Kubernetes & GitOps

Weaveworks

Follow along in this free workshop and experience GitOps! AGENDA:  Welcome - Tamao Nakahara, Head of DX (Weaveworks)  Introduction to Kubernetes & GitOps - Mark Emeis, Principal Engineer (Weaveworks)  Weave Gitops Overview - Tamao Nakahara  Free Gitops Workshop - David Harris, Product Manager (Weaveworks) If you're new to Kubernetes and GitOps, we'll give you a brief introduction to both and how GitOps is the natural evolution of Kubernetes. Weave GitOps Core is a continuous delivery product to run apps in any Kubernetes. It is free and open source, and you can get started today! https://www.weave.works/product/gitops-core If you’re stuck, also come talk to us at our Slack channel! #weave-gitops http://bit.ly/WeaveGitOpsSlack (If you need to invite yourself to the Slack, visit https://slack.weave.works/)

No Risk, No Reward: The Joys Of Testing In Production.

UXDXConf

Natalia and Fabio will share their story of how the HBC DevOps culture and mindset evolved. Follow their journey from moving to a distributed microservice architecture to enable simple, quick and frequent production roll-outs, through phased deployments to dark and live canary nodes, to a radical 'no staging environment' approach. Hear about the benefits and caveats they've experienced along the way.

From zero to hero with Kubernetes and Istio

Sergii Bishyr

S1P: Spring Cloud on PKS

Mauricio (Salaboy) Salatino

GumGum: Multi-Region Cassandra in AWS

DataStax Academy

GumGum relies heavily on Cassandra for storing different kinds of metadata. Currently GumGum reaches 1 billion unique visitors per month using 3 Cassandra datacenters in Amazon Web Services spread across the globe. This presentation will detail how we scaled out from one local Cassandra datacenter to a multi-datacenter Cassandra cluster and all the problems we encountered and choices we made while implementing it. How did we architect multi-region Cassandra in AWS? What were our experiences in implementing multi-datacenter Cassandra? How did we achieve low latency with multi-region Cassandra and the Datastax Driver? What are the different Cassandra use cases at GumGum? How did we integrate our Cassandra with Spark?

Citi Tech Talk: Hybrid Cloud

confluent

What's hot

Meetup 23 - 03 - Application Delivery on K8S with GitOps

Vietnam Open Infrastructure User Group

Bare Metal Cluster with Kubernetes, Istio and Metallb | Nguyen Phuong An, Ngu...

Vietnam Open Infrastructure User Group

APIOps: Automated Processes for Even Better APIs

OlyaSurits

Building successful business Java apps: How to deliver more, code less, and c...

Red Hat Developers

DevOps monitoring: Best Practices using OpenShift combined with Icinga & Big ...

Icinga

Kubernetes on AWS @ Zalando Tech

Michael Dürgner

The Power of GitOps with Flux & GitOps Toolkit

Weaveworks

Visual Studio로 Kubernetes 사용하기

충섭 김

Migrating from oracle soa suite to microservices on kubernetes

Konveyor Community

Serverless integration anatomy

Christina Lin

Cloud Native Testing, 2020 Edition: A Modern Blueprint for Pre-production Tes...

OlyaSurits

Cicd pixelfederation

Juraj Hantak

London Java Community: Spring Cloud -> Docker -> Kubernetes

Mauricio (Salaboy) Salatino

CNCF Keynote - What is cloud native?

Weaveworks

Migrating a Large Fortune 100 Healthcare Company to Kubernetes in 7 months

Konveyor Community

Beyond OpenStack | OpenStack in Real Life

Opsta

Free GitOps Workshop + Intro to Kubernetes & GitOps

Weaveworks

No Risk, No Reward: The Joys Of Testing In Production.

UXDXConf

From zero to hero with Kubernetes and Istio

Sergii Bishyr

S1P: Spring Cloud on PKS

Mauricio (Salaboy) Salatino

What's hot (20)

Meetup 23 - 03 - Application Delivery on K8S with GitOps

Bare Metal Cluster with Kubernetes, Istio and Metallb | Nguyen Phuong An, Ngu...

APIOps: Automated Processes for Even Better APIs

Building successful business Java apps: How to deliver more, code less, and c...

DevOps monitoring: Best Practices using OpenShift combined with Icinga & Big ...

Kubernetes on AWS @ Zalando Tech

The Power of GitOps with Flux & GitOps Toolkit

Visual Studio로 Kubernetes 사용하기

Migrating from oracle soa suite to microservices on kubernetes

Serverless integration anatomy

Cloud Native Testing, 2020 Edition: A Modern Blueprint for Pre-production Tes...

Cicd pixelfederation

London Java Community: Spring Cloud -> Docker -> Kubernetes

CNCF Keynote - What is cloud native?

Migrating a Large Fortune 100 Healthcare Company to Kubernetes in 7 months

Beyond OpenStack | OpenStack in Real Life

Free GitOps Workshop + Intro to Kubernetes & GitOps

No Risk, No Reward: The Joys Of Testing In Production.

From zero to hero with Kubernetes and Istio

S1P: Spring Cloud on PKS

Similar to Building Bridges Between Applications and Data

GumGum: Multi-Region Cassandra in AWS

DataStax Academy

Citi Tech Talk: Hybrid Cloud

confluent

Cloud-native .NET Microservices mit Kubernetes

QAware GmbH

BASTA! 2017, Mainz: Talk von Mario-Leander Reimer (@LeanderReimer, Cheftechnologe bei QAware). Cloud-Größen wie Google, Twitter und Netflix haben die Kernbausteine ihrer Infrastruktur quelloffen verfügbar gemacht. Das Resultat aus vielen Jahren Cloud-Erfahrung ist nun frei zugänglich, und jeder kann seine eigenen Cloud-nativen Anwendungen entwickeln – Anwendungen, die in der Cloud zuverlässig laufen und fast beliebig skalieren. Die einzelnen Bausteine wachsen zu einem großen Ganzen zusammen, dem Cloud-Native-Stack. In dieser Session stellen wir die wichtigsten Konzepte und aktuellen Schlüsseltechnologien kurz vor. Anschließend implementieren wir einen einfachen Microservice mit .NET Core und Steeltoe OSS und bringen ihn zusammen mit ausgewählten Bausteinen für Service-Discovery und Konfiguration schrittweise auf einem Kubernetes-Cluster zum Laufen.

ApacheCon 20190722 - CassKop : The Cassandra operator for Kubernetes

Jean-Armel Luce

Apache Cassandra Lunch #93: K8ssandra on Digital Ocean

Anant Corporation

In Cassandra Lunch #93, we will discuss how to use k8ssandra on Digital Ocean Accompanying Blog: Coming Soon! Accompanying YouTube: https://youtu.be/i1C81vYqiOw Sign Up For Our Newsletter: http://eepurl.com/grdMkn Join Cassandra Lunch Weekly at 12 PM EST Every Wednesday: https://www.meetup.com/Cassandra-DataStax-DC/events/ Cassandra.Link: https://cassandra.link/ Follow Us and Reach Us At: Anant: https://www.anant.us/ Awesome Cassandra: https://github.com/Anant/awesome-cassandra Cassandra.Lunch: https://github.com/Anant/Cassandra.Lunch Email: solutions@anant.us LinkedIn: https://www.linkedin.com/company/anant/ Twitter: https://twitter.com/anantcorp Eventbrite: https://www.eventbrite.com/o/anant-1072927283 Facebook: https://www.facebook.com/AnantCorp/ Join The Anant Team: https://www.careers.anant.us

A Hitchhiker’s Guide to the Cloud Native Stack. #CDS17

Mario-Leander Reimer

Cloud native applications are popular these days. They promise superior reliability and almost arbitrary scalability. They follow three key principles: they are built and composed as microservices. They are packaged and distributed in containers. The containers are executed dynamically in the cloud. But which technology is best to build this kind of application? This talk will be your guidebook. In this hands-on session, we will briefly introduce the core concepts and some key technologies of the cloud native stack and then show how to build, package, containerize, compose and orchestrate a cloud native showcase application on top of a cluster operating system such as Kubernetes or OpenShift. Throughout the session we will be using an off-the-shelf MIDI controller to visualize the concepts and to remote control the cluster. Container Days 2017 conference. @ConDaysEU #CDS17 #qaware #CloudNativeNerd @LeanderReimer

A hitchhiker‘s guide to the cloud native stack

QAware GmbH

Container Days 2017, Hamburg: Vortrag von Mario-Leander Reimer (@LeanderReimer, Cheftechnologe bei QAware). Abstract: Cloud-Größen wie Google, Twitter und Netflix haben die Kernbausteine ihrer Infrastruktur quelloffen verfügbar gemacht. Das Resultat aus vielen Jahren Cloud-Erfahrung ist nun frei zugänglich, und jeder kann seine eigenen Cloud-nativen Anwendungen entwickeln – Anwendungen, die in der Cloud zuverlässig laufen und fast beliebig skalieren. Die einzelnen Bausteine wachsen zu einem großen Ganzen zusammen, dem Cloud Native Stack. In dieser Session stellen wir die wichtigsten Konzepte und Schlüsseltechnologien vor und bringen dann eine Spring-Cloud-basierte Beispielanwendung schrittweise auf Kubernetes und DC/OS zum Laufen. Dabei diskutieren wir verschiedene praktikable Architekturalternativen.

Multi-cluster k8ssandra

KubernetesCommunityD

Presented by Rags Srinivas, Developer Advocate/Architect at Datastax at Kubernetes Community Days, Washington DC, September 14, 2022. Cassandra is designed for multi-region ● Partition tolerant ● Each node in the cluster maintains the full topology ● Nodes automatically route traffic to nearby neighbors ● Data is automatically and asynchronously replicated ● The cluster is homogenous ● Any node can service any client request ● Clients can be configured to automatically route traffic to the local datacenter Kubernetes was not designed for multi-region ● Increased latencies ● The cost is higher consensus request latency from crossing data center boundaries ● Loss of connectivity to ectd could cause outages ● Services should route traffic to nearby endpoints

Top 10 present and future innovations in the NoSQL Cassandra ecosystem (2022)

Cédrick Lunven

Are you new to Apache Cassandra® and wondering what all the excitement is about? Or a veteran Cassandra user interested in understanding what’s new in the project? Attend our live webinar on October 18 to learn about the latest Cassandra release and why it represents a big step forward but also all the initiative and new projects rising in the ecosystem, DataStax Director of Developer Relations Cedrick Lunven will walk you through new features in version 4.1. Get the inside scoop on how version 4.1 adds exciting new features for operators and improves the security posture, without compromising the stability achieved in Cassandra 4.0. Get some insights about projects actually in progress to make Cassandra more easy to use (Stargate) but also to deploy (K8ssandra). You will learn: System-wide Guardrails Denylisting Partition Keys Diagnostic events via CQL, not just JMX CQLSH Auth support for LDAP, Kerberos and more Lots of new, pluggable extension points Also, celebrate our open source community with highlights from the 2022 Apache Cassandra World Party and a look ahead to Cassandra 5.0!

CN Asturias - Stateful application for kubernetes

Cédrick Lunven

Dayta AI Seminar - Kubernetes, Docker and AI on Cloud

Jung-Hong Kim

ResumeAlen Badel

Best Practices for Building Open Source Data Layers

IBMCompose

Dok Talks #119 - Cloud-Native Data Pipelines

DoKC

Link to the full talk: https://youtu.be/HGinlidNWZU https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK This talk walks you through our stack, architecture, and processes. We develop tools to deploy and run data-driven applications in a cloud-native environment. We will give a whirlwind tour on developing a Java Quarkus application, a CICD stack powered by GitHub Actions / ArgoCD, building and deploying containerized Kafka Streams applications at runtime with Jib container builder. Having introduced the above common understanding, we will give a high-level overview of how we utilize modern Kubernetes and Cloud tooling to manage multiple clusters in different organizations together with our customers. BIO DataCater commoditizes data pipeline development lifecycle by applying software engineering and cloud native practices to data work. Hakan is a Software / Data Engineer and CTO of DataCater. He worked and built his knowledge around Software, Data Engineering, and Cloud-Native Computing in severely different environments. From early start-up to hyper-scaler AWS. From sports media companies to highly regulated FSI enterprises. The experiences gained, problems encountered, and solutions found led to him co-founding DataCater to enhance tooling in the Data space.

Episode 3: Kubernetes and Big Data Services

Mesosphere Inc.

Kubernetes is great for deploying stateless containers, but what about the big data ecosystem? Episode 3 of our Kubernetes series covers how DC/OS enables you to connect your Kubernetes-based applications to co-located big data services. Slides cover: 1. Why persistence is challenging in distributed architectures How DC/OS helps you take advantage of the services available in the big data ecosystem 2. How to connect Kubernetes to your data services through networking 3. How Apache Flink and Apache Spark work with Kubernetes to enable real-time data processing on DC/OS

Data relay introduction to big data clusters

Chris Adkin

MayaData Datastax webinar - Operating Cassandra on Kubernetes with the help ...

MayaData Inc

In this webinar experts from DataStax - the lead developer of Cassandra - and from MayaData - the lead developer of OpenEBS and LitmusChaos - will discuss and demonstrate ways to ensure the ease of use and resilience of Cassandra on Kubernetes. Topics to be discussed and demonstrated include: Provisioning underlying storage - how to make it consistent irrespective of the underlying hardware or cloud? Are there are ever reasons to have the storage replicate across nodes or is dynamic LocalPV the best choice in all cases? Cass Operator - DataStax Kubernetes Operator for Apache Cassandra Resilience - how to proactively assess the overall environment including the underlying Kubernetes with the help of Litmus

Introducing Scylla Cloud

ScyllaDB

ScyllaDB recently launched our Scylla Cloud database as a service, which combines the speed and power of the Scylla NoSQL database with the ease of a fully managed cloud service. Scylla Cloud relieves your team of day-to-day cluster management so you can focus on creating modern, interactive applications that respond to queries in milliseconds. Join us for an overview of Scylla Cloud, including a live demo of how to launch and connect to a cluster, how to create and query a table, and how to run a few operations, all in minutes.

2018 04-06 kubernetes ingress in production

Sandor Szuecs

Talk Abstract At Zalando we run 84 Kubernetes clusters in AWS. Ingress objects are enough to provision ALBs, do advanced HTTP routing and create DNS records. I will show how to support green-blue deployments, A/B testing, shadow traffic and feature toggles with the current ingress and our Open Source tools. Talk Description One of the hottest topics in Kubernetes is how to do ingress traffic done right. This is an opinionated talk that shows how one of the biggest online shops in Europe does it. All tools are Open Source and can be used by the audience. Presented use cases are production relevant. notes - Since 7 years I work as system and software engineer for Zalando. - I work in the team that runs 84 Kubernetes clusters for Zalando. - I am one of the core developers, which implements all the ingress features being presented.

Automatically scaling your Kubernetes workloads - SVC210-S - Santa Clara AWS ...

Amazon Web Services

We begin this session by taking a brief tour through the history of infrastructure and the evolution of our ability to scale. This includes what provisioning and scaling look like when working with physical servers. We then discuss the technologies that made automatic scaling possible. We also provide an overview of the most common scaling that is available today. Finally, we discuss how to monitor the things that matter. Using this framework, we can determine what metrics we should scale on for different types of applications and workloads.

Similar to Building Bridges Between Applications and Data (20)

GumGum: Multi-Region Cassandra in AWS

Citi Tech Talk: Hybrid Cloud

Cloud-native .NET Microservices mit Kubernetes

ApacheCon 20190722 - CassKop : The Cassandra operator for Kubernetes

Apache Cassandra Lunch #93: K8ssandra on Digital Ocean

A Hitchhiker’s Guide to the Cloud Native Stack. #CDS17

A hitchhiker‘s guide to the cloud native stack

Multi-cluster k8ssandra

Top 10 present and future innovations in the NoSQL Cassandra ecosystem (2022)

CN Asturias - Stateful application for kubernetes

Dayta AI Seminar - Kubernetes, Docker and AI on Cloud

Resume

Best Practices for Building Open Source Data Layers

Dok Talks #119 - Cloud-Native Data Pipelines

Episode 3: Kubernetes and Big Data Services

Data relay introduction to big data clusters

MayaData Datastax webinar - Operating Cassandra on Kubernetes with the help ...

Introducing Scylla Cloud

2018 04-06 kubernetes ingress in production

Automatically scaling your Kubernetes workloads - SVC210-S - Santa Clara AWS ...

Recently uploaded

Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx

ShamsuddeenMuhammadA

Vitthal Shirke Java Microservices Resume.pdf

Vitthal Shirke

Large Language Models and the End of Programming

Matt Welsh

GraphSummit Paris - The art of the possible with Graph Technology

Neo4j

Vitthal Shirke Microservices Resume Montevideo

Vitthal Shirke

Lecture 1 Introduction to games development

abdulrafaychaudhry

Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...

Globus

The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.

A Sighting of filterA in Typelevel Rite of Passage

Philip Schwarz

Essentials of Automations: The Art of Triggers and Actions in FME

Safe Software

In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation. We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios. Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!

In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...

Juraj Vysvader

First Steps with Globus Compute Multi-User Endpoints

Globus

In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.

BoxLang: Review our Visionary Licenses of 2024

Ortus Solutions, Corp

Webinar: Salesforce Document Management 2.0 - Smarter, Faster, Better

XfilesPro

How Recreation Management Software Can Streamline Your Operations.pptx

wottaspaceseo

Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.

Providing Globus Services to Users of JASMIN for Environmental Data Analysis

Globus

JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.

Developing Distributed High-performance Computing Capabilities of an Open Sci...

Globus

COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.

Orion Context Broker introduction 20240604

Fermin Galan

Cyaniclab : Software Development Agency Portfolio.pdf

Cyanic lab

CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.

2024 RoOUG Security model for the cloud.pptx

Georgi Kodinov

Enhancing Research Orchestration Capabilities at ORNL.pdf

Globus

Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.

Recently uploaded (20)

Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx

Vitthal Shirke Java Microservices Resume.pdf

Large Language Models and the End of Programming

GraphSummit Paris - The art of the possible with Graph Technology

Vitthal Shirke Microservices Resume Montevideo

Lecture 1 Introduction to games development

Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...

A Sighting of filterA in Typelevel Rite of Passage

Essentials of Automations: The Art of Triggers and Actions in FME

In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...

First Steps with Globus Compute Multi-User Endpoints

BoxLang: Review our Visionary Licenses of 2024

Webinar: Salesforce Document Management 2.0 - Smarter, Faster, Better

How Recreation Management Software Can Streamline Your Operations.pptx

Providing Globus Services to Users of JASMIN for Environmental Data Analysis

Developing Distributed High-performance Computing Capabilities of an Open Sci...

Orion Context Broker introduction 20240604

Cyaniclab : Software Development Agency Portfolio.pdf

2024 RoOUG Security model for the cloud.pptx

Enhancing Research Orchestration Capabilities at ORNL.pdf

Building Bridges Between Applications and Data

1. THE CLOUD CONNECTIVITY COMPANY Destination: Automation Building Bridges, Connecting Data San Francisco, CA July 2020 THE CLOUD CONNECTIVITY COMPANY 1

2. THE CLOUD CONNECTIVITY COMPANY Christopher Bradford Product Manager Astra, DataStax Kubernetes Operator for Apache Cassandra 2

3. THE CLOUD CONNECTIVITY COMPANY Jim Dickinson Lead Software Engineer Astra, DataStax Kubernetes Operator for Apache Cassandra 3

4. THE CLOUD CONNECTIVITY COMPANY 4

5. THE CLOUD CONNECTIVITY COMPANY Topics ● Cassandra Architecture ● Cassandra on Kubernetes ● Speaking the Native Protocol - CQL ● Routing with Kong ● Troubleshooting ● Questions 5

6. THE CLOUD CONNECTIVITY COMPANY Cassandra Architecture 6 AP Database, FTW!

7. THE CLOUD CONNECTIVITY COMPANY The Ring Replicated Fault Tolerant Scalable 7 Node E Node C Node A Node BNode F Node D Cassandra Cluster

8. THE CLOUD CONNECTIVITY COMPANY The Ring Replicated Fault Tolerant Scalable 8 Node E Node C Node A Node BNode F Node D Cassandra Cluster

9. THE CLOUD CONNECTIVITY COMPANY The Ring Replicated Fault Tolerant Scalable 9 Node E Node C Node A Node BNode F Node D East DC West DC Replication Sync & Async

10. THE CLOUD CONNECTIVITY COMPANY Driver Native Protocol - CQL 10 TCP, frames, versions, OH MY

11. THE CLOUD CONNECTIVITY COMPANY CQL Protocol Documented in-tree apache/cassandra 11

12. THE CLOUD CONNECTIVITY COMPANY Driver Communication Topology Aware Smart Routing 12 Node E Node C Node A DC Traditional LB

13. THE CLOUD CONNECTIVITY COMPANY Driver Communication Topology Aware Smart Routing 13 Node E Node C Node A

14. THE CLOUD CONNECTIVITY COMPANY Deploying Cassandra on Kubernetes 14 DataStax Kubernetes Operator for Apache Cassandra

15. THE CLOUD CONNECTIVITY COMPANY Cass Operator Features 15 ● Datacenter provisioning ● Node conﬁguration ● Node replacement ● Automatic upgrades ● Scale up

16. THE CLOUD CONNECTIVITY COMPANY Describes a logical C* Datacenter ■ Size ■ Racks (and Availability Zones) ■ Type: OSS or DSE ■ C* version ■ Conﬁguration overrides Submitted to the k8s cluster with kubectl 16 Custom Resource CassandraDatacenter apiVersion: datastax.com/v1beta1 kind: CassandraDatacenter metadata: name: dc1 spec: clusterName: cluster1 serverType: cassandra serverVersion: 3.11.6 size: 3 config: cassandra-yaml: num_tokens: 12

17. THE CLOUD CONNECTIVITY COMPANY helm repo add datastax https://datastax.github.io/charts helm repo update helm install cass-operator datastax/cass-operator 17 Install with Helm

18. THE CLOUD CONNECTIVITY COMPANY cass-operator 18

19. THE CLOUD CONNECTIVITY COMPANY Cluster Deployment 19

20. THE CLOUD CONNECTIVITY COMPANY Cluster Deployment 20

21. THE CLOUD CONNECTIVITY COMPANY Cluster Deployment 21

22. THE CLOUD CONNECTIVITY COMPANY Cluster Deployment 22

23. THE CLOUD CONNECTIVITY COMPANY Cluster Deployment 23

24. THE CLOUD CONNECTIVITY COMPANY Kubernetes Encapsulated 24 Node E Node C Node A

25. THE CLOUD CONNECTIVITY COMPANY Node E Node C Node A Application Outside the Cluster 25

26. THE CLOUD CONNECTIVITY COMPANY Deployments with Kong 26 Where are the clients, and how do we communicate with them?

27. THE CLOUD CONNECTIVITY COMPANY Internal Traﬃc 27 Node E Node C Node A DC

28. THE CLOUD CONNECTIVITY COMPANY External Ingress 28 Node E Node C Node A DC

29. THE CLOUD CONNECTIVITY COMPANY External Ingress with TLS 29 Node E Node C Node A DC 🔒 SNI

30. THE CLOUD CONNECTIVITY COMPANY External Ingress with TLS 30 Node E Node C Node A DC 🔒 SNI & mTLS

31. THE CLOUD CONNECTIVITY COMPANY There be dragons / Demo (time permitting) 31 Pitfalls, watch out!

32. THE CLOUD CONNECTIVITY COMPANY 32 Checkout Questions? https://bit.ly/CassOperator cass-operator on GitHub Managed Cassandra as a Service https://astra.datastax.com DataStax Astra

Building Bridges Between Applications and Data

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Building Bridges Between Applications and Data

Similar to Building Bridges Between Applications and Data (20)

Recently uploaded

Recently uploaded (20)

Building Bridges Between Applications and Data