Pivotal Container Service (PKS) provides an enterprise-grade Kubernetes platform that can be deployed on any cloud infrastructure using the open source BOSH tool. PKS handles operations tasks like provisioning and upgrading Kubernetes clusters, integrates with VMware technologies for networking and security, and provides a centralized control plane for managing multiple clusters and tenants. It aims to deliver the benefits of Kubernetes to enterprises by adding capabilities for high availability, multi-tenancy, security and automation.
Should you use traditional REST APIs to bind services together? Or is it better to use a richer, more loosely-coupled protocol? This talk will dig into how we piece services together in event driven systems, how we use a distributed log (event hub) to create a central, persistent history of events and what benefits we achieve from doing so. Apache Kafka is a perfect match for building such an asynchronous, loosely-coupled event-driven backbone. Events trigger processing logic, which can be implemented in a more traditional as well as in a stream processing fashion. The talk will show the difference between a request-driven and event-driven communication and show when to use which. It highlights how the modern stream processing systems can be used to
hold state both internally as well as in a database and how this state can be used to further increase independence of other services, the primary goal of a Microservices architecture.
Full recorded presentation at https://www.youtube.com/watch?v=2UfAgCSKPZo for Tetrate Tech Talks on 2022/05/13.
Envoy's support for Kafka protocol, in form of broker-filter and mesh-filter.
Contents:
- overview of Kafka (usecases, partitioning, producer/consumer, protocol);
- proxying Kafka (non-Envoy specific);
- proxying Kafka with Envoy;
- handling Kafka protocol in Envoy;
- Kafka-broker-filter for per-connection proxying;
- Kafka-mesh-filter to provide front proxy for multiple Kafka clusters.
References:
- https://adam-kotwasinski.medium.com/deploying-envoy-and-kafka-8aa7513ec0a0
- https://adam-kotwasinski.medium.com/kafka-mesh-filter-in-envoy-a70b3aefcdef
Free GitOps Workshop + Intro to Kubernetes & GitOpsWeaveworks
Follow along in this free workshop and experience GitOps!
AGENDA:
Welcome - Tamao Nakahara, Head of DX (Weaveworks)
Introduction to Kubernetes & GitOps - Mark Emeis, Principal Engineer (Weaveworks)
Weave Gitops Overview - Tamao Nakahara
Free Gitops Workshop - David Harris, Product Manager (Weaveworks)
If you're new to Kubernetes and GitOps, we'll give you a brief introduction to both and how GitOps is the natural evolution of Kubernetes.
Weave GitOps Core is a continuous delivery product to run apps in any Kubernetes. It is free and open source, and you can get started today!
https://www.weave.works/product/gitops-core
If you’re stuck, also come talk to us at our Slack channel! #weave-gitops http://bit.ly/WeaveGitOpsSlack (If you need to invite yourself to the Slack, visit https://slack.weave.works/)
Using the New Apache Flink Kubernetes Operator in a Production DeploymentFlink Forward
Flink Forward San Francisco 2022.
Running natively on Kubernetes, using the new Apache Flink Kubernetes Operator is a great way to deploy and manage Flink application and session deployments. In this presentation, we provide: - A brief overview of Kubernetes operators and their benefits. - Introduce the five levels of the operator maturity model. - Introduce the newly released Apache Flink Kubernetes Operator and FlinkDeployment CRs - Dockerfile modifications you can make to swap out UBI images and Java of the underlying Flink Operator container - Enhancements we're making in: - Versioning/Upgradeability/Stability - Security - Demo of the Apache Flink Operator in-action, with a technical preview of an upcoming product using the Flink Kubernetes Operator. - Lessons learned - Q&A
by
James Busche & Ted Chang
Should you use traditional REST APIs to bind services together? Or is it better to use a richer, more loosely-coupled protocol? This talk will dig into how we piece services together in event driven systems, how we use a distributed log (event hub) to create a central, persistent history of events and what benefits we achieve from doing so. Apache Kafka is a perfect match for building such an asynchronous, loosely-coupled event-driven backbone. Events trigger processing logic, which can be implemented in a more traditional as well as in a stream processing fashion. The talk will show the difference between a request-driven and event-driven communication and show when to use which. It highlights how the modern stream processing systems can be used to
hold state both internally as well as in a database and how this state can be used to further increase independence of other services, the primary goal of a Microservices architecture.
Full recorded presentation at https://www.youtube.com/watch?v=2UfAgCSKPZo for Tetrate Tech Talks on 2022/05/13.
Envoy's support for Kafka protocol, in form of broker-filter and mesh-filter.
Contents:
- overview of Kafka (usecases, partitioning, producer/consumer, protocol);
- proxying Kafka (non-Envoy specific);
- proxying Kafka with Envoy;
- handling Kafka protocol in Envoy;
- Kafka-broker-filter for per-connection proxying;
- Kafka-mesh-filter to provide front proxy for multiple Kafka clusters.
References:
- https://adam-kotwasinski.medium.com/deploying-envoy-and-kafka-8aa7513ec0a0
- https://adam-kotwasinski.medium.com/kafka-mesh-filter-in-envoy-a70b3aefcdef
Free GitOps Workshop + Intro to Kubernetes & GitOpsWeaveworks
Follow along in this free workshop and experience GitOps!
AGENDA:
Welcome - Tamao Nakahara, Head of DX (Weaveworks)
Introduction to Kubernetes & GitOps - Mark Emeis, Principal Engineer (Weaveworks)
Weave Gitops Overview - Tamao Nakahara
Free Gitops Workshop - David Harris, Product Manager (Weaveworks)
If you're new to Kubernetes and GitOps, we'll give you a brief introduction to both and how GitOps is the natural evolution of Kubernetes.
Weave GitOps Core is a continuous delivery product to run apps in any Kubernetes. It is free and open source, and you can get started today!
https://www.weave.works/product/gitops-core
If you’re stuck, also come talk to us at our Slack channel! #weave-gitops http://bit.ly/WeaveGitOpsSlack (If you need to invite yourself to the Slack, visit https://slack.weave.works/)
Using the New Apache Flink Kubernetes Operator in a Production DeploymentFlink Forward
Flink Forward San Francisco 2022.
Running natively on Kubernetes, using the new Apache Flink Kubernetes Operator is a great way to deploy and manage Flink application and session deployments. In this presentation, we provide: - A brief overview of Kubernetes operators and their benefits. - Introduce the five levels of the operator maturity model. - Introduce the newly released Apache Flink Kubernetes Operator and FlinkDeployment CRs - Dockerfile modifications you can make to swap out UBI images and Java of the underlying Flink Operator container - Enhancements we're making in: - Versioning/Upgradeability/Stability - Security - Demo of the Apache Flink Operator in-action, with a technical preview of an upcoming product using the Flink Kubernetes Operator. - Lessons learned - Q&A
by
James Busche & Ted Chang
Zeus: Uber’s Highly Scalable and Distributed Shuffle as a ServiceDatabricks
Zeus is an efficient, highly scalable and distributed shuffle as a service which is powering all Data processing (Spark and Hive) at Uber. Uber runs one of the largest Spark and Hive clusters on top of YARN in industry which leads to many issues such as hardware failures (Burn out Disks), reliability and scalability challenges.
Apache Iceberg - A Table Format for Hige Analytic DatasetsAlluxio, Inc.
Data Orchestration Summit
www.alluxio.io/data-orchestration-summit-2019
November 7, 2019
Apache Iceberg - A Table Format for Hige Analytic Datasets
Speaker:
Ryan Blue, Netflix
For more Alluxio events: https://www.alluxio.io/events/
Spark Summit EU 2015: Lessons from 300+ production usersDatabricks
At Databricks, we have a unique view into over a hundred different companies trying out Spark for development and production use-cases, from their support tickets and forum posts. Having seen so many different workflows and applications, some discernible patterns emerge when looking at common performance and scalability issues that our users run into. This talk will discuss some of these common common issues from an engineering and operations perspective, describing solutions and clarifying misconceptions.
Best Practices for ETL with Apache NiFi on Kubernetes - Albert Lewandowski, G...GetInData
Did you like it? Check out our E-book: Apache NiFi - A Complete Guide
https://ebook.getindata.com/apache-nifi-complete-guide
Apache NiFi is one of the most popular services for running ETL pipelines otherwise it’s not the youngest technology. During the talk, there are described all details about migrating pipelines from the old Hadoop platform to the Kubernetes, managing everything as the code, monitoring all corner cases of NiFi and making it a robust solution that is user-friendly even for non-programmers.
Author: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Flink powered stream processing platform at PinterestFlink Forward
Flink Forward San Francisco 2022.
Pinterest is a visual discovery engine that serves over 433MM users. Stream processing allows us to unlock value from realtime data for pinners. At Pinterest, we adopt Flink as the unified streaming processing engine. In this talk, we will share our journey in building a stream processing platform with Flink and how we onboarding critical use cases to the platform. Pinterest has supported 90+near realtime streaming applications. We will cover the problem statement, how we evaluate potential solutions and our decision to build the framework.
by
Rainie Li & Kanchi Masalia
This talk discusses the core concepts behind the Kubernetes extensibility model. We are going to see how to implement new CRDs, operators and when to use them to automate the most critical aspects of your Kubernetes clusters.
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...Flink Forward
Flink Forward San Francisco 2022.
Flink consumers read from Kafka as a scalable, high throughput, and low latency data source. However, there are challenges in scaling out data streams where migration and multiple Kafka clusters are required. Thus, we introduced a new Kafka source to read sharded data across multiple Kafka clusters in a way that conforms well with elastic, dynamic, and reliable infrastructure. In this presentation, we will present the source design and how the solution increases application availability while reducing maintenance toil. Furthermore, we will describe how we extended the existing KafkaSource to provide mechanisms to read logical streams located on multiple clusters, to dynamically adapt to infrastructure changes, and to perform transparent cluster migrations and failover.
by
Mason Chen
Apache Spark Streaming in K8s with ArgoCD & Spark OperatorDatabricks
Over the last year, we have been moving from a batch processing jobs setup with Airflow using EC2s to a powerful & scalable setup using Airflow & Spark in K8s.
The increasing need of moving forward with all the technology changes, the new community advances, and multidisciplinary teams, forced us to design a solution where we were able to run multiple Spark versions at the same time by avoiding duplicating infrastructure and simplifying its deployment, maintenance, and development.
Building Streaming Data Applications Using Apache KafkaSlim Baltagi
Apache Kafka evolved from an enterprise messaging system to a fully distributed streaming data platform for building real-time streaming data pipelines and streaming data applications without the need for other tools/clusters for data ingestion, storage and stream processing.
In this talk you will learn more about:
1. A quick introduction to Kafka Core, Kafka Connect and Kafka Streams: What is and why?
2. Code and step-by-step instructions to build an end-to-end streaming data application using Apache Kafka
This presentation covers how to setup an Airflow instance as a cluster which spans multiple machines instead of the traditional 1 machine distribution. In addition, it covers an added step you can take to ensure High Availability in that cluster.
Container Network Interface: Network Plugins for Kubernetes and beyondKubeAcademy
With the rise of modern containers comes new problems to solve – especially in networking. Numerous container SDN solutions have recently entered the market, each best suited for a particular environment. Combined with multiple container runtimes and orchestrators available today, there exists a need for a common layer to allow interoperability between them and the network solutions.
As different environments demand different networking solutions, multiple vendors and viewpoints look to a specification to help guide interoperability. Container Network Interface (CNI) is a specification started by CoreOS with the input from the wider open source community aimed to make network plugins interoperable between container execution engines. It aims to be as common and vendor-neutral as possible to support a wide variety of networking options — from MACVLAN to modern SDNs such as Weave and flannel.
CNI is growing in popularity. It got its start as a network plugin layer for rkt, a container runtime from CoreOS. Today rkt ships with multiple CNI plugins allowing users to take advantage of virtual switching, MACVLAN and IPVLAN as well as multiple IP management strategies, including DHCP. CNI is getting even wider adoption with Kubernetes adding support for it. Kubernetes accelerates development cycles while simplifying operations, and with support for CNI is taking the next step toward a common ground for networking. For continued success toward interoperability, Kubernetes users can come to this session to learn the CNI basics.
This talk will cover the CNI interface, including an example of how to build a simple plugin. It will also show Kubernetes users how CNI can be used to solve their networking challenges and how they can get involved.
KubeCon schedule link: http://sched.co/4VAo
Storing State Forever: Why It Can Be Good For Your AnalyticsYaroslav Tkachenko
State is an essential part of the modern streaming pipelines: it enables a variety of foundational capabilities like windowing, aggregation, enrichment, etc. But usually, the state is either transient, so we only keep it until the window is closed, or it's fairly small and doesn't grow much. But what if we treat the state differently? The keyed state in Flink can be scaled vertically and horizontally, it's reliable and fault-tolerant... so is scaling a stateful Flink application that different from scaling any data store like Kafka or MySQL?
At Shopify, we've worked on a massive analytical data pipeline that's needed to support complex streaming joins and correctly handle arbitrarily late-arriving data. We came up with an idea to never clear state and support joins this way. We've made a successful proof of concept, ingested all historical transactional Shopify data and ended up storing more than 10 TB of Flink state. In the end, it allowed us to achieve 100% data correctness.
Cross Data Center Replication with Redis using Redis EnterpriseCihan Biyikoglu
Redis Enterprise comes with a number of Replication technologies tuned for local (LAN) or cross geo (WAN) replication. The talk explains the architecture and capabilities provided for applications looking to deploy across multiple data centers for data locality or for geo-redundency
Tuning Apache Kafka Connectors for Flink.pptxFlink Forward
Flink Forward San Francisco 2022.
In normal situations, the default Kafka consumer and producer configuration options work well. But we all know life is not all roses and rainbows and in this session we’ll explore a few knobs that can save the day in atypical scenarios. First, we'll take a detailed look at the parameters available when reading from Kafka. We’ll inspect the params helping us to spot quickly an application lock or crash, the ones that can significantly improve the performance and the ones to touch with gloves since they could cause more harm than benefit. Moreover we’ll explore the partitioning options and discuss when diverging from the default strategy is needed. Next, we’ll discuss the Kafka Sink. After browsing the available options we'll then dive deep into understanding how to approach use cases like sinking enormous records, managing spikes, and handling small but frequent updates.. If you want to understand how to make your application survive when the sky is dark, this session is for you!
by
Olena Babenko
Pivotal Container Service (PKS) at SF Cloud Foundry Meetupcornelia davis
Overview of Pivotal Container Service (PKS), built on the open source Cloud Foundry Container Runtime (CFCR). Covers what Kubernetes is, how PKS presents a complete platform that includes Kubernetes and much more, and key cloud principles.
Presented at the San Francisco-Bay Area Cloud Foundry meetup.
Persistent Storage for stateful applications on Kubernetes made easy with Ope...MayaData Inc
In this webinar, Director of Community of Rancher Labs Jason van Brackel joins forces with Sr. Developer Advocate Patrick Hoolboom from MayaData to talk about benefits of OpenEBS and Rancher as a combined solution.
Rancher's multi-cluster Kubernetes management solution allows development teams to iterate fast, deploy efficiently and operate at scale. Kubernetes allows you to orchestrate containers that are highly available. However, in the case of container reschedule, Kubernetes does not provide a great set of primitives to manage your persistent data along with your application containers. In this webinar, we will present some of the challenges associated with managing persistent data in Kubernetes and how we can make day 2 operations easier to manage. We will briefly introduce the combined offering and talk about a couple of approaches to solving data persistence problems in multi-cloud environments with Rancher and OpenEBS. During the demos, we will showcase how we address data availability with OpenEBS.
We will also talk about project updates in the latest releases and preview of upcoming Kubecon announcements.
Zeus: Uber’s Highly Scalable and Distributed Shuffle as a ServiceDatabricks
Zeus is an efficient, highly scalable and distributed shuffle as a service which is powering all Data processing (Spark and Hive) at Uber. Uber runs one of the largest Spark and Hive clusters on top of YARN in industry which leads to many issues such as hardware failures (Burn out Disks), reliability and scalability challenges.
Apache Iceberg - A Table Format for Hige Analytic DatasetsAlluxio, Inc.
Data Orchestration Summit
www.alluxio.io/data-orchestration-summit-2019
November 7, 2019
Apache Iceberg - A Table Format for Hige Analytic Datasets
Speaker:
Ryan Blue, Netflix
For more Alluxio events: https://www.alluxio.io/events/
Spark Summit EU 2015: Lessons from 300+ production usersDatabricks
At Databricks, we have a unique view into over a hundred different companies trying out Spark for development and production use-cases, from their support tickets and forum posts. Having seen so many different workflows and applications, some discernible patterns emerge when looking at common performance and scalability issues that our users run into. This talk will discuss some of these common common issues from an engineering and operations perspective, describing solutions and clarifying misconceptions.
Best Practices for ETL with Apache NiFi on Kubernetes - Albert Lewandowski, G...GetInData
Did you like it? Check out our E-book: Apache NiFi - A Complete Guide
https://ebook.getindata.com/apache-nifi-complete-guide
Apache NiFi is one of the most popular services for running ETL pipelines otherwise it’s not the youngest technology. During the talk, there are described all details about migrating pipelines from the old Hadoop platform to the Kubernetes, managing everything as the code, monitoring all corner cases of NiFi and making it a robust solution that is user-friendly even for non-programmers.
Author: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Flink powered stream processing platform at PinterestFlink Forward
Flink Forward San Francisco 2022.
Pinterest is a visual discovery engine that serves over 433MM users. Stream processing allows us to unlock value from realtime data for pinners. At Pinterest, we adopt Flink as the unified streaming processing engine. In this talk, we will share our journey in building a stream processing platform with Flink and how we onboarding critical use cases to the platform. Pinterest has supported 90+near realtime streaming applications. We will cover the problem statement, how we evaluate potential solutions and our decision to build the framework.
by
Rainie Li & Kanchi Masalia
This talk discusses the core concepts behind the Kubernetes extensibility model. We are going to see how to implement new CRDs, operators and when to use them to automate the most critical aspects of your Kubernetes clusters.
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...Flink Forward
Flink Forward San Francisco 2022.
Flink consumers read from Kafka as a scalable, high throughput, and low latency data source. However, there are challenges in scaling out data streams where migration and multiple Kafka clusters are required. Thus, we introduced a new Kafka source to read sharded data across multiple Kafka clusters in a way that conforms well with elastic, dynamic, and reliable infrastructure. In this presentation, we will present the source design and how the solution increases application availability while reducing maintenance toil. Furthermore, we will describe how we extended the existing KafkaSource to provide mechanisms to read logical streams located on multiple clusters, to dynamically adapt to infrastructure changes, and to perform transparent cluster migrations and failover.
by
Mason Chen
Apache Spark Streaming in K8s with ArgoCD & Spark OperatorDatabricks
Over the last year, we have been moving from a batch processing jobs setup with Airflow using EC2s to a powerful & scalable setup using Airflow & Spark in K8s.
The increasing need of moving forward with all the technology changes, the new community advances, and multidisciplinary teams, forced us to design a solution where we were able to run multiple Spark versions at the same time by avoiding duplicating infrastructure and simplifying its deployment, maintenance, and development.
Building Streaming Data Applications Using Apache KafkaSlim Baltagi
Apache Kafka evolved from an enterprise messaging system to a fully distributed streaming data platform for building real-time streaming data pipelines and streaming data applications without the need for other tools/clusters for data ingestion, storage and stream processing.
In this talk you will learn more about:
1. A quick introduction to Kafka Core, Kafka Connect and Kafka Streams: What is and why?
2. Code and step-by-step instructions to build an end-to-end streaming data application using Apache Kafka
This presentation covers how to setup an Airflow instance as a cluster which spans multiple machines instead of the traditional 1 machine distribution. In addition, it covers an added step you can take to ensure High Availability in that cluster.
Container Network Interface: Network Plugins for Kubernetes and beyondKubeAcademy
With the rise of modern containers comes new problems to solve – especially in networking. Numerous container SDN solutions have recently entered the market, each best suited for a particular environment. Combined with multiple container runtimes and orchestrators available today, there exists a need for a common layer to allow interoperability between them and the network solutions.
As different environments demand different networking solutions, multiple vendors and viewpoints look to a specification to help guide interoperability. Container Network Interface (CNI) is a specification started by CoreOS with the input from the wider open source community aimed to make network plugins interoperable between container execution engines. It aims to be as common and vendor-neutral as possible to support a wide variety of networking options — from MACVLAN to modern SDNs such as Weave and flannel.
CNI is growing in popularity. It got its start as a network plugin layer for rkt, a container runtime from CoreOS. Today rkt ships with multiple CNI plugins allowing users to take advantage of virtual switching, MACVLAN and IPVLAN as well as multiple IP management strategies, including DHCP. CNI is getting even wider adoption with Kubernetes adding support for it. Kubernetes accelerates development cycles while simplifying operations, and with support for CNI is taking the next step toward a common ground for networking. For continued success toward interoperability, Kubernetes users can come to this session to learn the CNI basics.
This talk will cover the CNI interface, including an example of how to build a simple plugin. It will also show Kubernetes users how CNI can be used to solve their networking challenges and how they can get involved.
KubeCon schedule link: http://sched.co/4VAo
Storing State Forever: Why It Can Be Good For Your AnalyticsYaroslav Tkachenko
State is an essential part of the modern streaming pipelines: it enables a variety of foundational capabilities like windowing, aggregation, enrichment, etc. But usually, the state is either transient, so we only keep it until the window is closed, or it's fairly small and doesn't grow much. But what if we treat the state differently? The keyed state in Flink can be scaled vertically and horizontally, it's reliable and fault-tolerant... so is scaling a stateful Flink application that different from scaling any data store like Kafka or MySQL?
At Shopify, we've worked on a massive analytical data pipeline that's needed to support complex streaming joins and correctly handle arbitrarily late-arriving data. We came up with an idea to never clear state and support joins this way. We've made a successful proof of concept, ingested all historical transactional Shopify data and ended up storing more than 10 TB of Flink state. In the end, it allowed us to achieve 100% data correctness.
Cross Data Center Replication with Redis using Redis EnterpriseCihan Biyikoglu
Redis Enterprise comes with a number of Replication technologies tuned for local (LAN) or cross geo (WAN) replication. The talk explains the architecture and capabilities provided for applications looking to deploy across multiple data centers for data locality or for geo-redundency
Tuning Apache Kafka Connectors for Flink.pptxFlink Forward
Flink Forward San Francisco 2022.
In normal situations, the default Kafka consumer and producer configuration options work well. But we all know life is not all roses and rainbows and in this session we’ll explore a few knobs that can save the day in atypical scenarios. First, we'll take a detailed look at the parameters available when reading from Kafka. We’ll inspect the params helping us to spot quickly an application lock or crash, the ones that can significantly improve the performance and the ones to touch with gloves since they could cause more harm than benefit. Moreover we’ll explore the partitioning options and discuss when diverging from the default strategy is needed. Next, we’ll discuss the Kafka Sink. After browsing the available options we'll then dive deep into understanding how to approach use cases like sinking enormous records, managing spikes, and handling small but frequent updates.. If you want to understand how to make your application survive when the sky is dark, this session is for you!
by
Olena Babenko
Pivotal Container Service (PKS) at SF Cloud Foundry Meetupcornelia davis
Overview of Pivotal Container Service (PKS), built on the open source Cloud Foundry Container Runtime (CFCR). Covers what Kubernetes is, how PKS presents a complete platform that includes Kubernetes and much more, and key cloud principles.
Presented at the San Francisco-Bay Area Cloud Foundry meetup.
Persistent Storage for stateful applications on Kubernetes made easy with Ope...MayaData Inc
In this webinar, Director of Community of Rancher Labs Jason van Brackel joins forces with Sr. Developer Advocate Patrick Hoolboom from MayaData to talk about benefits of OpenEBS and Rancher as a combined solution.
Rancher's multi-cluster Kubernetes management solution allows development teams to iterate fast, deploy efficiently and operate at scale. Kubernetes allows you to orchestrate containers that are highly available. However, in the case of container reschedule, Kubernetes does not provide a great set of primitives to manage your persistent data along with your application containers. In this webinar, we will present some of the challenges associated with managing persistent data in Kubernetes and how we can make day 2 operations easier to manage. We will briefly introduce the combined offering and talk about a couple of approaches to solving data persistence problems in multi-cloud environments with Rancher and OpenEBS. During the demos, we will showcase how we address data availability with OpenEBS.
We will also talk about project updates in the latest releases and preview of upcoming Kubecon announcements.
Pivotal Container Service : la nuova soluzione per gestire Kubernetes in aziendaVMware Tanzu
Le applicazioni moderne vengono distribuite in poche ore anziché giorni o settimane, consentendo alle aziende di accelerare il time-to-value e fornire una migliore esperienza al loro cliente finale. Uno dei modi più rapidi per passare dall'ideazione alla produzione è quello di disporre di una piattaforma di gestione dei container coerente e affidabile che aiuti gli sviluppatori a erogare il software più velocemente e all'IT di semplificare le operazioni
VMware e Pivotal mettono insieme le nostre competenze combinate per offrire una soluzione di gestione dei container completa con Pivotal Container Service (PKS).
Unisciti ai tuoi colleghi in questo evento gratuito della durata di un'ora per sapere in che modo le aziende possono implementare i containers su vSphere con PKS, semplificando la gestione di un ambiente Kubernetes dall’installazione (day 1) fino all’aggiornamento ed evoluzione infrastrutturale (day 2).
Agenda del webinar:
- Kubernetes e l'orchestrazione dei container
- La gestione dei container e di Kubernetes in ambienti di produzione con VMware e -
- Pivotal Container Service (PKS)
- La modernizzazione delle applicazioni con PKS
- Demo di Pivotal Container Service e delle integrazioni con l'infrastruttura VMware
- Chiusura del webinar e Q/A
Presenters :
Fabio Chiodini, Advisory Platform Architect EMEA, Pivotal Ruggero Citterio, Senior System Engineer, VMware
The Kubernetes cloud native landscape is vast. Delivering a solution requires managing a puzzling array of required tooling, monitoring, disaster recovery, and other solutions that lie outside the realm of the central cluster. The governing body of Kubernetes, the Cloud Native Computing Foundation, has developed guidance for organizations interested in this topic by publishing the Cloud Native Landscape, but while a list of options is helpful it does not give operations and DevOps professionals the knowledge they need to execute.
Learn best practices of setting up and managing the tools needed around Kubernetes. This presentation covers popular open source options (to avoid lock in) and how one can implement and manage these tools on an ongoing basis. Learn from, and do not repeat, the mistakes of previous centralized platforms.
In this session, attendees will learn:
1. Cloud Native Landscape 101 - Prometheus, Sysdig, NGINX, and more. Where do they all fit in Kubernetes solution?
2. Avoiding the OpenStack sprawl of managing a multiverse of required tooling in the Kubernetes world.
3. Leverage technology like Kubernetes, now available on DC/OS, to provide part of the infrastructure framework that helps manage cloud native application patterns.
Pivotal Cloud Foundry 2.6: A First LookVMware Tanzu
Join Dan Baskette and Jared Ruckle for a view into Pivotal Cloud Foundry (PCF) 2.6 capabilities with demos and expert Q&A. We’ll review the latest features for Pivotal’s flagship app platform, including:
CUSTOM SIDECAR PROCESSES (BETA)
In Pivotal Application ServiceⓇ 2.6 (PAS), developers can run custom sidecar processes in the same container as their application. This simplifies development for all kinds of “wire” use cases, including proxy forwarding, client-side load balancing, timeouts, and retries.
MULTI-CLOUD CONTINUOUS DELIVERY WITH SPINNAKER
PCF now integrates nicely with the most popular CD tool, Spinnaker. Spinnaker 1.14 now supports several advanced CD scenarios with PCF. As a result, large development teams can more easily deploy to production to improve outcomes. Use Spinnaker with PAS as well as Enterprise PKSⓇ. (This integration is backed by community support.)
NEW PERMISSIONS MODEL IN CONCOURSE FOR PCF (coming soon) Concourse for PCF 5.2 will include a powerful new permissions model to better segment access to build pipelines. The new release will add compatibility with CredHub for secrets management as well.
MULTI-DATACENTER REPLICATION CAPABILITIES FOR MySQL (coming soon) MySQL for PCF 2.7 will add multi-DC replication capabilities as a beta feature. This will offer more stability and scalability for your database apps.
Plus much more!
Pivotal Platform: A First Look at the October ReleaseVMware Tanzu
Join Dan Baskette and Jared Ruckle for a first look at the latest Pivotal Platform capabilities with demos and expert Q&A. Attend this session and learn how you can put these new updates to work for your enterprise.
Build apps atop Kubernetes with:
● Azure Spring Cloud, a complete runtime for Spring apps atop Azure Kubernetes Service
● Pivotal Build Service, an automated workflow for code-to-container builds
● Container Services Manager for Pivotal Platform, a bridge between Pivotal Application Service and PKS
Build apps atop a self-managed platform with:
● Pivotal Application Service 2.7, and its additional app deployment capabilities
● Pivotal Service Instance Manager, a new tool to help you manage backing services at scale
Get your apps to production with CI/CD tools like:
● Pivotal Continuous Delivery with Spinnaker
● Pivotal Concourse 5.5
We’ll also review Pivotal Spring Cloud Gateway and Pivotal Cloud Cache 1.9!
Presenter : Dan Baskette, Director, Technical Marketing & Jared Ruckle, Director, Product Marketing
Eseguire Applicazioni Cloud-Native con Pivotal Cloud Foundry su Google Cloud ...VMware Tanzu
Eseguire Applicazioni Cloud-Native con Pivotal Cloud Foundry su Google Cloud Platform (Pivotal Cloud-Native Workshop: Milan)
Fabio Marinelli
7 February 2018
Cloud-Native Operations with Kubernetes and CI/CDVMware Tanzu
Operations practices have historically lagged behind development. Agile and Extreme Programming have become common practice for development teams. In the last decade, the DevOps and SRE movements have brought these concepts to operations, borrowing heavily from Lean principles such as Kanban and Value Stream Mapping. So, how does all of this play out if we’re using Kubernetes?
In this class, Paul Czarkowski, Principal Technologist at Pivotal, will explain how Kubernetes enables a new cloud-native way of operating software. Attend to learn:
● what cloud-native operations are;
● how to build a cloud-native CI/CD stack; and
● how to deploy and upgrade an application from source to production on Kubernetes.
Presenter:
Paul Czarkowski, Principal Technologist, Pivotal Software
Pivotal Cloud Foundry 2.1: Making Transformation Real WebinarVMware Tanzu
The Pivotal Cloud Foundry (PCF) platform has expanded and now includes a family of products to rapidly deliver apps, containers and functions. This evolution reflects today's IT reality — you need to use the right abstraction for each scenario.
Join us for a discussion of PCF 2.1: the first release that includes updates to the PCF family:Pivotal Application Service (PAS), Pivotal Container Service (PKS), Pivotal Function Service (PFS), and the Services Marketplace.
PCF 2.1 release highlights include: PAS for Windows, PKS 1.0, Steeltoe 2.0, Spring Cloud Data Flow for PCF 1.0, and much more. We'll also discuss a slew of highlights to PAS, including essential enhancements to Operations Manager, security, routing, and built-in services.
Presenter : Jared Ruckle & Pieter Humphrey, Pivotal
GCP Meetup #3 - Approaches to Cloud Native Architecturesnine
Talk by Daniel Leahy and Nic Gibson, given at the Google Cloud Meetup on March 3, 2020, hosted by Nine Internet Solutions AG - Your Swiss Managed Cloud Service Provider.
PKS: The What and How of Enterprise-Grade KubernetesVMware Tanzu
SpringOne Platform 2017
Cornelia Davis, Pivotal; Fred Melo, Pivotal
Because of its well thought out and powerful abstractions, robust and cloud-native architecture, and the vibrant community around it, the use of Kubernetes for containerized workloads has surged. And while Kubernetes is theoretically ready to run applications in production, the actual viability is highly dependent on how Kubernetes itself is managed. In this session Cornelia and Fred will cover role of the container orchestration system in your IT landscape, and they’ll dive under the covers to show how it provides the enterprise-class Kubernetes services you need to trust your most critical workloads to it. Yes, technical details revealed!
Continuous Everything in a Multi-cloud and Multi-platform EnvironmentVMware Tanzu
This presentation is all about Concourse, Spinnaker, Harbor and Pivotal Build Service, and how combining these tools can help you build an efficient, secure, and trusted container pipeline.
Red Hat multi-cluster management & what's new in OpenShiftKangaroot
More and more organisations are not only using container platforms but starting to run multiple clusters of containers. And with that comes new headaches of maintaining, securing, and updating those multiple clusters. In this session we'll look into how Red Hat has solved multi-cluster management, covering cluster lifecycle, app lifecycle, and governance/risk/compliance.
Pivotal Container Service il modo più semplice per gestire Kubernetes in azie...VMware Tanzu
Pivotal Container Service il modo più semplice per gestire Kubernetes in azienda (Pivotal Cloud-Native Workshop: Milan)
Fabio Marinelli & Mattia Gandolfi
7 February 2018
Similar to Pivotal Container Service Overview (20)
The Tanzu Developer Connect is a hands-on workshop that dives deep into TAP. Attendees receive a hands on experience. This is a great program to leverage accounts with current TAP opportunities.
The Tanzu Developer Connect is a hands-on workshop that dives deep into TAP. Attendees receive a hands on experience. This is a great program to leverage accounts with current TAP opportunities.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Software Engineering, Software Consulting, Tech Lead.
Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Security,
Spring Transaction, Spring MVC,
Log4j, REST/SOAP WEB-SERVICES.
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...Hivelance Technology
Cryptocurrency trading bots are computer programs designed to automate buying, selling, and managing cryptocurrency transactions. These bots utilize advanced algorithms and machine learning techniques to analyze market data, identify trading opportunities, and execute trades on behalf of their users. By automating the decision-making process, crypto trading bots can react to market changes faster than human traders
Hivelance, a leading provider of cryptocurrency trading bot development services, stands out as the premier choice for crypto traders and developers. Hivelance boasts a team of seasoned cryptocurrency experts and software engineers who deeply understand the crypto market and the latest trends in automated trading, Hivelance leverages the latest technologies and tools in the industry, including advanced AI and machine learning algorithms, to create highly efficient and adaptable crypto trading bots
Designing for Privacy in Amazon Web ServicesKrzysztofKkol1
Data privacy is one of the most critical issues that businesses face. This presentation shares insights on the principles and best practices for ensuring the resilience and security of your workload.
Drawing on a real-life project from the HR industry, the various challenges will be demonstrated: data protection, self-healing, business continuity, security, and transparency of data processing. This systematized approach allowed to create a secure AWS cloud infrastructure that not only met strict compliance rules but also exceeded the client's expectations.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar
The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month.
The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies.
However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News.
Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!
Experience our free, in-depth three-part Tendenci Platform Corporate Membership Management workshop series! In Session 1 on May 14th, 2024, we began with an Introduction and Setup, mastering the configuration of your Corporate Membership Module settings to establish membership types, applications, and more. Then, on May 16th, 2024, in Session 2, we focused on binding individual members to a Corporate Membership and Corporate Reps, teaching you how to add individual members and assign Corporate Representatives to manage dues, renewals, and associated members. Finally, on May 28th, 2024, in Session 3, we covered questions and concerns, addressing any queries or issues you may have.
For more Tendenci AMS events, check out www.tendenci.com/events
3. Operational
Efficiency
● Employ 500:1 developer
to operator ratio
● Perform zero-downtime
upgrades
● Runs the same way
on every public/private
cloud
Developer
Productivity
Comprehensive
Security
● Accelerate feedback
loops by improving
delivery velocity
● Focus on applications,
not infrastructure
● Give developers the
tools and frameworks
to build resilient apps
● Adopt a defense-in-
depth approach
● Continuously update
platforms to limit
threat impact
● Apply the 3 R’s →
repair, repave, rotate
● Run platforms that
stay online under
all circumstances
● Scale up and down,
in and out, through
automation
● Deploy multi-cloud
resilience patterns
High Availability
Platform Team Delivering Real Value
4. Can we realize these benefits for other workloads too?
MONOLITHIC
APPLICATIONS
More
MICROSERVICES
Stateful
or
Clusters
CONTAINERS COTS
DATA SERVICES
MICROSERVICES
MONOLITHIC
APPLICATIONS
Some Some .NET APPLICATIONS
5. Can we realize these benefits for other workloads too?
MONOLITHIC
APPLICATIONS
More
MICROSERVICES
Stateful
or
Clusters
CONTAINERS COTS
DATA SERVICES
MICROSERVICES
MONOLITHIC
APPLICATIONS
Some Some .NET APPLICATIONS
12. Storage NetworkingCompute
Dev / Apps
App User
IT / Ops
> kubectl
Kubernetes Dashboard
Load Balancing / Routing
Container Image
Registry
App Monitoring
App Logging
OS Updates
OS Images
K8S Updates
K8S Images
Log & Monitor
Recover & Restart
Backup & Restore
External
Data Services
Cluster
Provisioning
Provision & Scale
Command Line
/ API
Management
GUI
Monitoring GUI
...but Kubernetes alone is not enough for enterprises
13. Storage NetworkingCompute
Pivotal Container Service (PKS) provides what’s missing
Dev / Apps
App User
IT / Ops
> kubectl
Kubernetes Dashboard
Load Balancing / Routing
OS Updates
OS Images
K8S Updates
K8S Images
Log & Monitor
Recover & Restart
Backup & Restore
External
Data Services
Cluster
Provisioning
Provision & Scale
PKS Control Plane
> pks
Operations Manager
vRealize Operations*
Container Image
Registry
vRealize
LogInsight
14. on any Cloud
Dev / Apps
App User
IT / Ops
> kubectl
Kubernetes Dashboard
Load Balancing / Routing
Container Image
Registry
OS Updates
OS Images
K8S Updates
K8S Images
Log & Monitor
Recover & Restart
Backup & Restore
External
Data Services
Cluster
Provisioning
Provision & Scale
vRealize
LogInsight
PKS Control Plane
> pks
Operations Manager
vRealize Operations*
15. What PKS adds to Kubernetes
PKS value-
added
features
Built into
Kubernetes
Multi-container pods
Stateful Sets of pods
Persistent disks
Single tenant ingress
Pod scaling and high availability
Rolling upgrades to pods
Cluster provisioning and scaling
Embedded, hardened Operating System
Monitoring and recovery of cluster VMs and processes
Rolling upgrades to cluster infrastructure
Secure multi-tenant ingress
Secure container registry
16. “On a good day our DIY k8s solution could provision a cluster
in 45 minutes. On a bad day it would fail to deploy ~30
times” - IT Operator
17. Cover w/ Image
Turnkey solution. All the things you need to use
and operate a container runtime. In one package.
On every cloud.
Enterprise readiness, control and security.
Continually updated platform, embedded OS,
secure container registry, policy-driven networking,
integrated IdM. Controllable and customizable by
service plans.
Developer empowerment. Consume app services,
popular tools “just work” with vanilla Kubernetes via
constant compatibility. Developers get the
Kubernetes they want, for their choice of workloads.
Multi cloud. Run it on any infrastructure of choice.
On premises or in the public cloud.
Our investment themes
18. Is your network ready?
(How mature is your SDN?)
Answer: Probably not; if you require
tickets/manual process for network or
firewall rules, you aren’t ready
● Microsegmentation with NSX-T
● Automated IP allocation and
load balancer provisioning
● Monitoring & troubleshooting
with familiar VMware tooling
How many clusters are
you going to need?
Answer: More than one required to
provide true multi-tenancy
● Self-service, on-demand
provisioning of clusters
● Pre-defined T-shirt size
clusters
● Scale clusters up and down
How many times does
K8s release per year?
Answer: Major releases quarterly;
minor releases/patches frequently
● Automated upgrades on-
demand with BOSH
● Automatic patching with
Concourse pipelines
● Self-healing nodes on failure
Upgrades Multi-tenancy Network
PIVOTAL’S OPINION PIVOTAL’S OPINION
Three Questions
PIVOTAL’S OPINION
20. BOSH
Reliable and consistent operational experience for any cloud.
BOSH
Harbor
NSX-T
Kubernetes
K8s Cluster
K8s Cluster
K8s Cluster
PKS Control Plane
Use the PKS CLI and API to
create, operate, and scale your
clusters.
VMware GCP Azure Openstack AWS
PKSControlPlane
Built with open-source
Kubernetes
Constant compatibility with the
latest stable release Kubernetes
—no proprietary extensions.
Harbor
An enterprise-class container registry.
Includes vulnerability scanning,
identity management, and more.
NSX-T
Network management, security, and
load balancing out-of-the-box with
VMware NSX-T. Multi-cloud, multi-
hypervisor.
Enterprise-Grade Kubernetes
24. BOSH is an open source
tool for release
engineering, deployment,
lifecycle management,
and monitoring of
distributed systems.
25. ➔ Health monitoring (server & processes)
➔ Self-healing w/ Resurrector
➔ Storage management
➔ Rolling upgrades with canaries
➔ Easy scaling of clusters
➔ Repeatability and Consistency
➔ Packaging w/ embedded OS
➔ Server provisioning on any IaaS
➔ Software deployment across AZs
28. Availability Zone B
Availability Zone A
Bosh distributes deployments across AZ’s
Availability Zone A
Availability Zone B
Kubelet
Kube-proxy
Pod
Pod
K8s Worker
Pod
API Server
Kube Scheduler
K8s Master
Controller
Manager
Kubelet watches and restart containers
Bosh agent watches and restarts processes
Bosh director watches and restarts nodes
Bosh agent
Bosh agent
Bosh Health
Manager
PKS Health Management
31. Two models supported
Multi-tenant clusters
● Leverage Kubernetes namespaces
Limitations with Kubernetes alone
● Noisy neighbors (workloads can affect other
tenants)
● Share the same network
● Share DNS
● Shared Configuration
● ...
We add
● Network microsegmentation with NSX-T
○ Eliminating “Share the same network”
Multi (Single-tenant) clusters
It is having an API for creation and management
that enables this!!!
● Every tenant gets their own cluster
Addresses limitations
● Single tenant worker VMs (depend on the
hypervisor to ensure host is properly shared)
● Every cluster has own network segment
● Every cluster has own DNS
● Every cluster has own configuration
● ...
No other “on
prem” solution
has this!!!
32. Two models supported
Multi-tenant clusters
● Leverage Kubernetes namespaces
Limitations with Kubernetes alone
● Noisy neighbors (workloads can affect other
tenants)
● Share the same network
● Share DNS
● Shared Configuration
● ...
We add
● Network microsegmentation with NSX-T
○ Eliminating “Share the same network”
Multi (Single-tenant) clusters
It is having an API for creation and management
that enables this!!!
● Every tenant gets their own cluster
Addresses limitations
● Single tenant worker VMs (depend on the
hypervisor to ensure host is properly shared)
● Every cluster has own network segment
● Every cluster has own DNS
● Every cluster has own configuration
● ...
No other “on
prem” solution
has this!!!
33.
34. Multi-Tenancy - Single Cluster vs Multi-Cluster
Shared Compute/Storage/Network
Tenant 1
Tenant 2
Kube API
Server
Kube SchedulerDNS Cont. Mgr.
Kubelet Kubelet Kubelet
Worker Worker Worker
PKS Control Plane
Master Node(s)
Shared Compute/Storage/Network
Kube API
Server
Kube
Scheduler
DNS
Cont. Mgr.
Kubelet Kubelet Kubelet
Worker Worker Worker
PKS Control Plane
Master Node(s)
Kube API
Server
Kube
Scheduler
DNS
Cont. Mgr.
Master Node(s)
Kubelet
Worker
Tenant 1 Tenant 2
41. Role-Based Access Control (RBAC)
LDAP/AD Integration
Image Vulnerability Scanning (Clair)
Notary Image Signing
Policy-Based Image Replication
Graphical User Portal & RESTful API
Image Deletion & Garbage Collection
Auditing
An enterprise-class registry server for
Docker images
Build Image
Push
Image
Scan
Image
for CVEs
Sign
Image
kubectl
run
Dev Team
Image
Registry
Clair Notary
R
B
A
C
UAA
Auth
R
E
P
L
42. Physical Infrastructure
Container
Registry
vSphere vSAN, NFS, iSCSI, FC Datastores
NSX-T
BOSH
masteretcd workermasteretcd worker
PKS Control Plane
Integrations w/ VMware
vRealize
Automation
vRealize
Log Insight
vRealize
Operations
vRealize
Network
Insight
Wavefront
by VMware
44. ● Install/Manage PKS
● Configure cluster plans
● Apply a patch / update
● Onboard Cluster Owner
via RBAC
● Operate Bosh
Platform Operator (Alana)
A Day in Life with PKS
● Create a cluster
● Scale a cluster
● Create Network Policy
● Onboard App Dev via
RBAC
● ...
Cluster Owner (Cody)
● Deploy an app
● Expose app with service
type: LoadBalancer
● Expose app with Ingress
● ...
App Dev (Naomi)
● Health Management (server & process)
● Network Automation
Automation
49. PKS User Interaction
● The PKS Management VM runs the PKS API
together with the Broker, UAA and a MySQL DB.
● The PKS API orchestrates the initial kubernetes
cluster deployments and scaling of those clusters.
● A single PKS VM can manage hundreds of
Kubernetes cluster.
● The PKS CLI is a single binary that can be installed
on a Mac, Windows, or Linux to drive the PKS API.
PKS CLI
PKS
Control Plane
PKS API
50. Creating a new K8s Cluster
Platform User
PKSControlPlane
CLI
API
PKS CREATE CLUSTER
BOSH
deploy
Kubernetes cluster
Create
Harbor
NSX-T
Master
Worker
WorkerWorker
etcd Worker
Master
etcd
51. Scaling a Kubernetes Cluster
Platform User
PKSControlPlane
CLI
API
PKS SCALE CLUSTER
BOSH
deploy
Kubernetes cluster
Scale
Harbor
NSX-T
Master
Worker
WorkerWorker
etcd
Worker
Master
etcd
Worker