Kubernetes is making the promise of changing the datacenter from being a group of computer to "a computer" itself. This presentation outlines the new features in K8S with 1.1 and 1.2 release.
A small introduction to get started on Kubernetes as a user. This explains the main concepts like pod, deployment and services and gives some hints to help you use kubectl command.
These slides were presented in Grenoble Docker meetup in November 2017.
WSO2Con US 2015 Kubernetes: a platform for automating deployment, scaling, an...Brian Grant
Kubernetes can run application containers on clusters of physical or virtual machines.
It can also do much more than that.
Kubernetes satisfies a number of common needs of applications running in production, such as co-locating helper processes, mounting storage systems, distributing secrets, application health checking, replicating application instances, horizontal auto-scaling, load balancing, rolling updates, and resource monitoring.
However, even though Kubernetes provides a lot of functionality, there are always new scenarios that would benefit from new features. Ad hoc orchestration that is acceptable initially often requires robust automation at scale. Application-specific workflows can be streamlined to accelerate developer velocity.
This is why Kubernetes was also designed to serve as a platform for building an ecosystem of components and tools to make it easier to deploy, scale, and manage applications. The Kubernetes control plane is built upon the same APIs that are available to developers and users, implementing resilient control loops that continuously drive the current state towards the desired state. This design has enabled Apache Stratos and a number of other Platform as a Service and Continuous Integration and Deployment systems to build atop Kubernetes.
This presentation introduces Kubernetes’s core primitives, shows how some of its better known features are built on them, and introduces some of the new capabilities that are being added.
Kubernetes has been a key component for many companies to reduce technical debt in infrastructure by:
• Fostering the Adoption of Docker
• Simplifying Container Management
• Onboarding Developers On Infrastructure
• Unlocking Continuous Integration and Delivery
During this meetup we are going to discuss the following topics and share some best practices
• What's new with Kubernetes 1.3
• Generate Cluster Configuration using CloudFormation
• Deploy Kubernetes Clusters on AWS
• Scaling the Cluster
• Integrating Ingress with Elastic Load Balancer
• Using Internal ELB's as Kubernetes' Service
• Using EBS for persistent volumes
• Integrating Route53
KubeCon EU 2016: A Practical Guide to Container SchedulingKubeAcademy
Containers are at the forefront of a new wave of technology innovation but the methods for scheduling and managing them are still new to most developers. In this talk we'll look at the kind of problems that container scheduling solves and at how maximising efficiency and maiximising QoS don't have to be exclusive goals. We'll take a behind the scenes look at the Kubernetes scheduler: How does it prioritize? What about node selection and external dependencies? How do you schedule based on your own specific needs? How does it scale and what’s in it both for developers already using containers and for those that aren't? We’ll use a combination of slides, code, demos to answer all these questions and hopefully all of yours.
Sched Link: http://sched.co/6BZa
Kubernetes is a great tool to run (Docker) containers in a clustered production environment. When deploying often to production we need fully automated blue-green deployments, which makes it possible to deploy without any downtime. We also need to handle external HTTP requests and SSL offloading. This requires integration with a load balancer like Ha-Proxy. Another concern is (semi) auto scaling of the Kubernetes cluster itself when running in a cloud environment. E.g. partially scale down the cluster at night.
In this technical deep dive you will learn how to setup Kubernetes together with other open source components to achieve a production ready environment that takes code from git commit to production without downtime.
A small introduction to get started on Kubernetes as a user. This explains the main concepts like pod, deployment and services and gives some hints to help you use kubectl command.
These slides were presented in Grenoble Docker meetup in November 2017.
WSO2Con US 2015 Kubernetes: a platform for automating deployment, scaling, an...Brian Grant
Kubernetes can run application containers on clusters of physical or virtual machines.
It can also do much more than that.
Kubernetes satisfies a number of common needs of applications running in production, such as co-locating helper processes, mounting storage systems, distributing secrets, application health checking, replicating application instances, horizontal auto-scaling, load balancing, rolling updates, and resource monitoring.
However, even though Kubernetes provides a lot of functionality, there are always new scenarios that would benefit from new features. Ad hoc orchestration that is acceptable initially often requires robust automation at scale. Application-specific workflows can be streamlined to accelerate developer velocity.
This is why Kubernetes was also designed to serve as a platform for building an ecosystem of components and tools to make it easier to deploy, scale, and manage applications. The Kubernetes control plane is built upon the same APIs that are available to developers and users, implementing resilient control loops that continuously drive the current state towards the desired state. This design has enabled Apache Stratos and a number of other Platform as a Service and Continuous Integration and Deployment systems to build atop Kubernetes.
This presentation introduces Kubernetes’s core primitives, shows how some of its better known features are built on them, and introduces some of the new capabilities that are being added.
Kubernetes has been a key component for many companies to reduce technical debt in infrastructure by:
• Fostering the Adoption of Docker
• Simplifying Container Management
• Onboarding Developers On Infrastructure
• Unlocking Continuous Integration and Delivery
During this meetup we are going to discuss the following topics and share some best practices
• What's new with Kubernetes 1.3
• Generate Cluster Configuration using CloudFormation
• Deploy Kubernetes Clusters on AWS
• Scaling the Cluster
• Integrating Ingress with Elastic Load Balancer
• Using Internal ELB's as Kubernetes' Service
• Using EBS for persistent volumes
• Integrating Route53
KubeCon EU 2016: A Practical Guide to Container SchedulingKubeAcademy
Containers are at the forefront of a new wave of technology innovation but the methods for scheduling and managing them are still new to most developers. In this talk we'll look at the kind of problems that container scheduling solves and at how maximising efficiency and maiximising QoS don't have to be exclusive goals. We'll take a behind the scenes look at the Kubernetes scheduler: How does it prioritize? What about node selection and external dependencies? How do you schedule based on your own specific needs? How does it scale and what’s in it both for developers already using containers and for those that aren't? We’ll use a combination of slides, code, demos to answer all these questions and hopefully all of yours.
Sched Link: http://sched.co/6BZa
Kubernetes is a great tool to run (Docker) containers in a clustered production environment. When deploying often to production we need fully automated blue-green deployments, which makes it possible to deploy without any downtime. We also need to handle external HTTP requests and SSL offloading. This requires integration with a load balancer like Ha-Proxy. Another concern is (semi) auto scaling of the Kubernetes cluster itself when running in a cloud environment. E.g. partially scale down the cluster at night.
In this technical deep dive you will learn how to setup Kubernetes together with other open source components to achieve a production ready environment that takes code from git commit to production without downtime.
Overview of kubernetes and its use as a DevOps cluster management framework.
Problems with deployment via kube-up.sh and improving kubernetes on AWS via custom cloud formation template.
KubeCon EU 2016: Multi-Tenant KubernetesKubeAcademy
Today Kubernetes is mostly employed in single tenant deployment, either private cloud, or as a COE on top of IaaS. By leveraging virtualized container like Hyper, Kubernetes will be the core of multi-tenant Container-as-a-Service. This talk will present Hypernetes, a secure Kubernetes distro focusing on the public container hosting service.
Sched Link: http://sched.co/6BYD
This talk will focus on a brief history, including a demo and overview of how we at Superbalist use Kubernetes, and how Kubernetes uses Docker, does load balancing, deployments, and data migrations.
Talk from Cape Town DevOps meetup on Jun 21, 2016:
https://www.meetup.com/Cape-Town-DevOps/events/231530172/
Code: https://github.com/zoidbergwill/kubernetes-examples
Slides as markdown: http://www.zoidbergwill.com/presentations/2016/kubernetes-1.2-and-spread/index.md
I am glad to share the presentation of the Kubernetes Pune meetup organized on 29 July 2017. One of the good response from the Pune folks to the community.
Monitoring, Logging and Tracing on KubernetesMartin Etmajer
In this presentation, I'll describe a variety of tools, like the Kubernetes Dashboard, Heapster, Grafana, Fluentd, Elasticsearch, Kibana, Jolokia and OpenTracing to bring Monitoring, Logging and Tracing to the Kubernetes container platform.
- Archeology: before and without Kubernetes
- Deployment: kube-up, DCOS, GKE
- Core Architecture: the apiserver, the kubelet and the scheduler
- Compute Model: the pod, the service and the controller
Top 3 reasons why you should run your Enterprise workloads on GKESreenivas Makam
This deck covers top 3 reasons why Google Kubernetes engine is best suited to run containerized workloads. The reasons covered are Security, Observability and Maturity.
Kubernetes has two simple but powerful network concepts: every Pod is connected to the same network, and Services let you talk to a Pod by name. Bryan will take you through how these concepts are implemented - Pod Networks via the Container Network Interface (CNI), Service Discovery via kube-dns and Service virtual IPs, then on to how Services are exposed to the rest of the world.
Cloud native applications are popular these days – applications that run in the cloud reliably und scale almost arbitrarily. They follow three key principles: they are built and composed as micro services. They are packaged and distributed in containers. The containers are executed dynamically in the cloud. Kubernetes is an open-source cluster manager for the automated deployment, scaling and management of cloud native applications. In this hands-on session we will introduce the core concepts of Kubernetes and then show how to build, package and operate a cloud native showcase application on top of Kubernetes step-by-step. Throughout this session we will be using an off-the-shelf MIDI controller to demonstrate and visualize the concepts and to remote control Kubernetes. This session has been presented at the ContainerCon Europe 2016 in Berlin. #qaware #cloudnativenerd #LinuxCon #ContainerCon
Orchestration tool roundup kubernetes vs. docker vs. heat vs. terra form vs...Nati Shalom
Video recording: https://www.youtube.com/watch?v=tGlIgUeoGz8
It’s no news that containers represent a portable unit of deployment, and OpenStack has proven an ideal environment for running container workloads. However, where it usually becomes more complex is that many times an application is often built out of multiple containers. What’s more, setting up a cluster of container images can be fairly cumbersome because you need to make one container aware of another and expose intimate details that are required for them to communicate which is not trivial especially if they’re not on the same host.
These scenarios have instigated the demand for some kind of orchestrator. The list of container orchestrators is growing fairly fast. This session will compare the different orchestation projects out there - from Heat to Kubernetes to TOSCA - and help you choose the right tool for the job.
Session link from teh summit: https://openstacksummitmay2015vancouver.sched.org/event/abd484e0dedcb9774edda1548ad47518#.VV5eh5NViko
Arkena's video-on-demand platform is used as backend by major european channels (TF1 / beIN SPORTS / Elisa) to propose a non-linear experience to their customers.
Previously hosted on Heroku, the number of our users is increasing constantly. In order to optimize resources we decided to move on a bare metal infrastructure powered by Kubernetes.
We'll share thoughts, feedbacks and technical details about this successful transition.
Sched Link:
KubeCon CloudNativeCon 2016 Seattle - a reportKrishna-Kumar
KubeCon / CloudNativeCon Seattle summary report - Just to recapture some of the items from the event - Few of the items are copied from other blogs from reference - pictures are just for FUN!
Overview of kubernetes and its use as a DevOps cluster management framework.
Problems with deployment via kube-up.sh and improving kubernetes on AWS via custom cloud formation template.
KubeCon EU 2016: Multi-Tenant KubernetesKubeAcademy
Today Kubernetes is mostly employed in single tenant deployment, either private cloud, or as a COE on top of IaaS. By leveraging virtualized container like Hyper, Kubernetes will be the core of multi-tenant Container-as-a-Service. This talk will present Hypernetes, a secure Kubernetes distro focusing on the public container hosting service.
Sched Link: http://sched.co/6BYD
This talk will focus on a brief history, including a demo and overview of how we at Superbalist use Kubernetes, and how Kubernetes uses Docker, does load balancing, deployments, and data migrations.
Talk from Cape Town DevOps meetup on Jun 21, 2016:
https://www.meetup.com/Cape-Town-DevOps/events/231530172/
Code: https://github.com/zoidbergwill/kubernetes-examples
Slides as markdown: http://www.zoidbergwill.com/presentations/2016/kubernetes-1.2-and-spread/index.md
I am glad to share the presentation of the Kubernetes Pune meetup organized on 29 July 2017. One of the good response from the Pune folks to the community.
Monitoring, Logging and Tracing on KubernetesMartin Etmajer
In this presentation, I'll describe a variety of tools, like the Kubernetes Dashboard, Heapster, Grafana, Fluentd, Elasticsearch, Kibana, Jolokia and OpenTracing to bring Monitoring, Logging and Tracing to the Kubernetes container platform.
- Archeology: before and without Kubernetes
- Deployment: kube-up, DCOS, GKE
- Core Architecture: the apiserver, the kubelet and the scheduler
- Compute Model: the pod, the service and the controller
Top 3 reasons why you should run your Enterprise workloads on GKESreenivas Makam
This deck covers top 3 reasons why Google Kubernetes engine is best suited to run containerized workloads. The reasons covered are Security, Observability and Maturity.
Kubernetes has two simple but powerful network concepts: every Pod is connected to the same network, and Services let you talk to a Pod by name. Bryan will take you through how these concepts are implemented - Pod Networks via the Container Network Interface (CNI), Service Discovery via kube-dns and Service virtual IPs, then on to how Services are exposed to the rest of the world.
Cloud native applications are popular these days – applications that run in the cloud reliably und scale almost arbitrarily. They follow three key principles: they are built and composed as micro services. They are packaged and distributed in containers. The containers are executed dynamically in the cloud. Kubernetes is an open-source cluster manager for the automated deployment, scaling and management of cloud native applications. In this hands-on session we will introduce the core concepts of Kubernetes and then show how to build, package and operate a cloud native showcase application on top of Kubernetes step-by-step. Throughout this session we will be using an off-the-shelf MIDI controller to demonstrate and visualize the concepts and to remote control Kubernetes. This session has been presented at the ContainerCon Europe 2016 in Berlin. #qaware #cloudnativenerd #LinuxCon #ContainerCon
Orchestration tool roundup kubernetes vs. docker vs. heat vs. terra form vs...Nati Shalom
Video recording: https://www.youtube.com/watch?v=tGlIgUeoGz8
It’s no news that containers represent a portable unit of deployment, and OpenStack has proven an ideal environment for running container workloads. However, where it usually becomes more complex is that many times an application is often built out of multiple containers. What’s more, setting up a cluster of container images can be fairly cumbersome because you need to make one container aware of another and expose intimate details that are required for them to communicate which is not trivial especially if they’re not on the same host.
These scenarios have instigated the demand for some kind of orchestrator. The list of container orchestrators is growing fairly fast. This session will compare the different orchestation projects out there - from Heat to Kubernetes to TOSCA - and help you choose the right tool for the job.
Session link from teh summit: https://openstacksummitmay2015vancouver.sched.org/event/abd484e0dedcb9774edda1548ad47518#.VV5eh5NViko
Arkena's video-on-demand platform is used as backend by major european channels (TF1 / beIN SPORTS / Elisa) to propose a non-linear experience to their customers.
Previously hosted on Heroku, the number of our users is increasing constantly. In order to optimize resources we decided to move on a bare metal infrastructure powered by Kubernetes.
We'll share thoughts, feedbacks and technical details about this successful transition.
Sched Link:
KubeCon CloudNativeCon 2016 Seattle - a reportKrishna-Kumar
KubeCon / CloudNativeCon Seattle summary report - Just to recapture some of the items from the event - Few of the items are copied from other blogs from reference - pictures are just for FUN!
9 ways to consume kubernetes on open stack in 15 mins (k8s meetup)Stacy Véronneau
Like that title states, this is a quick slide deck to help people consume OpenStack resources from Kubernetes. It covers elements running on a laptop to consuming a full production cloud.
Vadim Solovey is a CTO of DoiT International has helped to implement Google BigQuery as a cloud data warehouse for many medium and large sized data and analytics initiatives. BigQuery’s serverless architecture had redefined what it means to be fully managed for hundreds of Israeli's startups.
Recently, Google announced an update to BigQuery that dramatically advances cloud data analytics for large-scale businesses such as BigQuery now support Standard SQL, implementing the SQL 2011 standard as well as new ODBC drivers making it possible to use BigQuery with a number of tools ranging from Microsoft Excel to traditional business intelligence systems such as Microstrategy and Qlik.
Agenda:
• Partitioned tables
• The ability to update, delete rows and columns using SQL
• Integration with IAM for fine-grained security policies
• Monitoring w/ StackDriver to track performance and usage
• Query sharing via links, to foster knowledge within orgs
• Cost optimisation strategies
AWS Athena vs. Google BigQuery for interactive SQL QueriesDoiT International
During the re:Invent 2016, AWS has released the Amazon Athena - an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Athena is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run.
We took a look on AWS Athena and compared it to the Google BigQuery - another player of serverless interactive data analysis.
Would you like to know which one is the right tool for you? Join us for this meetup to learn AWS Athena and for the test drive of querying exactly the same dataset using AWS Athena and Google BigQuery to see where each one shines (or totally blows it).
This 1-day course provides hands-on skills in ingesting, analyzing, transforming and visualizing data using AWS Athena and getting the best performance when using it at scale.
Audience:
This class is intended for data engineers, analysts and data scientists responsible for: analyzing and visualizing big data, implementing cloud-based big data solutions, deploying or migrating big data applications to the public cloud, implementing and maintaining large-scale data storage environments, and transforming/processing big data.
Cloud Spanner is the first and only relational database service that is both strongly consistent and horizontally scalable. With Cloud Spanner you enjoy all the traditional benefits of a relational database: ACID transactions, relational schemas (and schema changes without downtime), SQL queries, high performance, and high availability. But unlike any other relational database service, Cloud Spanner scales horizontally, to hundreds or thousands of servers, so it can handle the highest of transactional workloads.
We believe that security *IS* a shared responsibility, - when we give developers the power to create infrastructure, security became their responsibility, too.
During this meetup, we'd like to share our experience with implementing security best practices, to be implemented directly by development teams to build more robust and secure cloud environments. Make cloud security your team's sport!
GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...Oleg Shalygin
Kubernetes provides an automated platform to deployment, scaling and operations of applications across a cluster of hosts. Complementing Kubernetes with a series of build scripts in conjunction with Travis-CI, GitHub, Artifactory, and Google Cloud Platform, we can take code from a merged pull request to a deployed environment with no manual intervention on a highly scaleable and robust infrastructure.
Lean Drupal Repositories with Composer and DrushPantheon
Composer is the industry-standard PHP dependency manager that is now in use in Drupal 8 core. This session will show the current best practices for using Composer, drupal-composer, drupal-scaffold, Drush, Drupal Console and Drush site-local aliases to streamline your Drupal 7 and Drupal 8 site repositories for optimal use on teams.
You may know Google for search, YouTube, Android, Chrome, and Gmail, but that's only as an end-user of OUR apps. Did you know you can also integrate Google technologies into YOUR apps? We have many APIs and open source libraries that help you do that! If you have tried and found it challenging, didn't find not enough examples, run into roadblocks, got confused, or just curious about what Google APIs can offer, join us to resolve any blockers. Code samples will be in Python and/or Node.js/JavaScript. This session focuses on showing you how to access Google Cloud APIs from one of Google Cloud's compute platforms, whether serverless or otherwise.
Learn from the dozens of large-scale deployments how to get the most out of your Kubernetes environment:
- Container images optimization
- Organizing namespaces
- Readiness and Liveness probes
- Resource requests and limits
- Failing with grace
- Mapping external services
- Upgrading clusters with zero downtime
Mete Atamel "Resilient microservices with kubernetes"IT Event
Talk description: Creating a single microservice is a well understood problem. Creating a cluster of load-balanced microservices that are resilient and self-healing is not so easy. Managing that cluster with rollouts and rollbacks, scaling individual services on demand, securely sharing secrets and configuration among services is even harder.
Presented at AI NEXTCon Seattle 1/17-20, 2018
http://aisea18.xnextcon.com
join our free online AI group with 50,000+ tech engineers to learn and practice AI technology, including: latest AI news, tech articles/blogs, tech talks, tutorial videos, and hands-on workshop/codelabs, on machine learning, deep learning, data science, etc..
Docker Orchestration: Welcome to the Jungle! Devoxx & Docker Meetup Tour Nov ...Patrick Chanezon
In two years, Docker hit the sweet spot for devs and ops, with tools for building, shipping, and running distributed apps architected as a set of collaborating microservices packaged as Linux containers. One area of the Docker ecosystem that saw a lot of innovation in the past year is container orchestration systems. This session compares and contrasts various Docker orchestration systems (Swarm, Machine, and Compose), the batteries included with Docker itself, Mesos, Kubernetes, CoreOS/Fleet, Deis, Cloud Foundry, and Tutum. It includes a demo of how to deploy a Java 8 app with MongoDB on several of these systems. The goal of the session is to give you a framework to help evaluate how these systems can meet your particular requirements.
Demo code at https://github.com/chanezon/docker-tips/blob/master/orchestration-networking/README.md
Speakers: Vic Iglesias, Benjamin Good, Karl Isenberg
Venue: Google Cloud Next '19
Video: https://www.youtube.com/watch?v=rt287-94Pq4
Continuous Integration and Delivery allows companies to quickly iterate on and deploy their ideas to customers. In doing so, they should strive to have environments that closely match production. Using Kubernetes as the target platform across cloud providers and on-premises environments can help to mitigate some difficulties when ensuring environment parity but many other concerns can arise.
In this talk we will dive into the tools and methodologies available to ensure your code and deployment artifacts can smoothly transition among the various people, environments, and platforms that make up your CI/CD process.
Designing flexible apps deployable to App Engine, Cloud Functions, or Cloud Runwesley chun
Many people ask, "Which one is better for me: App Engine, Cloud Functions, or Cloud Run?" To help you learn more about them, understand their differences, appropriate use cases, etc., why not deploy the same app to all 3? With this "test drive," you only need to make minor config changes between platforms. You'll also learn one of Google Cloud's AI/ML "building block" APIs as a bonus as the sample app is a simple "mini" Google Translate "MVP". This is a 45- 60-minute talk that reviews the Google Cloud serverless compute platforms then walks through the same app and its deployments. The code is maintained at https://github.com/googlecodelabs/cloud-nebulous-serverless-python
The recent launch of the Docker Init command has significantly simplified the process of generating Dockerfiles and Docker Compose templates for containerized applications. This presentation aims to explore the evolution of Docker deployment resources generation process, comparing its approach prior to the Docker Init command release and discussing the way forward. Before the introduction of the Docker Init command, I've been delivering some projects like the "alfresco-docker-installer"[1], which provides custom scripts and configurations to streamline the process of deploying Alfresco in Docker containers. These kinds of projects use tools like Yeoman or raw Python. There are some differences between a Docker Template for a technology (Go, Python, Node or Rust) and a Docker Template for a product (like Alfresco) that may be covered when generating automatic deployment resources. This presentation will delve into the methodologies employed before the Docker Init command:
Custom Dockerfile Extension
Compose Template for a complete product deployment, including a set of services like the database, content repository, search engine, or web application
Configuration Management, including techniques such as environment variable injection, externalized configuration files, and configuration overrides
Following the release of the Docker Init command, this presentation will provide insights into the possibilities and advantages it brings to complex products Docker deployment process. A PoC of a Docker Plugin, including this product-oriented approach for docker init, will be demoed live. >> Note that the Open Source Alfresco product is used only to explain the concepts of building a Docker Compose generator with a real example.
30-45-min tech talk given at user groups or technical conferences to introducing developers to integrating with Google APIs from Python .
ABSTRACT
Want to integrate Google technologies into the web+mobile apps that you build? Google has various open source libraries & developer tools that help you do exactly that. Users who have run into roadblocks like authentication or found our APIs confusing/challenging, are welcome to come and make these non-issues moving forward. Learn how to leverage the power of Google technologies in the next apps you build!!
Similar to Kubernetes - State of the Union (Q1-2016) (20)
Training Generative Adversarial Networks is a gentle process. Recent advances in GANs research resulted in incredible results of generated images. However, training GANs still remains tricky and slow.
With TensorFlow and NVidia’s new Volta architecture it is possible to reduce training time by up to 50% using simplified code.
1. Double Orchestration of Redis Enterprise cluster on Kubernetes
**************
In this session we'll display how we deploy a highly available database on Kubernetes. The considerations we took when deploying a stateful application, and the challenges of answering different clients' demands for different k8s environments.
2. Operators to the rescue: stateful applications made easy with operators
**************
Kubernetes 1.7 introduced an import feature called custom controllers. This allows you to customise your Kubernetes installation and add your own resources to be managed in the native Kubernetes manner.
The session will display the operator concept and cover our journey with developing the Redis Labs operator - why we chose it and how we use it.
An Open-Source Platform to Connect, Manage, and Secure MicroservicesDoiT International
Services are at the core of modern software architecture. Deploying a series of modular, small (micro-)services rather than big monoliths gives developers the flexibility to work in different languages, technologies and release cadence across the system; resulting in higher productivity and velocity, especially for larger teams.
With the adoption of microservices, however, new problems emerge due to the sheer number of services that exist in a larger system. Problems that had to be solved once for a monolith, like security, load balancing, monitoring, and rate limiting need to be handled for each service.
Istio, announced at GlueCon 2017, addresses these problems in a fundamental way through a service mesh framework. With Istio, developers can implement the core logic for the microservices, and let the framework take care of the rest – traffic management, discovery, service identity and security, and policy enforcement. Better yet, this can be also done for existing microservices without rewriting or recompiling any of their parts. Istio uses Envoy as its runtime proxy component and provides an extensible intermediation layer which allows global cross-cutting policy enforcement and telemetry collection.
So you are deployed to production (or soon to be) with Elasticsearch running and powering important application features. Or maybe used for centralized logging for effective debugging.
Was your Elastic cluster deployed correctly? Is it stable? Can it hold the throughput you expect it to?
How did you do capacity planning? How to tell if the cluster is healthy and what to monitor? How to apply effective multi-tenancy? and what would be an ideal cluster topology and data ingestion architecture?
We already trust artificial intelligence to drive our car, but we still configure thresholds and thrift through logs manually. In this talk, Ronny Lehmann, Loom CTO will discuss how he spent months analyzing modern-ops work, until he finally was able to extract the common-basis practices; and how we used this understanding to build a machine that complements ops teams, automating much of the work which is more suitable for machines - leaving for "humans" just the parts which require humans. What we built saves you time spent on parsers, on configuring and tuning rules and alerts, on conducting root-cause analysis and triage - and finally - on figuring out what to do.
This meeting we'll host a discussion on Google Cloud Platform and Amazon Web Services to bring light to similarities and differences between platforms. If you have questions about how our platforms compare this is the meeting to attend!
Cloud Dataflow - A Unified Model for Batch and Streaming Data ProcessingDoiT International
Dataflow is a unified programming model and a managed service for developing and executing a wide range of data processing patterns including ETL, batch computation, and continuous computation. Cloud Dataflow frees you from operational tasks like resource management and performance optimization.
Dataflow - A Unified Model for Batch and Streaming Data ProcessingDoiT International
Batch and Streaming Data Processing and Vizualize 300Tb in 5 Seconds meetup on April 18th, 2016 (http://www.meetup.com/Big-things-are-happening-here/events/229532500)
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
1. Section Slide Template Option 2
Put your subtitle here. Feel free to pick from the handful of pretty Google colors available to you.
Make the subtitle something clever. People will think it’s neat.
Kubernetes - State of the Union (Q1-2016)
Vadim Solovey - CTO, DoIT International
Google Cloud Developer Expert | Authorized Trainer
vadim@doit-intl.com
2. Google confidential │ Do not distribute
Agenda
Introduction to Containers & Kubernetes
What’s new and coming soon
Q&A
1
2
3
3. • Usage of micro-services
• Declarative management
• Highly flexible and scalable
• Automation-friendly
• Good for complex architectures
• Development for “Google scale”
KubernetesPackaging containersApps in Containers
Containers
5. Copyright 2016 Google Inc
How Can We Scale Out Container Workloads?
Node Node
Cluster
Node
???
• Placement?
• Scale?
• Node failure?
• Container failure?
• Application upgrades?
How to handle...
Containers
6. Managed Base OS
Node Container
Manager
Scheduled Containers
Cluster Scheduler
Schedule containers across
machines
Replication and resizing
Service naming and discovery
Cluster schedulingKubernetes
Containers
7. A datacenter is not a group
of computers,
a datacenter is a
computer.
The promise
8. Copyright 2015 Google Inc
Replication controllers create
new pod "replicas" from a
template and ensures that a
configurable number of
those pods are running.
A Service offers low overhead
way to route requests to a
logical set of pod backends
in the cluster based on a
label selector.
Replication
Controllers
ServicesLabels
Labels are metadata that
are attached to objects,
such as pods.
They enable organization
and selection of subsets
of objects with a cluster.
Pods
Pods are ephemeral units
that are used to manage
one or more tightly
coupled containers.
They enable data sharing
and communication
among their constituent
components.
Moving parts
Kubernetes
9. Copyright 2015 Google Inc
Namespaces AnnotationsSecretsVolumes
More moving parts
Kubernetes
Persistent
Volumes
Selectors
Load
Balancers
10. Copyright 2015 Google Inc
Autoscalers
Ingress
Jobs
Daemon
Sets
New kids in the town
Kubernetes
Deployments
11. Section Slide Template Option 2
Put your subtitle here. Feel free to pick from the handful of pretty Google colors available to you.
Make the subtitle something clever. People will think it’s neat.
Daemon Sets
12. Daemon Sets
A Daemon Set ensures that all (or some) nodes run a copy of a pod.
Node 1 Node 2 Node 3
pod pod pod
Popular use-cases:
● running a cluster storage daemon, such as glusterd or ceph
● running a logs collection daemon on every node, such as fluentd or logstash
● running a node monitoring daemon on every node collectd, new relic, ganglia
Alternatives:
● init script of your religion, - init, upstartd, systemd
● bare pods
13. Section Slide Template Option 2
Put your subtitle here. Feel free to pick from the handful of pretty Google colors available to you.
Make the subtitle something clever. People will think it’s neat.
Deployments
14. Deployments
A Deployment provides declarative update for Pods and ReplicationControllers.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 3
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
A typical use case is:
● Create a deployment to bring up a replication controller and pods.
● Later, update that deployment to recreate the pods (for ex: to use a
new image).
$ kubectl create -f app.yaml
deployment "app" created..
$ kubectl get deployments
NAME UPDATEDREPLICAS AGE
app 3/3 1m
15. Section Slide Template Option 2
Put your subtitle here. Feel free to pick from the handful of pretty Google colors available to you.
Make the subtitle something clever. People will think it’s neat.
Horizontal Pod Autoscaling
16. Pod Autoscaling
Horizontal pod autoscaling allows the number of pods in a replication controller or deployment
to scale automatically based on observed CPU utilization
Pod 1
Details:
● Control loop (targetNumOfPods = ceil(sum(currentPodsCPUUtilization) / target)
● --horizontal-pod-autoscaler-sync-period
● Autoscaling during rolling update
Pod 2 Pod .. Pod N
RC / Deployment Autoscaler
17. Section Slide Template Option 2
Put your subtitle here. Feel free to pick from the handful of pretty Google colors available to you.
Make the subtitle something clever. People will think it’s neat.
Ingress
18. Copyright 2016 Google Inc
The Ingress
Services
Internet
Services
Internet
Ingress
is collection of rules that allow inbound
connections to reach the cluster services
19. Copyright 2016 Google Inc
The Ingress Resource
Services
Internet
Ingress
Few potential use-cases include:
● Externally reachable urls for services
● Traffic Load Balancing
● Terminate SSL
● Name based virtual hosting
● More more as it evolves..
Available Controllers:
● GCE L7 LB
● nginx
● Write your own
20. Copyright 2016 Google Inc
The Ingress Resource
Services
Internet
Ingress
Minimal Ingress Resource may look like this:
01. apiVersion: extensions/v1beta1
02. kind: Ingress
03. metadata:
04. name: test-ingress
05. spec:
06. rules:
07. - http:
08. paths:
09. - path: /testpath
10. backend:
11. serviceName: test
12. servicePort: 80
21. Copyright 2016 Google Inc
Creating Ingress Resource
Services
Internet
Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-ingress
spec:
backend:
serviceName: testsvc
servicePort: 80
$ kubectl get ing
NAME RULE BACKEND ADDRESS
test-ingress - testsvc:80 107.178.254.228
23. Copyright 2016 Google Inc
Simple Fan Out
Simple edge accepting ingress
traffic and proxying it to the right
endpoints
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /foo
backend:
serviceName: s1
servicePort: 80
- path: /bar
backend:
serviceName: s2
servicePort: 80
$ kubectl get ing
NAME RULE BACKEND ADDRESS
test -
foo.bar.com
/foo s1:80
/bar s2:80
foo.bar.com
178.91.123.132
/foo
s1:80
/bar
s2:80
24. Copyright 2016 Google Inc
Name based virtual hosting
Name-based virtual hosts use
multiple host names for the same
IP address
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
spec:
rules:
- host: foo.bar.com
http:
paths:
- backend:
serviceName: s1
servicePort: 80
- host: bar.foo.com
http:
paths:
- backend:
serviceName: s2
servicePort: 80
foo.bar.com
178.91.123.132
foo.bar.com
s1:80
bar.foo.com
s2:80
bar.foo.com
25. Copyright 2016 Google Inc
Alternatives
You can expose a Service in multiple ways that don't directly involve the Ingress resource:
● Use Service.Type=LoadBalancer
● Use Service.Type=NodePort (30K-32K ports)
● Use a Port Proxy
● Deploy the Service Loadbalancer. This allows you to share a single IP among multiple
services and achieve more advanced load balancing through service annotations.
26. Copyright 2016 Google Inc
Gotchas
● The Ingress resource is not available in Kubernetes < 1.1
● You need an Ingress Controller to satisfy an Ingress.
○ Simply creating the resource will have no effect.
● On GCE/GKE there is a L7 LB controller, on other platforms you either need to write
your own or deploy an existing controller as a pod.
● The resource currently does not support HTTPS, but will do so before it leaves beta
(March/April 2016)
27. Copyright 2016 Google Inc
Future Work
● Various modes of HTTPS/TLS support (edge termination, sni etc)
● Requesting an IP or Hostname via claims
● Combining L4 and L7 Ingress
● More Ingress controllers (haproxy, vulcan, zuul, etc)
28. Section Slide Template Option 2
Put your subtitle here. Feel free to pick from the handful of pretty Google colors available to you.
Make the subtitle something clever. People will think it’s neat.
Jobs
29. Section Slide Template Option 2
Put your subtitle here. Feel free to pick from the handful of pretty Google colors available to you.
Make the subtitle something clever. People will think it’s neat.
Going forward
30. Jobs
A job creates one or more pods and ensures that a specified number of them successfully
terminate.
Details:
● .restartPolicy, .parallelism & .completions
● replication controller vs jobs
● cron
apiVersion: extensions/v1beta1
kind: Job
metadata:
name: pi
spec:
selector:
matchLabels:
app: pi
template:
metadata:
name: pi
labels:
app: pi
spec:
containers:
- name: pi
image: perl
command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
restartPolicy: Never
$ kubectl create -f ./job.yaml
jobs/pi
$ kubectl logs pi-aiw0a
3.141592653589793238462643383279502884197169399
37510582097494459230781640628620899862803482534
21170679821480865132823066470938446095505822317
25359408128481117450284102701938521105559644622
94895493038196442881097566593344612847564823371
31. Copyright 2016 Google Inc
Going forward in 2016
● version 1.2 would also enable multi-zone
● version 1.4 will allow multi-clustering (Ubernetes)
32. Section Slide Template Option 2
Put your subtitle here. Feel free to pick from the handful of pretty Google colors available to you.
Make the subtitle something clever. People will think it’s neat.
Q & A
Vadim Solovey - CTO, DoIT International
Google Cloud Developer Expert | Authorized Trainer
vadim@doit-intl.com
33. Section Slide Template Option 2
Put your subtitle here. Feel free to pick from the handful of pretty Google colors available to you.
Make the subtitle something clever. People will think it’s neat.
meetup.com/googlecloud
Editor's Notes
Questions to audience:
How many people are using containers in some environment (dev, ci, production)?
How many people are using some container orchestration engine (ecs, k8s, swarm, mesos)?
How many people know a little bit about Kubernetes?
Microservices take the Unix philosophy to your application design. Write programs that do one thing, and do it well. Write programs that work together.
Apps in containers provide ideal infrastructure for micro-services, it’s flexible, very automation friendly and built for complex architectures and scale.
So far, sounds familiar, right?
We can then create a node that hosts many containers.
This is much better.
My app & libraries get isolation through their containers,
and the container spins up on the order of a process (not booting a VM per app)
My app is kept portable, as containers that run on any modern linux stack.
We reduce the number of redundant OS kernels.
But
Google could not run, if we programmed and operated at individual Node level.
We have to write our apps with a higher level construct, we have to program at the cluster level
As we saw when clusters came into Google
the number of services proliferates, as ops & dev have better tools that cleave at the right abstraction layer
We have to be cluster first.
GCE does not natively support any way to manage deployment, scaling and reliability of container based workloads.
How to handle replication?
What about node failure?
What about container failure?
How do we manage application upgrades?
Managed Base OS
Node Container Manager
Common services: log rotation, watchdog restarting
Containers:
System container for shared daemons. Statically defined.
Dynamically scheduled containers
Cluster Scheduler
Schedules work (tasks) onto nodes
Work specified based on intents
Surfaces data about running tasks, restarts, etc.
Essentially, the promise of Kubernetes is to make a datacenter not a group of computers but for a datacenter to become a computer in itself.
Pods are ephemeral units that are used to manage one or more tightly coupled containers.
They enable data sharing and communication among their constituent components.
Labels are metadata that are attached to objects, such as pods.
They enable organization and selection of subsets of objects with a cluster.
Replication controllers create new pod "replicas" from a template and ensures that a configurable number of those pods are running.
A Service offers low overhead way to route requests to a logical set of pod backends in the cluster based on a label selector.
Services also provide a mechanism for surfacing legacy components such as databases with a cluster
Pods are ephemeral units that are used to manage one or more tightly coupled containers.
They enable data sharing and communication among their constituent components.
Labels are metadata that are attached to objects, such as pods.
They enable organization and selection of subsets of objects with a cluster.
Replication controllers create new pod "replicas" from a template and ensures that a configurable number of those pods are running.
A Service offers low overhead way to route requests to a logical set of pod backends in the cluster based on a label selector.
Services also provide a mechanism for surfacing legacy components such as databases with a cluster
But there are also new functionality coming up in 2016. Most of it is already available as beta feature in 1.1 release and all of them will be GA with 1.2 release scheduled for March/April 2016.
In a simple case, one Daemon Set, covering all nodes, would be used for each type of daemon. A more complex setup might use multiple DaemonSets would be used for a single type of daemon, but with different flags and/or different memory and cpu requests for different hardware types.
It is certainly possible to run daemon processes by directly starting them on a node (e.g using init, upstartd, or systemd). This is perfectly fine. However, there are several advantages to running such processes via a DaemonSet:
Ability to monitor and manage logs for daemons in the same way as applications.
Same config language and tools (e.g. pod templates, kubectl) for daemons and applications.
Future versions of Kubernetes will likely support integration between DaemonSet-created pods and node upgrade workflows.
Running daemons in containers with resource limits increases isolation between daemons from app containers. However, this can also be accomplished by running the daemons in a container but not in a pod (e.g. start directly via Docker).
Bare Pods
It is possible to create pods directly which specify a particular node to run on. However, a Daemon Set replaces pods that are deleted or terminated for any reason, such as in the case of node failure or disruptive node maintenance, such as a kernel upgrade. For this reason, you should use a Daemon Set rather than creating individual pods.
We already have cluster resize with 1.1 release on GCE and now we are adding pod autoscaling
Possible use-case for default backend: 404 page if none of the Hosts in your Ingress match the Host in the request header, and/or none of the Paths match the url of the request
A job creates one or more pods and ensures that a specified number of them successfully terminate. As pods successfully complete, the job tracks the successful completions. When a specified number of successful completions is reached, the job itself is complete. Deleting a Job will cleanup the pods it created.
A simple case is to create 1 Job object in order to reliably run one Pod to completion. A Job can also be used to run multiple pods in parallel.
Multiple Completions
By default, a Job is complete when one Pod runs to successful completion. You can also specify that this needs to happen multiple times by specifying .spec.completions with a value greater than 1. When multiple completions are requested, each Pod created by the Job controller has an identical spec. In particular, all pods will have the same command line and the same image, the same volumes, and mostly the same environment variables. It is up to the user to arrange for the pods to do work on different things. For example, the pods might all access a shared work queue service to acquire work units.
To create multiple pods which are similar, but have slightly different arguments, environment variables or images, use multiple Jobs.
Parallelism
You can suggest how many pods should run concurrently by setting .spec.parallelism to the number of pods you would like to have running concurrently. This number is a suggestion. The number running concurrently may be lower or higher for a variety of reasons. For example, it may be lower if the number of remaining completions is less, or as the controller is ramping up, or if it is throttling the job due to excessive failures. It may be higher for example if a pod is gracefully shutdown, and the replacement starts early.
If you do not specify .spec.parallelism, then it defaults to .spec.completions.
Everyone is invited for Google Cloud meetup to follow up on next events and workshops