Ambassador is an open source Kubernetes-native API Gateway built on the Envoy proxy. We talked about why and how we built Ambassador during the Envoy salon at KubeCon.
DevOps Days Boston 2017: Real-world Kubernetes for DevOpsAmbassador Labs
DevOps Days Boston 2017
Microservices is an increasingly popular approach to building cloud-native applications. Dozens of new technologies that streamline adopting microservices development such as Docker, Kubernetes, and Envoy have been released over the past few years. But how do you actually use these technologies together to develop, deploy, and run microservices?
In this presentation, we’ll cover the nuances of deploying containerized applications on Kubernetes, including creating a Kubernetes manifest, debugging and logging, and how to build an automated continuous deployment pipeline. Then, we’ll do a brief tour of some of the advanced concepts related to microservices, including service mesh, canary deployments, resilience, and security.
Microservices, Kubernetes and Istio - A Great Fit!Animesh Singh
Microservices and containers are now influencing application design and deployment patterns. Sixty percent of all new applications will use cloud-enabled continuous delivery microservice architectures and containers. Service discovery, registration, and routing are fundamental tenets of microservices. Kubernetes provides a platform for running microservices. Kubernetes can be used to automate the deployment of Microservices and leverage features such as Kube-DNS, Config Maps, and Ingress service for managing those microservices. This configuration works fine for deployments up to a certain size. However, with complex deployments consisting of a large fleet of microservices, additional features are required to augment Kubernetes.
2017 Microservices Practitioner Virtual Summit: The Mechanics of Deploying En...Ambassador Labs
Abstract: The idea of the "service mesh" is becoming very popular in microservice design circles. However, the mechanics of deploying one into an existing infrastructure are far from simple. In this talk we will cover the logistical details of how Envoy was developed and deployed incrementally at Lyft, focusing primarily on the evolution of service mesh configuration management. We will also discuss why high level systems such as Istio are likely to be the main mechanism by which most customers ultimately get access to service mesh technology.
This talk was presented as part of the Microservices Practitioner Virtual Summit, https://www.microservices.com/summit/
DCEU 18: App-in-a-Box with Docker Application PackagesDocker, Inc.
Michael Irwin - Application Architect, Virginia Tech
Docker Application Packages is an experimental tool that makes it easy to share multi-service applications. Create a Compose file, package it in an image, and voilà! You now have an "app-in-a-box"! Not convinced yet? No worries! It took a while for me to be convinced too! In this session, we'll start off by diving into how Docker Application Packages actually works, which will help us understand the use cases. We'll see how dev environments can hook in to this app-in-a-box by replacing the service being worked on with a dev container. Then we'll move on to see how end-to-end functional tests are much easier to run. And, finally, we'll see how to maintain an "app-in-a-box" with the latest versions of each component in a CI/CD pipeline, allowing for a unique app-in-a-box for each feature branch under development. Lots of good material! And lots of live demos!
DevOps Days Boston 2017: Developer first workflows for KubernetesAmbassador Labs
Kubernetes is a powerful, operational platform for containerized applications. However, the developer workflow on Kubernetes – how you code, deploy, update, and monitor your services – is much less mature.
How should you lay out your Git repo? How do you create loosely coupled services? How do you support deploying your service at any time?
In this talk, we’ll talk about these questions and more. We’ll discuss the journey towards a rapid development workflow, discuss best practices, and, talk about the process we followed to get to a rapid development workflow.
DevOps Days Boston 2017
DevOps Days Boston 2017: Real-world Kubernetes for DevOpsAmbassador Labs
DevOps Days Boston 2017
Microservices is an increasingly popular approach to building cloud-native applications. Dozens of new technologies that streamline adopting microservices development such as Docker, Kubernetes, and Envoy have been released over the past few years. But how do you actually use these technologies together to develop, deploy, and run microservices?
In this presentation, we’ll cover the nuances of deploying containerized applications on Kubernetes, including creating a Kubernetes manifest, debugging and logging, and how to build an automated continuous deployment pipeline. Then, we’ll do a brief tour of some of the advanced concepts related to microservices, including service mesh, canary deployments, resilience, and security.
Microservices, Kubernetes and Istio - A Great Fit!Animesh Singh
Microservices and containers are now influencing application design and deployment patterns. Sixty percent of all new applications will use cloud-enabled continuous delivery microservice architectures and containers. Service discovery, registration, and routing are fundamental tenets of microservices. Kubernetes provides a platform for running microservices. Kubernetes can be used to automate the deployment of Microservices and leverage features such as Kube-DNS, Config Maps, and Ingress service for managing those microservices. This configuration works fine for deployments up to a certain size. However, with complex deployments consisting of a large fleet of microservices, additional features are required to augment Kubernetes.
2017 Microservices Practitioner Virtual Summit: The Mechanics of Deploying En...Ambassador Labs
Abstract: The idea of the "service mesh" is becoming very popular in microservice design circles. However, the mechanics of deploying one into an existing infrastructure are far from simple. In this talk we will cover the logistical details of how Envoy was developed and deployed incrementally at Lyft, focusing primarily on the evolution of service mesh configuration management. We will also discuss why high level systems such as Istio are likely to be the main mechanism by which most customers ultimately get access to service mesh technology.
This talk was presented as part of the Microservices Practitioner Virtual Summit, https://www.microservices.com/summit/
DCEU 18: App-in-a-Box with Docker Application PackagesDocker, Inc.
Michael Irwin - Application Architect, Virginia Tech
Docker Application Packages is an experimental tool that makes it easy to share multi-service applications. Create a Compose file, package it in an image, and voilà! You now have an "app-in-a-box"! Not convinced yet? No worries! It took a while for me to be convinced too! In this session, we'll start off by diving into how Docker Application Packages actually works, which will help us understand the use cases. We'll see how dev environments can hook in to this app-in-a-box by replacing the service being worked on with a dev container. Then we'll move on to see how end-to-end functional tests are much easier to run. And, finally, we'll see how to maintain an "app-in-a-box" with the latest versions of each component in a CI/CD pipeline, allowing for a unique app-in-a-box for each feature branch under development. Lots of good material! And lots of live demos!
DevOps Days Boston 2017: Developer first workflows for KubernetesAmbassador Labs
Kubernetes is a powerful, operational platform for containerized applications. However, the developer workflow on Kubernetes – how you code, deploy, update, and monitor your services – is much less mature.
How should you lay out your Git repo? How do you create loosely coupled services? How do you support deploying your service at any time?
In this talk, we’ll talk about these questions and more. We’ll discuss the journey towards a rapid development workflow, discuss best practices, and, talk about the process we followed to get to a rapid development workflow.
DevOps Days Boston 2017
Continuous Delivery the hard way with KubernetesLuke Marsden
This talk shows three increasingly advanced levels of continuous delivery with Kubernetes and GitLab (as an example), arguing for a continuous delivery architecture which has an explicit _Release Manager_ component. We then propose Flux, the open source project which powers the _Deploy_ feature of Weave Cloud, as an implementation of that idea. This approach is the precursor to GitOps.
DockerCon EU 2015: Placing a container on a train at 200mphDocker, Inc.
Presented by Casper S. Jensen, Software Engineer, Uber
At Uber, we've been introducing Docker to give service owners more control over their environments. However, everything at Uber is moving very fast so we have had to do it a way such that Docker fitted into the existing infrastructure and services could be migrated seamlessly to Docker without any service interruptions. In this talk we will talk about the challenges we faced while doing this, such as handling both non-Docker and Docker builds, image replication, integration with our deployment systems and other challenges when deploying Docker at scale.
Docker Online Meetup: Announcing Docker CE + EEDocker, Inc.
Docker Community Edition (CE) and Enterprise Edition (EE) are the best expressions of the Docker Platform to date. Whether you’re a developer, an ops team or a enterprise IT-team member, and no matter the infrastructure, Docker CE and EE gives you a way to install, upgrade and maintain Docker with the support and assurances required for your particular workload.
Both Docker CE and EE are available on a wide range of popular operating systems (including Windows Server 2016) and cloud infrastructure. Developers and devOps have the freedom to run Docker on their favorite infrastructure without risk of lock-in.
Michael Friis will give an overview of both editions and highlight the big enhancements to the lifecycle, maintainability and upgradability of Docker.
Docker Enterprise Edition: Building a Secure Supply Chain for the Enterprise ...Docker, Inc.
Learn from the development team as we dive into some of the latest and upcoming features in Docker EE, our enterprise container management solution. We will focus on the architecture and configuration of the features and how they can be used with both modern apps and containerized legacy apps. Stay for some tips on monitoring and troubleshooting to help you prevent your production environment from going sideways.
Taking Docker from Local to Production at Intuit JanJaap Lahpor, Intuit and H...Docker, Inc.
In this talk, we will share how a small team at Intuit moved Docker from local to production serving real and critical workloads. We will share how we addressed the organization challenges of running Docker at large enterprises by building a business case for a pilot project to prove the value of containers and its real world application. Next, we will share how we solved the technical challenges that present themselves when taking Docker from local to production in a corporate data center. We will share the blueprint for the business case and the associated pilot which laser focused on running stateless back-end services throughout the entire SDLC. Finally, we will highlight our crawl-walk-run approach that allowed us to make inexpensive mistakes before investing in the right areas as our Docker knowledge increased. We will share the major technical issues we encountered, how we overcame them and the lessons we learned.
Weaveworks discusses Microservices and best practices
Visit Weave Cloud: https://www.weave.works/product/cloud/
For more free talks, join our Weave Online User Group: https://www.meetup.com/Weave-User-Group/
DCEU 18: 5 Patterns for Success in Application TransformationDocker, Inc.
Elton Stoneman - Developer Advocate, Docker
Legacy applications weren't designed to run in a modern distributed platform like Docker. They have their own ideas about logging, configuration and health which don't translate to the world of containers and make transformation projects hard.
This session shows you how to bring your old applications into the modern world, and integrate them with Docker - without changing code. We'll cover patterns for all the core application concerns:
* logging
* configuration
* monitoring
* health
* dependency management
The sample applications will be in .NET and Java, and will show you how to turn your existing applications into good Docker citizens.
Empower Your Docker Containers with Watson - DockerCon 2017 AustinPhil Estes
A community theater talk given at DockerCon in Austin, Texas on April 18th, 2017 by Lin Sun and Phil Estes from IBM Cloud. This talk first describes the growth of Watson cognitive services and APIs since its origin beating human participants in Jeopardy years ago, and then takes a simple containerized application and adds cognitive capability by using the Watson Conversation service.
How to Install and Use Kubernetes by Weaveworks Weaveworks
Kubernetes is exploding with over 10,000 people in the Kubernetes Slack channel and hundreds joining every day. Why is it so popular with software developers and DevOps folks? This talk covers:
• The benefits of using containers and microservices
• An overview of Kubernetes concepts including Pods, Replica Sets, Deployments, Services and Selectors, and how they all fit together
• How to install your own Kubernetes cluster onto any machine running Linux
• How to deploy the microservices sample app, the Sock Shop, to a Kubernetes cluster
For more information read our blog: https://www.weave.works/blog/k8s-future-simplified-kubernetes-installation/
Interested in future Weave Events? Please join our Meetup group: https://www.meetup.com/Weave-User-Group/
PHPIDOL#80: Kubernetes 101 for PHP Developer. Yusuf Hadiwinata - VP Operation...Yusuf Hadiwinata Sutandar
Sesi Terakhir sebelum libur PHPID-OL memasuki Bulan Puasa Ramadhan. Kita akan ketemu lagi 19 April 2021.
Topik penutup yang akan diisi oleh Om Yusuf Hadiwinata, Praktisi Teknologi terkemuka dan ternama di lingkungan Industri IT Indonesia...
Ciyaooo.... Maju Terus PHP Indonesia
Link Video: https://fb.me/e/hzWbd0FeW
Immutable Awesomeness by John Willis and Josh CormanDocker, Inc.
This presentation will show the combination of two ideas that can create 2 to 3 order of magnitude efficiencies in service delivery. We will discuss an example used in an insurance company that has experienced these efficiencies. Josh Corman will present the concept of using Open Source and Toyota Supply Chain principles as a weapon for eliminating operational costs of service delivery. By applying first order principles like fewer suppliers (e.g, less logging frameworks) and image manifests (i.e., bill of materials) he will show how an organization can cut down on bugs and issue resolution times. John Willis will then cover how these principles fit like peanut butter and chocolate when used in an immutable delivery model based on Docker. This presentation was the third highest rated session at the 2015 Devops Enterprise Summit.
KubeCon EU 2016: ITNW (If This Now What): Orchestrating an EnterpriseKubeAcademy
With growing demand for containers in the enterprise, build pipelines are a bottleneck to success. Traditional workflows can't release application candidates quickly enough to fulfill demand. With over 400 development teams across many different business units, Pearson had to move away from massive installs of traditional build pipeline tools and rethink the entire concept. In this talk we'll demonstrate how we have built in security compliance, performance testing, quality assurance, abstracted away complexity, reduced overhead, aim to recover 10% of developers time and turned build tools into cattle.
This represents the story to date of an in-flight engineering project to modernise the digital estate of a global enterprise organisation and how scale of the operation is leading us to challenge many established beliefs. Attendees will walk away with everything from workflows to code which they can use to get started in their own endeavors.
Sched Link:
We are on the cusp of a new era of application development software: instead of bolting on operations as an after-thought to the software development process, Kubernetes promises to bring development and operations together by design.
Docker Store: The New Destination for Enterprise Software - Lily Guo and Alfr...Docker, Inc.
Docker Store is the place to find trusted community and enterprise content. Independent software vendors, startups and developers alike now have a marketplace to create and distribute Enterprise-ready content through the Store. Join Chinmayee and Alfred from the Docker Store team to learn how ISVs, big and small, are using Docker Store. They will cover: How to publish and distribute high quality, reusable containers and plugins; and How Enterprise customers can simplify procurement and management of their software assets using Docker Store.
Troubleshooting tips from docker support engineersDocker, Inc.
Docker makes everything easier. But even with the easiest platforms, sometimes you run into problems. In this session, you'll learn first-hand from someone whose job is helping customers fix these problems. In this session, you'll learn how to:
Ask better questions & identify the real problems you want to solve
Use command line tools to inspect internal Swarm and Kubernetes components
Debug and troubleshoot interactions between Enterprise Edition components
Troubleshoot Docker Enterprise Edition Windows nodes
Apply these skills to troubleshooting Docker Enterprise Edition
Don't Assume Your API Gateway is Ready for MicroservicesAmbassador Labs
Is your API Gateway ready for microservices and Kubernetes?
In this 30-minute webinar, Flynn, Principal Software Engineer at Datawire, explains the key capabilities you need in an API Gateway for microservices running on Kubernetes. He’ll also introduce service meshes, a new layer of infrastructure for distributed services.
Running large Kubernetes clusters is challenging. At large scales, practitioners need to adapt and tune both their architectures and component configurations in specialized ways.
Our organisation has been running large scale Kubernetes clusters (up to 2000 nodes, and growing) for more than a year, and we have learned several lessons the hard way. This talk will dive into complex runtime and networking issues that occur when running Kubernetes in production at scale. We will provide examples of how to improve the architecture of clusters to increase scalability and performance, both on the control plane and the data plane. Further, tools from the greater ecosystem will be examined, as they are rarely tested within the context of very large clusters.
Finally, the talk will also discuss the mutually beneficial relationship we built with the larger Kubernetes community by providing feedback on the tools and contributing both fixes and improvements upstream.
Istio is a service mesh—a modernized service networking layer that provides a transparent and language-independent way to flexibly and easily automate application network functions. Istio is designed to run in a variety of environments: on-premise, cloud-hosted, in Kubernetes containers.
Continuous Delivery the hard way with KubernetesLuke Marsden
This talk shows three increasingly advanced levels of continuous delivery with Kubernetes and GitLab (as an example), arguing for a continuous delivery architecture which has an explicit _Release Manager_ component. We then propose Flux, the open source project which powers the _Deploy_ feature of Weave Cloud, as an implementation of that idea. This approach is the precursor to GitOps.
DockerCon EU 2015: Placing a container on a train at 200mphDocker, Inc.
Presented by Casper S. Jensen, Software Engineer, Uber
At Uber, we've been introducing Docker to give service owners more control over their environments. However, everything at Uber is moving very fast so we have had to do it a way such that Docker fitted into the existing infrastructure and services could be migrated seamlessly to Docker without any service interruptions. In this talk we will talk about the challenges we faced while doing this, such as handling both non-Docker and Docker builds, image replication, integration with our deployment systems and other challenges when deploying Docker at scale.
Docker Online Meetup: Announcing Docker CE + EEDocker, Inc.
Docker Community Edition (CE) and Enterprise Edition (EE) are the best expressions of the Docker Platform to date. Whether you’re a developer, an ops team or a enterprise IT-team member, and no matter the infrastructure, Docker CE and EE gives you a way to install, upgrade and maintain Docker with the support and assurances required for your particular workload.
Both Docker CE and EE are available on a wide range of popular operating systems (including Windows Server 2016) and cloud infrastructure. Developers and devOps have the freedom to run Docker on their favorite infrastructure without risk of lock-in.
Michael Friis will give an overview of both editions and highlight the big enhancements to the lifecycle, maintainability and upgradability of Docker.
Docker Enterprise Edition: Building a Secure Supply Chain for the Enterprise ...Docker, Inc.
Learn from the development team as we dive into some of the latest and upcoming features in Docker EE, our enterprise container management solution. We will focus on the architecture and configuration of the features and how they can be used with both modern apps and containerized legacy apps. Stay for some tips on monitoring and troubleshooting to help you prevent your production environment from going sideways.
Taking Docker from Local to Production at Intuit JanJaap Lahpor, Intuit and H...Docker, Inc.
In this talk, we will share how a small team at Intuit moved Docker from local to production serving real and critical workloads. We will share how we addressed the organization challenges of running Docker at large enterprises by building a business case for a pilot project to prove the value of containers and its real world application. Next, we will share how we solved the technical challenges that present themselves when taking Docker from local to production in a corporate data center. We will share the blueprint for the business case and the associated pilot which laser focused on running stateless back-end services throughout the entire SDLC. Finally, we will highlight our crawl-walk-run approach that allowed us to make inexpensive mistakes before investing in the right areas as our Docker knowledge increased. We will share the major technical issues we encountered, how we overcame them and the lessons we learned.
Weaveworks discusses Microservices and best practices
Visit Weave Cloud: https://www.weave.works/product/cloud/
For more free talks, join our Weave Online User Group: https://www.meetup.com/Weave-User-Group/
DCEU 18: 5 Patterns for Success in Application TransformationDocker, Inc.
Elton Stoneman - Developer Advocate, Docker
Legacy applications weren't designed to run in a modern distributed platform like Docker. They have their own ideas about logging, configuration and health which don't translate to the world of containers and make transformation projects hard.
This session shows you how to bring your old applications into the modern world, and integrate them with Docker - without changing code. We'll cover patterns for all the core application concerns:
* logging
* configuration
* monitoring
* health
* dependency management
The sample applications will be in .NET and Java, and will show you how to turn your existing applications into good Docker citizens.
Empower Your Docker Containers with Watson - DockerCon 2017 AustinPhil Estes
A community theater talk given at DockerCon in Austin, Texas on April 18th, 2017 by Lin Sun and Phil Estes from IBM Cloud. This talk first describes the growth of Watson cognitive services and APIs since its origin beating human participants in Jeopardy years ago, and then takes a simple containerized application and adds cognitive capability by using the Watson Conversation service.
How to Install and Use Kubernetes by Weaveworks Weaveworks
Kubernetes is exploding with over 10,000 people in the Kubernetes Slack channel and hundreds joining every day. Why is it so popular with software developers and DevOps folks? This talk covers:
• The benefits of using containers and microservices
• An overview of Kubernetes concepts including Pods, Replica Sets, Deployments, Services and Selectors, and how they all fit together
• How to install your own Kubernetes cluster onto any machine running Linux
• How to deploy the microservices sample app, the Sock Shop, to a Kubernetes cluster
For more information read our blog: https://www.weave.works/blog/k8s-future-simplified-kubernetes-installation/
Interested in future Weave Events? Please join our Meetup group: https://www.meetup.com/Weave-User-Group/
PHPIDOL#80: Kubernetes 101 for PHP Developer. Yusuf Hadiwinata - VP Operation...Yusuf Hadiwinata Sutandar
Sesi Terakhir sebelum libur PHPID-OL memasuki Bulan Puasa Ramadhan. Kita akan ketemu lagi 19 April 2021.
Topik penutup yang akan diisi oleh Om Yusuf Hadiwinata, Praktisi Teknologi terkemuka dan ternama di lingkungan Industri IT Indonesia...
Ciyaooo.... Maju Terus PHP Indonesia
Link Video: https://fb.me/e/hzWbd0FeW
Immutable Awesomeness by John Willis and Josh CormanDocker, Inc.
This presentation will show the combination of two ideas that can create 2 to 3 order of magnitude efficiencies in service delivery. We will discuss an example used in an insurance company that has experienced these efficiencies. Josh Corman will present the concept of using Open Source and Toyota Supply Chain principles as a weapon for eliminating operational costs of service delivery. By applying first order principles like fewer suppliers (e.g, less logging frameworks) and image manifests (i.e., bill of materials) he will show how an organization can cut down on bugs and issue resolution times. John Willis will then cover how these principles fit like peanut butter and chocolate when used in an immutable delivery model based on Docker. This presentation was the third highest rated session at the 2015 Devops Enterprise Summit.
KubeCon EU 2016: ITNW (If This Now What): Orchestrating an EnterpriseKubeAcademy
With growing demand for containers in the enterprise, build pipelines are a bottleneck to success. Traditional workflows can't release application candidates quickly enough to fulfill demand. With over 400 development teams across many different business units, Pearson had to move away from massive installs of traditional build pipeline tools and rethink the entire concept. In this talk we'll demonstrate how we have built in security compliance, performance testing, quality assurance, abstracted away complexity, reduced overhead, aim to recover 10% of developers time and turned build tools into cattle.
This represents the story to date of an in-flight engineering project to modernise the digital estate of a global enterprise organisation and how scale of the operation is leading us to challenge many established beliefs. Attendees will walk away with everything from workflows to code which they can use to get started in their own endeavors.
Sched Link:
We are on the cusp of a new era of application development software: instead of bolting on operations as an after-thought to the software development process, Kubernetes promises to bring development and operations together by design.
Docker Store: The New Destination for Enterprise Software - Lily Guo and Alfr...Docker, Inc.
Docker Store is the place to find trusted community and enterprise content. Independent software vendors, startups and developers alike now have a marketplace to create and distribute Enterprise-ready content through the Store. Join Chinmayee and Alfred from the Docker Store team to learn how ISVs, big and small, are using Docker Store. They will cover: How to publish and distribute high quality, reusable containers and plugins; and How Enterprise customers can simplify procurement and management of their software assets using Docker Store.
Troubleshooting tips from docker support engineersDocker, Inc.
Docker makes everything easier. But even with the easiest platforms, sometimes you run into problems. In this session, you'll learn first-hand from someone whose job is helping customers fix these problems. In this session, you'll learn how to:
Ask better questions & identify the real problems you want to solve
Use command line tools to inspect internal Swarm and Kubernetes components
Debug and troubleshoot interactions between Enterprise Edition components
Troubleshoot Docker Enterprise Edition Windows nodes
Apply these skills to troubleshooting Docker Enterprise Edition
Don't Assume Your API Gateway is Ready for MicroservicesAmbassador Labs
Is your API Gateway ready for microservices and Kubernetes?
In this 30-minute webinar, Flynn, Principal Software Engineer at Datawire, explains the key capabilities you need in an API Gateway for microservices running on Kubernetes. He’ll also introduce service meshes, a new layer of infrastructure for distributed services.
Running large Kubernetes clusters is challenging. At large scales, practitioners need to adapt and tune both their architectures and component configurations in specialized ways.
Our organisation has been running large scale Kubernetes clusters (up to 2000 nodes, and growing) for more than a year, and we have learned several lessons the hard way. This talk will dive into complex runtime and networking issues that occur when running Kubernetes in production at scale. We will provide examples of how to improve the architecture of clusters to increase scalability and performance, both on the control plane and the data plane. Further, tools from the greater ecosystem will be examined, as they are rarely tested within the context of very large clusters.
Finally, the talk will also discuss the mutually beneficial relationship we built with the larger Kubernetes community by providing feedback on the tools and contributing both fixes and improvements upstream.
Istio is a service mesh—a modernized service networking layer that provides a transparent and language-independent way to flexibly and easily automate application network functions. Istio is designed to run in a variety of environments: on-premise, cloud-hosted, in Kubernetes containers.
AWS November Webinar Series - From Local Development to Production Using the ...Amazon Web Services
Running and managing large scale multi-container applications in production usually requires different tools than what is used for development.
In this webinar, we will show you how to use the Amazon EC2 Container Service CLI with Docker Compose to define and run multi-container applications in a local development environment. We will also show how you can eliminate the need to install, operate, and scale your own cluster management infrastructure by using Amazon ECS. We will then demonstrate how to schedule your multi-container application as defined by Compose across a production Amazon ECS cluster. We will also walk through some best practice patterns used by customers for running their microservices platforms or batch jobs.
Learning Objectives:
Understand the basics of the Amazon ECS CLI
Run multi-container applications defined by Docker Compose using the Amazon ECS CLI
Learn how to run and manage production applications using Amazon ECS
Who Should Attend:
Developers, system administrators, Docker users, container users
Donnie Prakoso, Technology Evangelist, ASEAN, AWS.
Container technology provides unparalleled improvements in efficiency and agility of packaging and deploying applications. Containers offer VM-like isolation and process-like efficiency and hence are becoming the de-facto method for deploying micro-services. However, using containers for running services at scale has required that operations team handle complex, dynamically changing infrastructure requirements, or run the risk or under- or over-provisioning infrastructure. Sounds like going back to the days before Cloud? In this session, learn how AWS services for containers take the pain out of managing infrastructure, and best practices for developing new services rapidly while running them at scale.
DevOps with Azure, Kubernetes, and Helm WebinarCodefresh
Watch the webinar here: https://codefresh.io/devops-azure-kubernetes-helm-lp/
Sign up for a FREE Codefresh account today: https://codefresh.io/codefresh-signup/
In this webinar, we will show you how you can use standard DevOps practices such as IaC, CI/CD, automated release and more in conjunction with Kubernetes (AKS) and Helm.
Developing Java based microservices ready for the world of containersClaus Ibsen
The so-called experts are saying microservices and containers will
change the way we build, maintain, operate, and integrate
applications. This talk is intended for Java developers who wants to hear and see how you can develop Java microservices that are ready to run in containers.
In this talk we will build a set of Java based Microservices that uses a mix of technologies with Apache Camel, Spring Boot and WildFly Swarm.
You will see how we can build small discrete microservices with these Java technologies and build and deploy on the Kubernets container platform.
We will discuss practices how to build distributed and fault tolerant microservices using technologies such as Kubernetes Services, Camel EIPs, and Netflixx Hysterix.
And the self healing and fault tolerant aspects of the Kubernetes platform is also discussed and demoed when we let the chaos monkeys loose killing containers.
This talk is a 50/50 mix between slides and demo.
The talk was presented at JDKIO on September 13th 2016.
Integrating Infrastructure as Code into a Continuous Delivery Pipeline | AWS ...Amazon Web Services
Ansible is a simple, but powerful automation tool with an agentless footprint that allows for the definition of architecture, intent, and policy as code that can be deployed across both on-prem and cloud infrastructure. This enables customers to extend their enterprise and applications into AWS in a way that maintains a consistent, secure posture as part of a continuous delivery pipeline. Customers can then natively integrate with AWS to seamlessly configure and deploy a range of AWS services such as Amazon Aurora, Amazon Redshift, Amazon EMR, Amazon Athena, Amazon CloudFront, Amazon Route 53, and Elastic Load Balancing from within Red Hat OpenShift across a secure, consistent hybrid cloud infrastructure. In this session, we will demonstrate how infrastructure can be instantiated with code as part of a continuous delivery pipeline and describe how that integrates with an OpenShift hybrid cloud deployment. Learn More: https://aws.amazon.com/government-education/
2016 - Easing Your Way Into Docker: Lessons From a Journey to Productiondevopsdaysaustin
Presentation by Steve Woodruff
The story of how SpareFoot broke up its monolithic application into micro services, deployed Docker into production, and established a "contract" between Dev and Ops.
In this WebHack talk I shared my experience about microservices, Docker, Kubernetes and Kong, an API gateway by Mashape. Since they are based on a real working system, this slides is majorly for how to build the whole thing up, not about detailed internal implementation. Although I included some details and reference in order to make it more comprehensive.
Kubernetes: від знайомства до використання у CI/CDStfalcon Meetups
Kubernetes: від знайомства до використання у CI/CD
Олександр Занічковський
Technical Lead у компанії SoftServe
14+ років досвіду розробки різноманітного програмного забезпечення, як для десктопа, так і для веб
Працював фріланс-програмістом та в команді
Цікавиться архітектурою ПЗ, автоматизацією процесів інтеграції та доставки нових версій продукту, хмарними технологіями
Віднедавна займається менторінгом майбутніх техлідів
У вільний від роботи час грає на гітарі і мріє про велику сцену
Олександр поділиться власним досвідом роботи з Kubernetes:
ознайомить з базовими поняттями та примітивами K8S
опише можливі сценарії використання Kubernetes для CI/CD на прикладі GitLab
покаже, як можна використовувати постійне сховище, збирати метрики контейнерів, використовувати Ingress для роутинга запитів за певними правилами
покаже, як можна самому встановити K8S для ознайомлення чи локальної роботи
La sécurité avec Kubernetes et les conteneurs Docker (June 19th, 2019)Alexandre Roman
Avec l’essor de Kubernetes dans le petit monde des moteurs d’orchestration de conteneurs, nous nous rendons compte à quel point nos logiciels, conteneurs et plateformes sont vulnérables. Toute l’attention portée sur Kubernetes et les images Docker amène à découvrir des failles de sécurité plus ou moins importantes, avec un rythme de plus en plus soutenu.
Est-ce que votre installation Kubernetes est à jour ? Quelle est votre stratégie de mise à jour ? Comment garantir la sécurité des images Docker, alors même que de nouvelles failles apparaissent chaque jour ?
Equifax, Tesla, Marriott : nombreux sont les acteurs qui, ces dernières années, ont dû faire face à des incidents de sécurité majeurs, avec à la clé des fuites de données sensibles en grande quantité. Un rapport a montré récemment que 10 des images Docker les plus populaires contiennent au moins 30 vulnérabilités.
En s’appuyant sur les technologies Pivotal, venez découvrir comment sécuriser les images Docker avec des outils modernes, et comment patcher un cluster K8s avec un correctif pour la faille runC, sans interruption.
As the demand for serverless computing continues to grow, the availability and choice of open source Function as a Service (FaaS) technologies are expanding rapidly. In this talk, we will present our experience in using some of the most popular open source FaaS platforms, compare them and discussing their features and characteristics.
The session will begin with an introduction to serverless computing and the key concepts behind FaaS. Throughout the talk, we will analyze various aspects such as ease of deployment, architecture, specific features, etc. We will examine real-world use cases and share insights on the strengths and limitations of each platform, allowing attendees to make informed decisions when choosing the right FaaS technology for their projects.
By the end of this session, attendees will have a better understanding of the open source FaaS landscape from our experience in production usage.
OpenShift in your own backyard
Installing OpenShift on your servers with Assisted Installer
OpenShift Container Platform is an open source enterprise-ready Kubernetes container platform.
There are multiple ways to use OpenShift, including in cloud provider environments such as Amazon Web Services, Google Cloud Platform or Microsoft Azure or on your own infrastructure like RHV, Openstack or Bare Metal.
Installing on bare metal servers or virtual machines can sometimes be hard. Having the ability to easily install OpenShift in your data center helps increase the productivity of the IT and Development teams.
The Assisted Installer is a SaaS solution that introduces a new way to deploy a new OpenShift cluster on bare metal basically by only booting the nodes that will be part of the cluster, with an ISO generated by a service hosted in cloud.redhat.com. The service will orchestrate the needed steps based on the user parameters and report about the installation progress. All that without the need of an additional bootstrap node.
In this session, we will explain about the Assisted Service flows and what are the network and hardware requirements, and the needed inputs from the user. We will discuss all the customizations available to the user, and about the possibility of running the Assisted Installer in a disconnected environment. Finally, we will do a demonstration showing how all come together to a running OpenShift cluster in your own backyard.
OpenShift in your own backyard - DevConf CZ 2021Freddy Rolland
OpenShift in your own backyard
Installing OpenShift on your servers with Assisted Installer
OpenShift Container Platform is an open source enterprise-ready Kubernetes container platform.
There are multiple ways to use OpenShift, including in cloud provider environments such as Amazon Web Services, Google Cloud Platform or Microsoft Azure or on your own infrastructure like RHV, Openstack or Bare Metal.
Installing on bare metal servers or virtual machines can sometimes be hard. Having the ability to easily install OpenShift in your data center helps increase the productivity of the IT and Development teams.
The Assisted Installer is a SaaS solution that introduces a new way to deploy a new OpenShift cluster on bare metal basically by only booting the nodes that will be part of the cluster, with an ISO generated by a service hosted in cloud.redhat.com. The service will orchestrate the needed steps based on the user parameters and report about the installation progress. All that without the need of an additional bootstrap node.
In this session, we will explain about the Assisted Service flows and what are the network and hardware requirements, and the needed inputs from the user. We will discuss all the customizations available to the user, and about the possibility of running the Assisted Installer in a disconnected environment. Finally, we will do a demonstration showing how all come together to a running OpenShift cluster in your own backyard.
Try the Assisted Installer here:
https://cloud.redhat.com/openshift/assisted-installer/clusters
Building Microservice Systems Without Cooking Your Laptop: Going “Remocal” wi...Ambassador Labs
When you adopt microservices, containers, and cloud native development, the technologies and architectures may change, but the need for fast feedback doesn’t. Kubernetes enables us to deploy and run applications at scale, but whether you’re coding or testing applications, you want to be able to get work done quickly without spinning up all of your microservices locally and driving your laptop fans into high speed!
Join me for a tour of coding, testing, and shipping microservices using remote-to-local “remocal” tools and techniques. You will:
Understand the challenges with scaling container-based application development – i.e. you can only run so many microservices locally before minikube melts your laptop.
Learn when to use various types of development practices and tooling based on your use case and requirements for production realism, speed, and practicality.
Explore how to utilize containerized dependencies and Docker for testing, including for both apps and services you own and those you don’t.
Learn how Telepresence can enable “remocal” development, expanding your local machine and Docker Desktop out into a remote Kubernetes cluster.
Ship Week 1: Intro to Continuous Delivery and GitOps
When building cloud native applications, software developers are no longer just responsible for coding new features. In the next module of Summer of Kubernetes, our expert guides (with the help of some special guests) will cover how to safely and effectively ship software without disrupting end users. To do this you will:
✅ Understand the basics of continuous delivery and GitOps
✅ Learn about how K8s enables declarative CD (via the use of reconciliation loops)
At GOTO Amsterdam in 2019 I presented how to create an effective cloud native developer workflow. Two years later and many new developer technologies have come and gone, but I still hear daily from cloud developers about the pain and friction associated with building, debugging, and deploying to the cloud. In this talk I'll share my latest learning on how to bring the fun and productivity back into delivering Kubernetes-based software.
In this talk, you will:
- Learn why the core tenets of continuous delivery -- speed and safety -- must be considered in all parts of the cloud native SDLC
- Explore how cloud native coding benefits from thinking separately about the inner development loop, continuous integration, continuous deployment, observability, and analysis
- Understand how cloud native best practices and tooling fit together. Learn about artifact syncing (e.g. Skaffold), dev environment bridging (e.g. Telepresence), GitOps (e.g. Argo), and observability-focused monitoring (e.g. Prometheus, Jaeger)
- Explore the importance of cultivating an effective cloud platform and associated team of experts
- Walk away with an overview of tools that can help you develop and debug effectively when using Kubernetes
Webinar: Accelerate Your Inner Dev Loop for Kubernetes Services Ambassador Labs
Many turn to static duplicate dev environments to shorten the dev loop and isolate code tests, but those bring about additional issues. The idea of safely sharing a dev environment and seeing your code changes in action immediately before sharing them probably seems impossible.
Service Preview, powered by Telepresence and the Ambassador Edge Stack, is here to help! This capability enables you to preview changes immediately and test locally with your tool of choice, while sharing a development cluster.
In this 45-minute webinar, Abhay Saxena will demonstrate using Service Preview to have a fast inner development loop while fixing a bug in a microservice, including stepping through the code in a debugger while other developers continue working unaffected.
[Confoo Montreal 2020] From Grief to Growth: The 7 Stages of Observability - ...Ambassador Labs
In this case-study talk, we will share Brent’s journey through the adoption of modern observability practices as he operated an architecture of distributed services. Facing difficulties using application logs as the primary tool to debug performance and reliability issues? Learn how to improve your company toolkit and engineering habits using existing monitoring tools with the addition of distributed tracing.
https://confoo.ca/en/yul2020/session/from-grief-to-growth-the-7-stages-of-observability
[Confoo Montreal 2020] Build Your Own Serverless with Knative - Alex GervaisAmbassador Labs
Google Cloud Run’s use of Knative introduced a portable Serverless solution built on top of Kubernetes. In this talk, we’ll recap the basic guidelines, use cases, and benefits of a Serverless architecture. Getting up and started, you will learn to take advantage of containers and the Ambassador API Gateway to serve event-driven application workloads and save costs using your existing Kubernetes resources.
https://confoo.ca/en/yul2020/session/build-your-own-serverless-with-knative
[QCon London 2020] The Future of Cloud Native API Gateways - Richard LiAmbassador Labs
The introduction of microservices, Kubernetes, and cloud technology has provided many benefits for developers. However, the age-old problem of getting user traffic routed correctly to the API of your backend applications can still be an issue, and may be complicated with the adoption of cloud native approaches: applications are now composed of multiple (micro)services that are built and released by independent teams; the underlying infrastructure is dynamically changing; services support multiple protocols, from HTTP/JSON to WebSockets and gRPC, and more; and many API endpoints require custom configuration of cross-cutting concerns, such as authn/z, rate limiting, and retry policies.
A cloud native API gateway is on the critical path of all requests, and also on the critical path for the workflow of any developer that is releasing functionality. Join this session to learn about the underlying technology and the required changes in engineering workflows. Key takeaways will include:
A brief overview of the evolution of API gateways over the past ten years, and how the original problems being solved have shifted in relation to cloud native technologies and workflow
Two important challenges when using an API gateway within Kubernetes: scaling the developer workflow; and supporting multiple architecture styles and protocols
Strategies for exposing Kubernetes services and APIs at the edge of your system
Insight into the (potential) future of cloud native API gateways
https://qconlondon.com/london2020/presentation/future-cloud-native-api-gateways
What's New in the Ambassador Edge Stack 1.0? Ambassador Labs
Before Kubernetes, the boundary between your users and your monolithic application was simple to manage. Now with Kubernetes, managing the edge has become dynamic and complex. More developers are involved, there are exponentially more edge operations, and each microservice has diverse requirements.
To fully capitalize on the benefits of Kubernetes, you need to provide a solution that supports the autonomy of application developers, the various requirements of your microservices, and your ability to scale.
You no longer need an API Gateway - you need a self-service, comprehensive edge stack.
In this 40 minute webinar on January 30th, we will discuss and demo the new functionality available with the Ambassador Edge Stack.
Edge Policy Console- graphical UI to visualize and manage all of your edge policies
Security Features- automatic TLS setup via ACME integration, OAuth/OpenID Connect integration, rate limiting, and fine-grained access control
Developer Onboarding- API catalog, Swagger/OpenAPI documentation support, and a fully customizable developer portal
Webinar: Effective Management of APIs and the Edge when Adopting Kubernetes Ambassador Labs
As you adopt Kubernetes, the requirements for your edge change. You now have teams working on multiple services all with different requirements. How can you make sure your edge is Kubernetes-ready?
[KubeCon NA 2018] Telepresence Deep Dive Session - Rafael Schloming & Luke Sh...Ambassador Labs
One of the challenges facing Telepresence is growing the contributor community. It’s a complex application that requires a good understanding of OS networking, VPNs, Kubernetes, and everything in between. We’ll kick off this meeting with a general architectural overview of Telepresence. We’ll talk about how we’ve managed the project to date, and our investments to make it easier. We want to then turn it over for an interactive discussion with participants to see what we can do to make it easier to contribute and grow the Telepresence community.
[KubeCon NA 2018] Effective Kubernetes Develop: Turbocharge Your Dev Loop - P...Ambassador Labs
Every software development cycle is rife with inefficiency. Seasoned devs know the pain of getting access to essential remote systems, waiting for tests to run (and then fail), or debugging with only log files. This talk teaches you how to best leverage Kubernetes, remote infrastructure and related tooling to create a dev cycle that maximizes velocity and minimizes developer friction and frustration.
Using tools such as Kubernetes, Docker and Telepresence, I will walk attendees through several advanced techniques that can be used to produce an effective developer experience and optimized dev loop. The goal of this is to eliminate many sources of frustrating inefficiency and reduce cycle time between releases. I will demonstrate how to incrementally adopt some of these techniques and how to approach introducing new and unfamiliar technology and techniques to skeptical dev teams.
The rise of Layer 7, microservices, and the proxy war with Envoy, NGINX, and ...Ambassador Labs
Modern cloud applications today are built as distributed microservices. These microservices talk to each other over L7 protocols: HTTP, gRPC, Redis, Kafka, and more. In this world, L7 proxies have assumed a crucial role in managing and observing L7 protocols. In this talk, I’ll discuss the evolution of service architectures, the role L7 proxies play in this world, and how there is now a battle raging between Envoy Proxy, HAProxy, and NGINX. I’ll wrap by talking about why we chose Envoy Proxy as the anchor of our Ambassador API Gateway and show how that has enabled a number of new capabilities.
The Simply Complex Task of Implementing Kubernetes Ingress - Velocity NYCAmbassador Labs
Getting traffic into a Kubernetes cluster should be simple, but it’s not. Richard Li explains how software architectures have evolved to take advantage of Kubernetes and discusses the implications that these changes have on ingress. Richard then covers some of the nuances of modern ingress, including authentication, resilience, and observability at the edge, explores how Kubernetes handles ingress today, with NodePorts, LoadBalancers, and ingress controllers, and shares his experience and lessons learned from using several real-world implementations of ingress on Kubernetes.
What’s the key to successfully adopting microservices on Kubernetes?
Building a development workflow that helps developers code faster.
In this webinar, we introduce the principles of a cloud-native development workflow where individual teams build and ship software independently from each other.
QCon SF 2017 - Microservices: Service-Oriented DevelopmentAmbassador Labs
Conventional wisdom is that microservices is an architecture that is the spiritual successor to service-oriented architecture. While true, this myopic view of microservices ignores some of the profound workflow shifts in today’s microservices organizations.
The reality is that microservices is an architecture _and_ workflow. In this talk, we’ll introduce the workflow of service-oriented development. Rafael will talk about how the real goal of microservices is to break up a monolithic development workflow. We’ll show you how, by breaking up your workflow, you can build software that lets you move fast and make things.
How Recreation Management Software Can Streamline Your Operations.pptxwottaspaceseo
Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.
Graspan: A Big Data System for Big Code AnalysisAftab Hussain
We built a disk-based parallel graph system, Graspan, that uses a novel edge-pair centric computation model to compute dynamic transitive closures on very large program graphs.
We implement context-sensitive pointer/alias and dataflow analyses on Graspan. An evaluation of these analyses on large codebases such as Linux shows that their Graspan implementations scale to millions of lines of code and are much simpler than their original implementations.
These analyses were used to augment the existing checkers; these augmented checkers found 132 new NULL pointer bugs and 1308 unnecessary NULL tests in Linux 4.4.0-rc5, PostgreSQL 8.3.9, and Apache httpd 2.2.18.
- Accepted in ASPLOS ‘17, Xi’an, China.
- Featured in the tutorial, Systemized Program Analyses: A Big Data Perspective on Static Analysis Scalability, ASPLOS ‘17.
- Invited for presentation at SoCal PLS ‘16.
- Invited for poster presentation at PLDI SRC ‘16.
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
Navigating the Metaverse: A Journey into Virtual Evolution"Donna Lenk
Join us for an exploration of the Metaverse's evolution, where innovation meets imagination. Discover new dimensions of virtual events, engage with thought-provoking discussions, and witness the transformative power of digital realms."
GraphSummit Paris - The art of the possible with Graph TechnologyNeo4j
Sudhir Hasbe, Chief Product Officer, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Software Engineering, Software Consulting, Tech Lead, Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Transaction, Spring MVC, OpenShift Cloud Platform, Kafka, REST, SOAP, LLD & HLD.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Crescat
Crescat is industry-trusted event management software, built by event professionals for event professionals. Founded in 2017, we have three key products tailored for the live event industry.
Crescat Event for concert promoters and event agencies. Crescat Venue for music venues, conference centers, wedding venues, concert halls and more. And Crescat Festival for festivals, conferences and complex events.
With a wide range of popular features such as event scheduling, shift management, volunteer and crew coordination, artist booking and much more, Crescat is designed for customisation and ease-of-use.
Over 125,000 events have been planned in Crescat and with hundreds of customers of all shapes and sizes, from boutique event agencies through to international concert promoters, Crescat is rigged for success. What's more, we highly value feedback from our users and we are constantly improving our software with updates, new features and improvements.
If you plan events, run a venue or produce festivals and you're looking for ways to make your life easier, then we have a solution for you. Try our software for free or schedule a no-obligation demo with one of our product specialists today at crescat.io
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
Software Engineering, Software Consulting, Tech Lead.
Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Security,
Spring Transaction, Spring MVC,
Log4j, REST/SOAP WEB-SERVICES.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
8. 8
How?
Configuration engine
Read K8s annotations and secrets, create
Envoy config
Update config as annotations change
Diagnostic service
Show user what we read, and the resulting
Envoy config
10. 10
How?
External Auth service
Ask an external service whether it’s OK for a
request to proceed
Envoy filter, so custom Envoy build for now
11. 11
How?
Kubernetes synchronization using hot restart
Less cool than xDS…
…but simpler to deploy and operate
Avoids instability due toV1 xDS polling latency
(keeping a careful eye onV2 though!)
14. 14
Challenges
Layers upon layers in debugging
You just got a 404 — quick, where do you
look?What caused it? How do you fix it
Might be Ambassador, Envoy, Kubernetes,
maybe Istio
Diagnostic service is a huge help here, but
we need more focus and visibility
15. 15
Challenges
Kubernetes development process
Reaching into the cluster to iterate and
debug adds a lot of complexity
In general, hunt for any tricks you can find to
streamline the dev cycle
Shameless plug: Datawire has tools to help
here