Puppet is IT automation tool. I was a speaker for this presentation in one of Meetup and it was received well. Sharing it with open source folks who want to collaborate, learn and Win.
Docker Engine 1.12 can be rightly called ” A Next Generation Docker Clustering & Distributed System”. Though Docker Engine 1.12 Final Release is around corner but the recent RC3 brings lots of improvements and exciting features. One of the major highlight of this release is Docker Swarm Mode which provides powerful yet optional ability to create coordinated groups of decentralized Docker Engines. Swarm Mode combines your engine in swarms of any scale. It’s self-organizing and self-healing. It enables infrastructure-agnostic topology.The newer version democratizes orchestration with out-of-box capabilities for multi-container on multi-host app deployments.
Docker Online Meetup #28: Production-Ready Docker SwarmDocker, Inc.
presented by Alexandre Beslic (@abronan)
Swarm v1.0 is now ready for running your apps in production!
Swarm is the easiest way to run Docker applications at large scale on a cluster. It turns a pool of Docker Engines into a single, virtual Engine. You don’t have to worry about where to put containers, or how they’re going to talk to each other - it just handles all that for you.
We’ve spent the last few months tirelessly hardening and tuning it, and in combination with multi-host networking and the new volume system in Docker Engine 1.9, we can confidently say that it’s ready for running your apps in production. In our tests, we’ve been running Swarm on EC2 with 1,000 nodes and 30,000 containers and it keeps on scheduling containers in less than half a second. Not even breaking a sweat! Keep an eye for a blog post soon with the full details.
Read more: http://blog.docker.com/2015/11/swarm-1-0/
Orchestrating Linux Containers while tolerating failuresDocker, Inc.
lthough containers are bringing a refreshing flexibility when deploying services in production, the management of those containers in such an environment still requires special care in order to keep the application up and running. In this regard, orchestration platforms like Docker, Kubernetes and Nomad have been trying to alleviate this responsibility, facilitating the task of deploying and maintaining the entire application stack in its desired state. This ensures that a service will be always running, tolerating machine failures, network erratic behavior or software updates and downtime. The purpose of this talk is to explain the mechanisms and architecture of the Docker Engine orchestration platform (using a framework called swarmkit) to tolerate failures of services and machines, from cluster state replication and leader-election to container re-scheduling logic when a host goes down.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2GtI22x.
Bret Fisher talks about how to get started with Docker and Swarm in production, using the latest versions. He shows tactics, example configs, real working infrastructure designs, and looks at the internals of Docker in production today. Filmed at qconsf.com.
Bret Fisher is a freelance DevOps and Docker consultant, trainer, speaker, and Open Source volunteer working from Virginia Beach. He is the author of the popular Docker Mastery series on Udemy with over 20,000 students. He has been a sysadmin and developer for 20+ years, and lately, he helps teams Dockerize their apps and orchestrate and automate their infrastructure.
Docker Swarm Is Dead: Long Live Docker SwarmElton Stoneman
From the Docker London MeetUp, presented on 27th June 2016. A walkthrough of Swarm Mode in Docker 1.12, the presentation introduces demos for creating a Docker Swarm using Azure virtual machines, and running a distributed application with a Node REST API, feeding analytics into Elasticsearch via a Redis queue.
Docker Engine 1.12 can be rightly called ” A Next Generation Docker Clustering & Distributed System”. Though Docker Engine 1.12 Final Release is around corner but the recent RC3 brings lots of improvements and exciting features. One of the major highlight of this release is Docker Swarm Mode which provides powerful yet optional ability to create coordinated groups of decentralized Docker Engines. Swarm Mode combines your engine in swarms of any scale. It’s self-organizing and self-healing. It enables infrastructure-agnostic topology.The newer version democratizes orchestration with out-of-box capabilities for multi-container on multi-host app deployments.
Docker Online Meetup #28: Production-Ready Docker SwarmDocker, Inc.
presented by Alexandre Beslic (@abronan)
Swarm v1.0 is now ready for running your apps in production!
Swarm is the easiest way to run Docker applications at large scale on a cluster. It turns a pool of Docker Engines into a single, virtual Engine. You don’t have to worry about where to put containers, or how they’re going to talk to each other - it just handles all that for you.
We’ve spent the last few months tirelessly hardening and tuning it, and in combination with multi-host networking and the new volume system in Docker Engine 1.9, we can confidently say that it’s ready for running your apps in production. In our tests, we’ve been running Swarm on EC2 with 1,000 nodes and 30,000 containers and it keeps on scheduling containers in less than half a second. Not even breaking a sweat! Keep an eye for a blog post soon with the full details.
Read more: http://blog.docker.com/2015/11/swarm-1-0/
Orchestrating Linux Containers while tolerating failuresDocker, Inc.
lthough containers are bringing a refreshing flexibility when deploying services in production, the management of those containers in such an environment still requires special care in order to keep the application up and running. In this regard, orchestration platforms like Docker, Kubernetes and Nomad have been trying to alleviate this responsibility, facilitating the task of deploying and maintaining the entire application stack in its desired state. This ensures that a service will be always running, tolerating machine failures, network erratic behavior or software updates and downtime. The purpose of this talk is to explain the mechanisms and architecture of the Docker Engine orchestration platform (using a framework called swarmkit) to tolerate failures of services and machines, from cluster state replication and leader-election to container re-scheduling logic when a host goes down.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2GtI22x.
Bret Fisher talks about how to get started with Docker and Swarm in production, using the latest versions. He shows tactics, example configs, real working infrastructure designs, and looks at the internals of Docker in production today. Filmed at qconsf.com.
Bret Fisher is a freelance DevOps and Docker consultant, trainer, speaker, and Open Source volunteer working from Virginia Beach. He is the author of the popular Docker Mastery series on Udemy with over 20,000 students. He has been a sysadmin and developer for 20+ years, and lately, he helps teams Dockerize their apps and orchestrate and automate their infrastructure.
Docker Swarm Is Dead: Long Live Docker SwarmElton Stoneman
From the Docker London MeetUp, presented on 27th June 2016. A walkthrough of Swarm Mode in Docker 1.12, the presentation introduces demos for creating a Docker Swarm using Azure virtual machines, and running a distributed application with a Node REST API, feeding analytics into Elasticsearch via a Redis queue.
In this overview presented to a gathering of directors for a large network equipment manufacturer, Chris discusses Docker, DevOps workflows, considerations for containers in production, and the extended Docker technology ecosystem.
Load Balancing Apps in Docker Swarm with NGINXNGINX, Inc.
On-demand webinar recording: http://bit.ly/2mRjk2g
Docker and other container technologies continue to gain in popularity. We recently surveyed the broad community of NGINX and NGINX Plus users and found that two-thirds of organizations are either investigating containers, using them in development, or using them in production. Why? Because abstracting your applications from the underlying infrastructure makes developing, distributing, and running software simpler, faster, and more robust than ever before.
But when you move from running your app in a development environment to deploying containers in production, you face new challenges – such as how to effectively run and scale an application across multiple hosts with the performance and uptime that your customers demand.
The latest Docker release, 1.12, supports multihost container orchestration, which simplifies deployment and management of containers across a cluster of Docker hosts. In a complex environment like this, load balancing plays an essential part in delivering your container-based application with reliability and high performance.
Join us in this webinar to learn:
* The basic built-in load balancing options available in Docker Swarm Mode
* The pros and cons of moving to an advanced load balancer like NGINX
* How to integrate NGINX and NGINX Plus with Swarm Mode to provide an advanced load-balancing solution for a cluster with orchestration
* How to scale your Docker-based application with Swarm Mode and NGINX Plus
Since its first 1.12 release on July 2016, Docker Swarm Mode has matured enough as a clustering and scheduling tool for IT administrators and developers who can easily establish and manage a cluster of Docker nodes as a single virtual system. Swarm mode integrates the orchestration capabilities of Docker Swarm into Docker Engine itself and help administrators and developers with the ability to add or subtract container iterations as computing demands change. With sophisticated but easy to implement features like built-in Service Discovery, Routing Mesh, Secrets, declarative service model, scaling of the services, desired state reconciliation, scheduling, filters, multi-host networking model, Load-Balancing, rolling updates etc. Docker 17.06 is all set for production-ready product today. Join me webinar organised by Docker Izmir, to get familiar with the current Swarm Mode capabilities & functionalities across the heterogeneous environments.
Docker Birthday #3 - Intro to Docker SlidesDocker, Inc.
High level overview of Docker + Birthday #3 overview (app and challenge portion)!
Learn more about Docker Birthday #3 celebrations here: https://www.docker.com/community/docker-birthday-3
It is the understanding of docker and how it was useful in day-to-day tasks, some basic difference between old and new. At last my learnings and mistake while doing it all.
Docker Datacenter Overview and Production Setup SlidesDocker, Inc.
An overview on Docker Data Center and Universal Control Plane. We will cover how to install for production and integrate Docker Trusted Registry.
Led by DDC + UCP Champ:
Vivek Saraswat
Experience Level: Attendees need no prior experience with Docker, but should be familiar with basic linux command-line.
Slides I presented at Docker Palo Alto Meetup 2/17/2016. They cover: (1) what is Swarm?, (2) how to set up Swarm, (3) an example microservice app based on Swarm.
Heart of the SwarmKit: Store, Topology & Object ModelDocker, Inc.
Heart of the SwarmKit: Store, Topology & Object Model by Aaron, Andrea, Stephen D (Docker)
Swarmkit repo - https://github.com/docker/swarmkit
Liveblogging: http://canopy.mirage.io/Liveblog/SwarmKitDDS2016
Docker Online Meetup: Infrakit update and Q&ADocker, Inc.
While working on Docker for AWS and Azure, we realized the need for a standard way to create and manage infrastructure state that was portable across any type of infrastructure, from different cloud providers to on-prem. One challenge is that each vendor has differentiated IP invested in how they handle certain aspects of their cloud infrastructure. It is not enough to just provision five servers; what IT ops teams need is a simple and consistent way to declare the number of servers, what size they should be, and what sort of base software configuration is required. And in the case of server failures (especially unplanned), that sudden change needs to be reconciled against the desired state to ensure that any required servers are re-provisioned with the necessary configuration. We started InfraKit to solves these problems and to provide the ability to create a self healing infrastructure for distributed systems.
Catching up with what has happened with logging in Docker since late 2014 all the way up to the recently released Docker 0.10. Also, presenting my view on a comprehensive approach to monitoring Docker using the API to get events, logs, stats with a little bit of self promotion in pointing out that we have recently released an implementation of comprehensive monitoring as part of a Sumo Logic collector source.
Using Docker Swarm Mode to Deploy Service Without Loss by Dongluo Chen & Nish...Docker, Inc.
Talk from Docker SF Meetup #50
Abstract:
Docker swarm mode enables users to manage their applications with service primitives. In this talk we demonstrate how to do service upgrades without impacting your application. The Healthcheck feature provides health indication for a container. Coming up in Docker 1.13 release, Docker Swarm can connect healthcheck result with load balancer to implement no-loss service upgrade.
Speaker Biographies:
Nishant Totla is a software engineer at Docker, and works on the core open source team. He is currently working on Docker SwarmKit and Docker Swarm. Prior to Docker, he was a PhD student at UC Berkeley, doing research on programming languages. In his spare time, he enjoys long-distance running, biking, and other outdoor activities. Nishant tweets at @nishanttotla.
Dongluo Chen is a software engineer at Docker focusing on orchestration and container development. Before Docker he was software engineer manager at Microsoft Azure building and automating global data centers. He worked at France Telecom (Orange) and the Ohio State University as research scientist in networking area.
Puppet at Scale – Case Study of PayPal's Learnings - PuppetConf 2013Puppet
"Puppet at Scale – Case Study of PayPal's Learnings" by Stan Hsu, Senior Dev Manager, PayPal.
Presentation Overview: Large scale and app level management pose challenges to any implementation of puppet. Come and learn some of the challenges PayPal Deployment Systems team faced and the how these were overcome.
Speaker Bio: Stan Hsu is the Senior Dev Manager for PayPal's deployment systems team. His team is currently responsible to build out a new deployment system based on puppet. In his tenure at eBay/PayPal, he's had the unique experience of having had access to all data centers in both eBay and PayPal to help build out of new deployment systems for production and QA environments. His interests include application at scale, scalability, performance tuning, and usability. In his previous roles he has managed teams at Tibco, Crossworlds, and HP.
As a PHP developer building web applications is besides making a living a lot of fun too, especially when you can deploy your apps to any kind of environment and on any platform. In this session I take a non-standard PHP application (based on Zend Framework) and deploy it to a bare metal environment running LAMP, Windows 2008 Server with IIS7 and to cloud instances like Azure and Amazon.
The goal is to provide information on how to deploy to these various environments manual and automatic, but also to show it doesn't really matter anymore what the targeted platform is, as long the apps are written in PHP.
In this overview presented to a gathering of directors for a large network equipment manufacturer, Chris discusses Docker, DevOps workflows, considerations for containers in production, and the extended Docker technology ecosystem.
Load Balancing Apps in Docker Swarm with NGINXNGINX, Inc.
On-demand webinar recording: http://bit.ly/2mRjk2g
Docker and other container technologies continue to gain in popularity. We recently surveyed the broad community of NGINX and NGINX Plus users and found that two-thirds of organizations are either investigating containers, using them in development, or using them in production. Why? Because abstracting your applications from the underlying infrastructure makes developing, distributing, and running software simpler, faster, and more robust than ever before.
But when you move from running your app in a development environment to deploying containers in production, you face new challenges – such as how to effectively run and scale an application across multiple hosts with the performance and uptime that your customers demand.
The latest Docker release, 1.12, supports multihost container orchestration, which simplifies deployment and management of containers across a cluster of Docker hosts. In a complex environment like this, load balancing plays an essential part in delivering your container-based application with reliability and high performance.
Join us in this webinar to learn:
* The basic built-in load balancing options available in Docker Swarm Mode
* The pros and cons of moving to an advanced load balancer like NGINX
* How to integrate NGINX and NGINX Plus with Swarm Mode to provide an advanced load-balancing solution for a cluster with orchestration
* How to scale your Docker-based application with Swarm Mode and NGINX Plus
Since its first 1.12 release on July 2016, Docker Swarm Mode has matured enough as a clustering and scheduling tool for IT administrators and developers who can easily establish and manage a cluster of Docker nodes as a single virtual system. Swarm mode integrates the orchestration capabilities of Docker Swarm into Docker Engine itself and help administrators and developers with the ability to add or subtract container iterations as computing demands change. With sophisticated but easy to implement features like built-in Service Discovery, Routing Mesh, Secrets, declarative service model, scaling of the services, desired state reconciliation, scheduling, filters, multi-host networking model, Load-Balancing, rolling updates etc. Docker 17.06 is all set for production-ready product today. Join me webinar organised by Docker Izmir, to get familiar with the current Swarm Mode capabilities & functionalities across the heterogeneous environments.
Docker Birthday #3 - Intro to Docker SlidesDocker, Inc.
High level overview of Docker + Birthday #3 overview (app and challenge portion)!
Learn more about Docker Birthday #3 celebrations here: https://www.docker.com/community/docker-birthday-3
It is the understanding of docker and how it was useful in day-to-day tasks, some basic difference between old and new. At last my learnings and mistake while doing it all.
Docker Datacenter Overview and Production Setup SlidesDocker, Inc.
An overview on Docker Data Center and Universal Control Plane. We will cover how to install for production and integrate Docker Trusted Registry.
Led by DDC + UCP Champ:
Vivek Saraswat
Experience Level: Attendees need no prior experience with Docker, but should be familiar with basic linux command-line.
Slides I presented at Docker Palo Alto Meetup 2/17/2016. They cover: (1) what is Swarm?, (2) how to set up Swarm, (3) an example microservice app based on Swarm.
Heart of the SwarmKit: Store, Topology & Object ModelDocker, Inc.
Heart of the SwarmKit: Store, Topology & Object Model by Aaron, Andrea, Stephen D (Docker)
Swarmkit repo - https://github.com/docker/swarmkit
Liveblogging: http://canopy.mirage.io/Liveblog/SwarmKitDDS2016
Docker Online Meetup: Infrakit update and Q&ADocker, Inc.
While working on Docker for AWS and Azure, we realized the need for a standard way to create and manage infrastructure state that was portable across any type of infrastructure, from different cloud providers to on-prem. One challenge is that each vendor has differentiated IP invested in how they handle certain aspects of their cloud infrastructure. It is not enough to just provision five servers; what IT ops teams need is a simple and consistent way to declare the number of servers, what size they should be, and what sort of base software configuration is required. And in the case of server failures (especially unplanned), that sudden change needs to be reconciled against the desired state to ensure that any required servers are re-provisioned with the necessary configuration. We started InfraKit to solves these problems and to provide the ability to create a self healing infrastructure for distributed systems.
Catching up with what has happened with logging in Docker since late 2014 all the way up to the recently released Docker 0.10. Also, presenting my view on a comprehensive approach to monitoring Docker using the API to get events, logs, stats with a little bit of self promotion in pointing out that we have recently released an implementation of comprehensive monitoring as part of a Sumo Logic collector source.
Using Docker Swarm Mode to Deploy Service Without Loss by Dongluo Chen & Nish...Docker, Inc.
Talk from Docker SF Meetup #50
Abstract:
Docker swarm mode enables users to manage their applications with service primitives. In this talk we demonstrate how to do service upgrades without impacting your application. The Healthcheck feature provides health indication for a container. Coming up in Docker 1.13 release, Docker Swarm can connect healthcheck result with load balancer to implement no-loss service upgrade.
Speaker Biographies:
Nishant Totla is a software engineer at Docker, and works on the core open source team. He is currently working on Docker SwarmKit and Docker Swarm. Prior to Docker, he was a PhD student at UC Berkeley, doing research on programming languages. In his spare time, he enjoys long-distance running, biking, and other outdoor activities. Nishant tweets at @nishanttotla.
Dongluo Chen is a software engineer at Docker focusing on orchestration and container development. Before Docker he was software engineer manager at Microsoft Azure building and automating global data centers. He worked at France Telecom (Orange) and the Ohio State University as research scientist in networking area.
Puppet at Scale – Case Study of PayPal's Learnings - PuppetConf 2013Puppet
"Puppet at Scale – Case Study of PayPal's Learnings" by Stan Hsu, Senior Dev Manager, PayPal.
Presentation Overview: Large scale and app level management pose challenges to any implementation of puppet. Come and learn some of the challenges PayPal Deployment Systems team faced and the how these were overcome.
Speaker Bio: Stan Hsu is the Senior Dev Manager for PayPal's deployment systems team. His team is currently responsible to build out a new deployment system based on puppet. In his tenure at eBay/PayPal, he's had the unique experience of having had access to all data centers in both eBay and PayPal to help build out of new deployment systems for production and QA environments. His interests include application at scale, scalability, performance tuning, and usability. In his previous roles he has managed teams at Tibco, Crossworlds, and HP.
As a PHP developer building web applications is besides making a living a lot of fun too, especially when you can deploy your apps to any kind of environment and on any platform. In this session I take a non-standard PHP application (based on Zend Framework) and deploy it to a bare metal environment running LAMP, Windows 2008 Server with IIS7 and to cloud instances like Azure and Amazon.
The goal is to provide information on how to deploy to these various environments manual and automatic, but also to show it doesn't really matter anymore what the targeted platform is, as long the apps are written in PHP.
Build your own clouds with Chef and MCollectiveJonathan Weiss
One important part of the DevOps movement is infrastructure automation, especially if you are running your application on top of services like Amazon EC2.
Everybody's dream is to be able to bootstrap and deploy hundreds or even thousands of machines with a few simple commands. This talk will tell you how you can do this using Open Source tools like Chef and mcollective. Chef manages your servers configuration using a nice Ruby DSL while mcollective orchestrates and commands all your nodes.
Introduction to orchestration using McollectivePuppet
"Introduction to orchestration using MCollective" by Pieter Loubser at Puppet Camp London 2013. Find the video here: http://puppetlabs.com/community/puppet-camp
Continuous Deployment Applied at MyHeritageRan Levy
Learn how continuous deployment was applied at MyHeritage. Check out how we automated the process using MCollective, RPM, Jenkins, unit and integration tests using JUnit, PHPUnit, Cucumber and more.
PuppetConf track overview: Modern InfrastructurePuppet
From containers to Docker, Mesos and Kubernetes — you'll hear about it at PuppetConf 2016 in San Diego. Learn more and register at https://puppet.com/puppetconf/.
LAS16-301: OpenStack on Aarch64, running in production, upstream improvements...Linaro
LAS16-301: OpenStack on Aarch64, running in production, upstream improvements, and interoperability
Speakers: Yibo Cai, Gema Gomez Solano, Jack He, Marcin Juskiewicz, Martin Stadtler
Date: September 28, 2016
★ Session Description ★
“OpenStack is at the heart of the next generation of the opensource
cloud on a global scale. During this presentation, we will touch on three themes, running an OpenStack based cloud in production by Gema Gomez and Andy Doan, followed by Marcin talking about the packaging and bug fixing on archives required to make that happen on AArch64. Jack He and Yibo Cai, will explain what it is like working with the the upstream project, the development environment, the current patches and what needs to be done next. Then Gema Gomez will Introduce the OpenStack Interop Working Group. Why is interoperability important for OpenStack? And What is Linaro doing to improve the interoperability of OpenStack?
★ Resources ★
Etherpad: pad.linaro.org/p/las16-301
Presentations & Videos: http://connect.linaro.org/resource/las16/las16-301/
★ Event Details ★
Linaro Connect Las Vegas 2016 – #LAS16
September 26-30, 2016
http://www.linaro.org
http://connect.linaro.org
Build cloud native solution using open source Nitesh Jadhav
Build cloud native solution using open source. I have tried to give a high level overview on How to build Cloud Native using CNCF graduated software's which are tested, proven and having many reference case studies and partner support for deployment
Sanger, upcoming Openstack for Bio-informaticiansPeter Clapham
Delivery of a new Bio-informatics infrastructure at the Wellcome Trust Sanger Center. We include how to programatically create, manage and provide providence for images used both at Sanger and elsewhere using open source tools and continuous integration.
A presentation on how applying Cloud Architecture Patterns using Docker Swarm as orchestrator is possible to create reliable, resilient and scalable FIWARE platforms.
TechWiseTV Workshop: Open NX-OS and Devops with Puppet LabsRobb Boyd
Two incredible engineers: Shane Corban from Cisco and Carl Caum from Puppet Labs came together to be our guest experts for this workshop. See the demos in the replay at bit.ly/1lJQm3A
Monitoring in Big Data Platform - Albert Lewandowski, GetInDataGetInData
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
The webinar was organized by GetinData on 2020. During the webinar we explaned the concept of monitoring and observability with focus on data analytics platforms.
Watch more here: https://www.youtube.com/watch?v=qSOlEN5XBQc
Whitepaper - Monitoring ang Observability for Data Platform: https://getindata.com/blog/white-paper-big-data-monitoring-observability-data-platform/
Speaker: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Tips Tricks and Tactics with Cells and Scaling OpenStack - May, 2015Belmiro Moreira
Tips Tricks and Tactics with Cells and Scaling OpenStack
OpenStack Design Summit, Paris - May, 2015
Belmiro Moreira - CERN
Matt Van Winkle - Rackspace
Sam Morrison - NeCTAR, University of Melbourne
About 94% of AI Adopters are planning to use containers in the next 1 year. What’s driving this exponential growth? Faster time to deployment and Faster AI workload processing are the two major reasons. You can use GPUs in big data applications such as machine learning, data analytics, and genome sequencing. Docker containerization makes it easier for you to package and distribute applications. You can enable GPU support when using YARN on Docker containers. In this talk, I will demonstrate how Docker accelerates the AI workload development and deployment over the IoT Edge devices in efficient manner
Delivering Container-based Apps to IoT Edge devicesAjeet Singh Raina
I presented it during Dockercon. This talk was all about AI + Docker + IoT. Showcased how Docker app talk to Sensors, GPUs and Camera module and demo'ed how sensors data can be visualized over Grafana dashboard - all running on a IoT Edge device.
Presented Keynotes last weekend to Collabnix Community . Talk about trending topics, Kubernetes Statistics and community growth. It was fun talking about how exciting the community is and where are we heading next to.
Quantifying Your World with AI & Docker on the Edge | OSCONF 2020 JaipurAjeet Singh Raina
When Docker, IoT and AI Meet Together....
Highlights:
- Opendatacam is 100% Open Source solution
- Quantifies and tracks moving objects with Live Video Analysis
- Runs on Linux and CUDA GPU enabled hardware
- Runs completely on Docker Containers as well as support K3s.
Read the Full Story - https://collabnix.com/object-detection-with-yolo-using-docker-19-03-on-nvidia-jetson-nano/
Hear me LIVE on 10th October while I talk about Running Docker Container which uses GPU on Jetson Nano for Implementing Object Detection and Analytics.
IoET Conference is run by IoET Planet which is purely a collaborative platform. IoET Planet is an innovative platform to bring people together for the first time under a single umbrella in order to learn, collaborate and contribute.
It is an open community dedicated to IoT Research, innovation and applications. Today it is a blogging site & covers a wide range of topics, from embedded systems to data analytics and machine learning implementation, having the objective of enabling a vast array of services in areas like e-health, mobility, energy, manufacturing and agriculture, just to cite a few examples. These topics are coming from the community users and open for industry experts, IoT enthusiast and DevOps Engineers.
OSCONF is a platform built to connect & bring all leading Meetup communities, Cloud native experts, Evangelists & DevOps together under one roof. It is a non-profit community event which allows you to interact with community leaders, ambassadors & open community contributors.
Accelerate Your Automation Testing Effort using TestProject & Docker | Docker...Ajeet Singh Raina
Accelerate Your Test Automation using TestProject & Docker
A recording of a live webinar hosted on May 17th, 2020 - Learn from Docker Captain Ajeet Singh Raina how you can leverage TestProject Agents running in Docker containers, easily setup virtual labs & run tests in parallel.
- Introduction to TestProject
- Why TestProject for Automation?
- A Brief about TestProject Agents
- Why Docker?
- Why run TestProject Agents inside Docker containers?
- Live Demo
- Game with Prizes!
The Year 2020 is going to be super exciting for "Developers". As rightly stated by Scott Johnston, Chief Product Officer, Docker. Inc "Addressing the needs of developers has been core to Docker since its inception. Developers want the freedom to choose their own tools, the flexibility to quickly try new technologies and the ability to deploy their applications anywhere, independent of the underlying infrastructure,” Going forward, Docker Inc is going to focus on Docker Desktop, Docker Hub & Docker Compose's roles in the developer workflow for modern apps. Docker will be investing in expanding cloud services to enable developers to quickly discover technologies for use when building applications, to easily share these apps with teammates and the community, and to run apps frictionlessly on any Kubernetes endpoint, whether locally or in the cloud.
This year starting Jan 18th, Docker Bangalore Meetup community kickstarted 2020 with really exciting topics around Docker, Kubernetes & latest announcement from Kubecon.
Links to get Started with Docker/Kubernetes:
Collabnix - https://collabnix.com
DockerLabs - http://dockerlabs.collabnix.com
Kubelabs - http://kubelabs.collabnix.com
Kubetools - http://kubetools.collabnix.com
Gopherlabs - http://gopherlabs.collabnix.com
#Docker #Kubernetes #dockerlabs #cloudnative
Collabnix Community conduct webinar on regular basis. Swapnasagar Pradhan, an engineer from VISA delivered a talk on Traefik this January 11th 2020. Check this out.
I presented these slides during Docker Bangalore Meetup which happened on 23-Nov in DellEMC in Bangalore. These slides talks about the recent acquisition of Docker Enterprise by Mirantis and talks about what Docker is going to focus next on.
Prashansa Kulshrestha is currently pursuing B. Tech in Computer Science with Specialization in Cloud Computing and Virtualization Technology at UPES, Dehradun. She is Red Hat OpenShift Certified Engineer as well as RHCE too. She has experience working on Docker, Docker Compose & Docker Swarm. She is an active contributor of Docker Labs. She is expert in Python and well acquainted with AWS.
She recently implemented Log analytics tool using Elastic Stack and Docker and want to share her learning and experience with Collabnix Community.
Simplifying Real Time Data Analytics with Docker, IoT & CloudAjeet Singh Raina
Last Thursday, I travelled to Dehradun(around 1500 miles) for 1 day session on " The Pico project" as well as Docker workshop on Raspberry Pi. It was an amazing experience where I got chance to interact with the University students for the first time.
Explore the innovative world of trenchless pipe repair with our comprehensive guide, "The Benefits and Techniques of Trenchless Pipe Repair." This document delves into the modern methods of repairing underground pipes without the need for extensive excavation, highlighting the numerous advantages and the latest techniques used in the industry.
Learn about the cost savings, reduced environmental impact, and minimal disruption associated with trenchless technology. Discover detailed explanations of popular techniques such as pipe bursting, cured-in-place pipe (CIPP) lining, and directional drilling. Understand how these methods can be applied to various types of infrastructure, from residential plumbing to large-scale municipal systems.
Ideal for homeowners, contractors, engineers, and anyone interested in modern plumbing solutions, this guide provides valuable insights into why trenchless pipe repair is becoming the preferred choice for pipe rehabilitation. Stay informed about the latest advancements and best practices in the field.
Welcome to WIPAC Monthly the magazine brought to you by the LinkedIn Group Water Industry Process Automation & Control.
In this month's edition, along with this month's industry news to celebrate the 13 years since the group was created we have articles including
A case study of the used of Advanced Process Control at the Wastewater Treatment works at Lleida in Spain
A look back on an article on smart wastewater networks in order to see how the industry has measured up in the interim around the adoption of Digital Transformation in the Water Industry.
Event Management System Vb Net Project Report.pdfKamal Acharya
In present era, the scopes of information technology growing with a very fast .We do not see any are untouched from this industry. The scope of information technology has become wider includes: Business and industry. Household Business, Communication, Education, Entertainment, Science, Medicine, Engineering, Distance Learning, Weather Forecasting. Carrier Searching and so on.
My project named “Event Management System” is software that store and maintained all events coordinated in college. It also helpful to print related reports. My project will help to record the events coordinated by faculties with their Name, Event subject, date & details in an efficient & effective ways.
In my system we have to make a system by which a user can record all events coordinated by a particular faculty. In our proposed system some more featured are added which differs it from the existing system such as security.
CFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptxR&R Consult
CFD analysis is incredibly effective at solving mysteries and improving the performance of complex systems!
Here's a great example: At a large natural gas-fired power plant, where they use waste heat to generate steam and energy, they were puzzled that their boiler wasn't producing as much steam as expected.
R&R and Tetra Engineering Group Inc. were asked to solve the issue with reduced steam production.
An inspection had shown that a significant amount of hot flue gas was bypassing the boiler tubes, where the heat was supposed to be transferred.
R&R Consult conducted a CFD analysis, which revealed that 6.3% of the flue gas was bypassing the boiler tubes without transferring heat. The analysis also showed that the flue gas was instead being directed along the sides of the boiler and between the modules that were supposed to capture the heat. This was the cause of the reduced performance.
Based on our results, Tetra Engineering installed covering plates to reduce the bypass flow. This improved the boiler's performance and increased electricity production.
It is always satisfying when we can help solve complex challenges like this. Do your systems also need a check-up or optimization? Give us a call!
Work done in cooperation with James Malloy and David Moelling from Tetra Engineering.
More examples of our work https://www.r-r-consult.dk/en/cases-en/
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Dr.Costas Sachpazis
Terzaghi's soil bearing capacity theory, developed by Karl Terzaghi, is a fundamental principle in geotechnical engineering used to determine the bearing capacity of shallow foundations. This theory provides a method to calculate the ultimate bearing capacity of soil, which is the maximum load per unit area that the soil can support without undergoing shear failure. The Calculation HTML Code included.
Student information management system project report ii.pdfKamal Acharya
Our project explains about the student management. This project mainly explains the various actions related to student details. This project shows some ease in adding, editing and deleting the student details. It also provides a less time consuming process for viewing, adding, editing and deleting the marks of the students.
Saudi Arabia stands as a titan in the global energy landscape, renowned for its abundant oil and gas resources. It's the largest exporter of petroleum and holds some of the world's most significant reserves. Let's delve into the top 10 oil and gas projects shaping Saudi Arabia's energy future in 2024.
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
Quality defects in TMT Bars, Possible causes and Potential Solutions.PrashantGoswami42
Maintaining high-quality standards in the production of TMT bars is crucial for ensuring structural integrity in construction. Addressing common defects through careful monitoring, standardized processes, and advanced technology can significantly improve the quality of TMT bars. Continuous training and adherence to quality control measures will also play a pivotal role in minimizing these defects.
2. • Infrastructure-as-a-Code
• Puppet is an open source configuration management utility
• It is written in Ruby and released as free software under the GPL
• Built to be cross-platform
• A Declarative Language
• A Model Driven Architecture
10. • Talks about Resource Management when agents connect
• Handles the “How” by knowing how different platforms and OS manages certain type of
resources
• Each types has number of providers
• A Providers contains “how” of managing packages using a Package Management tool.
• When agents connect , Puppet uses a tool called “Facter”. To return the information about that
agent, including what OS is it running.
• Puppet chooses the appropriate package provider for that OS. Reports Success or Failure
11. • A System Inventory tool
• Returns facts about each agent(Hostname, IP address, OS and version information)
• These facts are gathered when agent runs.
• Facts are sent to the Puppet master, and automatically created as variable available to Puppet
• How to run facter?
• Facts are made available as variables that can be used in your puppet configuration
• Helps Puppet understand on how to manage particular resources in an agent.
12.
13. • Resources - The core of the Puppet language is declaring resources. An Individual configuration
items
• Files – Physical files to serve to your agents
• Templates – Templates files that you can use to populate files
• Modules – Portable collection of resources. Reusable, sharable units of Puppet Code
• Classes – Modules can contain many Puppet classes. Groups of resource declarations and
conditional statements
• Manifests - Puppet code is saved in files called manifests, which are in turn stored in structured
directories called modules.
• Pre-built Puppet modules can be downloaded from the Puppet Forge, and most users will write at
least some of their own modules.
23. • All servers that are physical with 4 CPU, deploy ESX.
• All servers that are virtual with 1 CPU and 4GB of memory, deploy
CentOS, and handoff system to puppetmaster.cse.com for
management.
• All servers that are virtual with 32GB of memory, deploy Debian,
and handoff system to puppetmaster.cse.com for management.
24. - A Software tool for rapid provisioning of OS and Hypervisor
~ on both physical and virtual servers
- A policy-based bare-metal provisioning lets you inventory and manage the lifecycle of your
physical machines.
- Automatically discovers bare-metal hardware, dynamically configure operating systems
and/or hypervisors, and hand nodes off to PE for workload configuration
- Two major Components:
~ The Razor Server
(Ruby, MongoDB, Node.js)
~ The Razor Microkernel
(~20MB LInux Kernel, Facter, MCollective)
25. • - Discovery ( Tags, Matcher Rules)
• - Models(Defining OS Templates,..)
• - Policies(Rules that apply Models to Nodes based on Discovery)
• - Broker( Configuration Management)
26. When a new node appears, Razor discovers its characteristics by booting it with the Razor
microkernel and inventorying its facts.
The node is tagged based on its characteristics. Tags contain a match condition — a Boolean
expression that has access to the node’s facts and determines whether the tag should be applied to
the node or not.
27.
28.
29. • Install PE in Your Virtual Environment
• Install and Configure dnsmasq DHCP/TFTP Service
• Temporarily Disable SELinux to Enable PXE Boot
• Edit the dnsmasq Configuration File to Enable PXE Boot
• Install the Razor Server
– Load iPXE Software
– Verify the Razor Server
• Install and Set Up the Razor Client
• Setup Razor Provisioning
• Include Repos
• Include Brokers
• Include Tasks
• Create Policies
• Identify and Register Nodes
30.
31. - A Tag consists of a unique name and rule
- The tag matches a node if evaluating it against the tag's facts results in true.
- The syntax for rule expressions is defined in lib/razor/matcher.rb
32. - Policies orchestrate repos, brokers, and tasks to tell Razor what bits to install, where to get the
bits, how they should be configured, and how to communicate between a node and PE.
- Policies contain a good deal of information, it’s handy to save them in a JSON file that you run when
you create the policy
Example: It should be applied to the first 20 nodes that have no more than two processors that boot
33. - Create a file called policy.json and copy the following template text into it:
- Edit the options in the policy.json template with information specific to your environment.
- Apply the policy by executing: razor create-policy --json policy.json.
34. Step -1 – A Fresh Razor with no new node.
Step-2 – Create a new VM. It retrieves DHCP IP and loads microkernel
45. Step-11- A New Node starts loading as per the policy specified.
46. Step-12 - Verify the node2 policy attached through puppet master
47. Step-13 - The New OS comes up , shows that it has been installed through Razor.
48. - Deploy version 1.2.3 on my application to all 3000 systems
- Deploy version 1.2.5rc2 of my application to all 340 development systems
- Restart the Apache service on all the systems in North America zones
- What systems are online right now?
- Run puppet on all systems, ensuring that at most 10 runs are happenings at once
- Upgrade the Hadoop version from 0.1 to 1.1 on all those 2500 nodes
49. - A framework to build server orchestration or parallel job execution systems
- Uses a Publish Subscribe Middleware philosophy ~ real time discovery of network resources
using meta data and not hostnames
~ a messaging pattern where senders of messages, called publishers, do not program the messages to be sent directly to specific
receivers, called subscribers. Instead, published messages are characterized into classes, without knowledge of what, if any, subscribers
there may be. Similarly, subscribers express interest in one or more classes, and only receive messages that are of interest, without
knowledge of what, if any, publishers there are. “
- Uses a broadcast paradigm for request distribution.
~ all servers get all requests at the same time, requests have filters attached and only servers matching the filter will act on requests.
There is no central asset database to go out of sync, the network is the only source of truth “
50. An MCollective client can send requests to any number of servers, using a security plugin to encode and sign the request
and a connector plugin to publish it.
It can also receive replies from servers, and format the response data for a user or some other system.
Example : mco command-line client
51. An mCollective server is a computer which can be controlled via mCollective.
Servers runs the mcollecitve daemon (mcollectived) and have any number of agent plugins installed
52.
53.
54. • An Abstract Cloud of Magic
• Clients and servers don’t communicate directly. They publish messages
to the middleware, and subscribe to messages they are interested in.
• Middleware system ~ knows how to route messages.
• External to MCollective, and everything that interacts directly with it is
pushed out into a connector plugin (which needs some limited
knowledge about the topology and configuration of the middleware).
55. Apache ActiveMQ
- an open-source message broker that runs on the JVM;
- Installed with a wrapper script and init script that allow it to be managed as a normal
OS service.
- MCollective talks to ActiveMQ using the Stomp protocol
- This is the main middleware recommended for use with Mcollective
- most well-tested option, its security features are powerful and flexible enough to suit
nearly all needs, and it can scale by clustering once a deployment gets too big (we
recommend ~800 nodes per ActiveMQ server as a maximum). Its main drawback is
that it can be frustrating to configure; to help mitigate that, we provide a detailed
ActiveMQ config reference in our own docs
•
56. RabbitMQ
• An open-source message broker written in Erlang; MCollective talks to
RabbitMQ using the Stomp protocol. Although it works well with
MCollective, it isn’t as thoroughly tested with it as ActiveMQ is, so if your
site has no preference, you should default to ActiveMQ.
• The RabbitMQ connector ships with MCollective’s core and is available by
default.
59. ~ Lets you define broadcast domains and configure a mcollective server to
belong to one or many of these domains.
60. $mco ping – communicates to all the puppet agents
61. - Make requests to your servers.
- Capable of interacting with any standard Remote Procedure Call (RPC) agent.
How it work?
i. Perform discovery against the network and discover 10 servers
ii. Send the request and then show a progress bar of the replies
iii. Show any results that were out of the ordinary
iv. Show some statistics
62. • You can request the status for a specific service
63. • Report of all your IBM hardware listing hostname, serial number, and product
name
64. Are you using the default Web Server?
Limitation of WEBrick, the default web server used to enable Puppet’s web services connectivity, is
essentially a reference implementation, and becomes unreliable as number of nodes increases.
Challenge-1 - Scaling the Transport
Possible Solution - increase the performance and potential number of possible master and agent
connections.
Challenge-2 – Scaling SSL
Possible Solution - we implement good management of the SSL certificates that secure the
connection between the master and the agent
• -
65. Approach
• Replacing WEBRick Ruby-based HTTP server with the Apache web server on a single Puppet master
system
• Extending the strategy to multiple Puppet master systems working behind a load balancer with Apache
Web Server/Nginx.
66. Running Puppet Master with Apache and Passenger
An Apache module that allows the embedding of Ruby applications, much like mod_php or mod_perl allow the
embedding of PHP and Perl applications.
For networks of one to two thousand Puppet managed nodes, a single Puppet master system running inside of
Apache with Passenger is often sufficient
Approach:
- Install Apache and Passenger
- Configure apache to handle SSL authentication and Verification of Puppet Agent
- Connect Apache to the Puppet Master
70. Front End HTTP Load Balancer
- Front-end Apache Virtual Host to authorize incoming Puppet Agent requests and handle SSL encryption
and decryption of the traffic
- It will terminate the SSL connection, be responsible for authenticating the puppet agent request and then
present this authentication information to the back-end Puppet master workers for authorization.
- All traffic between the front-end load balancer and the back-end Puppet master systems are unencrypted
and in plain text
71. Puppet Master Worker Configuration
• When running the Puppet master behind a load balancer, there will be multiple
Puppet master processes running on different hosts behind the load balancer.
• The load balancer will listen on the Puppet port of 8140. Incoming requests will be
dispatched to available back-end worker processes, (configures the Puppet CA and
workers all on the same host using unique TCP ports bound to the loopback
interface)
- .
75. - Synchronize the Puppet CA directory across all of the worker systems
- Make one worker system the active Puppet CA service and a second worker system the hot
standby Puppet CA service.