This document summarizes Lucas Fontes' approach to deploying applications with Docker and Consul at Uken Studios, which has around 150 servers running 70 application servers and 40 databases. It outlines challenges with their previous deployment process and infrastructure, and describes how their new system addresses these using Docker for building and running applications, Consul for service discovery and configuration, and Consul Template to reload applications when configurations change. The workflow involves building Docker images, running containers securely, and releasing new versions by pushing images to a registry and notifying Consul to trigger reloads.
Slides from my ContainerCamp UK 2017 session.
These slides present a practical chaos engineering approach for resilience testing of Docker based software systems.
DockerCon EU 2015: The Glue is the Hard Part: Making a Production-Ready PaaSDocker, Inc.
Presented by Evan Krall, Site Reliability Engineer, Yelp
Docker is an amazing technology. In particular, its build-once-run-anywhere model unlocks the world of cluster schedulers like Mesos and Kubernetes. These solve many of the problems of running high-scale websites, but introduce new challenges that need addressing.
In this talk, Evan will describe PaaSTA, a PaaS built on top of open source tools including Docker, Mesos, Marathon, and Chronos. PaaSTA provides tooling for developers to quickly turn their microservice into a monitored, highly available application spanning multiple datacenters and cloud regions. Evan will give an overview of the open-source technologies that power PaaSTA, discuss how Yelp has glued these together to give developers control without burdening them with the complexities of the infrastructure, and show the workflow used by developers to update and maintain their services on PaaSTA.
In this webinar, Alex Casalboni will overview the main FaaS concepts and best practices (Function as a Service), explore the open-source FaaS options and discuss pros and cons of deploying and managing your own serverless platform on Kubernetes.
KubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipelineKubeAcademy
One of the most underrated features of Kubernetes is namespaces. In the market, instead of using this feature, people are still stuck with having different clusters for their environments. This talk will try to break this approach, and will introduce how we end up using ephemeral namespaces within our CI/CD pipeline. It will cover the architecture of our system for running the user acceptance tests on isolated ephemeral namespaces with every bits and pieces running within pods. While doing this, we will set up our CI/CD pipeline on top of TravisCI, GoCD, and Selenium that is controlled by Nightwatch.js.
Sched Link: http://sched.co/6Bcb
Slides from my ContainerCamp UK 2017 session.
These slides present a practical chaos engineering approach for resilience testing of Docker based software systems.
DockerCon EU 2015: The Glue is the Hard Part: Making a Production-Ready PaaSDocker, Inc.
Presented by Evan Krall, Site Reliability Engineer, Yelp
Docker is an amazing technology. In particular, its build-once-run-anywhere model unlocks the world of cluster schedulers like Mesos and Kubernetes. These solve many of the problems of running high-scale websites, but introduce new challenges that need addressing.
In this talk, Evan will describe PaaSTA, a PaaS built on top of open source tools including Docker, Mesos, Marathon, and Chronos. PaaSTA provides tooling for developers to quickly turn their microservice into a monitored, highly available application spanning multiple datacenters and cloud regions. Evan will give an overview of the open-source technologies that power PaaSTA, discuss how Yelp has glued these together to give developers control without burdening them with the complexities of the infrastructure, and show the workflow used by developers to update and maintain their services on PaaSTA.
In this webinar, Alex Casalboni will overview the main FaaS concepts and best practices (Function as a Service), explore the open-source FaaS options and discuss pros and cons of deploying and managing your own serverless platform on Kubernetes.
KubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipelineKubeAcademy
One of the most underrated features of Kubernetes is namespaces. In the market, instead of using this feature, people are still stuck with having different clusters for their environments. This talk will try to break this approach, and will introduce how we end up using ephemeral namespaces within our CI/CD pipeline. It will cover the architecture of our system for running the user acceptance tests on isolated ephemeral namespaces with every bits and pieces running within pods. While doing this, we will set up our CI/CD pipeline on top of TravisCI, GoCD, and Selenium that is controlled by Nightwatch.js.
Sched Link: http://sched.co/6Bcb
Kernel load-balancing for Docker containers using IPVSDocker, Inc.
Many companies use expensive proprietary hardware and software to provide load-balancing and routing for their users and services. I'm going to demonstrate how the same or even exceeding performance and feature set can be achieved using an open-source technology which has been a part of the mainline Linux kernel for over a decade – IPVS. Specifically, you'll see how IPVS can be used to automatically configure load balancing and routing for Docker containers using a simple Go daemon and a Docker plugin.
KubeCon EU 2016: Kubernetes and the Potential for Higher Level InterfacesKubeAcademy
Kubernetes provides rock-solid APIs for building and running your distributed systems. Pods, Services and ReplicationControllers provide trustworthy and scalable abstractions that make solving real-world infrastructure problems simpler. But that doesn’t mean interacting with those low-level primitives will be the only option for developers and operators.
Sched Link: http://sched.co/67dA
KubeCon EU 2016: Using Traffic Control to Test Apps in KubernetesKubeAcademy
Testing applications is important, as shown by the rise of continuous integration and automated testing. In this talk, I will focus on one area of testing that is difficult to automate: poor network connectivity. Developers usually work within reliable networking conditions so they might not notice issues that arise in other networking conditions. I will give examples of software that would benefit from test scenarios with varying connectivity. I will explain how traffic control on Linux can help to simulate various network connectivity. Finally, I will run a demo showing how an application running in Kubernetes behaves when changing network parameters.
Sched Link: http://sched.co/6Bb3
An introduction to Kubernetes and a look at how it leverages AWS IaaS features to provide its own virtual clustering, and demonstration of some of the behaviour inside the cluster that makes Kubernetes a popular choice for microservice deployments.
Dockercon 16 Wrap-up (Docker for Mac and Win, Docker 1.12, Swarm Mode, etc.)Nils De Moor
The Docker Belgium Meetup group held a Post-Dockercon16 meetup where the highlights of the conference were presented: Docker 1.12, Docker Datacenter, Docker Swarm Mode, Docker for AWS and Azure, DOcker for Mac and Windows.
Demo Notes: https://gist.github.com/ndemoor/7d72e0778f7a6a03b072e9b8c8e4781e
Shipping Applications to Production in Containers with DockerJérôme Petazzoni
Docker is an Open Source engine to build, run, and manage Linux Containers. Containers use less resources than virtual machines, they boot faster, but they have similar guarantees of portability and repeatability for Linux applications. Those features made Docker and Linux Containers extremely popular for development and testing environments. But what does it take to use Docker and Containers for production workloads?
Kernel load-balancing for Docker containers using IPVSDocker, Inc.
Many companies use expensive proprietary hardware and software to provide load-balancing and routing for their users and services. I'm going to demonstrate how the same or even exceeding performance and feature set can be achieved using an open-source technology which has been a part of the mainline Linux kernel for over a decade – IPVS. Specifically, you'll see how IPVS can be used to automatically configure load balancing and routing for Docker containers using a simple Go daemon and a Docker plugin.
KubeCon EU 2016: Kubernetes and the Potential for Higher Level InterfacesKubeAcademy
Kubernetes provides rock-solid APIs for building and running your distributed systems. Pods, Services and ReplicationControllers provide trustworthy and scalable abstractions that make solving real-world infrastructure problems simpler. But that doesn’t mean interacting with those low-level primitives will be the only option for developers and operators.
Sched Link: http://sched.co/67dA
KubeCon EU 2016: Using Traffic Control to Test Apps in KubernetesKubeAcademy
Testing applications is important, as shown by the rise of continuous integration and automated testing. In this talk, I will focus on one area of testing that is difficult to automate: poor network connectivity. Developers usually work within reliable networking conditions so they might not notice issues that arise in other networking conditions. I will give examples of software that would benefit from test scenarios with varying connectivity. I will explain how traffic control on Linux can help to simulate various network connectivity. Finally, I will run a demo showing how an application running in Kubernetes behaves when changing network parameters.
Sched Link: http://sched.co/6Bb3
An introduction to Kubernetes and a look at how it leverages AWS IaaS features to provide its own virtual clustering, and demonstration of some of the behaviour inside the cluster that makes Kubernetes a popular choice for microservice deployments.
Dockercon 16 Wrap-up (Docker for Mac and Win, Docker 1.12, Swarm Mode, etc.)Nils De Moor
The Docker Belgium Meetup group held a Post-Dockercon16 meetup where the highlights of the conference were presented: Docker 1.12, Docker Datacenter, Docker Swarm Mode, Docker for AWS and Azure, DOcker for Mac and Windows.
Demo Notes: https://gist.github.com/ndemoor/7d72e0778f7a6a03b072e9b8c8e4781e
Shipping Applications to Production in Containers with DockerJérôme Petazzoni
Docker is an Open Source engine to build, run, and manage Linux Containers. Containers use less resources than virtual machines, they boot faster, but they have similar guarantees of portability and repeatability for Linux applications. Those features made Docker and Linux Containers extremely popular for development and testing environments. But what does it take to use Docker and Containers for production workloads?
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...Codemotion
In less than two years Docker went from first line of code to major Open Source project with contributions from all the big names in IT. Everyone is excited, but what's in for me - as a Dev or Ops? In short, Docker makes creating Development, Test and even Production environments an order of magnitude simpler, faster and completely portable across both local and cloud infrastructure. We will start from Docker main concepts: how to create a Linux Container from base images, run your application in it, and version your runtimes as you would with source code, and finish with a concrete example.
JDO 2019: Tips and Tricks from Docker Captain - Łukasz LachPROIDEA
This session covers a bunch of tips and tricks for getting the most out of Docker. The tips were inspired by suggestions, blogs, and presentations and everyday challenges encountered by other Docker Captains but also the members of the Docker community. Come and see the unobvious and unexpected in terms of orchestration, image creation and management, also networking and volumes!
Docker is the Open Source container engine. It lets you author, run, and manage software containers. Escape from dependency hell, and make deployment a breeze! This presentation includes the standard Docker intro (actualized for Docker 0.11) as well as some insights about how to perform orchestration and multi-host container linking.
JDD2014: Docker.io - versioned linux containers for JVM devops - Dominik DornPROIDEA
This presentation will introduce you to Docker - the new shiny star on the Devops horizon. It will teach you everything you need to know to get started with Docker, why you'd want to use it and which tools to use to get the most out of it. Additionally to showing the basics, it will introduce helpful libraries available for the JVM and how they can be used together with Docker to create secure, scalable and maintainable cloud setups for your applications.
From Monolith to Docker Distributed ApplicationsCarlos Sanchez
Docker is revolutionizing the way people think about applications and deployments. It provides a simple way to run and distribute Linux containers for a variety of use cases, from lightweight virtual machines to complex distributed micro-services architectures.
Containers allow to run services in isolation with a minimum performance penalty, increased speed, easier configuration and less complexity, making it ideal for continuous integration and continuous delivery based workloads. But migrating an existing application to a distributed microservices architecture is no easy task, requiring a shift in the software development, networking and storage to accommodate the new architecture.
We will provide insight on our experience creating a Jenkins platform based on distributed Docker containers running on Apache Mesos and Marathon, applicable for all types of applications, but specially Java and JVM based nones.
Docker is a key player in the microservices movement and is arguably the leader in containerization technology.
That said, there are many ways to “do Docker”.
Between the leading cloud providers AWS, Azure, and Google; plus other platform stacks like Docker/Swarm, Apache Mesos – DC/OS, and Kubernetes; it can get confusing. In this session, Michele will bring her customer experiences building solutions across most of these platforms – to provide you with the highlights, the architecture topologies, and some perspective on the way she helps her customers choose the right platform for their cloud, on premise or hybrid solutions.
If you're not familiar with Docker yet, here is your chance to catch up: a quick overview of the Open Source Docker Engine, and its associated services delivered through the Docker Hub. It also includes Jérôme will also discuss the new features of Docker 1.0, and briefly explain how you can run and maintain Docker on Azure. In addition, an Azure team member will demonstrate how deploy docker to Azure. The presentation will be followed by a Q&A session!
Docker - Demo on PHP Application deployment Arun prasath
Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.
In this demo, I will show how to build a Apache image from a Dockerfile and deploy a PHP application which is present in an external folder using custom configuration files.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
4. Hello!
I am Lucas Fontes
Services Team Lead at Uken Studios
I take screenshots
Spirit Animal: Left Shark
internets: @lxfontes
5. Uken Studios
Rails shop
HTML5 and Unity3d
~ 150 servers (300 peak)
- ~ 40 Databases
- ~ 70 App servers
- SOA with nsq.io
6.
7. Static server list
Lock-step commands on all servers
HAProxy config via Chef
Snowflake servers. (DB operations/scheduled
tasks)
~15 clusters
Shared HAProxy cluster
8. Bare Metal
DBs (cpu speed / memory / ssd)
Web (cpu count / nic bonding)
Tons of unused network transfer
Monthly billing upgrades
idle hardware (redundancy)
15. What if auto-scaling kicks in mid-deploy?
What if instance dies mid-deploy?
Trust chef nodes will register themselves!
Static server list
Lock-step commands on all servers
HAProxy config via Chef
Snowflake servers. (DB operations/scheduled
tasks)
19. Ops Requirements
- Not tied to AWS
- Containers / packs
- Either straight LXC, docker, rkt
- Dynamic Load Balancing
- ELB only for SSL termination
- Easy to troubleshoot
- Heavy logging, unique-ids everywhere
- Configuration Decoupling
- Not tied to chef/ansible/puppet
- DB location, worker count
- Deploy anything
- Not just rails
26. BUILD
How? Dockerfile on each repo
docker build and done
Store? Docker Registry
Target Image? repo + prod|staging
registry.uken.int/uken/titans_prod
Target Version? Git revision
registry.uken.int/uken/titans_prod:cc3e68a
27. RUN
Sealed container only
docker --rm=true
Pre release
Database Migrations / Seeds
CDN Asset upload
Post release
Ephemeral shell sessions
28. RELEASE
Upload image to docker registry
Notify hosts
registry.uken.int/uken/titans_prod:cc3e68a
Go download this version!
Docker pull + restart on hosts
30. Distributed Quorum
Agents on all hosts / load balancers
Key Value Pairs
titans_prod/version: registry.uken.int/uken/titans_prod:cc3e68a
titans_prod/env/DB_HOST: db1.uken.int
DNS + HTTP API
kinda important
Service Registration + Health Checks
web1.node.consul => 10.28.3.1
titans_prod.service.consul => 10.28.3.1 port 5001
Consul
34. Use Consul DNS API
I said it was important
Lookup by repo + environment
titans_prod.service.consul
srv1.node.consul port 5100
srv2.node.consul port 4200
srv3.node.consul port 9000
proxy_pass http://$target;
gist
39. Consul + DNS
Run a local cache + consul chain (unbound)
stub-zone:
name: "consul"
stub-addr: 127.0.0.1@8600
CNAME databases / caches on alt domain
40. Docker
Filesystem Drivers
overlay / aufs / btrfs / devicemapper
STDOUT logging
Bad for long lived containers - fixed in 1.6!
Container linking
Meh. --net=host
Need volumes during build?
docker run && docker commit
