20 mins to Faking the DevOps Unicorn by Matt williams, Datadog

•Download as PPTX, PDF•

2 likes•545 views

Something changed in job ads over the last few years: everyone wants the DevOps Unicorn. What is that and why did this happen? You probably have a good amount of what is in that description, but is there an easy way to fill in the rest of the 100%? It turns out that it is possible to fake your way to being a DevOps Unicorn. All that you need is a way to know which metrics are the most important. And to know that you need a framework that applies everywhere. No really, it's easier than you think. There is some work needed on your part, but just a few minutes is enough to get started. In this 20 minute session, we will cover what changed in the market, what the framework looks like, and how to apply it to all of the containerized applications you need to monitor.

Technology

20 minutes to faking the
DevOps Unicorn
Matthew
Williams
Evangelist @
Datadog

Section One
Item one
Item two
Item three
Agenda
Section Two
Item one
Item two
Item three
Section Three
Item one
Item two
Item three

• SaaS based infrastructure and app monitoring
• Open Source Agent
• Time series data (metrics and events)
• Processing nearly a trillion data points per day
• Intelligent Alerting
• We’re hiring! (www.datadoghq.com/careers/)
Datadog Overview

Using and Sharing the
same metrics and
measurements across
teams is key to avoiding
misunderstandings.

Culture
Automation
Metrics
Sharing
Damon Edwards &
John Willis
DevOps Days LA

Collecting data is cheap;
not having it when you
need it can be expensive

Example: NGINX Metrics
Work Metrics:
• Requests Per Second
• Dropped Connections
• Request Time
• Error Rates (4xx or 5xx)
• Success (2xx)
Resource Metrics:
• Disk I/O
• Memory
• CPU
• Queue Length

Example: NGINX Events
• Configuration Change
• Code Deployment
• Service Started / Stopped
• etc

Informative and Actionable Alerts
• Why is this important?
• What do I do about it?
• Who do I call next if I get stuck?
• EVERY ALERT MUST BE ACTIONABLE

“Monitor all containers running image web
in region us-west-2 across all availability zones
that use more than 1.5x the average memory on
c3.xlarge”

What's hot

In this talk, we will provide a 10,000-ft. overview of the key concepts, architectures, and common deployment scenarios for stateful services. We will cover the Docker volumes and available storage options in the community including ClusterHQ’s Flocker volume manager. After getting the lay of the land, we'll see these concepts in action. Starting by deploying a database container on a single node with UCP, Flocker and VolumeHub. Then, using the features of Docker Swarm and Flocker, we will then allow Swarm to automatically reschedule the stateful service along with Flocker moving its volume when the node fails giving us a HA containerized database.

Everything You Need to Know About Docker and Storage by Ryan Wallner, ClusterHQ

Docker, Inc.

Cloud Native Patterns Using AWS - Practical Examples

Anderson Carvalho

56K.cloud Docker Training

Brian Christner

Containerized Databases for Enterprise Applications Containers are now being used in organizations of all sizes. From small startups to established enterprises, data persistence is necessary in many mission critical applications. “Containers are not for database applications” is a misconception and nothing could be further from the truth. This session aims to help practitioners navigate the minefield of database containerization and avoid some of the major pitfalls that can occur. Discussion includes traditional enterprise database concerns surrounding data persistence and data security, and how they mesh with containerized deployment.

DCSF19 Containerized Databases for Enterprise Applications

Docker, Inc.

Over the last several years, we’ve seen the emergence new application architecture – dubbed “cloud native” – that is highly distributed, elastic and composable with the container as the modular compute abstraction. With that, a new breed of tools has emerged to help deploy, manage and scale these applications. Cluster management, service discovery, scheduling, etc. – terms that previously were unknown or, at best, reserved for the realm of high-performance computing – are now becoming part of every IT organization’s lexicon. As the pace of innovation continues at breakneck speed, a taxonomy to help understand the elements of this new stack is helpful. The “Cloud-Native” Ecosystem presentation is the consequence of many conversations with developers, CIOs and founders who are playing a critical role in shaping this new application paradigm. It attempts to define the discreet components of the cloud-native stack and calls out the vendors, products and projects that comprise the ecosystem. Note, this is an ever-evolving document that’s meant to be collaborative and is by no means a hardened or exhaustive industry landscape. If you have suggestions, edits and/or know of products or companies that should be included please don’t hesitate to get in touch.

'Cloud-Native' Ecosystem - Aug 2015

Lenny Pruss

Using Rancher and Docker with RightScale at Industrie IT

RightScale

Olivier meetup-boston-2013-jan-21-v2

Olivier Eeckhoutte

DevSecOps in a cloudnative world

Karthik Gaekwad

Listen as experts from Storage Switzerland and HyperGrid discuss new alternatives to bi-modal IT that allow organizations to containerize legacy applications to create a completely agile data center. In this on demand webinar you will learn: * What are Containers * Why Should You Containerize Legacy Apps * What are the Challenges of Moving Legacy Apps To Containers * How to Overcome Container Challenges

Webinar: How and Why to Containerize Your Legacy Applications

Storage Switzerland

RTI International is an independent nonprofit research institute dedicated to improving the human condition, looking into areas of crime analytics, health economics and more. RTI International’s Center for Data Science sits at the confluence of academic research and innovative technologies, developing complex statistical and analytical applications derived from abstract research. In this talk, Keith Richards of the Center for Data Science will discuss some of the Center’s novel use cases of Docker technology throughout the data lifecycle for projects such as: developing and deploying software-as-a-service crime analytics, supporting public policy creation through horizontally scaled health economics microsimulations, and managing the dependencies of several machine learning applications

DCSF 19 Improving the Human Condition with Docker

Docker, Inc.

DockerCon 2016 - Structured Container Delivery

Oscar Renalias

Elton Stoneman - Developer Advocate, Docker Legacy applications weren't designed to run in a modern distributed platform like Docker. They have their own ideas about logging, configuration and health which don't translate to the world of containers and make transformation projects hard. This session shows you how to bring your old applications into the modern world, and integrate them with Docker - without changing code. We'll cover patterns for all the core application concerns: * logging * configuration * monitoring * health * dependency management The sample applications will be in .NET and Java, and will show you how to turn your existing applications into good Docker citizens.

DCEU 18: 5 Patterns for Success in Application Transformation

Docker, Inc.

The combination of StackPointCloud with NetApp creates NetApp Kubernetes Service, the industry’s first complete Kubernetes platform for multi-cloud deployments and a complete cloud-based stack for Azure, Google Cloud, AWS, and NetApp HCI. Further, Trident is a fully supported open source project maintained by NetApp, designed from the ground up to help meet the sophisticated persistence demands of containerized applications.

Containers and Kubernetes

Altoros

10 tips for Cloud Native Security

Karthik Gaekwad

Terrascan - Cloud Native Security Tool

sangam biradar

I would like to introduce Elara, which is a centralized storage solution for building a huge container-based cloud platform and it can work as a Docker volume plugin. Elara can work with openstack (Cinder, Manila, Swift) and storages such as SAN and NAS, it has several advantages: - No central control node (Decentralized) - Microservice architecture - Supports Docker volume plugin with extended volume operations (snapshot, migration, QoS etc.) - Supports different kinds of backend storage including block device, filesystem storage and object storage - Supports storage resource scheduler (storage pools) - Easy to install and use

A New Centralized Volume Storage Solution for Docker and Container Cloud by W...

Docker, Inc.

Intro - Cloud Native

Albert Suwandhi

Empower Your Docker Containers with Watson - DockerCon 2017 Austin

Phil Estes

DockerCon 16 General Session Day 1

Docker, Inc.

ISACA SV Chapter: Securing Software Supply Chains

Jim Bugwadia

What's hot (20)

Everything You Need to Know About Docker and Storage by Ryan Wallner, ClusterHQ

Cloud Native Patterns Using AWS - Practical Examples

56K.cloud Docker Training

DCSF19 Containerized Databases for Enterprise Applications

'Cloud-Native' Ecosystem - Aug 2015

Using Rancher and Docker with RightScale at Industrie IT

Olivier meetup-boston-2013-jan-21-v2

DevSecOps in a cloudnative world

Webinar: How and Why to Containerize Your Legacy Applications

DCSF 19 Improving the Human Condition with Docker

DockerCon 2016 - Structured Container Delivery

DCEU 18: 5 Patterns for Success in Application Transformation

Containers and Kubernetes

10 tips for Cloud Native Security

Terrascan - Cloud Native Security Tool

A New Centralized Volume Storage Solution for Docker and Container Cloud by W...

Intro - Cloud Native

Empower Your Docker Containers with Watson - DockerCon 2017 Austin

DockerCon 16 General Session Day 1

ISACA SV Chapter: Securing Software Supply Chains

Viewers also liked

There are literally tens of thousands of images available today in the public DockerHub repository. These images cover almost every possible distro and common open source tool, server, or application that exists today. But there has been one drawback--all of these images assume you are on the most commonly used platform: 64-bit Intel Linux. In late 2015 and 2016 the Docker development community and Docker distribution team have enabled a new registry image specification that supports packaging multiple architecture and OS layer images within the same repository name and tag. This allows a common image, say "mysql:latest", to contain references to images for all supported OS and architectures that the packager determines to make available. Now a `docker run` or `docker pull` of "mysql:latest" will work seamlessly across all supported architectures. In this talk we will demonstrate the packaging and running of a multi-architecture containerized application on several different supported Docker platforms like ARM, POWER, and System z.

Containerize All the (Multi-Platform) Things! by Phil Estes

Docker, Inc.

Tyrion Cannister Neural Styles by Dora Korpar and Siphan Bou

Docker, Inc.

Interoperable Containers

Docker, Inc.

Dockerizing WordPress

Docker, Inc.

Docker 1.11 @ Docker SF Meetup

Docker, Inc.

DockerCon 14 Keynote Day 2

Docker, Inc.

DockerCon EU 2015: What is it we want in containers anyway?

Docker, Inc.

Docker at RelateIQ

Docker, Inc.

DockerCon SF 2015: Panel Discussion Birds of a Different Feather Soar Together

Docker, Inc.

Docker Online Meetup #3: Docker in Production

Docker, Inc.

Presented by Andrew Hately, CTO - Cloud Architecture, IBM and Scott Porter, Sr. Developer, Electronic Arts Learn how Docker can be used to achieve near bare-metal performance and a scale-out architecture that enables game backends to scale and stay responsive during load spikes. Game popularity can change with every feature and content pack release, and IBM and Electronic Arts have transitioned a mobile game engine to leverage Docker to enable rapid rollouts while handling more game users. In this session you'll learn design tips from the development of this next-gen gaming platform in an industry where user loyalty and performance are everything. Docker packaging of the game services is enabling a transition to a more flexible, micro-service based architecture, and this session will discuss the development lessons learned during that transition as well as the transition to using Docker in production.

DockerCon EU 2015: It's in the game: the path to micro-services at Electronic...

Docker, Inc.

DockerCon EU 2015: Nesting Containers: Real Life Observations

Docker, Inc.

Distributed, Real-time Web Apps

Docker, Inc.

A community is one of the key components of an open source software project. The success of an open source project like Docker is highly dependent on a large and active community. The speakers will share their experience of how to successfully build a local community by the example of how they raised a Docker community at their University (Univ. of Bamberg, Germany). They summon their best practices as a result of the mistakes they made, illustrated by story telling. This is a talk from me as a student, which is an underrepresented group at DockerCon.

How to Successfully Build a Local Docker Community by Mathias Renner

Docker, Inc.

A Gentle Introduction to Docker and Containers

Docker, Inc.

WOT Cloud Computing Architect Summit

Docker, Inc.

Introduction to Docker and Containers

Docker, Inc.

DockerCon14 Contributing to Docker by Tianon

Docker, Inc.

Securing your Containers (Meetup at Docker HQ 4/7)

Docker, Inc.

Presentation by Dr. Marius Millea, Cosmologist and Postdoctoral Fellow at the Institut Lagrange de Paris Cosmology@Home is a project which uses volunteer computing to analyze cosmological data and answer questions about our universe such as "how much dark matter is there?" and "under what conditions did the Big Bang occur?" We recently began using Docker by taking each job which we would normally send to our volunteer computers, and packaging it up inside a Docker container. The volunteer computers themselves come from interested users all over the world who download and run the software allowing them to become volunteers (called BOINC). The system is working exceedingly well and using Docker has made it massively easier for us to develop and run it. I will explain some of the technical details of the implementation, which involves a customized boot2docker ISO, as well give a brief summary of the scientific questions we are trying to answer and how these results made possible by Docker are helping analyze data from, e.g. the European Space Agency's Planck satellite.

DockerCon EU 2015: Finding a Theory of the Universe with Docker and Volunteer...

Docker, Inc.

Viewers also liked (20)

Containerize All the (Multi-Platform) Things! by Phil Estes

Tyrion Cannister Neural Styles by Dora Korpar and Siphan Bou

Interoperable Containers

Dockerizing WordPress

Docker 1.11 @ Docker SF Meetup

DockerCon 14 Keynote Day 2

DockerCon EU 2015: What is it we want in containers anyway?

Docker at RelateIQ

DockerCon SF 2015: Panel Discussion Birds of a Different Feather Soar Together

Docker Online Meetup #3: Docker in Production

DockerCon EU 2015: It's in the game: the path to micro-services at Electronic...

DockerCon EU 2015: Nesting Containers: Real Life Observations

Distributed, Real-time Web Apps

How to Successfully Build a Local Docker Community by Mathias Renner

A Gentle Introduction to Docker and Containers

WOT Cloud Computing Architect Summit

Introduction to Docker and Containers

DockerCon14 Contributing to Docker by Tianon

Securing your Containers (Meetup at Docker HQ 4/7)

DockerCon EU 2015: Finding a Theory of the Universe with Docker and Volunteer...

Similar to 20 mins to Faking the DevOps Unicorn by Matt williams, Datadog

Think you have big data? What about high availability requirements? At DataDog we process billions of data points every day including metrics and events, as we help the world monitor the their applications and infrastructure. Being the world’s monitoring system is a big responsibility, and thanks to Redis we are up to the task. Join us as we discuss how the DataDog team monitors and scales Redis to power our SaaS based monitoring offering. We will discuss our usage and deployment patterns, as well as dive into monitoring best practices for production Redis workloads

Monitoring and Scaling Redis at DataDog - Ilan Rabinovitch, DataDog

Redis Labs

On 2020-12-09 Laurens Vijnck and Jonny Daenen gave a workshop at PXL. During this session, we collectively provisioned a streaming ingestion pipeline in mere minutes. The technology stack included Pub/Sub, Dataflow, and BigQuery. Hereafter, students had the opportunity to perform interactive queries on their own real-time data to answer a series of business questions. These questions were borrowed from real-life cases that we encountered at Selligent Marketing Cloud. Google Colab (Free Jupyter Notebooks) and Google Data Studio have proven to be excellent tools to facilitate these kinds of interactive sessions.

PXL Data Engineering Workshop By Selligent

Jonny Daenen

How to Use Big Data to Transform IT Operations

ExtraHop Networks

The presentation topic for this meet-up was covered in two sections without any breaks in-between Section 1: Business Aspects (20 mins) Speaker: Rasmi Mohapatra, Product Owner, Experian https://www.linkedin.com/in/rasmi-m-428b3a46/ Once your data science application is in the production, there are many typical data science operational challenges experienced today - across business domains - we will cover a few challenges with example scenarios Section 2: Tech Aspects (40 mins, slides & demo, Q&A ) Speaker: Santanu Dey, Solution Architect, Iguazio https://www.linkedin.com/in/santanu/ In this part of the talk, we will cover how these operational challenges can be overcome e.g. automating data collection & preparation, making ML models portable & deploying in production, monitoring and scaling, etc. with relevant demos.

Challenges of Operationalising Data Science in Production

iguazio

Shikha fdp 62_14july2017

Dr. Shikha Mehta

The key to unlocking the Value in the IoT? Managing the Data!

DataWorks Summit/Hadoop Summit

Watch full webinar here: https://bit.ly/2EpHGyd Presented at Data Champions, Online Asia 2020 Businesses and individuals around the world are experiencing the impact of a global pandemic. With many workers and potential shoppers still sequestered, COVID-19 is proving to have a momentous impact on the global economy. Regardless of the current situation and post-pandemic era, real-time data becomes even more critical to healthcare practitioners, business owners, government officials, and the public at large where holistic and timely information are important to make quick decisions. It enables doctors to make quick decisions about where to focus the care, business owners to alter production schedules to meet the demand, government agencies to contain the epidemic, and the public to be informed about prevention. In this on-demand session, you will learn about the capabilities of data virtualization as a modern data integration technique and how can organisations: - Rapidly unify information from disparate data sources to make accurate decisions and analyse data in real-time - Build a single engine for security that provides audit and control by geographies - Accelerate delivery of insights from your advanced analytics project

A Key to Real-time Insights in a Post-COVID World (ASEAN)

Denodo

DBTA Data Summit : Eliminating the data constraint in Application Development

Kyle Hailey

Monitoring Containerized Micro-Services In Azure

Alex Bulankou

Data in Motion - tech-intro-for-paris-hackathon

Cisco DevNet

Morningstar’s Risk Model project is created by stitching together statistical and machine learning models to produce risk and performance metrics for millions of financial securities. Previously, we were running a single version of this application, but needed to expand it to allow for customizations based on client demand. With the goal of running hundreds of custom Risk Model runs at once at an output size of around 1TB of data each, we had a challenging technical problem on our hands! In this presentation, we’ll talk about the challenges we faced replatforming this application to Spark, how we solved them, and the benefits we saw. Some things we’ll touch on include how we created customized models, the architecture of our machine learning application, how we maintain an audit trail of data transformations (for rigorous third party audits), and how we validate the input data our model takes in and output data our model produces. We want the attendees to walk away with some key ideas of what worked for us when productizing a large scale machine learning platform.

Lessons Learned Replatforming A Large Machine Learning Application To Apache ...

Databricks

FullStack London July 15th, 2016 Monitoring is complicated, and in most organizations consists of far too many tools owned by many teams. These tools consist of monitoring tools each looking at a component myopically. These tools metrics and logs from devices and software emitting them. Increasingly modern companies are creating their own instrumentation, but there is a large base of generic instrumentation of software. Fixing monitoring issues requires people, process, and technology. In this talk we will cover many common issues seen in the real world. For example decisions on what should be monitored or collected from a technology and a business perspective. This requires process and coordination. We will investigate what instrumentation is most scalable and effective across languages this includes the commonly used APIs and possibilities to capture data from common languages like Java, .NET and PHP, but we’ll also go into methods which work with Python, Node.js, and golang. We will cover browser and mobile instrumentation techniques. How these are done? which APIs are being used? What open source tools and frameworks can be leveraged? Most importantly how to coordinate and communicate requirements across your organization. Attendees of this session will walk away with a clear understanding of: What is instrumentation, and what do I instrument, collect, and store? The understanding of overhead and how this can be accomplished on common software stacks? How to work with application owners to collect business data. How correlation works in custom open source or packaged monitoring tools.

The differing ways to monitor and instrument

Jonah Kowall

Supporting Contractors with NIST SP 800-171 Compliance

SolarWinds

IT OT Integration_Vishnu_Murali_05262016_UPDATED

Vishnu Murali

Monitoring in 2017 - TIAD Camp Docker

The Incredible Automation Day

It's harder than ever to predict the load your application will need to handle in advance, so how do you design your architecture so you can afford to implement as you go and be ready for whatever comes your way. It's easy to focus on optimizing each part of your application but your application architecture determines the options you have to make big leaps in scalability. In this talk we'll cover practical patterns you can build today to meet the needs of rapid development while still creating systems that can scale up and out. Specific code examples will focus on .NET but the principles apply across many technologies. Real world systems will be discussed based on our experience helping customers around the world optimize their enterprise applications.

Scaling Systems: Architectures that grow

Gibraltar Software

Séminaire Big Data Alter Way - Elasticsearch - octobre 2014

ALTER WAY

Msst 2019 v4

Nisha Talagala

Charles Mahler [InfluxData] | Use Case: Networking Monitoring | InfluxDays 2022

InfluxData

Machine Learning & Predictive Maintenance

Arnab Biswas

Similar to 20 mins to Faking the DevOps Unicorn by Matt williams, Datadog (20)

Monitoring and Scaling Redis at DataDog - Ilan Rabinovitch, DataDog

PXL Data Engineering Workshop By Selligent

How to Use Big Data to Transform IT Operations

Challenges of Operationalising Data Science in Production

Shikha fdp 62_14july2017

The key to unlocking the Value in the IoT? Managing the Data!

A Key to Real-time Insights in a Post-COVID World (ASEAN)

DBTA Data Summit : Eliminating the data constraint in Application Development

Monitoring Containerized Micro-Services In Azure

Data in Motion - tech-intro-for-paris-hackathon

Lessons Learned Replatforming A Large Machine Learning Application To Apache ...

The differing ways to monitor and instrument

Supporting Contractors with NIST SP 800-171 Compliance

IT OT Integration_Vishnu_Murali_05262016_UPDATED

Monitoring in 2017 - TIAD Camp Docker

Scaling Systems: Architectures that grow

Séminaire Big Data Alter Way - Elasticsearch - octobre 2014

Msst 2019 v4

Charles Mahler [InfluxData] | Use Case: Networking Monitoring | InfluxDays 2022

Machine Learning & Predictive Maintenance

More from Docker, Inc.

Raymond Arifianto, AccelByte and Mark Mandel, Google - We have been deploying containerized micro-services for our Game Backend Services for a while. Now we are tackling the challenge to scale up fleets of game dedicated servers in multiple regions, multiple data centers and multiple providers - some in bare metal, some in Cloud. So we leverage docker containerization to deploy Game Servers to achieve Portability, Fast Deployment and Predictability, enabling us to scale up to thousands of servers, on demand, without a sweat.

Containerize Your Game Server for the Best Multiplayer Experience

Docker, Inc.

Nicholas Dille, Haufe-Lexware + Docker Captain - Docker continues to be the standard tool for building container images. For more than a year Docker ships with BuildKit as an alternative image builder, providing advanced features for secret and cache management. These features help to make image builds faster and more secure. In this session, Docker Captain Nicholas Dille will teach you how to use Buildkit features to your advantage.

How to Improve Your Image Builds Using Advance Docker Build

Docker, Inc.

Lukonde Mwila, Entelect - As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud. In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics: - Docker Compose - Containerizing an Nginx Server - Containerizing an React App - Containerizing an Node.JS App - Containerizing anMongoDB App - Runing Multi-Container App Locally - Creating a CI/CD Pipeline - Adding a build stage to test containers and push images to Docker Hub - Deploying Multi-Container App to AWS Elastic Beanstalk Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking. After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.

Build & Deploy Multi-Container Applications to AWS

Docker, Inc.

Kevin Jones, NGNIX - NGINX is one of the most popular images on Docker Hub and has been at the forefront of the web since the early 2000's. In this talk we will discuss how and why NGINX's lightweight and powerful architecture makes it a very popular choice for securing containerized applications as a sidecar reverse proxy within containers. We will highlight important aspects of application security that NGINX can help with, such as TLS, HTTP, AuthN, AuthZ and traffic control.

Securing Your Containerized Applications with NGINX

Docker, Inc.

Kathleen Juell, Digital Ocean - Containers are an essential part of today's microservice ecosystem, as they allow developers and operators to maintain standards of reliability and reproducibility in fast-paced deployment scenarios. And while there are best practices that extend across stacks in containerized environments, there are also things that make each stack distinct, starting with the application image itself. This talk will dive into some of these particularities, both at the image and service level, while also covering general best practices for building and running Node applications with database backends using Docker and Compose.

How To Build and Run Node Apps with Docker and Compose

Docker, Inc.

Hands-on Helm

Docker, Inc.

Jeff Hajewski, Salesforce - There is a wealth of information on building deep learning models with PyTorch or TensorFlow. Anyone interested in building a deep learning model is only a quick search away from a number of clear and well written tutorials that will take them from zero knowledge to having a working image classifier. But what happens when you need to deploy these models in a production setting? At Salesforce, we use TensorFlow models to help us provide customers with insights into their data, and we do this as close to real-time as possible. Designing these systems in a scalable manner requires overcoming a number of design challenges, but the core component is Docker. Docker enables us to design highly scalable systems by allowing us to focus on service interactions, rather than how our services will interact with the hardware. Docker is also at the core of our test infrastructure, allowing developers and data scientists to build and test the system in an end to end manner on their local machines. While some of this may sound complex, the core message is simplicity - Docker allows us to focus on the aspects of the system that matter, greatly simplifying our lives.

Distributed Deep Learning with Docker at Salesforce

Docker, Inc.

James Fuller, webcomposite s.r.o. - Curl is the venerable (yet very modern) 'swiss army knife' command line tool and library for transferring data with URLs. Recently we (the Curl team) decided to build a release for Docker Hub. This talk will outline our current development workflow with respect to the docker image and provide insights on what it takes to build a docker image for mass public consumption. We are also keen to learn from users and other developers how we might improve and enhance the official curl docker image.

The First 10M Pulls: Building The Official Curl Image for Docker Hub

Docker, Inc.

Fabian Stäber, Instana - In recent years, we saw a great paradigm shift in software engineering away from static monolithic applications towards dynamic distributed horizontally scalable architectures. Docker is one of the key technologies enabling this development. This shift poses a lot of new challenges for application monitoring, ranging from practical issues (need for automation) to technical challenges (Docker networking) to organizational topics (blurring line between software engineers and operations) to fundamental questions (define what is an application). In this talk we show how Docker changed the way we do monitoring, how modern application monitoring systems work, and what future developments we expect.

Monitoring in a Microservices World

Docker, Inc.

Clemente Biondo, Engineering Ingegneria Informatica - When the COVID 19 pandemic started, Engineering Ingegneria Informatica Group (1.25 billion euros of revenues, 65 offices around the world, 12.000 employees) was forced to put their digital transformation to the test in order to maintain operational continuity. In this session, Clemente Biondo, the Tech Lead of the Information Systems Department, will share how his company is reacting to this unforeseeable scenario and how Docker-driven digital transformation had paved the path for work to continue remotely. Clemente will discuss learnings moving from colocated teams, manual approaches, email based-business processes, and a monolithic application to a mature DevOps culture characterized by a distributed autonomous workforce and a continuous deployment process that deploys backward-compatible Docker containerized microservices into hybrid multi cloud datacenters an average of twice a day with zero-downtime. He will detail how they use Docker to unify dev, test and production environments, and as an efficient and automated mechanism for deploying applications. Lastly, Clemente shares how, in our darkest hour, he and others are working to shine their brightest light.

COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...

Docker, Inc.

Chris Lauer, NOAA Space Weather Prediction Center - This is the story of how adopting a containerized workflow changed the way our small software team works at NOAA’s Space Weather Prediction Center. Our old architecture, a big ball of mud shared-database integration, just wasn’t cutting it - it was killing our agility. Over the past two years, our small team has adopted a microservice style architecture, using Docker with docker-compose and environment files as our deployment strategy for all new development. We’ve discovered the joys of using containers for identical dev, staging, and production environments. We work closely with scientists: much of the code we’re running has complicated and conflicting library dependencies. Docker captures these beautifully - we’ve even had some success teaching our scientists to use it! I’ll share what we’ve learned, some of the persistent challenges we face, and one place we really got it wrong. This talk builds off of a popular hallway track from DockerCon 2019.

Predicting Space Weather with Docker

Docker, Inc.

Brian Christner, 56k + Docker Captain - In this session, we will unlock the full potential of using Microsoft Visual Studio Code (VS Code) and Docker Desktop to turn you into a Docker Power User. When we expand and utilize the VS Code Docker plugin, we can take our projects and Docker skills to the next level. In addition to using VS Code, we streamline our Docker Desktop development workflow with less context switching and built-in shortcuts. You will learn how to bootstrap new projects, quickly write Dockerfiles utilizing templates, build, run, and interact with containers all from VS Code.

Become a Docker Power User With Microsoft Visual Studio Code

Docker, Inc.

How to Use Mirroring and Caching to Optimize your Container Registry

Docker, Inc.

Ashish Sharma, SS&C Eze - SS&C Eze provides various products in the stock market domain. We spent the last couple of years building Eclipse which is an investment suite born in cloud. The journey so far has been very interesting. The very first version of the product were a bunch of monolithic windows services and deployed using Octopus tool. We successfully managed to bring all the monolithic problem to the cloud and created a nightmare for ourselves. We then started applying microservices architecture principles and started breaking the monolithic into small services. Very soon we realized that we need a better packaging/deployment tool. Docker looked like a magical solution to our problem. Since its adoption, It has not only solved the deployment problem for us but has made a deep impact on different aspects of SDLC. It allowed us to use heterogeneous technology stacks, simplified development environment setup, simplified our testing strategy, improved our speed of delivery, and made our developers more productive. In this talk I would like to share our experience of using Docker and its positive impact on our SDLC.

Monolithic to Microservices + Docker = SDLC on Steroids!

Docker, Inc.

Ara Pulido, Datadog - Container technologies, although not new, have increased their popularity in the past few years, with container orchestrators allowing companies around the world to adopt these technologies to help them ship and scale microservices with precision and velocity. Kubernetes is currently the most popular container orchestration platform, and while many organizations are migrating their workloads to it, Kubernetes is still relatively immature. New corner cases, errors, and quirks are regularly discovered as users push the boundaries of size and scale. When Datadog adopted Kubernetes we discovered some of these boundaries the hard way, and we continuously challenge and modify our infrastructure decisions in order to fit our use case. Join me in this talk for our story on what we learned while we scaled our Kubernetes clusters, the contributions to Kubernetes we made along the way, and how you can apply those learnings when growing your Kubernetes clusters from a handful to hundreds or thousands of nodes.

Kubernetes at Datadog Scale

Docker, Inc.

Andy Clemenko, StackRox - One underutilized, and amazing, thing about the docker image scheme is labels. Labels are a built in way to document all aspects about the image itself. Think about all the information that the tags inside your clothing carry. If you care to look you can find out everything about the garment. All that information can be very valuable. Now think about how we can leverage labels to carry similar information. We can even use the labels to contain Docker Compose or even Kubernetes Yaml. We can even include labels into the CI/CD process making things more secure and smoother. Come find out some fun techniques on how to leverage labels to do some fun and amazing things.

Labels, Labels, Labels

Docker, Inc.

Patrick Deloulay, Micro Focus - Micro Focus started their digital transformation 3 years ago, moving the entire portfolio into hundreds of container images. Leveraging Docker Hub as our primary registry service, we will cover how we ended up building a simple but secure push/pull model to publish and deliver our premium assets to our customers and partners to both meet the high agility of our DevOps teams while greatly simplifying the deployment of our applications.

Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model

Docker, Inc.

Lukonde Mwila, Entelect As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud. In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics: - Docker Compose - Containerizing an Nginx Server - Containerizing an React App - Containerizing an Node.JS App - Containerizing anMongoDB App - Runing Multi-Container App Locally - Creating a CI/CD Pipeline - Adding a build stage to test containers and push images to Docker Hub - Deploying Multi-Container App to AWS Elastic Beanstalk Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking. After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.

Build & Deploy Multi-Container Applications to AWS

Docker, Inc.

Elton Stoneman, Docker Captain + Container Consultant and Trainer How do you provide a SaaS offering when your product is a 10-year old Fortran app, currently built to run on Windows 10? With Docker and Kubernetes of course - and you can do it in a week (... to prototype level at least). In this session I'll walk through the processes and practicalities of taking an older Windows app, making it run in containers with Kubernetes, and then building a simple API wrapper to host the whole stack as a cloud-based SaaS product. There's a lot of technology here from a real world case study, and I'll focus on: - running Windows apps in Docker containers - building a .NET Core API which can run in Linux or Windows containers - running the stack in Kubernetes with Docker Desktop locally and AKS in the cloud - configuring AKS workloads in Azure to burst out to Azure Container Instances And there's a core theme to this session: Docker and Kubernetes are complex technologies, but they're the key to modern development. If you invest time learning them, they make projects like this simple, portable, fast and fun.

From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...

Docker, Inc.

This virtual meetup introduces the concepts and best practices of using Docker containers for software development for the Arm architecture across a variety of hardware systems. Using Docker Desktop on Windows or Mac, Amazon Web Services (AWS) A1 instances, and embedded Linux, we will demonstrate the latest Docker features to build, share, and run multi-architecture images with transparent support for Arm.

Developing with Docker for the Arm Architecture

Docker, Inc.

More from Docker, Inc. (20)

Containerize Your Game Server for the Best Multiplayer Experience

How to Improve Your Image Builds Using Advance Docker Build

Build & Deploy Multi-Container Applications to AWS

Securing Your Containerized Applications with NGINX

How To Build and Run Node Apps with Docker and Compose

Hands-on Helm

Distributed Deep Learning with Docker at Salesforce

The First 10M Pulls: Building The Official Curl Image for Docker Hub

Monitoring in a Microservices World

COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...

Predicting Space Weather with Docker

Become a Docker Power User With Microsoft Visual Studio Code

How to Use Mirroring and Caching to Optimize your Container Registry

Monolithic to Microservices + Docker = SDLC on Steroids!

Kubernetes at Datadog Scale

Labels, Labels, Labels

Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model

Build & Deploy Multi-Container Applications to AWS

From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...

Developing with Docker for the Arm Architecture

Recently uploaded

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on the deployment of external web forms using Jotform for Bonterra Impact Management. This solution can be customized to your organization’s needs and deployed to support the common use cases below: - Intake and consent - Assessments - Surveys - Applications - Program registration Interested in deploying web form automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Jeffrey Haguewood

MINDCTI Revenue Release Quarter One 2024

MIND CTI

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

Six Myths about Ontologies: The Basics of Formal Ontology

johnbeverley2021

JohnPollard-hybrid-app-RailsConf2024.pptx

JohnPollard37

AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)

Samir Dash

Discover the innovative features and strategic vision that keep WSO2 an industry leader. Explore the exciting 2024 roadmap of WSO2 API management, showcasing innovations, unified APIM/APK control plane, natural language API interaction, and cloud native agility. Discover how open source solutions, microservices architecture, and cloud native technologies unlock seamless API management in today's dynamic landscapes. Leave with a clear blueprint to revolutionize your API journey and achieve industry success!

WSO2's API Vision: Unifying Control, Empowering Developers

WSO2

The microservices honeymoon is over. When starting a new project or revamping a legacy monolith, teams started looking for alternatives to microservices. The Modular Monolith, or 'Modulith', is an architecture that reaps the benefits of (vertical) functional decoupling without the high costs associated with separate deployments. This talk will delve into the advantages and challenges of this progressive architecture, beginning with exploring the concept of a 'module', its internal structure, public API, and inter-module communication patterns. Supported by spring-modulith, the talk provides practical guidance on addressing the main challenges of a Modultith Architecture: finding and guarding module boundaries, data decoupling, and integration module-testing. You should not miss this talk if you are a software architect or tech lead seeking practical, scalable solutions. About the author With two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Victor Rentea

TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....

rightmanforbloodline

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Zilliz

💥 You’re lucky! We’ve found two different (lead) developers that are willing to share their valuable lessons learned about using UiPath Document Understanding! Based on recent implementations in appealing use cases at Partou and SPIE. Don’t expect fancy videos or slide decks, but real and practical experiences that will help you with your own implementations. 📕 Topics that will be addressed: • Training the ML-model by humans: do or don't? • Rule-based versus AI extractors • Tips for finding use cases • How to start 👨‍🏫👨‍💻 Speakers: o Dion Morskieft, RPA Product Owner @Partou o Jack Klein-Schiphorst, Automation Developer @Tacstone Technology

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

UiPathCommunity

Dubai, known for its towering skyscrapers, luxurious lifestyle, and relentless pursuit of innovation, often finds itself in the global spotlight. However, amidst the glitz and glamour, the emirate faces its own set of challenges, including the occasional threat of flooding. In recent years, Dubai has experienced sporadic but significant floods, disrupting normalcy and posing unique challenges to its infrastructure. Among the critical nodes in this bustling metropolis is the Dubai International Airport, a vital hub connecting the world. This article delves into the intersection of Dubai flood events and the resilience demonstrated by the Dubai International Airport in the face of such challenges.

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Orbitshub

Architecting Cloud Native Applications

WSO2

Introduction to use of FHIR Documents in ABDM

Kumar Satyam

Following the popularity of "Cloud Revolution: Exploring the New Wave of Serverless Spatial Data," we're thrilled to announce this much-anticipated encore webinar. In this sequel, we'll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you're building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Safe Software

In the dynamic field of DevOps, the quest for efficiency and productivity is endless. This talk introduces a revolutionary toolkit: Large Language Models (LLMs), including ChatGPT, Gemini, and Claude, extending far beyond traditional coding assistance. We'll explore how LLMs can automate not just code generation, but also transform day-to-day operations such as crafting compelling cover letters for TPS reports, streamlining client communications, and architecting innovative DevOps solutions. Attendees will learn effective prompting strategies and examine real-life use cases, demonstrating LLMs' potential to redefine productivity in the DevOps landscape. Join us to discover how to harness the power of LLMs for a comprehensive productivity boost across your DevOps activities.

ChatGPT and Beyond - Elevating DevOps Productivity

VictorSzoltysek

At its core, the challenge of managing Human Resources data is an integration challenge: estimates range from 2-3 HR systems in use at a typical SMB, up to a few dozen systems implemented amongst enterprise HR departments, and these systems seldom integrate seamlessly between themselves. Providing a multi-tenant, cloud-native solution to integrate these hundreds of HR-related systems, normalize their disparate data models and then render that consolidated information for stakeholder decision making has been a substantial undertaking, but one significantly eased by leveraging Ballerina. In this session, we’ll cover: The overall software architecture for VHR’s Cloud Data Platform Critical decision points leading to adoption of Ballerina for the CDP Ballerina’s role in multiple evolutionary steps to the current architecture Roadmap for the CDP architecture and plans for Ballerina WSO2’s partnership in bringing continual success for the CD

Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform

WSO2

The presentation was made in “Web3 Fusion: Embracing AI and Beyond” is more than a conference; it's a journey into the heart of digital transformation. The conference a provided a platform where the future of technology meets practical application. This three-day hybrid event, set in the heart of innovation, served as a gateway to the latest trends and transformative discussions in AI, Blockchain, IoT, AR/VR, and their collective impact on the information space.

AI in Action: Real World Use Cases by Anitaraj

AnitaRaj43

Decarbonising Commercial Real Estate: The Role of Operational Performance

IES VE

WSO2 Micro Integrator for Enterprise Integration in a Decentralized, Microser...

WSO2

Recently uploaded (20)

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

MINDCTI Revenue Release Quarter One 2024

Why Teams call analytics are critical to your entire business

Six Myths about Ontologies: The Basics of Formal Ontology

JohnPollard-hybrid-app-RailsConf2024.pptx

AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)

WSO2's API Vision: Unifying Control, Empowering Developers

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Architecting Cloud Native Applications

Introduction to use of FHIR Documents in ABDM

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

ChatGPT and Beyond - Elevating DevOps Productivity

Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform

AI in Action: Real World Use Cases by Anitaraj

Decarbonising Commercial Real Estate: The Role of Operational Performance

WSO2 Micro Integrator for Enterprise Integration in a Decentralized, Microser...

20 mins to Faking the DevOps Unicorn by Matt williams, Datadog

1. 20 minutes to faking the DevOps Unicorn Matthew Williams Evangelist @ Datadog

2. Section One Item one Item two Item three Agenda Section Two Item one Item two Item three Section Three Item one Item two Item three

3. • SaaS based infrastructure and app monitoring • Open Source Agent • Time series data (metrics and events) • Processing nearly a trillion data points per day • Intelligent Alerting • We’re hiring! (www.datadoghq.com/careers/) Datadog Overview

6. Using and Sharing the same metrics and measurements across teams is key to avoiding misunderstandings.

7. Culture Automation Metrics Sharing Damon Edwards & John Willis DevOps Days LA

8. Collecting data is cheap; not having it when you need it can be expensive

9. Instrument all the things!

10.

11.

12.

13.

14.

15.

16. Example: NGINX Metrics Work Metrics: • Requests Per Second • Dropped Connections • Request Time • Error Rates (4xx or 5xx) • Success (2xx) Resource Metrics: • Disk I/O • Memory • CPU • Queue Length

17. Example: NGINX Events • Configuration Change • Code Deployment • Service Started / Stopped • etc

18.

19.

20.

21.

22. Cryptic Alerts

23. Cryptic Alerts

24. Informative and Actionable Alerts • Why is this important? • What do I do about it? • Who do I call next if I get stuck? • EVERY ALERT MUST BE ACTIONABLE

25.

26.

27.

28. “Monitor all containers running image web in region us-west-2 across all availability zones that use more than 1.5x the average memory on c3.xlarge”

29. Thank you!

20 mins to Faking the DevOps Unicorn by Matt williams, Datadog

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to 20 mins to Faking the DevOps Unicorn by Matt williams, Datadog

Similar to 20 mins to Faking the DevOps Unicorn by Matt williams, Datadog (20)

More from Docker, Inc.

More from Docker, Inc. (20)

Recently uploaded

Recently uploaded (20)

20 mins to Faking the DevOps Unicorn by Matt williams, Datadog