Monitoring your shiny new docker environment
•
0 likes•169 views
You’ve spent considerable time picking your orchestrator, choosing the right cloud provider and configuring all the intricate details of your new Docker environment, but what about monitoring? In this talk we will cover the tools available on the market: upsides, downsides and upcoming changes. We’ll open the floor to questions, comments and feedback for each tool, so you have a complete view on the monitoring landscape.
Report
Share
Report
Share
Recommended
State of Ethereum, and Mining
This document summarizes the state of Ethereum and cryptocurrency mining in 2014. It discusses features enabled by Ethereum like decentralized applications and smart contracts. It outlines progress made in technical areas like client implementations and applications built on Ethereum. It also discusses challenges remaining around efficiency, security, and client implementations. The document concludes that Ethereum is at the halfway point and more work is needed to address remaining issues.
Windows Azure Tables e NoSQL
This document provides information about Windows Azure Storage. It notes that Windows Azure Storage stores over 9 trillion objects and can handle over 900,000 requests per second on average, totaling 2.5 trillion requests per month. It then describes the different data storage options in Windows Azure Storage including Blobs, Tables, and Queues. It also provides URLs for accessing these different data stores and links to documentation and blog posts about Windows Azure Storage.
nebulaconf
This document discusses Pedro Dias's experience with OpenNebula. It describes how he used OpenNebula at his hosting company PTisp to replace their previous IaaS solution. OpenNebula met their requirements of being open, avoiding vendor lock-in, and having a low maintenance stack. The document outlines some customizations made to OpenNebula, such as using LVM instead of cLVM and developing tools like ONE Janitor to help with storage and VM sanitization. It also mentions contributions to the OpenNebula project and developing a Node.js addon and multi-tenant web panel called Cloudash.
Cassandra meetup slides - Oct 15 Santa Monica Coloft
This document summarizes Shift.com's migration from MongoDB to Cassandra. Shift is a platform that enables marketers to communicate across organizations. The initial database stack included MongoDB, but it was replaced with Cassandra for better operational benefits like easier node management, better control of data storage, and improved long-term scalability. The migration goals were zero downtime and no loss of performance. The strategy involved carefully structuring the Cassandra data model and schema to match MongoDB's performance. Benefits of Cassandra included its familiar CQL query language and improved support for features like time series data storage.
Proteon - DevOps Live 2019 - OpenShift Pitfalls
In these slides Alwyn explains various pitfalls to look out for in OpenShift, and our recommended approach.
FOSSAsia 2016 - Shared storage management in the virtualization world
This document discusses storage management in oVirt virtualization. It introduces oVirt, which manages virtualization resources including storage. It explains oVirt storage concepts like images, storage domains, and storage pools. It then covers shared storage management, describing the Storage Pool Manager which performs shared storage operations and can be a single point of failure, and the Storage Domain Manager architecture which separates metadata and data operations for better load balancing without single points of failure.
Fluentd Intro for OpenShift Commons Briefing
This document discusses logging in cloud native environments and introduces Fluentd and Fluent Bit as solutions. Fluentd is well-suited for log aggregation due to its pluggable architecture, reliability, and integration with Docker and Kubernetes. Fluent Bit is designed as a lightweight log forwarder that can send logs from edge nodes to Fluentd for aggregation. Using Fluent Bit for forwarding reduces resource usage on edge nodes. Both support features important for cloud environments like backpressure handling.
SqliteToRealm
This document discusses migrating an Android app's database from SQLite to Realm. It describes how the author modified their webtoon app source code on GitHub to replace SQLite components like SqlBrite with Realm and related libraries. Performance tests showed improvements like reduced insertion and search times when using Realm compared to the original SQLite implementation. Some challenges mentioned include Realm's larger APK size, potential out of memory errors with large transactions, and ensuring compatibility with different Realm versions. Overall, the document recommends considering Realm from the beginning of a new project rather than migrating later.
Recommended
State of Ethereum, and Mining
This document summarizes the state of Ethereum and cryptocurrency mining in 2014. It discusses features enabled by Ethereum like decentralized applications and smart contracts. It outlines progress made in technical areas like client implementations and applications built on Ethereum. It also discusses challenges remaining around efficiency, security, and client implementations. The document concludes that Ethereum is at the halfway point and more work is needed to address remaining issues.
Windows Azure Tables e NoSQL
This document provides information about Windows Azure Storage. It notes that Windows Azure Storage stores over 9 trillion objects and can handle over 900,000 requests per second on average, totaling 2.5 trillion requests per month. It then describes the different data storage options in Windows Azure Storage including Blobs, Tables, and Queues. It also provides URLs for accessing these different data stores and links to documentation and blog posts about Windows Azure Storage.
nebulaconf
This document discusses Pedro Dias's experience with OpenNebula. It describes how he used OpenNebula at his hosting company PTisp to replace their previous IaaS solution. OpenNebula met their requirements of being open, avoiding vendor lock-in, and having a low maintenance stack. The document outlines some customizations made to OpenNebula, such as using LVM instead of cLVM and developing tools like ONE Janitor to help with storage and VM sanitization. It also mentions contributions to the OpenNebula project and developing a Node.js addon and multi-tenant web panel called Cloudash.
Cassandra meetup slides - Oct 15 Santa Monica Coloft
This document summarizes Shift.com's migration from MongoDB to Cassandra. Shift is a platform that enables marketers to communicate across organizations. The initial database stack included MongoDB, but it was replaced with Cassandra for better operational benefits like easier node management, better control of data storage, and improved long-term scalability. The migration goals were zero downtime and no loss of performance. The strategy involved carefully structuring the Cassandra data model and schema to match MongoDB's performance. Benefits of Cassandra included its familiar CQL query language and improved support for features like time series data storage.
Proteon - DevOps Live 2019 - OpenShift Pitfalls
In these slides Alwyn explains various pitfalls to look out for in OpenShift, and our recommended approach.
FOSSAsia 2016 - Shared storage management in the virtualization world
This document discusses storage management in oVirt virtualization. It introduces oVirt, which manages virtualization resources including storage. It explains oVirt storage concepts like images, storage domains, and storage pools. It then covers shared storage management, describing the Storage Pool Manager which performs shared storage operations and can be a single point of failure, and the Storage Domain Manager architecture which separates metadata and data operations for better load balancing without single points of failure.
Fluentd Intro for OpenShift Commons Briefing
This document discusses logging in cloud native environments and introduces Fluentd and Fluent Bit as solutions. Fluentd is well-suited for log aggregation due to its pluggable architecture, reliability, and integration with Docker and Kubernetes. Fluent Bit is designed as a lightweight log forwarder that can send logs from edge nodes to Fluentd for aggregation. Using Fluent Bit for forwarding reduces resource usage on edge nodes. Both support features important for cloud environments like backpressure handling.
SqliteToRealm
This document discusses migrating an Android app's database from SQLite to Realm. It describes how the author modified their webtoon app source code on GitHub to replace SQLite components like SqlBrite with Realm and related libraries. Performance tests showed improvements like reduced insertion and search times when using Realm compared to the original SQLite implementation. Some challenges mentioned include Realm's larger APK size, potential out of memory errors with large transactions, and ensuring compatibility with different Realm versions. Overall, the document recommends considering Realm from the beginning of a new project rather than migrating later.
Fluent Bit: Log Forwarding at Scale
This document summarizes a presentation about log forwarding at scale. It discusses how logging works internally and requires understanding the logging pipeline of parsing, filtering, buffering and routing logs. It then introduces Fluent Bit as a lightweight log forwarder that can be used to cheaply forward logs from edge nodes to log aggregators in a scalable way, especially in cloud native environments like Kubernetes. Hands-on demos show how Fluent Bit can parse and add metadata to Kubernetes logs.
Netflix Titus WASP October 2017
This document discusses Netflix's use of containers and the Titus container management platform. It provides the following key points:
1. Titus provides container scheduling, resource management, and execution capabilities. It helps Netflix achieve consistent developer experiences, faster innovation, and simpler deployments.
2. Titus has scaled to support over 1,000,000 daily container launches across multiple AWS regions. It integrates containers into Netflix's infrastructure and provides features like VPC networking and AWS integration.
3. Looking ahead, Netflix aims to further improve Titus' performance, operations, reliability, and scheduling capabilities to support even larger container workloads. Deeper security isolation and integration with services like ALB are also goals.
Dynamic routing in microservice oriented architecture
When splitting an application into different micro-services and each application access URL is dynamically generated, the hell gets loose. If you are tired of manually setting a route in your ngnix, come see linkerd in action.
GlusterD - Daemon refactoring
This presentation talks about how did we refactor the existing daemons which glusterd currently manage
Approaches for duplicating Kubernetes Storage with Gluster
This document discusses approaches for duplicating Kubernetes storage using Gluster. It provides an introduction to Gluster and Kubernetes storage basics. It then covers use cases for cloning storage, how cloning can be initiated with the Container Data Importer, limitations of cloning with Gluster volume snapshots, and improved cloning based on reflinking files on the filesystem. The document concludes by discussing the future of cloning with the Container Storage Interface.
Corwin on containers
A deep dive into the history of containers as well as an introduction to how they work under the covers. This includes a discussion around Control Groups and Process Namespaces, as well as touching on some underlying syscalls, such as Fork and Clone.
WebKitGTK+ status and roadmap to WebKit2 (FOSDEM 2012)
By Mario Sánchez Prada and Philippe Normand.
In the talk we will talk about WebKitGTK+ [1] current status and work done in the last months since last desktop summit. A lot of things have happened in these months, we will summarize the various WebKitGTK+ improvements in rendering (accelerated compositing), accessibility, networking, etc. Also we will cover WebKit2 architecture and initial version of the API, we will detail the roadmap for WebKit2 support for WebKitGTK+. And or course some nice demos :).
The talk should be interesting for embedders and those considering integration of the web with the free desktop.
OpenStack und Containers
The world of Linux Containers might be the hottest technology helping businesses to build cloud ready applications and services. In this talk we will provide the current status of OpenStack support for containers and containers support for OpenStack.In the first part we will dive into two container-focused OpenStack projects: Magnum and Kolla and show how Magnum as an API service can help to provide "Containers as a Service" on top of OpenStack leveraging the capabilities of Kubernetes, Docker, Heat and Flannel, and show how Kolla is going to improve OpenStack operations by containerizing OpenStack components into micro services for simplified upgrades and deployment consistency, portability and scaling.In the second part we will see Magnum in action and learn how to get the whole thing running on top of his own, on an existing OpenStack Kilo environment!
Speaker: Daniel Bäurer
Noch mehr Vorträge von uns: https://www.inovex.de/de/content-pool/vortraege/
DSD-INT 2017 The use of big data for dredging - De Boer
Presentation by Gerben de Boer (van Oord) at the Symposium Earth Observation and Data Science, during Delft Software Days - Edition 2017. Thursday, 2 November 2017, Delft.
OpenStack Swift的性能调优
CloudConnect 云计算大会 China 2016
OpenStack Swift的性能调优
Performance Tuning of OpenStack Swift
李明宇 奥思数据 创始⼈& CTO
Email: li.mingyu@ostorage.com.cn
Swift is good at storing small objects like photos and documents.
RIPE74 - ARouteServer - IXP Automation Made Easy
Presentation of ARouteServer, a Python tool to automatically build (and test) feature-rich configurations for BGP route servers.
RIPE74, 10 May 2017
Logstash
The document discusses evaluating and implementing a log management system using Logstash. It outlines requirements for live log parsing, open source software, and scaling to 50GB per day. The proof of concept tested Logstash and its architecture includes parsing logs with Rsyslog and Logstash, storing in Elasticsearch, and visualizing with Kibana. Future plans are to add EBS support, aggregated metrics, and GeoIP.
MongoDB IoT City Tour EINDHOVEN: Managing the Database Complexity
The value of the fast growing class of NoSQL databases is the ability to handle high velocity and volumes of data while enabling greater agility with dynamic schemas. MongoDB gives you those benefits while also providing a rich querying capability and a document model for developer productivity. Arthur Viegers will outline the reasons for MongoDB's popularity in IoT applications and how you can leverage the core concepts of NoSQL to build robust and highly scalable IoT applications.
AWS Foundation Course - Outline
Prepare yourself on Sound Footing for AWS certifications.
This outline is for Cloud Manthan's Virtual Instructor Led Outline course on AWS.
Course is divided into various modules
Module 1- Introduction to AWS
Module 2 - Identity and Access Management (IAM)
Module 3 - Elastic Compute Cloud (EC2) and EBS
Module 4 - Virtual Private Cloud (VPC)
Module 5 - Storage - S3 , EBS (Review) , EFS
Module 6 - Relational Database Service (RDS)
Module 7 - Using Command Line Interface (CLI)
Module 8 - Monitoring - CloudWatch , CloudTrail , Config
Module 9 - Scaling your Environment !
Module 10 - Project and Exam !
Docker @haufe lexware tech lunch
This document provides an overview of Docker components, how to create and store Docker images and containers, use containers, integrate Docker with continuous integration tools like Jenkins, build multi-container applications, orchestrate containers, and access the Docker API. It also lists documentation resources for learning more about Docker.
Timur Shemsedinov "Эволюция архитектуры ИС"
The document traces the evolution of application architecture from early data files to modern web and mobile applications. It outlines key stages including file servers, local databases, client-server, and three-tier architectures. Emerging trends are seen moving to cloud-based, distributed, high-load, interactive, and flexible applications. The future may see more database-centric, peer-to-peer, offline, low-latency, and big data capable architectures that synchronize data across high volumes of interconnected users and servers. The full article provides additional details on these stages and directions for application evolution.
DSD-INT 2018 What's new in XBeach X - Nederhoff
Presentation by Kees Nederhoff, Deltares
at the XBeach User Day 2018, during Delft Software Days - Edition 2018. Thursday, 15 November 2018, Delft.
Storing your data in the cloud: doing right reversim 2018
This document provides an overview of cloud object storage, including its key features like multipart uploads, tagging, versioning, lifecycles, prefixes, static websites, security measures, and geo-replication for disaster recovery. It discusses when object storage is suitable compared to block and file storage, how to properly use features like multipart uploads, encryption, and access control, and cautions about ensuring data security when using public cloud storage services.
KDB+ Lite
Kdb+ is a database and analytics software designed for processing large, diverse financial and market data in real-time. It uses the q programming language which allows for less code and faster execution compared to other languages. Kdb+ includes components for ingesting data from multiple sources, storing real-time and historical data, performing analytics using triggers without slowing performance, and returning query results in various formats. The q language provides SQL-like and time series querying capabilities along with built-in functions to minimize programming and data transferred over networks for efficient analytics. Users can get started with kdb+ by downloading and installing it, then executing queries from the command line or IDE.
KDB database (EPAM tech talks, Sofia, April, 2015)
KDB is an in-memory column-oriented database that provides high-performance for real-time and historical large volumes of data. It is used widely in the financial industry. KDB supports the Q programming language for querying and manipulating data, and can be deployed in a distributed environment. The Java API provides simple connection and query methods to access a KDB database. KDB is well-suited for use cases involving capturing market data feeds and analyzing FIX messages.
Monitoring hybrid container environments
With the support for Windows containers in Docker Swarm and Kubernetes recently hitting beta, we’re entering a world where Hybrid container environments are no longer a strange sight. Microsoft is now an important player in the container world and for this talk I’d like to spend some time explaining what that means for us in the monitoring space. I’ll show you how to install and configure a Hybrid Kubernetes and Docker Swarm environment. We will setup the opensource Prometheus tooling and the commercial CoScale platform to monitor this unique environment. We’ll take a look at infrastructure monitoring and application monitoring, and the differences between Linux and Windows nodes.
Scheduling a fuller house - Talk at QCon NY 2016
Lessons learned building a container cloud at Netflix, presented at QCon NY 2016 along with colleague Andrew Spyker.
More Related Content
What's hot
Fluent Bit: Log Forwarding at Scale
This document summarizes a presentation about log forwarding at scale. It discusses how logging works internally and requires understanding the logging pipeline of parsing, filtering, buffering and routing logs. It then introduces Fluent Bit as a lightweight log forwarder that can be used to cheaply forward logs from edge nodes to log aggregators in a scalable way, especially in cloud native environments like Kubernetes. Hands-on demos show how Fluent Bit can parse and add metadata to Kubernetes logs.
Netflix Titus WASP October 2017
This document discusses Netflix's use of containers and the Titus container management platform. It provides the following key points:
1. Titus provides container scheduling, resource management, and execution capabilities. It helps Netflix achieve consistent developer experiences, faster innovation, and simpler deployments.
2. Titus has scaled to support over 1,000,000 daily container launches across multiple AWS regions. It integrates containers into Netflix's infrastructure and provides features like VPC networking and AWS integration.
3. Looking ahead, Netflix aims to further improve Titus' performance, operations, reliability, and scheduling capabilities to support even larger container workloads. Deeper security isolation and integration with services like ALB are also goals.
Dynamic routing in microservice oriented architecture
When splitting an application into different micro-services and each application access URL is dynamically generated, the hell gets loose. If you are tired of manually setting a route in your ngnix, come see linkerd in action.
GlusterD - Daemon refactoring
This presentation talks about how did we refactor the existing daemons which glusterd currently manage
Approaches for duplicating Kubernetes Storage with Gluster
This document discusses approaches for duplicating Kubernetes storage using Gluster. It provides an introduction to Gluster and Kubernetes storage basics. It then covers use cases for cloning storage, how cloning can be initiated with the Container Data Importer, limitations of cloning with Gluster volume snapshots, and improved cloning based on reflinking files on the filesystem. The document concludes by discussing the future of cloning with the Container Storage Interface.
Corwin on containers
A deep dive into the history of containers as well as an introduction to how they work under the covers. This includes a discussion around Control Groups and Process Namespaces, as well as touching on some underlying syscalls, such as Fork and Clone.
WebKitGTK+ status and roadmap to WebKit2 (FOSDEM 2012)
By Mario Sánchez Prada and Philippe Normand.
In the talk we will talk about WebKitGTK+ [1] current status and work done in the last months since last desktop summit. A lot of things have happened in these months, we will summarize the various WebKitGTK+ improvements in rendering (accelerated compositing), accessibility, networking, etc. Also we will cover WebKit2 architecture and initial version of the API, we will detail the roadmap for WebKit2 support for WebKitGTK+. And or course some nice demos :).
The talk should be interesting for embedders and those considering integration of the web with the free desktop.
OpenStack und Containers
The world of Linux Containers might be the hottest technology helping businesses to build cloud ready applications and services. In this talk we will provide the current status of OpenStack support for containers and containers support for OpenStack.In the first part we will dive into two container-focused OpenStack projects: Magnum and Kolla and show how Magnum as an API service can help to provide "Containers as a Service" on top of OpenStack leveraging the capabilities of Kubernetes, Docker, Heat and Flannel, and show how Kolla is going to improve OpenStack operations by containerizing OpenStack components into micro services for simplified upgrades and deployment consistency, portability and scaling.In the second part we will see Magnum in action and learn how to get the whole thing running on top of his own, on an existing OpenStack Kilo environment!
Speaker: Daniel Bäurer
Noch mehr Vorträge von uns: https://www.inovex.de/de/content-pool/vortraege/
DSD-INT 2017 The use of big data for dredging - De Boer
Presentation by Gerben de Boer (van Oord) at the Symposium Earth Observation and Data Science, during Delft Software Days - Edition 2017. Thursday, 2 November 2017, Delft.
OpenStack Swift的性能调优
CloudConnect 云计算大会 China 2016
OpenStack Swift的性能调优
Performance Tuning of OpenStack Swift
李明宇 奥思数据 创始⼈& CTO
Email: li.mingyu@ostorage.com.cn
Swift is good at storing small objects like photos and documents.
RIPE74 - ARouteServer - IXP Automation Made Easy
Presentation of ARouteServer, a Python tool to automatically build (and test) feature-rich configurations for BGP route servers.
RIPE74, 10 May 2017
Logstash
The document discusses evaluating and implementing a log management system using Logstash. It outlines requirements for live log parsing, open source software, and scaling to 50GB per day. The proof of concept tested Logstash and its architecture includes parsing logs with Rsyslog and Logstash, storing in Elasticsearch, and visualizing with Kibana. Future plans are to add EBS support, aggregated metrics, and GeoIP.
MongoDB IoT City Tour EINDHOVEN: Managing the Database Complexity
The value of the fast growing class of NoSQL databases is the ability to handle high velocity and volumes of data while enabling greater agility with dynamic schemas. MongoDB gives you those benefits while also providing a rich querying capability and a document model for developer productivity. Arthur Viegers will outline the reasons for MongoDB's popularity in IoT applications and how you can leverage the core concepts of NoSQL to build robust and highly scalable IoT applications.
AWS Foundation Course - Outline
Prepare yourself on Sound Footing for AWS certifications.
This outline is for Cloud Manthan's Virtual Instructor Led Outline course on AWS.
Course is divided into various modules
Module 1- Introduction to AWS
Module 2 - Identity and Access Management (IAM)
Module 3 - Elastic Compute Cloud (EC2) and EBS
Module 4 - Virtual Private Cloud (VPC)
Module 5 - Storage - S3 , EBS (Review) , EFS
Module 6 - Relational Database Service (RDS)
Module 7 - Using Command Line Interface (CLI)
Module 8 - Monitoring - CloudWatch , CloudTrail , Config
Module 9 - Scaling your Environment !
Module 10 - Project and Exam !
Docker @haufe lexware tech lunch
This document provides an overview of Docker components, how to create and store Docker images and containers, use containers, integrate Docker with continuous integration tools like Jenkins, build multi-container applications, orchestrate containers, and access the Docker API. It also lists documentation resources for learning more about Docker.
Timur Shemsedinov "Эволюция архитектуры ИС"
The document traces the evolution of application architecture from early data files to modern web and mobile applications. It outlines key stages including file servers, local databases, client-server, and three-tier architectures. Emerging trends are seen moving to cloud-based, distributed, high-load, interactive, and flexible applications. The future may see more database-centric, peer-to-peer, offline, low-latency, and big data capable architectures that synchronize data across high volumes of interconnected users and servers. The full article provides additional details on these stages and directions for application evolution.
DSD-INT 2018 What's new in XBeach X - Nederhoff
Presentation by Kees Nederhoff, Deltares
at the XBeach User Day 2018, during Delft Software Days - Edition 2018. Thursday, 15 November 2018, Delft.
Storing your data in the cloud: doing right reversim 2018
This document provides an overview of cloud object storage, including its key features like multipart uploads, tagging, versioning, lifecycles, prefixes, static websites, security measures, and geo-replication for disaster recovery. It discusses when object storage is suitable compared to block and file storage, how to properly use features like multipart uploads, encryption, and access control, and cautions about ensuring data security when using public cloud storage services.
KDB+ Lite
Kdb+ is a database and analytics software designed for processing large, diverse financial and market data in real-time. It uses the q programming language which allows for less code and faster execution compared to other languages. Kdb+ includes components for ingesting data from multiple sources, storing real-time and historical data, performing analytics using triggers without slowing performance, and returning query results in various formats. The q language provides SQL-like and time series querying capabilities along with built-in functions to minimize programming and data transferred over networks for efficient analytics. Users can get started with kdb+ by downloading and installing it, then executing queries from the command line or IDE.
KDB database (EPAM tech talks, Sofia, April, 2015)
KDB is an in-memory column-oriented database that provides high-performance for real-time and historical large volumes of data. It is used widely in the financial industry. KDB supports the Q programming language for querying and manipulating data, and can be deployed in a distributed environment. The Java API provides simple connection and query methods to access a KDB database. KDB is well-suited for use cases involving capturing market data feeds and analyzing FIX messages.
What's hot (20)
Dynamic routing in microservice oriented architecture
Dynamic routing in microservice oriented architecture
Approaches for duplicating Kubernetes Storage with Gluster
Approaches for duplicating Kubernetes Storage with Gluster
WebKitGTK+ status and roadmap to WebKit2 (FOSDEM 2012)
WebKitGTK+ status and roadmap to WebKit2 (FOSDEM 2012)
DSD-INT 2017 The use of big data for dredging - De Boer
DSD-INT 2017 The use of big data for dredging - De Boer
MongoDB IoT City Tour EINDHOVEN: Managing the Database Complexity
MongoDB IoT City Tour EINDHOVEN: Managing the Database Complexity
Storing your data in the cloud: doing right reversim 2018
Storing your data in the cloud: doing right reversim 2018
KDB database (EPAM tech talks, Sofia, April, 2015)
KDB database (EPAM tech talks, Sofia, April, 2015)
Similar to Monitoring your shiny new docker environment
Monitoring hybrid container environments
With the support for Windows containers in Docker Swarm and Kubernetes recently hitting beta, we’re entering a world where Hybrid container environments are no longer a strange sight. Microsoft is now an important player in the container world and for this talk I’d like to spend some time explaining what that means for us in the monitoring space. I’ll show you how to install and configure a Hybrid Kubernetes and Docker Swarm environment. We will setup the opensource Prometheus tooling and the commercial CoScale platform to monitor this unique environment. We’ll take a look at infrastructure monitoring and application monitoring, and the differences between Linux and Windows nodes.
Scheduling a fuller house - Talk at QCon NY 2016
Lessons learned building a container cloud at Netflix, presented at QCon NY 2016 along with colleague Andrew Spyker.
Netflix Container Scheduling and Execution - QCon New York 2016
Scheduling a Fuller House: Container Management At Netflix
Customers from over all over the world streamed Forty Two Billion hours of Netflix content last year. Various Netflix batch jobs and an increasing number of service applications use containers for their processing. In this talk Netflix will present a deep dive on the motivations and the technology powering container deployment on top of the AWS EC2 service. The talk will cover our approach to cloud resource management and scheduling with the open source Fenzo library, along with details on docker execution engine as a part of project Titus. As well, the talk will share some of the results so far, lessons learned, and end with a brief look at the developer experience for containers.
Kubernetes @ Squarespace (SRE Portland Meetup October 2017)
In this presentation I talk about our motivation to converting our microservices to run on Kubernetes. I discuss many of the technical challenges we encountered along the way, including networking issues, Java issues, monitoring and alerting, and managing all of our resources!
Kubernetes for Beginners
Kubernetes 101 provides an overview of containers, Kubernetes architecture, and Kubernetes objects. It discusses how containers evolved from virtualization to improve efficiency. Kubernetes is introduced as a container orchestration tool to manage deployments, scaling, networking, security etc. of containers across clusters. Key Kubernetes components like the control plane, nodes, and objects like pods, deployments, services, storage and secrets are explained at a high level.
Container Landscape in 2019
Containers are widely used today for consolidation and cloud native applications, with emerging uses in edge computing and serverless technologies. Containers provide efficiency, repeatability, isolation, and portability. Traditional uses include consolidation to reduce infrastructure costs and improve server utilization. Cloud native applications are also well suited to containers due to their microservices architecture enabling rapid deployment and scalability. For day 2 operations, container orchestration platforms like Kubernetes are used for lifecycle management, placement, and service discovery. Container storage interfaces provide pluggable storage solutions, while observability tools help debug complex microservices interactions through application performance monitoring and distributed tracing. Open APIs and standards are important to avoid vendor lock-in and allow flexibility in building, deploying and
WSO2 Kubernetes Reference Architecture - Nov 2017
This document provides an overview of WSO2's reference architecture for deploying their middleware products on Kubernetes. It begins with introductions to containers and Kubernetes, explaining concepts like pods, services, deployments, etc. It then outlines WSO2's approach for container orchestration, service discovery, configuration management, load balancing, security, updates, and monitoring in a Kubernetes environment. Specific practices and Kubernetes resources are recommended for areas like pod security policies, horizontal pod autoscaling, ingress definitions, and more. Overall the document serves as a guide for architecting and operating WSO2 products on Kubernetes according to best practices.
MongoDB World 2019: Packing Up Your Data and Moving to MongoDB Atlas
Moving to a new home is daunting. Packing up all your things, getting a vehicle to move it all, unpacking it, updating your mailing address, and making sure you did not leave anything behind. Well, the move to MongoDB Atlas is similar, but all the logistics are already figured out for you by MongoDB.
Service Discovery In Kubernetes
Service Discovery in kubernetes is all about how services of kubernetes get discovered internally and externally. How does a single POD communicate to another POD the within the cluster and how does a user request reach to a specific POD in the cluster? These are some questions that are answered by this TOPIC.
Getting started with kubernetes
Join us to learn how to deploy your first containerized application on the most popular orchestration engine. You will understand the basic concepts of Kubernetes along with the terminology and the deployment architecture. We will show you everything from building a Docker image to going live with your application. Each attendee gets $300 credit to start using Google Container Engine!
Deploying and managing container-based applications with OpenStack and Kubern...
Linux containers have recently taken the industry by storm, offering a lightweight, powerful, portable and upgradeable alternative to traditional app deployment on a host OS/VM.
Managing Docker containers on OpenStack VMs is possible today with Mirantis OpenStack, with the Murano Application Catalog radically simplifying the job of placing multiple application containers in an environment, installing apps in them from public resources such as Docker Hub, and deploying the environment on VMs for use. For managing containers at large scales, Mirantis and Google are now working jointly to enable Murano to configure and deploy Kubernetes — the Google-initiated open source project to build and refine cluster orchestration for containers on infrastructure.
In this presentation the core concepts of OpenStack, Docker and Kubernetes will be described, as well as demonstrated abilities to deploy containerized applications, managed by Kubernetes on above of OpenStack cloud.
Unleashing k8 s to reduce complexities of an entire middleware platform
WSO2 uses Kubernetes to provide multi-tenancy for its middleware platform. Kubernetes namespaces isolate each tenant's resources, while quotas control how much CPU and memory each tenant can use. Kubernetes also provides health monitoring, rolling updates, secret sharing between pods, and autoscaling that help reduce the complexity of WSO2's platform. WSO2's identity server integrates with Kubernetes to provide access management for tenants and users.
Kubernetes @ Squarespace: Kubernetes in the Datacenter
The document discusses Kubernetes adoption at Squarespace as their engineering organization grew. It describes the challenges of a monolithic architecture and how microservices addressed these challenges. It then discusses how Kubernetes helped solve operational challenges of provisioning and scaling microservices. Key Kubernetes concepts like pods, deployments, services and namespaces are explained. Monitoring, networking and security with Kubernetes are also covered.
Cloud Native Applications on Kubernetes: a DevOps Approach
Slides from my talk at Voxxed Days Ticino 2017. Developing cloud native applications on Kubernetes and Openshift. From Fabric8 to Openshift.io.
Containerization & Docker - Under the Hood
This presentation looks deep into the concept of containerization. What is containerization, how is it different from VMs, how containerization is achieved using Linux containers (LXC), control groups (cgroups) and copy on write file systems and current trends in containerization/docker are described.
OpenEBS hangout #4
The OpenEBS Hangout #4 was held on 22nd December 2017 at 11:00 AM (IST and PST) where a live demo of cMotion was shown . Storage policies of OpenEBS 0.5 were also explained
QConSF18 - Disenchantment: Netflix Titus, its Feisty Team, and Daemons
Disenchantment is a Netflix show following the medieval misadventures of a hard-drinking princess, her feisty elf, and her personal demon. In this talk, we will follow the story of Netflix’s container management platform, Titus, which powers critical aspects of the Netflix business (video encoding & streaming, big data, recommendations & machine learning, and other workloads). We’ll cover the challenges growing Titus from 10’s to 1000’s of workloads. We’ll talk about our feisty team’s work across container runtimes, scheduling & control plane, and cloud infrastructure integration. We’ll talk about the demons we’ve found on this journey covering operability, security, reliability and performance.
Kubernetes: My BFF
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It provides mechanisms for scheduling containers, load balancing, storage orchestration, and declarative deployments. The document provides examples of how Kubernetes can help manage containerized applications through concepts like pods, services, replication controllers, deployments, jobs, secrets and configmaps. It also compares Kubernetes to other orchestration systems and container platforms like OpenShift, AWS ECS, Azure Container Service and OpenStack.
Janus & docker: friends or foe
This document discusses Janus, a WebRTC server, and its deployment using Docker containers. It begins by providing background on microservices and Docker. It then describes issues that can arise with Docker's default networking modes for Janus due to network-related WebRTC failures. Specifically, Docker's NAT functionality does not always behave predictably for ICE negotiations. The document recommends using dedicated IP addresses for containers to avoid these issues. It provides examples of complex Docker-based architectures for deploying Janus at IETF meetings and for recording production using Docker Swarm.
Future of Microservices - Jakub Hadvig
Poďme sa porozprávať prečo sú najväčší poskytovatelia webových služieb tak úspešný, aké technológie používajú, čo sú to tie mikroservisy a ako do toho všetkého zapadá nový OpenShift 3.
Jakub je vývojár OpenShift platformy, open-source nadšenec, ktorý okrem programovania v rozličných jayzkoch ako Go, Ruby, JavaScript nemá problém ani s grafikou a designom.
Similar to Monitoring your shiny new docker environment (20)
Netflix Container Scheduling and Execution - QCon New York 2016
Netflix Container Scheduling and Execution - QCon New York 2016
Kubernetes @ Squarespace (SRE Portland Meetup October 2017)
Kubernetes @ Squarespace (SRE Portland Meetup October 2017)
MongoDB World 2019: Packing Up Your Data and Moving to MongoDB Atlas
MongoDB World 2019: Packing Up Your Data and Moving to MongoDB Atlas
Deploying and managing container-based applications with OpenStack and Kubern...
Deploying and managing container-based applications with OpenStack and Kubern...
Unleashing k8 s to reduce complexities of an entire middleware platform
Unleashing k8 s to reduce complexities of an entire middleware platform
Kubernetes @ Squarespace: Kubernetes in the Datacenter
Kubernetes @ Squarespace: Kubernetes in the Datacenter
Cloud Native Applications on Kubernetes: a DevOps Approach
Cloud Native Applications on Kubernetes: a DevOps Approach
QConSF18 - Disenchantment: Netflix Titus, its Feisty Team, and Daemons
QConSF18 - Disenchantment: Netflix Titus, its Feisty Team, and Daemons
Recently uploaded
5th LF Energy Power Grid Model Meet-up Slides
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
Public CyberSecurity Awareness Presentation 2024.pptx
Cyber security awareness slides for a busisness by TreeTop Security
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
Predictive maintenance is a proactive approach that anticipates equipment failures before they happen. At the forefront of this innovative strategy is Artificial Intelligence (AI), which brings unprecedented precision and efficiency. AI in predictive maintenance is transforming industries by reducing downtime, minimizing costs, and enhancing productivity.
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
SAP S/4 HANA sourcing and procurement to Public cloud
SAP S4 HANA to Public cloud data object differences
GNSS spoofing via SDR (Criptored Talks 2024)
In the realm of cybersecurity, offensive security practices act as a critical shield. By simulating real-world attacks in a controlled environment, these techniques expose vulnerabilities before malicious actors can exploit them. This proactive approach allows manufacturers to identify and fix weaknesses, significantly enhancing system security.
This presentation delves into the development of a system designed to mimic Galileo's Open Service signal using software-defined radio (SDR) technology. We'll begin with a foundational overview of both Global Navigation Satellite Systems (GNSS) and the intricacies of digital signal processing.
The presentation culminates in a live demonstration. We'll showcase the manipulation of Galileo's Open Service pilot signal, simulating an attack on various software and hardware systems. This practical demonstration serves to highlight the potential consequences of unaddressed vulnerabilities, emphasizing the importance of offensive security practices in safeguarding critical infrastructure.
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Folding is a recent technique for building efficient recursive SNARKs. Several elegant folding protocols have been proposed, such as Nova, Supernova, Hypernova, Protostar, and others. However, all of them rely on an additively homomorphic commitment scheme based on discrete log, and are therefore not post-quantum secure. In this work we present LatticeFold, the first lattice-based folding protocol based on the Module SIS problem. This folding protocol naturally leads to an efficient recursive lattice-based SNARK and an efficient PCD scheme. LatticeFold supports folding low-degree relations, such as R1CS, as well as high-degree relations, such as CCS. The key challenge is to construct a secure folding protocol that works with the Ajtai commitment scheme. The difficulty, is ensuring that extracted witnesses are low norm through many rounds of folding. We present a novel technique using the sumcheck protocol to ensure that extracted witnesses are always low norm no matter how many rounds of folding are used. Our evaluation of the final proof system suggests that it is as performant as Hypernova, while providing post-quantum security.
Paper Link: https://eprint.iacr.org/2024/257
AWS Cloud Cost Optimization Presentation.pptx
This presentation provides valuable insights into effective cost-saving techniques on AWS. Learn how to optimize your AWS resources by rightsizing, increasing elasticity, picking the right storage class, and choosing the best pricing model. Additionally, discover essential governance mechanisms to ensure continuous cost efficiency. Whether you are new to AWS or an experienced user, this presentation provides clear and practical tips to help you reduce your cloud costs and get the most out of your budget.
Presentation of the OECD Artificial Intelligence Review of Germany
Consult the full report at https://www.oecd.org/digital/oecd-artificial-intelligence-review-of-germany-609808d6-en.htm
Generating privacy-protected synthetic data using Secludy and Milvus
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
JavaLand 2024: Application Development Green Masterplan
My presentation slides I used at JavaLand 2024
Programming Foundation Models with DSPy - Meetup Slides
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Fueling AI with Great Data with Airbyte Webinar
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Trusted Execution Environment for Decentralized Process Mining
Presentation of the paper "Trusted Execution Environment for Decentralized Process Mining" given during the CAiSE 2024 Conference in Cyprus on June 7, 2024.
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI, built on the robust SAP Business Technology Platform (SAP BTP), is the latest and most advanced version of our AI development, reaffirming our commitment to delivering top-tier AI solutions. Skybuffer AI harnesses all the innovative capabilities of the SAP BTP in the AI domain, from Conversational AI to cutting-edge Generative AI and Retrieval-Augmented Generation (RAG). It also helps SAP customers safeguard their investments into SAP Conversational AI and ensure a seamless, one-click transition to SAP Business AI.
With Skybuffer AI, various AI models can be integrated into a single communication channel such as Microsoft Teams. This integration empowers business users with insights drawn from SAP backend systems, enterprise documents, and the expansive knowledge of Generative AI. And the best part of it is that it is all managed through our intuitive no-code Action Server interface, requiring no extensive coding knowledge and making the advanced AI accessible to more users.
Recently uploaded (20)
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
SAP S/4 HANA sourcing and procurement to Public cloud
SAP S/4 HANA sourcing and procurement to Public cloud
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Monitoring your shiny new docker environment
- 1. Monitoring your shiny new Docker environment 1
- 3. Quick question ● Docker in production? ● Orchestrator? ● Monitoring? 3
- 4. Source: “Hope vs. reality, Containers in 2016”, Global perception study by CloudFoundry, June 2016
- 5. Framework 5
- 6. The environment Kubernetes ● 3 master nodes ○ Manages nodes, services, containers ● 3 infrastructure nodes ○ Manages inbound traffic ● 5 compute nodes ○ Runs application containers ● (Bastion management node) ● (Hardware load balancer) 6 Docker ● Kubernetes internal services ± 5 containers per node ± 1 container per application container ○ DNS ○ Proxy ○ Managers ● Your application ± 30 containers per node ○ Microservices ○ Databases ○ Caching ○ Queuing ○ ...
- 7. 7
- 8. 8 Why?
- 9. 9
- 10. The environment Kubernetes / Openshift / Docker EE ● 3 master nodes ○ Manages nodes, services, containers ● 3 infrastructure nodes ○ Manages inbound traffic ● 5 compute nodes ○ Runs application containers ● (Bastion management node) ● (Hardware load balancer) 10 Docker ● Kubernetes internal services ± 5 containers per node ± 1 container per application container ○ DNS ○ Proxy ○ Managers ● Your application ± 30 containers per node ○ Microservices ○ Databases ○ Caching ○ Queuing ○ ...
- 11. 11
- 12. 12
- 13. 13
- 14. 14
- 15. 15
- 16. How is it different? 16
- 17. Docker Container != VM, but not far from ● Disk ● CPU ● Memory ● Application 17 In CoScale: - 67 OS - 35 Docker (/ container)
- 18. 18
- 19. 19 Orchestration ● Automagical management of resources ● Self-healing ● Easily scalable
- 20. 20
- 21. Node B Node CNode A Dynamic behavior 21 Services
- 23. Metrics 35 Docker metrics + 10 Orchestration metrics / Container x 350 = 15750 metrics 23
- 24. The what? 24
- 25. 25 (Virtualized) OS System / Infrastructure monitoring Container resource monitoring Container Application component Container Application component Container Application component CPU, Memory, Storage, Network, ... Orchestration monitoring Services, volumes, replication controllers, ... CPU, memory, storage, network, ... In-container / Application monitoring Web service, database, caching, ... Orchestrator
- 26. The how? 26
- 29. 29 Dimensions 29 Old ● Metric Network - server1 ● Metric Network - server2 …. ● Metric Network - container x x 350 New Metric Network Dimension Server Dimension Interface
- 31. The who? 31
- 32. Prometheus 32
- 33. Dynatrace 33
- 34. Sysdig 34
- 35. CoScale 35 APP APP APP APP APP API APP APP RUM Postgresql Metadata Cassandra Metric data Elasticsearch Event data HaProxy Loadbalancer HTTPS handling Analysis workers Alerting workers Data workers RUM Boomerang.js Agent Log & api parsing
- 36. Questions? 36
- 37. The future 37
- 38. Thanks for your time 38 ● samuel.vandamme@coscale.com ● @kidk (twitter) ● https://www.coscale.com