Docker Swarm scheduling in 1.12

•Download as PPTX, PDF•

2 likes•1,909 views

This document discusses Docker 1.12 Swarm scheduling. It describes the basic concepts of swarm scheduling including labeling nodes, applying constraints, and strategies. It also demonstrates scheduling criteria like resource availability and awareness of labels and constraints. The demo section shows a setup with 3 EC2 instances, 1 manager and 2 workers with different engine labels, to demonstrate scheduling.

Software

Docker 1.12 Swarm
Scheduling
Atharva Chauthaiwale
Juniper Networks.

Agenda
 Basic Swarm Scheduling Concepts
 Scheduling criteria
 Resource Awareness
 Labels
 Constraints
 Strategy
 Demo
 WARNING : We are covering docker 1.12 Swarm mode only !!!

Swarm Scheduler
Manager
Swarm init
Worker 1
swarm join
Worker 2
swarm join
Worker 3
swarm join
Worker 2
swarm join
docker service create –-name myredis –-replicas 3 redis:latest
Scheduler
kicks in Here !
Main job of scheduler is to decide which node to use when running docker container/service.
docker service update docker service scale

Scheduling Criteria
 Resource Availability
 Scheduler is aware of resources available on nodes.
 Labels and Constraints
 Label : attribute of the node
 E.g. environment = test, storage = ssd
 Constraints
 Restrictions applied by Operator while creating a service.
 E.g. docker service create --constraint node.labels.storage==ssd …

Engine Labels & Constraints
 Attached to docker Engine
 Adding/Modifying requires Docker engine restart
 https://docs.docker.com/engine/admin/#/configuring-docker
 How to specify Engine labels ( Linux Machine )
 Edit file /etc/default/docker modify DOCKER_OPTS and restart
daemon
 Engine constraints
 docker service create --constraint engine.labels.storage==ssd …

Node Labels & Constraints
 Can be added by cluster admin for operational purpose
 Doesn’t require docker engine restart
 docker node update --label-add security=high
 docker node update –label-rm security
 Node constraints
 docker service create --constraint node.labels.deployment.environment
!= test

Strategy & Affinity
 Strategy : What happens when two nodes are similar
 Spread strategy : schedule tasks on the least loaded nodes, provided
they meet the constraints and resource requirements.
 Swarm standalone supported Bean pack and Random strategy
NOT SUPPORTED IN SWARM MODE YET !
 Affinity and Anti Affinity
 Affinity: two containers should be together
 Anti Affinity : two containers should not be together
 Supported in swarm standalone

Demo
 Setup
 AWS : 3 EC2 instances – Ubuntu 15.10
 1 Manager and 2 workers
Manager
Worker 1
Worker 2
Engine label: com.storage.class=ssd
Engine label: com.storage.class=hdd
Engine label: com.storage.class=hdd

References
 Swarm mode scheduler :
https://github.com/docker/swarmkit/blob/master/README.md
 What’s new in Docker 1.12 Scheduling – Docker Captain Ajeet Singh Raina
 Docker1.12 and AWS – Atharva
 Github repo for the demo - https://github.com/achautha/docker-
experiments
Thank you !!

What's hot

Docker has revolutionized the way people think about applications and deployments. It provides a simple way to run and distribute Linux containers for a variety of use cases, from lightweight virtual machines to complex distributed micro-services architectures. Containers allow to run services in isolation with a minimum performance penalty, increased speed, easier configuration and less complexity, making it ideal for continuous integration and continuous delivery based workloads. But testing a distributed micro-services architecture is no easy task, requiring a shift in mindset and tooling to accommodate the new architecture. We will provide insight on our experience creating a Jenkins platform based on distributed Docker containers running on Apache Mesos and Marathon, applicable for all types of applications, but specially Java and JVM based ones.

Divide and Conquer: Easier Continuous Delivery using Micro-Services

Carlos Sanchez

Docker is revolutionizing the way people think about applications and deployments. It provides a simple way to run and distribute Linux containers for a variety of use cases, from lightweight virtual machines to complex distributed micro-services architectures. Containers allow to run services in isolation with a minimum performance penalty, increased speed, easier configuration and less complexity, making it ideal for continuous integration and continuous delivery based workloads. But migrating an existing application to a distributed microservices architecture is no easy task, requiring a shift in the software development, networking and storage to accommodate the new architecture. We will provide insight on our experience creating a Jenkins platform based on distributed Docker containers running on Apache Mesos and Marathon, applicable for all types of applications, but specially Java and JVM based nones.

From Monolith to Docker Distributed Applications

Carlos Sanchez

Automation and Collaboration Across Multiple Swarms Using Docker Cloud - Marc...

Docker, Inc.

Docker orchestration

Open Source Consulting

Brett Inman - Manager, Infrastructure Engineering, Docker Loke Norlin Johannessen - Senior System Specialist, Alm. Brand Sune Keller - IT Architect, Alm. Brand At both Alm. Brand and Docker, we’ve been running Docker Enterprise in production since the first beta. Both teams have gathered several learnings that were ripe for being codified, both to save time and provide certainty in daily operations. This is the story of how each company moved to a more automated and declarative approach. In this talk, we will share how Alm. Brand and Docker use tools like Packer, Terraform, Ansible, Cloudformation, Salt, and GitLab to build and upgrade Docker Enterprise clusters, comparing how both companies have solved for automating the Docker Enterprise infrastructure lifecycle. This approach focuses on immutable or disposable infrastructure, software components, and most importantly the processes to help you build your own automation solution for Docker Enterprise to fit your infrastructure needs.

DCEU 18: Automating Docker Enterprise: Hands-off Install and Upgrade

Docker, Inc.

virtualization-vs-containerization-paas

rajdeep

Docker 1.9 Feature Overview

Sreenivas Makam

Continuous delivery with Jenkins, Docker and Mesos/Marathon - jbcnconf

Julia Mateo

What’s New in Docker - Victor Vieux, Docker

Docker, Inc.

Docker Mentorweek beginner workshop notes

Sreenivas Makam

Docker Security Overview

Sreenivas Makam

What's New in Docker 1.12 by Nishant Totla for Docker SF Meetup 08.03.16

Docker, Inc.

True microservices are more than simply bolting a REST interface on your legacy application, packing it in a Docker container and hoping for the best. Security is a key component when designing and building out any new architecture, and it must be considered from top to bottom. Umpa Lumpas might not be considered "real" microservices, but Willy Wonka still has them locked down tight! In this talk, Aaron will briefly touch on the idea and security benefits of microservices before diving into practical and real world examples of creating a secure microservices architecture. We'll start with designing and building high security Docker containers, using and examining the latest security features in Docker (such as User Namespaces and seccomp-bpf) as well as examine some typically forgotten security principals. Aaron will end on exploring related challenges and solutions in the areas of network security, secrets management and application hardening. Finally, while this talk is geared towards Microservices, it should prove informational for all Docker users, building a PaaS or otherwise.

The Golden Ticket: Docker and High Security Microservices by Aaron Grattafiori

Docker, Inc.

Catching up with what has happened with logging in Docker since late 2014 all the way up to the recently released Docker 0.10. Also, presenting my view on a comprehensive approach to monitoring Docker using the API to get events, logs, stats with a little bit of self promotion in pointing out that we have recently released an implementation of comprehensive monitoring as part of a Sumo Logic collector source.

Comprehensive Monitoring for Docker

Christian Beedgen

Containers orchestrators: Docker vs. Kubernetes

Dmitry Lazarenko

Virtual Meetup Docker + Arm: Building Multi-arch Apps with Buildx

Docker, Inc.

Docker 1.11 Presentation

Sreenivas Makam

Docker is an open source project which developed in the Linux world, but that doesn't mean Windows and .NET projects can't make use of it. With the Core CLR you can run .NET inside a container running Linux, and with Windows Server 2016 you'll be able to run Docker containers on Windows. In this session we'll start by looking at how Docker works, how containers are defined and run, and what's already out there in public repositories for you to use. We'll see where Docker can fit in integration projects - running core components like message queues and caches, and hosting .NET projects too.

Integration with Docker and .NET Core

Sriram Hariharan

Continuous Deployment with Jenkins on Kubernetes

Matt Baldwin

Docker Ecosystem on Azure

Patrick Chanezon

What's hot (20)

Divide and Conquer: Easier Continuous Delivery using Micro-Services

From Monolith to Docker Distributed Applications

Automation and Collaboration Across Multiple Swarms Using Docker Cloud - Marc...

Docker orchestration

DCEU 18: Automating Docker Enterprise: Hands-off Install and Upgrade

virtualization-vs-containerization-paas

Docker 1.9 Feature Overview

Continuous delivery with Jenkins, Docker and Mesos/Marathon - jbcnconf

What’s New in Docker - Victor Vieux, Docker

Docker Mentorweek beginner workshop notes

Docker Security Overview

What's New in Docker 1.12 by Nishant Totla for Docker SF Meetup 08.03.16

The Golden Ticket: Docker and High Security Microservices by Aaron Grattafiori

Comprehensive Monitoring for Docker

Containers orchestrators: Docker vs. Kubernetes

Virtual Meetup Docker + Arm: Building Multi-arch Apps with Buildx

Docker 1.11 Presentation

Integration with Docker and .NET Core

Continuous Deployment with Jenkins on Kubernetes

Docker Ecosystem on Azure

Viewers also liked

Docker is revolutionizing the way people think about applications and deployments. It provides a simple way to run and distribute Linux containers for a variety of use cases, from lightweight virtual machines to complex distributed micro-services architectures. Kubernetes is an open source project to manage a cluster of Linux containers as a single system, managing and running Docker containers across multiple Docker hosts, offering co-location of containers, service discovery and replication control. It was started by Google and now it is supported by Microsoft, RedHat, IBM and Docker Inc amongst others. Jenkins Continuous Integration environment can be dynamically scaled by using the Kubernetes and Docker plugins, using containers to run slaves and jobs, and also isolate job execution.

Scaling Jenkins with Docker and Kubernetes

Carlos Sanchez

Small is the new big, and for good reason. The benefits of microservices and service-oriented architecture have been extolled for a number of years, yet many forge ahead without thinking of the impact the users of the services. Consuming on micro services can be enjoyable as long as the developer experience has been crafted as finely as the service itself. But just like with any other product, there isn’t a single kind of consumer. Together we will walk through some typical kinds of consumers, what their needs are, and how we can create a great developer experience using brains and tools like Docker.

Making friendly-microservices

Michele Titolo

Building Jenkins Pipelines at Scale

Julien Pivotto

HashiCorpのNomadを使ったコンテナのスケジューリング手法

Masahito Zembutsu

Docker swarm

Alberto Guimarães Viana

This talk explains how microservices (Restfull Endpoint) could be secured using a Policy based approach to intercept the HTTP request. A less intrusive pattern is proposed at the level of the Web Container using Contrants mapping the Web Resources with JAAS API & Roles. Finally we will investigate how such Security design can be developed using an external API Management platform which reenforce the Security and Governance aspect.

Security enforcement of Microservices with API Management

Charles Moulliard

Docker swarm introduction

Evan Lin

Docker Swarm Introduction

rajdeep

Introction to docker swarm

Hsi-Kai Wang

Docker 1.12 & Swarm Mode Introduction Docker の新しい技術と swarm モードの紹介平成28年6月29日(水) Dockercon 2016報告会 #tistudy201606 発表資料 http://thinkit.connpass.com/event/28529/ ■捕捉 Docker 1.12: swarm モードと Ingress Load Balancing 概要 · Pocketstudio Technology Log https://pocketstudio.net/2016/06/23/docker-1-12-swarm-mode-and-ingress-load-balancing/ Swarm モード概要 — Docker-docs-ja 1.12.RC ドキュメント http://docs.docker.jp/engine/swarm/index.html

Docker 1.12 & Swarm Mode Introduction ～ Docker の新しい技術と swarm モードの紹介

Masahito Zembutsu

Achieving CI/CD with Kubernetes

Ramit Surana

Consul is a service discovery system that provides a microservice style interface to services, service topology and service health. With service discovery you can look up services which are organized in the topology of your datacenters. Consul uses client agents and RAFT to provide a consistent view of services. Consul provides a consistent view of configuration as well also using RAFT. Consul provides a microservice interface to a replicated view of your service topology and its configuration. Consul can monitor and change services topology based on health of individual nodes. Consul provides scalable distributed health checks. Consul only does minimal datacenter to datacenter communication so each datacenter has its own Consul cluster. Consul provides a domain model for managing topology of datacenters, server nodes, and services running on server nodes along with their configuration and current health status. Consul is like combining the features of a DNS server plus Consistent Key/Value Store like etcd plus features of ZooKeeper for service discovery, and health monitoring like Nagios but all rolled up into a consistent system. Essentially, Consul is all the bits you need to have a coherent domain service model available to provide service discovery, health and replicated config, service topology and health status. Consul also provides a nice REST interface and Web UI to see your service topology and distributed service config. Consul organizes your services in a Catalog called the Service Catalog and then provides a DNS and REST/HTTP/JSON interface to it. To use Consul you start up an agent process. The Consul agent process is a long running daemon on every member of Consul cluster. The agent process can be run in server mode or client mode. Consul agent clients would run on every physical server or OS virtual machine (if that makes more sense). Client runs on server hosting services. The clients use gossip and RPC calls to stay in sync with Consul. A client, consul agent running in client mode, forwards request to a server, consul agent running in server mode. Clients are mostly stateless. The client does LAN gossip to the server nodes to communicate changes. A server, consul agent running in server mode, is like a client agent but with more tasks. The consul servers use the RAFT quorum mechanism to see who is the leader. The consul servers maintain cluster state like the Service Catalog. The leader manages a consistent view of config key/value pairs, and service health and topology. Consul servers also handle WAN gossip to other datacenters. Consul server nodes forwards queries to leader, and forward queries to other datacenters. A Datacenter is fairly obvious. It is anything that allows for fast communication between nodes, with as few or no hops, little or no routing, and in short: high speed communication. This could be an Amazon EC2 availability zone, a networking environment like a subnet, or any private, low latency, high

Consul: Microservice Enabling Microservices and Reactive Programming

Rick Hightower

from the Docker Mountain View Meetup on 2/24 Docker Swarm turns a pool of Docker hosts into a single, virtual Docker host. It's a different approach to clustering that aims for simplicity, flexibility and high scale. This talk covers the new Swarm features and demonstrate a realistic microservice style application running on Swarm. Topics: • How to deploy a complex multi-container application on Swarm • Deployment patterns for AWS or Vagrant • Load balancing and scaling N web frontends with Interlock+ha_proxy • Independently scaling backend workers All code used in the demo is available at https://github.com/mgoelzer/swarm-demo-voting-app and can be used as a starting point for your own applications. Learn more about Docker Swarm: https://www.docker.com/products/docker-swarm

Docker Swarm: Docker Native Clustering

Docker, Inc.

presented by Alexandre Beslic (@abronan) Swarm v1.0 is now ready for running your apps in production! Swarm is the easiest way to run Docker applications at large scale on a cluster. It turns a pool of Docker Engines into a single, virtual Engine. You don’t have to worry about where to put containers, or how they’re going to talk to each other - it just handles all that for you. We’ve spent the last few months tirelessly hardening and tuning it, and in combination with multi-host networking and the new volume system in Docker Engine 1.9, we can confidently say that it’s ready for running your apps in production. In our tests, we’ve been running Swarm on EC2 with 1,000 nodes and 30,000 containers and it keeps on scheduling containers in less than half a second. Not even breaking a sweat! Keep an eye for a blog post soon with the full details. Read more: http://blog.docker.com/2015/11/swarm-1-0/

Docker Online Meetup #28: Production-Ready Docker Swarm

Docker, Inc.

Service discovery in a microservice architecture using consul

Jos Dirksen

This talk introduces Clocker and shows how to bootstrap a Docker Cloud that is responsive and scalable, across a dynamic cluster of hosts and cloud providers. Clocker is an Apache licensed open source project that demonstrates intelligent placement, on-demand provisioning and autonomic management of containers using Apache Brooklyn as the central nervous system. The Clocker stack enhances the standard Docker installation using best practices for configuration and integrates Weave networking capabilities plus Apache jclouds for provisioning on any infrastructure. We will show how to use Clocker to deploy, monitor and scale complex applications defined using Brooklyn blueprints across a network of Docker containers in the cloud.

Clocker: Managing Container Networking and Placement

Docker, Inc.

Docker Swarm 0.2.0

Docker, Inc.

MongoDB World 2016: Scaling MongoDB with Docker and cGroups

MongoDB

AWS is an elastic, secure, flexible, and developer-centric ecosystem that serves as an ideal platform for Docker deployments. AWS offers the scalable infrastructure, APIs, and SDKs that integrate tightly into a development lifecycle and accentuate the benefits of the lightweight and portable containers that Docker offers to its users. This session familiarizes you with the benefits of containers, introduce Amazon EC2 Container Service, and demonstrates how to use Amazon ECS to run containerized applications at scale in production.

Getting Started with Docker on AWS

Amazon Web Services

In this session, we will learn how to define and run multi-container applications with Docker Compose. Then, we will show how to deploy and scale them seamlessly to a cluster with Docker Swarm; and how Amazon EC2 Container Service (ECS) eliminates the need to install,operate, and scale your own cluster management infrastructure. We will also walk through some best practice patterns used by customers for running their microservices platforms or batch jobs. Sample code and Compose templates will be provided on GitHub afterwards.

From development environments to production deployments with Docker, Compose,...

Jérôme Petazzoni

Viewers also liked (20)

Scaling Jenkins with Docker and Kubernetes

Making friendly-microservices

Building Jenkins Pipelines at Scale

HashiCorpのNomadを使ったコンテナのスケジューリング手法

Docker swarm

Security enforcement of Microservices with API Management

Docker swarm introduction

Docker Swarm Introduction

Introction to docker swarm

Docker 1.12 & Swarm Mode Introduction ～ Docker の新しい技術と swarm モードの紹介

Achieving CI/CD with Kubernetes

Consul: Microservice Enabling Microservices and Reactive Programming

Docker Swarm: Docker Native Clustering

Docker Online Meetup #28: Production-Ready Docker Swarm

Service discovery in a microservice architecture using consul

Clocker: Managing Container Networking and Placement

Docker Swarm 0.2.0

MongoDB World 2016: Scaling MongoDB with Docker and cGroups

Getting Started with Docker on AWS

From development environments to production deployments with Docker, Compose,...

Similar to Docker Swarm scheduling in 1.12

Nats meetup oct 2016 docker 112

Nirmal Mehta

Docker Swarm for Beginner

Shahzad Masud

"Join Laura Frank and Stephen Day as they explain and examine technical concepts behind container orchestration systems, like distributed consensus, object models, and node topology. These concepts build the foundation of every modern orchestration system, and each technical explanation will be illustrated using SwarmKit and Kubernetes as a real-world example. Gain a deeper understanding of how orchestration systems work in practice and walk away with more insights into your production applications."

Container orchestration from theory to practice

Docker, Inc.

Since its first 1.12 release on July 2016, Docker Swarm Mode has matured enough as a clustering and scheduling tool for IT administrators and developers who can easily establish and manage a cluster of Docker nodes as a single virtual system. Swarm mode integrates the orchestration capabilities of Docker Swarm into Docker Engine itself and help administrators and developers with the ability to add or subtract container iterations as computing demands change. With sophisticated but easy to implement features like built-in Service Discovery, Routing Mesh, Secrets, declarative service model, scaling of the services, desired state reconciliation, scheduling, filters, multi-host networking model, Load-Balancing, rolling updates etc. Docker 17.06 is all set for production-ready product today. Join me webinar organised by Docker Izmir, to get familiar with the current Swarm Mode capabilities & functionalities across the heterogeneous environments.

Deep Dive into Docker Swarm Mode

Ajeet Singh Raina

New Docker Features for Orchestration and Containers

Jeff Anderson

Docker is a key player in the microservices movement and is arguably the leader in containerization technology. That said, there are many ways to “do Docker”. Between the leading cloud providers AWS, Azure, and Google; plus other platform stacks like Docker/Swarm, Apache Mesos – DC/OS, and Kubernetes; it can get confusing.In this session, Michele will bring her customer experiences building solutions across most of these platforms – to provide you with the highlights, the architecture topologies, and some perspective on the way she helps her customers choose the right platform for their cloud, on premise or hybrid solutions.

.NET Developer Days - So many Docker platforms, so little time...

Michele Leroux Bustamante

Docker 1.12 and swarm mode

Wesley Charles Blake

Kubernetes Operators: Rob Szumski

Redis Labs

Docker swarm - An introduction to Docker native clustering

Shantanu Deshpande

Docker

SangtongPeesing

Docker Engine 1.12 can be rightly called ” A Next Generation Docker Clustering & Distributed System”. Though Docker Engine 1.12 Final Release is around corner but the recent RC3 brings lots of improvements and exciting features. One of the major highlight of this release is Docker Swarm Mode which provides powerful yet optional ability to create coordinated groups of decentralized Docker Engines. Swarm Mode combines your engine in swarms of any scale. It’s self-organizing and self-healing. It enables infrastructure-agnostic topology.The newer version democratizes orchestration with out-of-box capabilities for multi-container on multi-host app deployments.

What's New in Docker 1.12?

Ajeet Singh Raina

What's New in Docker 1.12 (June 20, 2016) by Mike Goelzer & Andrea Luzzardi

Mike Goelzer

Mike Goelzer is a developer, hacker, author and the open source product management lead for Docker’s Core Runtime. He currently works on the open source Docker Engine and Docker’s open source container orchestration technologies. Andrea Luzzardi is a Software Engineer at Docker and was part of the original team that built the project. He is currently working on Swarm, a Docker-native clustering system. They discuss what is new in Docker.

What's New in Docker 1.12 by Mike Goelzer and Andrea Luzzardi

Docker, Inc.

Uno degli argomenti più importanti per chi utilizza wordpress è la condivisione dello stesso server, brutto a dirsi ma in questo settore si tende ad installare su una sola macchina un numero molto alto di siti. Vediamo come Docker può venirci incontro per organizzare al meglio le nostre installazioni e come può diventare un opportunità per gestire al meglio le risorse anche in vista di una crescita improvvisa e di una necessità di scalata orizzontale.

Gianluca Arbezzano Wordpress: gestione delle installazioni e scalabilità con ...

Codemotion

Lee Calcote, Solar Winds Running a few containers? No problem. Running hundreds or thousands? Enter the container orchestrator. Let’s take a look at the characteristics of the four most popular container orchestrators and what makes them alike, yet unique. Swarm Nomad Kubernetes Mesos+Marathon We’ll take a structured looked at these container orchestrators, contrasting them across these categories: Genesis & Purpose Support & Momentum Host & Service Discovery Scheduling Modularity & Extensibility Updates & Maintenance Health Monitoring Networking & Load-Balancing High Availability & Scale

Characterizing and Contrasting Kuhn-tey-ner Awr-kuh-streyt-ors

Sonatype

What's New in Docker 1.13?

Michael Irwin

Docker Networking - Common Issues and Troubleshooting Techniques

Sreenivas Makam

Swarm mode

Dharmit Shah

Docker: A New Way to Turbocharging Your Apps Development

msyukor

Docker 進階實務班

Philip Zheng

Similar to Docker Swarm scheduling in 1.12 (20)

Nats meetup oct 2016 docker 112

Docker Swarm for Beginner

Container orchestration from theory to practice

Deep Dive into Docker Swarm Mode

New Docker Features for Orchestration and Containers

.NET Developer Days - So many Docker platforms, so little time...

Docker 1.12 and swarm mode

Kubernetes Operators: Rob Szumski

Docker swarm - An introduction to Docker native clustering

Docker

What's New in Docker 1.12?

What's New in Docker 1.12 (June 20, 2016) by Mike Goelzer & Andrea Luzzardi

What's New in Docker 1.12 by Mike Goelzer and Andrea Luzzardi

Gianluca Arbezzano Wordpress: gestione delle installazioni e scalabilità con ...

Characterizing and Contrasting Kuhn-tey-ner Awr-kuh-streyt-ors

What's New in Docker 1.13?

Docker Networking - Common Issues and Troubleshooting Techniques

Swarm mode

Docker: A New Way to Turbocharging Your Apps Development

Docker 進階實務班

Recently uploaded

In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey. Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience. Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system. Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.

Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL

Natan Silnitsky

Advanced Flow Concepts Every Developer Should Know

Peter Caitens

Large Language Models and the End of Programming

Matt Welsh

Globus Compute Introduction - GlobusWorld 2024

Globus

Into the Box 2024 - Keynote Day 2 Slides.pdf

Ortus Solutions, Corp

AI/ML Infra Meetup May. 23, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Lu Qiu (Data & AI Platform Tech Lead, @Alluxio) - Siyuan Sheng (Senior Software Engineer, @Alluxio) Speed and efficiency are two requirements for the underlying infrastructure for machine learning model development. Data access can bottleneck end-to-end machine learning pipelines as training data volume grows and when large model files are more commonly used for serving. For instance, data loading can constitute nearly 80% of the total model training time, resulting in less than 30% GPU utilization. Also, loading large model files for deployment to production can be slow because of slow network or storage read operations. These challenges are prevalent when using popular frameworks like PyTorch, Ray, or HuggingFace, paired with cloud object storage solutions like S3 or GCS, or downloading models from the HuggingFace model hub. In this presentation, Lu and Siyuan will offer comprehensive insights into improving speed and GPU utilization for model training and serving. You will learn: - The data loading challenges hindering GPU utilization - The reference architecture for running PyTorch and Ray jobs while reading data from S3, with benchmark results of training ResNet50 and BERT - Real-world examples of boosting model performance and GPU utilization through optimized data access

AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...

Alluxio, Inc.

2024 RoOUG Security model for the cloud.pptx

Georgi Kodinov

Breaking the Code : A Guide to WhatsApp Business API.pdf

Meon Technology

The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month. The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies. However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News. Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!

SOCRadar Research Team: Latest Activities of IntelBroker

SOCRadar

COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.

Developing Distributed High-performance Computing Capabilities of an Open Sci...

Globus

Using IESVE for Room Loads Analysis - Australia & New Zealand

IES VE

A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf

kalichargn70th171

The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.

Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...

Globus

WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation

WSO2

Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...

informapgpstrackings

Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos https://www.amb-review.com/tubetrivia-ai Exclusive Features: AI-Powered Questions, Wide Range of Categories, Adaptive Difficulty, User-Friendly Interface, Multiplayer Mode, Regular Updates. #TubeTriviaAI #QuizVideoMagic #ViralQuizVideos #AIQuizGenerator #EngageExciteExplode #MarketingRevolution #BoostYourTraffic #SocialMediaSuccess #AIContentCreation #UnlimitedTraffic

Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf

AMB-Review

Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ? Venez le découvrir lors de cette session ignite

Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...

Anthony Dahanne

NetSage is an open privacy-aware network measurement, analysis, and visualization service designed to help end-users visualize and reason about large data transfers. NetSage traditionally has used a combination of passive measurements, including SNMP and flow data, as well as active measurements, mainly perfSONAR, to provide longitudinal network performance data visualization. It has been deployed by dozens of networks world wide, and is supported domestically by the Engagement and Performance Operations Center (EPOC), NSF #2328479. We have recently expanded the NetSage data sources to include logs for Globus data transfers, following the same privacy-preserving approach as for Flow data. Using the logs for the Texas Advanced Computing Center (TACC) as an example, this talk will walk through several different example use cases that NetSage can answer, including: Who is using Globus to share data with my institution, and what kind of performance are they able to achieve? How many transfers has Globus supported for us? Which sites are we sharing the most data with, and how is that changing over time? How is my site using Globus to move data internally, and what kind of performance do we see for those transfers? What percentage of data transfers at my institution used Globus, and how did the overall data transfer performance compare to the Globus users?

Understanding Globus Data Transfers with NetSage

Globus

Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...

Abortion Clinic

GlobusWorld 2024 Opening Keynote session

Globus

Recently uploaded (20)