SlideShare a Scribd company logo
github.com/gianarb/orbiter
Gianluca Arbezzano
@gianarb
Building clouds at InfluxDB (SRE)
Why?
● To create a use case around metrics, InfluxDB and Docker.
● Docker Swarm at the moment doesn’t provider autoscale capabilities.
● To design an application easy to manage and well integrated in Docker Swarm.
Orbiter is a general-purpose multi vendor autoscaler
Orbiter is a general-purpose multi vendor autoscaler
Orbiter is a Docker Swarm autoscaler
swarm cluster
manager
get metrics:
● cadvidor
● telegraf
● ….
alert:
● custom
● kapacitor
● alert
manager
Why Docker Swarm doesn’t have autoscaling?
● I don’t know.
● As you can see the architecture required is not small.
● Probably it will have autoscaling at some point.
Orbiter is designed to be Only-Swarm
● To make it easy to start
● To remove abstractions
● Because less is more! cit unknown but for sure a lazy person
What does it?
● It lists all your services and it gets only who has label orbiter=true
● Expose and HTTP API to trigger up and down scale event
○ curl -X POST -v http://$(IP):8081/v1/orbiter/handle/autoswarm/stack_micro/up
○ curl -X POST -v http://$(IP):8081/v1/orbiter/handle/autoswarm/stack_micro/down
● It uses Docker Swarm Api to keep in sync the managed services
● You can configure a service via label orbiter.up=3, orbiter.donw=2, orbiter.cooldown = 2
Requirement:
● Deploy it in a manager.
Deploy it as service in
Docker Swarm
version: '3'
services:
orbiter:
image: gianarb/orbiter
command: daemon
ports:
- 8081:8000
volumes:
- /var/run/docker.sock:/var/run/docker.sock
deploy:
placement:
constraints:
- node.role == manager
mode: replicated
replicas: 1
How Orbiter communicate with Swarm
func NewSwarmProvider(c map[string]string) (autoscaler.Provider, error) {
var p autoscaler.Provider
client, err := docker.NewEnvClient()
if err != nil {
logrus.WithField("error", err).Warn("problem to communicate with docker")
return p, err
} else {
logrus.Info("Successfully connected to a Docker daemon")
}
p = SwarmProvider{
dockerClient: client,
}
return p, nil
}
Scale a service - 1.Find the service
service, _, err := p.dockerClient.ServiceInspectWithRaw(ctx, serviceId, types.ServiceInspectOptions{
InsertDefaults: true,
})
if err != nil {
logrus.WithFields(logrus.Fields{
"error": err.Error(),
"provider": "swarm",
}).Debugf("Service %s didn't scale. We didn't get it from docker.", serviceId)
return err
}
filters := filters.NewArgs()
filters.Add("service", serviceId)
tasks, err := p.dockerClient.TaskList(ctx, types.TaskListOptions{Filters: filters})
if err != nil { … }
err = p.isAcceptable(tasks, target, direction)
Scale a service - 2.Update the Spec
spec := service.Spec
var ptrFromSystem uint64
base := p.calculateActiveTasks(tasks)
if direction == true {
ptrFromSystem = uint64(base + target)
} else {
ptrFromSystem = uint64(base - target)
}
spec.Mode.Replicated.Replicas = &ptrFromSystem
_, err = p.dockerClient.ServiceUpdate(ctx, serviceId, service.Version, spec, types.ServiceUpdateOptions{})
if err != nil {...}
Background job to sync services with Docker
services, err := dockerClient.ServiceList(ctx, types.ServiceListOptions{})
if err != nil {
logrus.WithField("error", err).Debug("Bad communication with Docker.")
return
}
prov, _ := provider.NewSwarmProvider(map[string]string{})
for _, service := range services {
s, err := getAutoscalerByService(prov, service.Spec.Annotations)
if err != nil {
continue
}
c.Autoscalers[fmt.Sprintf("autoswarm/%s", service.Spec.Annotations.Name)] = s
}
Is orbiter just a wrapper on Docker Swarm API?
Let’s put that in a cooler way.
Orbiter provides easy to use rest API to
scale your services up and down.
T-shirt time
Demo
Thanks a lot
https://twitter.com/gianarb
https://github.com/gianarb
https://gianarb.it

More Related Content

What's hot

A dive into akka streams: from the basics to a real-world scenario
A dive into akka streams: from the basics to a real-world scenarioA dive into akka streams: from the basics to a real-world scenario
A dive into akka streams: from the basics to a real-world scenario
Gioia Ballin
 
CLS & asyncListener: asynchronous observability for Node.js
CLS & asyncListener: asynchronous observability for Node.jsCLS & asyncListener: asynchronous observability for Node.js
CLS & asyncListener: asynchronous observability for Node.js
Forrest Norvell
 
Callbacks and control flow in Node js
Callbacks and control flow in Node jsCallbacks and control flow in Node js
Callbacks and control flow in Node js
Thomas Roch
 
Asynchronous handlers in asp.net
Asynchronous handlers in asp.netAsynchronous handlers in asp.net
Asynchronous handlers in asp.net
Abhishek Sur
 
Using akka streams to access s3 objects
Using akka streams to access s3 objectsUsing akka streams to access s3 objects
Using akka streams to access s3 objects
Mikhail Girkin
 
Using SaltStack to orchestrate microservices in application containers at Sal...
Using SaltStack to orchestrate microservices in application containers at Sal...Using SaltStack to orchestrate microservices in application containers at Sal...
Using SaltStack to orchestrate microservices in application containers at Sal...
Love Nyberg
 
Go With The Flow
Go With The FlowGo With The Flow
Go With The Flow
PhilWinstanley
 
Elixir at Evercam (By Milos Mosic)
Elixir at Evercam (By Milos Mosic)Elixir at Evercam (By Milos Mosic)
Asynchronní programování
Asynchronní programováníAsynchronní programování
Asynchronní programování
PeckaDesign.cz
 
Neighborly nagios
Neighborly nagiosNeighborly nagios
Neighborly nagios
David Josephsen
 
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
MongoDB
 
Javascript internals
Javascript internalsJavascript internals
Javascript internals
Ayush Sharma
 
Reactive mistakes - ScalaDays Chicago 2017
Reactive mistakes -  ScalaDays Chicago 2017Reactive mistakes -  ScalaDays Chicago 2017
Reactive mistakes - ScalaDays Chicago 2017
Petr Zapletal
 
Unity 3D Runtime Animation Generation
Unity 3D Runtime Animation GenerationUnity 3D Runtime Animation Generation
Unity 3D Runtime Animation Generation
Dustin Graham
 
Converting a naive flow to akka streams
Converting a naive flow to akka streamsConverting a naive flow to akka streams
Converting a naive flow to akka streams
Gal Topper
 
Async Web QA
Async Web QAAsync Web QA
Async Web QA
Vlad Maniak
 
Workshop 22: React-Redux Middleware
Workshop 22: React-Redux MiddlewareWorkshop 22: React-Redux Middleware
Workshop 22: React-Redux Middleware
Visual Engineering
 
Functional Programming with JavaScript
Functional Programming with JavaScriptFunctional Programming with JavaScript
Functional Programming with JavaScript
Mark Shelton
 
Gearman & PHP
Gearman & PHPGearman & PHP
Gearman & PHP
Nemanja Krivokapic
 
Hadoop Map Reduce Arch
Hadoop Map Reduce ArchHadoop Map Reduce Arch
Hadoop Map Reduce Arch
Jeff Hammerbacher
 

What's hot (20)

A dive into akka streams: from the basics to a real-world scenario
A dive into akka streams: from the basics to a real-world scenarioA dive into akka streams: from the basics to a real-world scenario
A dive into akka streams: from the basics to a real-world scenario
 
CLS & asyncListener: asynchronous observability for Node.js
CLS & asyncListener: asynchronous observability for Node.jsCLS & asyncListener: asynchronous observability for Node.js
CLS & asyncListener: asynchronous observability for Node.js
 
Callbacks and control flow in Node js
Callbacks and control flow in Node jsCallbacks and control flow in Node js
Callbacks and control flow in Node js
 
Asynchronous handlers in asp.net
Asynchronous handlers in asp.netAsynchronous handlers in asp.net
Asynchronous handlers in asp.net
 
Using akka streams to access s3 objects
Using akka streams to access s3 objectsUsing akka streams to access s3 objects
Using akka streams to access s3 objects
 
Using SaltStack to orchestrate microservices in application containers at Sal...
Using SaltStack to orchestrate microservices in application containers at Sal...Using SaltStack to orchestrate microservices in application containers at Sal...
Using SaltStack to orchestrate microservices in application containers at Sal...
 
Go With The Flow
Go With The FlowGo With The Flow
Go With The Flow
 
Elixir at Evercam (By Milos Mosic)
Elixir at Evercam (By Milos Mosic)Elixir at Evercam (By Milos Mosic)
Elixir at Evercam (By Milos Mosic)
 
Asynchronní programování
Asynchronní programováníAsynchronní programování
Asynchronní programování
 
Neighborly nagios
Neighborly nagiosNeighborly nagios
Neighborly nagios
 
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
 
Javascript internals
Javascript internalsJavascript internals
Javascript internals
 
Reactive mistakes - ScalaDays Chicago 2017
Reactive mistakes -  ScalaDays Chicago 2017Reactive mistakes -  ScalaDays Chicago 2017
Reactive mistakes - ScalaDays Chicago 2017
 
Unity 3D Runtime Animation Generation
Unity 3D Runtime Animation GenerationUnity 3D Runtime Animation Generation
Unity 3D Runtime Animation Generation
 
Converting a naive flow to akka streams
Converting a naive flow to akka streamsConverting a naive flow to akka streams
Converting a naive flow to akka streams
 
Async Web QA
Async Web QAAsync Web QA
Async Web QA
 
Workshop 22: React-Redux Middleware
Workshop 22: React-Redux MiddlewareWorkshop 22: React-Redux Middleware
Workshop 22: React-Redux Middleware
 
Functional Programming with JavaScript
Functional Programming with JavaScriptFunctional Programming with JavaScript
Functional Programming with JavaScript
 
Gearman & PHP
Gearman & PHPGearman & PHP
Gearman & PHP
 
Hadoop Map Reduce Arch
Hadoop Map Reduce ArchHadoop Map Reduce Arch
Hadoop Map Reduce Arch
 

Similar to Orbiter and how to extend Docker Swarm

ILM - Pipeline in the cloud
ILM - Pipeline in the cloudILM - Pipeline in the cloud
ILM - Pipeline in the cloud
Aaron Carey
 
Easy Cloud Native Transformation using HashiCorp Nomad
Easy Cloud Native Transformation using HashiCorp NomadEasy Cloud Native Transformation using HashiCorp Nomad
Easy Cloud Native Transformation using HashiCorp Nomad
Bram Vogelaar
 
Docker swarm introduction
Docker swarm introductionDocker swarm introduction
Docker swarm introduction
Evan Lin
 
Building a serverless company on AWS lambda and Serverless framework
Building a serverless company on AWS lambda and Serverless frameworkBuilding a serverless company on AWS lambda and Serverless framework
Building a serverless company on AWS lambda and Serverless framework
Luciano Mammino
 
Microservices with docker swarm and consul
Microservices with docker swarm and consulMicroservices with docker swarm and consul
Microservices with docker swarm and consul
Nguyen Sy Thanh Son
 
DCEU 18: Docker Containers in a Serverless World
DCEU 18: Docker Containers in a Serverless WorldDCEU 18: Docker Containers in a Serverless World
DCEU 18: Docker Containers in a Serverless World
Docker, Inc.
 
Lightbend Lagom: Microservices Just Right
Lightbend Lagom: Microservices Just RightLightbend Lagom: Microservices Just Right
Lightbend Lagom: Microservices Just Right
mircodotta
 
Higher order infrastructure: from Docker basics to cluster management - Nicol...
Higher order infrastructure: from Docker basics to cluster management - Nicol...Higher order infrastructure: from Docker basics to cluster management - Nicol...
Higher order infrastructure: from Docker basics to cluster management - Nicol...
Codemotion
 
ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...
ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...
ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...
European Collaboration Summit
 
Dockerizing Rails
Dockerizing RailsDockerizing Rails
Dockerizing Rails
iGbanam
 
Build Your Kubernetes Operator with the Right Tool!
Build Your Kubernetes Operator with the Right Tool!Build Your Kubernetes Operator with the Right Tool!
Build Your Kubernetes Operator with the Right Tool!
Rafał Leszko
 
Build your operator with the right tool
Build your operator with the right toolBuild your operator with the right tool
Build your operator with the right tool
Rafał Leszko
 
Docker in production service discovery with consul - road to opscon 2015
Docker in production  service discovery with consul - road to opscon 2015Docker in production  service discovery with consul - road to opscon 2015
Docker in production service discovery with consul - road to opscon 2015
Giovanni Toraldo
 
Altitude SF 2017: Nomad and next-gen application architectures
Altitude SF 2017: Nomad and next-gen application architecturesAltitude SF 2017: Nomad and next-gen application architectures
Altitude SF 2017: Nomad and next-gen application architectures
Fastly
 
Serverless APIs with JavaScript - Matt Searle - ChocPanda
Serverless APIs with JavaScript - Matt Searle - ChocPandaServerless APIs with JavaScript - Matt Searle - ChocPanda
Serverless APIs with JavaScript - Matt Searle - ChocPanda
Paul Dykes
 
Swarm mode
Swarm modeSwarm mode
Swarm mode
Dharmit Shah
 
Docker 進階實務班
Docker 進階實務班Docker 進階實務班
Docker 進階實務班
Philip Zheng
 
Why Grails
Why GrailsWhy Grails
Why Grails
Yiguang Hu
 
Why Grails?
Why Grails?Why Grails?
Why Grails?
Yiguang Hu
 
Kubernetes #1 intro
Kubernetes #1   introKubernetes #1   intro
Kubernetes #1 intro
Terry Cho
 

Similar to Orbiter and how to extend Docker Swarm (20)

ILM - Pipeline in the cloud
ILM - Pipeline in the cloudILM - Pipeline in the cloud
ILM - Pipeline in the cloud
 
Easy Cloud Native Transformation using HashiCorp Nomad
Easy Cloud Native Transformation using HashiCorp NomadEasy Cloud Native Transformation using HashiCorp Nomad
Easy Cloud Native Transformation using HashiCorp Nomad
 
Docker swarm introduction
Docker swarm introductionDocker swarm introduction
Docker swarm introduction
 
Building a serverless company on AWS lambda and Serverless framework
Building a serverless company on AWS lambda and Serverless frameworkBuilding a serverless company on AWS lambda and Serverless framework
Building a serverless company on AWS lambda and Serverless framework
 
Microservices with docker swarm and consul
Microservices with docker swarm and consulMicroservices with docker swarm and consul
Microservices with docker swarm and consul
 
DCEU 18: Docker Containers in a Serverless World
DCEU 18: Docker Containers in a Serverless WorldDCEU 18: Docker Containers in a Serverless World
DCEU 18: Docker Containers in a Serverless World
 
Lightbend Lagom: Microservices Just Right
Lightbend Lagom: Microservices Just RightLightbend Lagom: Microservices Just Right
Lightbend Lagom: Microservices Just Right
 
Higher order infrastructure: from Docker basics to cluster management - Nicol...
Higher order infrastructure: from Docker basics to cluster management - Nicol...Higher order infrastructure: from Docker basics to cluster management - Nicol...
Higher order infrastructure: from Docker basics to cluster management - Nicol...
 
ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...
ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...
ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...
 
Dockerizing Rails
Dockerizing RailsDockerizing Rails
Dockerizing Rails
 
Build Your Kubernetes Operator with the Right Tool!
Build Your Kubernetes Operator with the Right Tool!Build Your Kubernetes Operator with the Right Tool!
Build Your Kubernetes Operator with the Right Tool!
 
Build your operator with the right tool
Build your operator with the right toolBuild your operator with the right tool
Build your operator with the right tool
 
Docker in production service discovery with consul - road to opscon 2015
Docker in production  service discovery with consul - road to opscon 2015Docker in production  service discovery with consul - road to opscon 2015
Docker in production service discovery with consul - road to opscon 2015
 
Altitude SF 2017: Nomad and next-gen application architectures
Altitude SF 2017: Nomad and next-gen application architecturesAltitude SF 2017: Nomad and next-gen application architectures
Altitude SF 2017: Nomad and next-gen application architectures
 
Serverless APIs with JavaScript - Matt Searle - ChocPanda
Serverless APIs with JavaScript - Matt Searle - ChocPandaServerless APIs with JavaScript - Matt Searle - ChocPanda
Serverless APIs with JavaScript - Matt Searle - ChocPanda
 
Swarm mode
Swarm modeSwarm mode
Swarm mode
 
Docker 進階實務班
Docker 進階實務班Docker 進階實務班
Docker 進階實務班
 
Why Grails
Why GrailsWhy Grails
Why Grails
 
Why Grails?
Why Grails?Why Grails?
Why Grails?
 
Kubernetes #1 intro
Kubernetes #1   introKubernetes #1   intro
Kubernetes #1 intro
 

More from Gianluca Arbezzano

Kubernetes debug like a pro
Kubernetes debug like a proKubernetes debug like a pro
Kubernetes debug like a pro
Gianluca Arbezzano
 
OSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoringOSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoring
Gianluca Arbezzano
 
Value of your metrics: goodbye monitoring, welcome observability
Value of your metrics: goodbye monitoring, welcome observabilityValue of your metrics: goodbye monitoring, welcome observability
Value of your metrics: goodbye monitoring, welcome observability
Gianluca Arbezzano
 
InfluxCloudi craft container orchestrator
InfluxCloudi craft container orchestratorInfluxCloudi craft container orchestrator
InfluxCloudi craft container orchestrator
Gianluca Arbezzano
 
Overview and Opentracing in theory by Gianluca Arbezzano
Overview and Opentracing in theory by Gianluca ArbezzanoOverview and Opentracing in theory by Gianluca Arbezzano
Overview and Opentracing in theory by Gianluca Arbezzano
Gianluca Arbezzano
 
Monitoring Pull vs Push, InfluxDB and Prometheus
Monitoring Pull vs Push, InfluxDB and PrometheusMonitoring Pull vs Push, InfluxDB and Prometheus
Monitoring Pull vs Push, InfluxDB and Prometheus
Gianluca Arbezzano
 
Open Tracing, to order and understand your mess. - ApiConf 2017
Open Tracing, to order and understand your mess. - ApiConf 2017Open Tracing, to order and understand your mess. - ApiConf 2017
Open Tracing, to order and understand your mess. - ApiConf 2017
Gianluca Arbezzano
 
Security Tips to run Docker in Production
Security Tips to run Docker in ProductionSecurity Tips to run Docker in Production
Security Tips to run Docker in Production
Gianluca Arbezzano
 
Jenkins in the real world - DevOpsCon 2017
Jenkins in the real world - DevOpsCon 2017Jenkins in the real world - DevOpsCon 2017
Jenkins in the real world - DevOpsCon 2017
Gianluca Arbezzano
 
Monitor your application and sleep
Monitor your application and sleepMonitor your application and sleep
Monitor your application and sleep
Gianluca Arbezzano
 
Tick Stack - Listen your infrastructure and please sleep
Tick Stack - Listen your infrastructure and please sleepTick Stack - Listen your infrastructure and please sleep
Tick Stack - Listen your infrastructure and please sleep
Gianluca Arbezzano
 
Docker Novosibirsk Meetup #3 - Docker in Production
Docker Novosibirsk Meetup #3 - Docker in ProductionDocker Novosibirsk Meetup #3 - Docker in Production
Docker Novosibirsk Meetup #3 - Docker in Production
Gianluca Arbezzano
 
DockerDublin Meetup - News about Docker 1.13
DockerDublin Meetup -  News about Docker 1.13DockerDublin Meetup -  News about Docker 1.13
DockerDublin Meetup - News about Docker 1.13
Gianluca Arbezzano
 
Docker 1.12 and SwarmKit
Docker 1.12 and SwarmKitDocker 1.12 and SwarmKit
Docker 1.12 and SwarmKit
Gianluca Arbezzano
 
Time Series Database and Tick Stack
Time Series Database and Tick StackTime Series Database and Tick Stack
Time Series Database and Tick Stack
Gianluca Arbezzano
 
Queue System and Zend\Queue implementation
Queue System and Zend\Queue implementationQueue System and Zend\Queue implementation
Queue System and Zend\Queue implementation
Gianluca Arbezzano
 
ZfDayIt 2014 - There is a module for everything
ZfDayIt 2014 - There is a module for everythingZfDayIt 2014 - There is a module for everything
ZfDayIt 2014 - There is a module for everything
Gianluca Arbezzano
 
Vagrant - PugMI
Vagrant - PugMIVagrant - PugMI
Vagrant - PugMI
Gianluca Arbezzano
 
Silex, iniziamo
Silex, iniziamoSilex, iniziamo
Silex, iniziamo
Gianluca Arbezzano
 

More from Gianluca Arbezzano (19)

Kubernetes debug like a pro
Kubernetes debug like a proKubernetes debug like a pro
Kubernetes debug like a pro
 
OSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoringOSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoring
 
Value of your metrics: goodbye monitoring, welcome observability
Value of your metrics: goodbye monitoring, welcome observabilityValue of your metrics: goodbye monitoring, welcome observability
Value of your metrics: goodbye monitoring, welcome observability
 
InfluxCloudi craft container orchestrator
InfluxCloudi craft container orchestratorInfluxCloudi craft container orchestrator
InfluxCloudi craft container orchestrator
 
Overview and Opentracing in theory by Gianluca Arbezzano
Overview and Opentracing in theory by Gianluca ArbezzanoOverview and Opentracing in theory by Gianluca Arbezzano
Overview and Opentracing in theory by Gianluca Arbezzano
 
Monitoring Pull vs Push, InfluxDB and Prometheus
Monitoring Pull vs Push, InfluxDB and PrometheusMonitoring Pull vs Push, InfluxDB and Prometheus
Monitoring Pull vs Push, InfluxDB and Prometheus
 
Open Tracing, to order and understand your mess. - ApiConf 2017
Open Tracing, to order and understand your mess. - ApiConf 2017Open Tracing, to order and understand your mess. - ApiConf 2017
Open Tracing, to order and understand your mess. - ApiConf 2017
 
Security Tips to run Docker in Production
Security Tips to run Docker in ProductionSecurity Tips to run Docker in Production
Security Tips to run Docker in Production
 
Jenkins in the real world - DevOpsCon 2017
Jenkins in the real world - DevOpsCon 2017Jenkins in the real world - DevOpsCon 2017
Jenkins in the real world - DevOpsCon 2017
 
Monitor your application and sleep
Monitor your application and sleepMonitor your application and sleep
Monitor your application and sleep
 
Tick Stack - Listen your infrastructure and please sleep
Tick Stack - Listen your infrastructure and please sleepTick Stack - Listen your infrastructure and please sleep
Tick Stack - Listen your infrastructure and please sleep
 
Docker Novosibirsk Meetup #3 - Docker in Production
Docker Novosibirsk Meetup #3 - Docker in ProductionDocker Novosibirsk Meetup #3 - Docker in Production
Docker Novosibirsk Meetup #3 - Docker in Production
 
DockerDublin Meetup - News about Docker 1.13
DockerDublin Meetup -  News about Docker 1.13DockerDublin Meetup -  News about Docker 1.13
DockerDublin Meetup - News about Docker 1.13
 
Docker 1.12 and SwarmKit
Docker 1.12 and SwarmKitDocker 1.12 and SwarmKit
Docker 1.12 and SwarmKit
 
Time Series Database and Tick Stack
Time Series Database and Tick StackTime Series Database and Tick Stack
Time Series Database and Tick Stack
 
Queue System and Zend\Queue implementation
Queue System and Zend\Queue implementationQueue System and Zend\Queue implementation
Queue System and Zend\Queue implementation
 
ZfDayIt 2014 - There is a module for everything
ZfDayIt 2014 - There is a module for everythingZfDayIt 2014 - There is a module for everything
ZfDayIt 2014 - There is a module for everything
 
Vagrant - PugMI
Vagrant - PugMIVagrant - PugMI
Vagrant - PugMI
 
Silex, iniziamo
Silex, iniziamoSilex, iniziamo
Silex, iniziamo
 

Recently uploaded

Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Precisely
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
saastr
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
AstuteBusiness
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Postman
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 

Recently uploaded (20)

Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 

Orbiter and how to extend Docker Swarm

  • 3. Why? ● To create a use case around metrics, InfluxDB and Docker. ● Docker Swarm at the moment doesn’t provider autoscale capabilities. ● To design an application easy to manage and well integrated in Docker Swarm.
  • 4. Orbiter is a general-purpose multi vendor autoscaler
  • 5. Orbiter is a general-purpose multi vendor autoscaler Orbiter is a Docker Swarm autoscaler
  • 6. swarm cluster manager get metrics: ● cadvidor ● telegraf ● …. alert: ● custom ● kapacitor ● alert manager
  • 7. Why Docker Swarm doesn’t have autoscaling? ● I don’t know. ● As you can see the architecture required is not small. ● Probably it will have autoscaling at some point.
  • 8. Orbiter is designed to be Only-Swarm ● To make it easy to start ● To remove abstractions ● Because less is more! cit unknown but for sure a lazy person
  • 9. What does it? ● It lists all your services and it gets only who has label orbiter=true ● Expose and HTTP API to trigger up and down scale event ○ curl -X POST -v http://$(IP):8081/v1/orbiter/handle/autoswarm/stack_micro/up ○ curl -X POST -v http://$(IP):8081/v1/orbiter/handle/autoswarm/stack_micro/down ● It uses Docker Swarm Api to keep in sync the managed services ● You can configure a service via label orbiter.up=3, orbiter.donw=2, orbiter.cooldown = 2
  • 10. Requirement: ● Deploy it in a manager.
  • 11. Deploy it as service in Docker Swarm version: '3' services: orbiter: image: gianarb/orbiter command: daemon ports: - 8081:8000 volumes: - /var/run/docker.sock:/var/run/docker.sock deploy: placement: constraints: - node.role == manager mode: replicated replicas: 1
  • 12. How Orbiter communicate with Swarm func NewSwarmProvider(c map[string]string) (autoscaler.Provider, error) { var p autoscaler.Provider client, err := docker.NewEnvClient() if err != nil { logrus.WithField("error", err).Warn("problem to communicate with docker") return p, err } else { logrus.Info("Successfully connected to a Docker daemon") } p = SwarmProvider{ dockerClient: client, } return p, nil }
  • 13.
  • 14. Scale a service - 1.Find the service service, _, err := p.dockerClient.ServiceInspectWithRaw(ctx, serviceId, types.ServiceInspectOptions{ InsertDefaults: true, }) if err != nil { logrus.WithFields(logrus.Fields{ "error": err.Error(), "provider": "swarm", }).Debugf("Service %s didn't scale. We didn't get it from docker.", serviceId) return err } filters := filters.NewArgs() filters.Add("service", serviceId) tasks, err := p.dockerClient.TaskList(ctx, types.TaskListOptions{Filters: filters}) if err != nil { … } err = p.isAcceptable(tasks, target, direction)
  • 15. Scale a service - 2.Update the Spec spec := service.Spec var ptrFromSystem uint64 base := p.calculateActiveTasks(tasks) if direction == true { ptrFromSystem = uint64(base + target) } else { ptrFromSystem = uint64(base - target) } spec.Mode.Replicated.Replicas = &ptrFromSystem _, err = p.dockerClient.ServiceUpdate(ctx, serviceId, service.Version, spec, types.ServiceUpdateOptions{}) if err != nil {...}
  • 16. Background job to sync services with Docker services, err := dockerClient.ServiceList(ctx, types.ServiceListOptions{}) if err != nil { logrus.WithField("error", err).Debug("Bad communication with Docker.") return } prov, _ := provider.NewSwarmProvider(map[string]string{}) for _, service := range services { s, err := getAutoscalerByService(prov, service.Spec.Annotations) if err != nil { continue } c.Autoscalers[fmt.Sprintf("autoswarm/%s", service.Spec.Annotations.Name)] = s }
  • 17. Is orbiter just a wrapper on Docker Swarm API? Let’s put that in a cooler way. Orbiter provides easy to use rest API to scale your services up and down.
  • 19. Demo
  • 20.