SlideShare a Scribd company logo
1 of 55
Download to read offline
Docker at Nuxeo
Who we are
What is Nuxeo platform
• A platform to develop you own content apps
• Highly pluggable thru extension points
• Model you content with Nuxeo Studio
• Point and click UI to define your business model
• with hot-reload
Nuxeo tech. stack
• Java stack : Seam + JSF for Nuxeo DM and DAM
• Rest API to build your JS Apps
• Nuxeo Studio (Saas) to model your business
• Thru data model (doc types, schemas)
• Thru Automation chains
Nuxeo in the cloud ?
Goals
• Customers ask to host their app
• Provide quick access to test instances (onboarding)
• Provide onDemand dev or CI instances
Goals
• Customers ask to host their app
• Provide quick access to test instances (onboarding)
• Provide onDemand dev or CI instances
This is the nuxeo.io promise
First attempt
Manual OnDemand
• Based on AWS CloudFormation
• Manually deployed
• Works but :
• not scalable
• manual admin & monitoring
• difficult upgrade process
Second attempt
AWS Automation
• Provisionning may be automated
• Working mockup (python, celery, boto) but…
• Strong adherence to AWS
• difficult to test
• slow provisionning (one EC2 instance per Nuxeo instance)
• upgrade process still difficult
• scaling out is difficult
• sensible to failures
This leads to our
challenge
Current challenge
• Build a scalable infrastructure
• Quick provisioning (less than a minute)
• Easy setup (no admin needed)
• Easy upgrade process
• Automated healing
• Easy to test
How Docker solves
some of these issues ?
Quick provisioning
• We don’t have to start
a server
• No VM overhead
• Start time is Nuxeo
start time
Data free containers
• Docker container
doesn’t hold data
• Upgrading is just using
a new version of the
image
• Healing means killing
and restarting
Nuxeo v5.9.3
S3Postgres
Data free containers
• Docker container
doesn’t hold data
• Upgrading is just using
a new version of the
image
• Healing means killing
and restarting
S3Postgres
Data free containers
• Docker container
doesn’t hold data
• Upgrading is just using
a new version of the
image
• Healing means killing
and restarting
S3Postgres
Nuxeo v5.9.4
Several containers per server
• Scaling is just a matter
of starting several
containers
• But means infinite CPU
/ RAM and IO
Nuxeo.io server
io
container
NXIO-0001
io
container
NXIO-0002
io
container
NXIO-0003
io
container
NXIO-XXXX
…
Challenge status
• Build a scalable infrastructure
• Quick provisioning (less than a minute)
• Easy setup (no admin needed)
• Easy upgrade process
• Automated healing
• Easy to test
How to solve the
other issues ?
… with a little help from my friends
The infinite resource computer…
… is a cluster
Docker is not sufficient
• Docker handles processes at the OS level
• On the cluster level we need a companion
• To load balance the processes
• To check the statuses of the processes
• We need a docker ps at the cluster level
Introducing CoreOS
• A minimal Linux distribution (small footprint)
• Designed to handle cluster
• Providing cluster tooling
• Docker
• etcd
• fleet
https://coreos.com/
etcd : service discovery
• etcd is
• small distributed key value store
• keys may have TTL
• using Raft protocol to elect cluster leader
• suitable for :
• shared configuration
• service discovery
fleet : clustered init system
• fleet is :
• an init system at the cluster level
• compatible with systemd unit files
• suitable for :
• launching docker containers
• distributing load across cluster
• get the status of the jobs
fleet demo
• Deploy our nuxeo
container in the cluster
Challenge status
• Build a scalable infrastructure
• Quick provisioning (less than a minute)
• Easy setup (no admin needed)
• Easy upgrade process
• Automated healing
• Easy to test
What are the admin tasks ?
• Launch the fleetctl command
• Setup DNS and reverse proxies
• Link container to external services (S3, Postgres etc…)
DNS and reverse proxy
• docker start containers and link host port to container
port
• The cluster has to redirect calls to mycustomDNS.io to the
running container meaning knowing
• the ip of the host
• the port on the host
Introducing Gogeta
!
/services/NXIO-0001/location:{host:172.31.51.1,port:49153}!
                /status/current:started!
                      /expected:started!
!! ! ! ! ! /alive:1!
/domains/mydomain.nuxeo.io/type:iocontainer!
                          /value:NXIO-0001
Given this etcd configuration
Cluster
CoreOS
172.31.51.20
etcd
io
container
NXIO-0098
systemd
CoreOS
172.31.51.1
etcd
io
container
NXIO-0001 Gogeta
systemd
… … … …
Fleet
…
Gogeta
…
Introducing Gogeta
Round robin load Balancer
etcd !
/services/NXIO-0098/location:{host:172.31.51.20,port:49153}!
                /status/current:started!
                      /expected:started!
!! ! ! ! ! /alive:1!
/domains/mydomain.nuxeo.io/type:iocontainer!
                          /value:NXIO-0001
Cluster
CoreOS
172.31.51.20
etcd
io
container
NXIO-0098
systemd
CoreOS
172.31.51.1
etcd
io
container
NXIO-0001 Gogeta
systemd
… … … …
Fleet
…
Gogeta
…
Introducing Gogeta
Round robin load Balancer
etcd !
/services/NXIO-0098/location:{host:172.31.51.20,port:49153}!
                /status/current:started!
                      /expected:started!
!! ! ! ! ! /alive:1!
/domains/mydomain.nuxeo.io/type:iocontainer!
                          /value:NXIO-0001
Cluster
CoreOS
172.31.51.20
etcd
io
container
NXIO-0098
systemd
CoreOS
172.31.51.1
etcd
io
container
NXIO-0001 Gogeta
systemd
… … … …
Fleet
…
Gogeta
…
Introducing Gogeta
Round robin load Balancer
etcd !
/services/NXIO-0098/location:{host:172.31.51.20,port:49153}!
                /status/current:started!
                      /expected:started!
!! ! ! ! ! /alive:1!
/domains/mydomain.nuxeo.io/type:iocontainer!
                          /value:NXIO-0001
Cluster
CoreOS
172.31.51.20
etcd
io
container
NXIO-0098
systemd
CoreOS
172.31.51.1
etcd
io
container
NXIO-0001 Gogeta
systemd
… … … …
Fleet
…
Gogeta
…
Introducing Gogeta
Round robin load Balancer
etcd !
/services/NXIO-0098/location:{host:172.31.51.20,port:49153}!
                /status/current:started!
                      /expected:started!
!! ! ! ! ! /alive:1!
/domains/mydomain.nuxeo.io/type:iocontainer!
                          /value:NXIO-0001
Cluster
CoreOS
172.31.51.20
etcd
io
container
NXIO-0098
systemd
CoreOS
172.31.51.1
etcd
io
container
NXIO-0001 Gogeta
systemd
… … … …
Fleet
…
Gogeta
…
Introducing Gogeta
Round robin load Balancer
etcd !
/services/NXIO-0098/location:{host:172.31.51.20,port:49153}!
                /status/current:started!
                      /expected:started!
!! ! ! ! ! /alive:1!
/domains/mydomain.nuxeo.io/type:iocontainer!
                          /value:NXIO-0001
Cluster
CoreOS
172.31.51.20
etcd
io
container
NXIO-0098
systemd
CoreOS
172.31.51.1
etcd
io
container
NXIO-0001 Gogeta
systemd
… … … …
Fleet
…
Gogeta
…
Introducing Gogeta
Round robin load Balancer
etcd !
/services/NXIO-0098/location:{host:172.31.51.20,port:49153}!
                /status/current:started!
                      /expected:started!
!! ! ! ! ! /alive:1!
/domains/mydomain.nuxeo.io/type:iocontainer!
                          /value:NXIO-0001
Introducing Gogeta
• Easy to start as a docker container
!
!
• Written in Go
• Available at : http://github.com/nuxeo/gogeta
/usr/bin/docker run --rm --name gogeta -p 7777:7777 nuxeo/gogeta
Application startup unit
!
[Unit]!
Description=Sample startup script!
!
[Service]!
ExecStartPre= Set starting status in etcd!
ExecStart= Launch docker container!
ExecStartPost= Wait container to be launched and set app location in etcd!
ExecStop= Stop docker container!
ExecStopPost= Set stopped status in etcd!
Linking services
• Docker recommend to use ambassadors to link services
• Service may be hosted on other servers or outside of the
cluster
• We’ve implemented dynamic ambassadors
• http://coreos.com/blog/docker-dynamic-ambassador-
powered-by-etcd/
Cluster
CoreOS
172.31.51.20
etcd
Postgres
systemd
CoreOS
172.31.51.1
etcd
io-container
NXIO-0001
systemd
… … … …
Fleet
postgres-amb
Dynamic ambassador
etcd !
/services/postgres/location:{host:172.31.51.20,port:49159}!
https://github.com/nuxeo/etcd-netfw
Cluster
CoreOS
172.31.51.20
etcd
Postgres
systemd
CoreOS
172.31.51.1
etcd
io-container
NXIO-0001
systemd
… … … …
Fleet
postgres-amb
Dynamic ambassador
etcd !
/services/postgres/location:{host:172.31.51.20,port:49159}!
https://github.com/nuxeo/etcd-netfw
Cluster
CoreOS
172.31.51.20
etcd
Postgres
systemd
CoreOS
172.31.51.1
etcd
io-container
NXIO-0001
systemd
… … … …
Fleet
postgres-amb
Dynamic ambassador
etcd !
/services/postgres/location:{host:172.31.51.20,port:49159}!
https://github.com/nuxeo/etcd-netfw
Dynamic ambassador
• Very easy to start
!
• Uses https://github.com/nuxeo/etcd-netfw
docker run --rm --name postgres-amb -P nuxeo/service-amb postgres-service
Launching commands
• A manager web app
now launches
commands :
• sets up basic etcd
values (like domain to
service resolution)
• launches fleet
commands (start & stop
apps)
• polls statuses
Challenge status
• Build a scalable infrastructure
• Quick provisioning (less than a minute)
• Easy setup (no admin needed)
• Easy upgrade process
• Automated healing
• Easy to test
Let’s resume
From nuxeo.io…
• We’ve built a cluster infrastructure that :
• knows how to deploy Docker containers (with fleet)
• Expose those containers thru domain name resolution (with
gogeta)
• Link dependent services to the container (with etcd-netfw)
From nuxeo.io…
• Given that a container should represent an application
• Running this apps means :
• Launch it with runtime parameters
• Link it to its dependent services
• Then we can build a metamodel around it
…
name: io-container!
version: 1.0!
params:!
- env_tech_id: !
- description: The id of the environment to run!
- type:string!
- clid:!
- description: a registered connect client ID!
- type:string!
!
buildimage:!
path: "./docker"!
!
dependencies:!
- postgres.service!
- s3.service!
!
… to arken
• to be able to run any application
!
• it will take care of :
• Submitting the model to the cluster
• Building the image if needed
• Launching the container
• Registering everything in etcd
arkenctl start iocontainer.ark —domain myapp.mycluster.io
Arken status
• We are extracting things from http://github.com/nuxeo/nuxeo.io
• http://github.com/arkenio
• home of gogeta and etcd
• provide cluster init instruction
• arken and arkenctl tools needs to be done
• new opensource projects (not in nuxeo repo)
• pull requests are welcome
Conclusion
• Docker helped us to enter the micro containers world
• Lots of new features : multi-repo, automatic /etc/hosts etc…
• CoreOS provided us cluster deployment
• Arken wants to provide a higher deployment model
Time for questions
@damienmetzler

More Related Content

What's hot

CloudStack Day Japan 2015 - Hypervisor Selection in CloudStack 4.5
CloudStack Day Japan 2015 - Hypervisor Selection in CloudStack 4.5CloudStack Day Japan 2015 - Hypervisor Selection in CloudStack 4.5
CloudStack Day Japan 2015 - Hypervisor Selection in CloudStack 4.5
Tim Mackey
 
Python Deployment with Fabric
Python Deployment with FabricPython Deployment with Fabric
Python Deployment with Fabric
andymccurdy
 
Erlang On Xen: Redefining the Cloud Software Stack
Erlang On Xen: Redefining the Cloud Software StackErlang On Xen: Redefining the Cloud Software Stack
Erlang On Xen: Redefining the Cloud Software Stack
Viktor Sovietov
 

What's hot (20)

Introduction to Docker storage, volume and image
Introduction to Docker storage, volume and imageIntroduction to Docker storage, volume and image
Introduction to Docker storage, volume and image
 
Apache Performance Tuning: Scaling Out
Apache Performance Tuning: Scaling OutApache Performance Tuning: Scaling Out
Apache Performance Tuning: Scaling Out
 
Docker on openstack by OpenSource Consulting
Docker on openstack by OpenSource ConsultingDocker on openstack by OpenSource Consulting
Docker on openstack by OpenSource Consulting
 
[오픈소스컨설팅] EFK Stack 소개와 설치 방법
[오픈소스컨설팅] EFK Stack 소개와 설치 방법[오픈소스컨설팅] EFK Stack 소개와 설치 방법
[오픈소스컨설팅] EFK Stack 소개와 설치 방법
 
OSv presentation from Linux Foundation Collaboration Summit
OSv presentation from Linux Foundation Collaboration SummitOSv presentation from Linux Foundation Collaboration Summit
OSv presentation from Linux Foundation Collaboration Summit
 
Live migrating a container: pros, cons and gotchas
Live migrating a container: pros, cons and gotchasLive migrating a container: pros, cons and gotchas
Live migrating a container: pros, cons and gotchas
 
AF Ceph: Ceph Performance Analysis and Improvement on Flash
AF Ceph: Ceph Performance Analysis and Improvement on FlashAF Ceph: Ceph Performance Analysis and Improvement on Flash
AF Ceph: Ceph Performance Analysis and Improvement on Flash
 
CloudStack Day Japan 2015 - Hypervisor Selection in CloudStack 4.5
CloudStack Day Japan 2015 - Hypervisor Selection in CloudStack 4.5CloudStack Day Japan 2015 - Hypervisor Selection in CloudStack 4.5
CloudStack Day Japan 2015 - Hypervisor Selection in CloudStack 4.5
 
Apache Traffic Server & Lua
Apache Traffic Server & LuaApache Traffic Server & Lua
Apache Traffic Server & Lua
 
An Updated Performance Comparison of Virtual Machines and Linux Containers
An Updated Performance Comparison of Virtual Machines and Linux ContainersAn Updated Performance Comparison of Virtual Machines and Linux Containers
An Updated Performance Comparison of Virtual Machines and Linux Containers
 
Python Deployment with Fabric
Python Deployment with FabricPython Deployment with Fabric
Python Deployment with Fabric
 
Cobbler - Fast and reliable multi-OS provisioning
Cobbler - Fast and reliable multi-OS provisioningCobbler - Fast and reliable multi-OS provisioning
Cobbler - Fast and reliable multi-OS provisioning
 
Cobbler, Func and Puppet: Tools for Large Scale Environments
Cobbler, Func and Puppet: Tools for Large Scale EnvironmentsCobbler, Func and Puppet: Tools for Large Scale Environments
Cobbler, Func and Puppet: Tools for Large Scale Environments
 
Running High Performance and Fault Tolerant Elasticsearch Clusters on Docker
Running High Performance and Fault Tolerant Elasticsearch Clusters on DockerRunning High Performance and Fault Tolerant Elasticsearch Clusters on Docker
Running High Performance and Fault Tolerant Elasticsearch Clusters on Docker
 
XPDS14 - Towards Massive Server Consolidation - Filipe Manco, NEC
XPDS14 - Towards Massive Server Consolidation - Filipe Manco, NECXPDS14 - Towards Massive Server Consolidation - Filipe Manco, NEC
XPDS14 - Towards Massive Server Consolidation - Filipe Manco, NEC
 
VMworld 2016: vSphere 6.x Host Resource Deep Dive
VMworld 2016: vSphere 6.x Host Resource Deep DiveVMworld 2016: vSphere 6.x Host Resource Deep Dive
VMworld 2016: vSphere 6.x Host Resource Deep Dive
 
To Build My Own Cloud with Blackjack…
To Build My Own Cloud with Blackjack…To Build My Own Cloud with Blackjack…
To Build My Own Cloud with Blackjack…
 
SaltConf14 - Oz Akan, Rackspace - Deploying OpenStack Marconi with SaltStack
SaltConf14 - Oz Akan, Rackspace - Deploying OpenStack Marconi with SaltStackSaltConf14 - Oz Akan, Rackspace - Deploying OpenStack Marconi with SaltStack
SaltConf14 - Oz Akan, Rackspace - Deploying OpenStack Marconi with SaltStack
 
Docker vs kvm
Docker vs kvmDocker vs kvm
Docker vs kvm
 
Erlang On Xen: Redefining the Cloud Software Stack
Erlang On Xen: Redefining the Cloud Software StackErlang On Xen: Redefining the Cloud Software Stack
Erlang On Xen: Redefining the Cloud Software Stack
 

Viewers also liked

Viewers also liked (10)

etcd based PostgreSQL HA Cluster
etcd based PostgreSQL HA Clusteretcd based PostgreSQL HA Cluster
etcd based PostgreSQL HA Cluster
 
Production Readiness Strategies in an Automated World
Production Readiness Strategies in an Automated WorldProduction Readiness Strategies in an Automated World
Production Readiness Strategies in an Automated World
 
Creating PostgreSQL-as-a-Service at Scale
Creating PostgreSQL-as-a-Service at ScaleCreating PostgreSQL-as-a-Service at Scale
Creating PostgreSQL-as-a-Service at Scale
 
KubeCon EU 2016: Full Automatic Database: PostgreSQL HA with Kubernetes
KubeCon EU 2016: Full Automatic Database: PostgreSQL HA with KubernetesKubeCon EU 2016: Full Automatic Database: PostgreSQL HA with Kubernetes
KubeCon EU 2016: Full Automatic Database: PostgreSQL HA with Kubernetes
 
Dynamic Database Credentials: Security Contingency Planning
Dynamic Database Credentials: Security Contingency PlanningDynamic Database Credentials: Security Contingency Planning
Dynamic Database Credentials: Security Contingency Planning
 
PostgreSQL High-Availability and Geographic Locality using consul
PostgreSQL High-Availability and Geographic Locality using consulPostgreSQL High-Availability and Geographic Locality using consul
PostgreSQL High-Availability and Geographic Locality using consul
 
FreeBSD: Dev to Prod
FreeBSD: Dev to ProdFreeBSD: Dev to Prod
FreeBSD: Dev to Prod
 
High Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando PatroniHigh Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando Patroni
 
gRPC: The Story of Microservices at Square
gRPC: The Story of Microservices at SquaregRPC: The Story of Microservices at Square
gRPC: The Story of Microservices at Square
 
Ingesting Drone Data into Big Data Platforms
Ingesting Drone Data into Big Data Platforms Ingesting Drone Data into Big Data Platforms
Ingesting Drone Data into Big Data Platforms
 

Similar to [NYC Meetup] Docker at Nuxeo

PDXPortland - Dockerize Django
PDXPortland - Dockerize DjangoPDXPortland - Dockerize Django
PDXPortland - Dockerize Django
Hannes Hapke
 
Meetup Docker@Nuxeo - Build a Cloud Platform with Docker
Meetup Docker@Nuxeo - Build a Cloud Platform with DockerMeetup Docker@Nuxeo - Build a Cloud Platform with Docker
Meetup Docker@Nuxeo - Build a Cloud Platform with Docker
Nuxeo
 

Similar to [NYC Meetup] Docker at Nuxeo (20)

Docker and kubernetes
Docker and kubernetesDocker and kubernetes
Docker and kubernetes
 
Docker and Puppet for Continuous Integration
Docker and Puppet for Continuous IntegrationDocker and Puppet for Continuous Integration
Docker and Puppet for Continuous Integration
 
SenchaCon 2016: Develop, Test & Deploy with Docker - Jonas Schwabe
SenchaCon 2016: Develop, Test & Deploy with Docker - Jonas Schwabe SenchaCon 2016: Develop, Test & Deploy with Docker - Jonas Schwabe
SenchaCon 2016: Develop, Test & Deploy with Docker - Jonas Schwabe
 
WSO2ConEU 2016 Tutorial - Deploying WSO2 Middleware on Containers
WSO2ConEU 2016 Tutorial - Deploying WSO2 Middleware on ContainersWSO2ConEU 2016 Tutorial - Deploying WSO2 Middleware on Containers
WSO2ConEU 2016 Tutorial - Deploying WSO2 Middleware on Containers
 
Deploying WSO2 Middleware on Containers
Deploying WSO2 Middleware on ContainersDeploying WSO2 Middleware on Containers
Deploying WSO2 Middleware on Containers
 
Reactive summit 2020 microsoft orleans the easy way
Reactive summit 2020   microsoft orleans the easy wayReactive summit 2020   microsoft orleans the easy way
Reactive summit 2020 microsoft orleans the easy way
 
PDXPortland - Dockerize Django
PDXPortland - Dockerize DjangoPDXPortland - Dockerize Django
PDXPortland - Dockerize Django
 
Docker and-daily-devops
Docker and-daily-devopsDocker and-daily-devops
Docker and-daily-devops
 
Docker & Daily DevOps
Docker & Daily DevOpsDocker & Daily DevOps
Docker & Daily DevOps
 
Meetup Docker@Nuxeo - Build a Cloud Platform with Docker
Meetup Docker@Nuxeo - Build a Cloud Platform with DockerMeetup Docker@Nuxeo - Build a Cloud Platform with Docker
Meetup Docker@Nuxeo - Build a Cloud Platform with Docker
 
Docker Seattle Meetup April 2015 - The Docker Orchestration Ecosystem on Azure
Docker Seattle Meetup April 2015 - The Docker Orchestration Ecosystem on AzureDocker Seattle Meetup April 2015 - The Docker Orchestration Ecosystem on Azure
Docker Seattle Meetup April 2015 - The Docker Orchestration Ecosystem on Azure
 
Docker New York Meetup May 2015 - The Docker Orchestration Ecosystem on Azure
Docker New York Meetup May 2015 - The Docker Orchestration Ecosystem on Azure Docker New York Meetup May 2015 - The Docker Orchestration Ecosystem on Azure
Docker New York Meetup May 2015 - The Docker Orchestration Ecosystem on Azure
 
DCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development PipelineDCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development Pipeline
 
Docker introduction
Docker introductionDocker introduction
Docker introduction
 
Boosting Sitecore Development With Sitecore Docker
Boosting Sitecore Development With Sitecore DockerBoosting Sitecore Development With Sitecore Docker
Boosting Sitecore Development With Sitecore Docker
 
PostgreSQL and Linux Containers
PostgreSQL and Linux ContainersPostgreSQL and Linux Containers
PostgreSQL and Linux Containers
 
Docker 1.11 Presentation
Docker 1.11 PresentationDocker 1.11 Presentation
Docker 1.11 Presentation
 
Postgre sql linuxcontainers by Jignesh Shah
Postgre sql linuxcontainers by Jignesh ShahPostgre sql linuxcontainers by Jignesh Shah
Postgre sql linuxcontainers by Jignesh Shah
 
Intro docker and demo monitor on docker
Intro docker and demo monitor on dockerIntro docker and demo monitor on docker
Intro docker and demo monitor on docker
 
Detailed Introduction To Docker
Detailed Introduction To DockerDetailed Introduction To Docker
Detailed Introduction To Docker
 

More from Nuxeo

Enabling Digital Transformation Amidst a Global Pandemic | Low-Code, Cloud, A...
Enabling Digital Transformation Amidst a Global Pandemic | Low-Code, Cloud, A...Enabling Digital Transformation Amidst a Global Pandemic | Low-Code, Cloud, A...
Enabling Digital Transformation Amidst a Global Pandemic | Low-Code, Cloud, A...
Nuxeo
 

More from Nuxeo (20)

Own the Digital Shelf Strategies Food and Beverage Companies
Own the Digital Shelf Strategies Food and Beverage CompaniesOwn the Digital Shelf Strategies Food and Beverage Companies
Own the Digital Shelf Strategies Food and Beverage Companies
 
How DAM Librarians Can Get Ready for the Uncertain Future
How DAM Librarians Can Get Ready for the Uncertain FutureHow DAM Librarians Can Get Ready for the Uncertain Future
How DAM Librarians Can Get Ready for the Uncertain Future
 
How Insurers Fueled Transformation During a Pandemic
How Insurers Fueled Transformation During a PandemicHow Insurers Fueled Transformation During a Pandemic
How Insurers Fueled Transformation During a Pandemic
 
Manage your Content at Scale with MongoDB and Nuxeo
Manage your Content at Scale with MongoDB and NuxeoManage your Content at Scale with MongoDB and Nuxeo
Manage your Content at Scale with MongoDB and Nuxeo
 
Accelerate the Digital Supply Chain From Idea to Support
Accelerate the Digital Supply Chain From Idea to SupportAccelerate the Digital Supply Chain From Idea to Support
Accelerate the Digital Supply Chain From Idea to Support
 
Where are you in the DAM Continuum
Where are you in the DAM ContinuumWhere are you in the DAM Continuum
Where are you in the DAM Continuum
 
Customer Experience in 2021
Customer Experience in 2021Customer Experience in 2021
Customer Experience in 2021
 
L’IA personnalisée, clé d’une gestion de l’information innovante
L’IA personnalisée, clé d’une gestion de l’information innovanteL’IA personnalisée, clé d’une gestion de l’information innovante
L’IA personnalisée, clé d’une gestion de l’information innovante
 
Gérer ses contenus avec MongoDB et Nuxeo
Gérer ses contenus avec MongoDB et NuxeoGérer ses contenus avec MongoDB et Nuxeo
Gérer ses contenus avec MongoDB et Nuxeo
 
Le DAM en 2021 : Tendances, points clés et critères d'évaluation
Le DAM en 2021 : Tendances, points clés et critères d'évaluationLe DAM en 2021 : Tendances, points clés et critères d'évaluation
Le DAM en 2021 : Tendances, points clés et critères d'évaluation
 
Enabling Digital Transformation Amidst a Global Pandemic | Low-Code, Cloud, A...
Enabling Digital Transformation Amidst a Global Pandemic | Low-Code, Cloud, A...Enabling Digital Transformation Amidst a Global Pandemic | Low-Code, Cloud, A...
Enabling Digital Transformation Amidst a Global Pandemic | Low-Code, Cloud, A...
 
Elevate your Customer's Experience and Stay Ahead of the Competition
Elevate your Customer's Experience and Stay Ahead of the CompetitionElevate your Customer's Experience and Stay Ahead of the Competition
Elevate your Customer's Experience and Stay Ahead of the Competition
 
Driving Brand Loyalty Through Superior Customer Experience
Driving Brand Loyalty Through Superior Customer Experience Driving Brand Loyalty Through Superior Customer Experience
Driving Brand Loyalty Through Superior Customer Experience
 
Drive Enterprise Speed and Scale with A Cloud-Native DAM
Drive Enterprise Speed and Scale with A Cloud-Native DAMDrive Enterprise Speed and Scale with A Cloud-Native DAM
Drive Enterprise Speed and Scale with A Cloud-Native DAM
 
The Big Picture: the Role of Video, Photography, and Content in Enhancing the...
The Big Picture: the Role of Video, Photography, and Content in Enhancing the...The Big Picture: the Role of Video, Photography, and Content in Enhancing the...
The Big Picture: the Role of Video, Photography, and Content in Enhancing the...
 
How Creatives Are Getting Creative in 2020 and Beyond
How Creatives Are Getting Creative in 2020 and BeyondHow Creatives Are Getting Creative in 2020 and Beyond
How Creatives Are Getting Creative in 2020 and Beyond
 
Digitalisation : Améliorez la collaboration et l’expérience client grâce au DAM
Digitalisation : Améliorez la collaboration et l’expérience client grâce au DAMDigitalisation : Améliorez la collaboration et l’expérience client grâce au DAM
Digitalisation : Améliorez la collaboration et l’expérience client grâce au DAM
 
Reimagine Your Claims Process with Future-Proof Technologies
Reimagine Your Claims Process with Future-Proof TechnologiesReimagine Your Claims Process with Future-Proof Technologies
Reimagine Your Claims Process with Future-Proof Technologies
 
Comment le Centre Hospitalier Laborit dématérialise ses processus administratifs
Comment le Centre Hospitalier Laborit dématérialise ses processus administratifsComment le Centre Hospitalier Laborit dématérialise ses processus administratifs
Comment le Centre Hospitalier Laborit dématérialise ses processus administratifs
 
Accelerating the Packaging Design Process with Artificial Intelligence
Accelerating the Packaging Design Process with Artificial IntelligenceAccelerating the Packaging Design Process with Artificial Intelligence
Accelerating the Packaging Design Process with Artificial Intelligence
 

Recently uploaded

Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
panagenda
 

Recently uploaded (20)

A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
Google I/O Extended 2024 Warsaw
Google I/O Extended 2024 WarsawGoogle I/O Extended 2024 Warsaw
Google I/O Extended 2024 Warsaw
 
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 

[NYC Meetup] Docker at Nuxeo

  • 3. What is Nuxeo platform • A platform to develop you own content apps • Highly pluggable thru extension points • Model you content with Nuxeo Studio • Point and click UI to define your business model • with hot-reload
  • 4. Nuxeo tech. stack • Java stack : Seam + JSF for Nuxeo DM and DAM • Rest API to build your JS Apps • Nuxeo Studio (Saas) to model your business • Thru data model (doc types, schemas) • Thru Automation chains
  • 5. Nuxeo in the cloud ?
  • 6. Goals • Customers ask to host their app • Provide quick access to test instances (onboarding) • Provide onDemand dev or CI instances
  • 7. Goals • Customers ask to host their app • Provide quick access to test instances (onboarding) • Provide onDemand dev or CI instances This is the nuxeo.io promise
  • 9. Manual OnDemand • Based on AWS CloudFormation • Manually deployed • Works but : • not scalable • manual admin & monitoring • difficult upgrade process
  • 11. AWS Automation • Provisionning may be automated • Working mockup (python, celery, boto) but… • Strong adherence to AWS • difficult to test • slow provisionning (one EC2 instance per Nuxeo instance) • upgrade process still difficult • scaling out is difficult • sensible to failures
  • 12. This leads to our challenge
  • 13. Current challenge • Build a scalable infrastructure • Quick provisioning (less than a minute) • Easy setup (no admin needed) • Easy upgrade process • Automated healing • Easy to test
  • 14. How Docker solves some of these issues ?
  • 15. Quick provisioning • We don’t have to start a server • No VM overhead • Start time is Nuxeo start time
  • 16. Data free containers • Docker container doesn’t hold data • Upgrading is just using a new version of the image • Healing means killing and restarting Nuxeo v5.9.3 S3Postgres
  • 17. Data free containers • Docker container doesn’t hold data • Upgrading is just using a new version of the image • Healing means killing and restarting S3Postgres
  • 18. Data free containers • Docker container doesn’t hold data • Upgrading is just using a new version of the image • Healing means killing and restarting S3Postgres Nuxeo v5.9.4
  • 19. Several containers per server • Scaling is just a matter of starting several containers • But means infinite CPU / RAM and IO Nuxeo.io server io container NXIO-0001 io container NXIO-0002 io container NXIO-0003 io container NXIO-XXXX …
  • 20. Challenge status • Build a scalable infrastructure • Quick provisioning (less than a minute) • Easy setup (no admin needed) • Easy upgrade process • Automated healing • Easy to test
  • 21. How to solve the other issues ? … with a little help from my friends
  • 22. The infinite resource computer…
  • 23. … is a cluster
  • 24. Docker is not sufficient • Docker handles processes at the OS level • On the cluster level we need a companion • To load balance the processes • To check the statuses of the processes • We need a docker ps at the cluster level
  • 25. Introducing CoreOS • A minimal Linux distribution (small footprint) • Designed to handle cluster • Providing cluster tooling • Docker • etcd • fleet https://coreos.com/
  • 26. etcd : service discovery • etcd is • small distributed key value store • keys may have TTL • using Raft protocol to elect cluster leader • suitable for : • shared configuration • service discovery
  • 27. fleet : clustered init system • fleet is : • an init system at the cluster level • compatible with systemd unit files • suitable for : • launching docker containers • distributing load across cluster • get the status of the jobs
  • 28. fleet demo • Deploy our nuxeo container in the cluster
  • 29. Challenge status • Build a scalable infrastructure • Quick provisioning (less than a minute) • Easy setup (no admin needed) • Easy upgrade process • Automated healing • Easy to test
  • 30. What are the admin tasks ? • Launch the fleetctl command • Setup DNS and reverse proxies • Link container to external services (S3, Postgres etc…)
  • 31. DNS and reverse proxy • docker start containers and link host port to container port • The cluster has to redirect calls to mycustomDNS.io to the running container meaning knowing • the ip of the host • the port on the host
  • 32. Introducing Gogeta ! /services/NXIO-0001/location:{host:172.31.51.1,port:49153}!                 /status/current:started!                       /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer!                           /value:NXIO-0001 Given this etcd configuration
  • 33. Cluster CoreOS 172.31.51.20 etcd io container NXIO-0098 systemd CoreOS 172.31.51.1 etcd io container NXIO-0001 Gogeta systemd … … … … Fleet … Gogeta … Introducing Gogeta Round robin load Balancer etcd ! /services/NXIO-0098/location:{host:172.31.51.20,port:49153}!                 /status/current:started!                       /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer!                           /value:NXIO-0001
  • 34. Cluster CoreOS 172.31.51.20 etcd io container NXIO-0098 systemd CoreOS 172.31.51.1 etcd io container NXIO-0001 Gogeta systemd … … … … Fleet … Gogeta … Introducing Gogeta Round robin load Balancer etcd ! /services/NXIO-0098/location:{host:172.31.51.20,port:49153}!                 /status/current:started!                       /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer!                           /value:NXIO-0001
  • 35. Cluster CoreOS 172.31.51.20 etcd io container NXIO-0098 systemd CoreOS 172.31.51.1 etcd io container NXIO-0001 Gogeta systemd … … … … Fleet … Gogeta … Introducing Gogeta Round robin load Balancer etcd ! /services/NXIO-0098/location:{host:172.31.51.20,port:49153}!                 /status/current:started!                       /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer!                           /value:NXIO-0001
  • 36. Cluster CoreOS 172.31.51.20 etcd io container NXIO-0098 systemd CoreOS 172.31.51.1 etcd io container NXIO-0001 Gogeta systemd … … … … Fleet … Gogeta … Introducing Gogeta Round robin load Balancer etcd ! /services/NXIO-0098/location:{host:172.31.51.20,port:49153}!                 /status/current:started!                       /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer!                           /value:NXIO-0001
  • 37. Cluster CoreOS 172.31.51.20 etcd io container NXIO-0098 systemd CoreOS 172.31.51.1 etcd io container NXIO-0001 Gogeta systemd … … … … Fleet … Gogeta … Introducing Gogeta Round robin load Balancer etcd ! /services/NXIO-0098/location:{host:172.31.51.20,port:49153}!                 /status/current:started!                       /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer!                           /value:NXIO-0001
  • 38. Cluster CoreOS 172.31.51.20 etcd io container NXIO-0098 systemd CoreOS 172.31.51.1 etcd io container NXIO-0001 Gogeta systemd … … … … Fleet … Gogeta … Introducing Gogeta Round robin load Balancer etcd ! /services/NXIO-0098/location:{host:172.31.51.20,port:49153}!                 /status/current:started!                       /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer!                           /value:NXIO-0001
  • 39. Introducing Gogeta • Easy to start as a docker container ! ! • Written in Go • Available at : http://github.com/nuxeo/gogeta /usr/bin/docker run --rm --name gogeta -p 7777:7777 nuxeo/gogeta
  • 40. Application startup unit ! [Unit]! Description=Sample startup script! ! [Service]! ExecStartPre= Set starting status in etcd! ExecStart= Launch docker container! ExecStartPost= Wait container to be launched and set app location in etcd! ExecStop= Stop docker container! ExecStopPost= Set stopped status in etcd!
  • 41. Linking services • Docker recommend to use ambassadors to link services • Service may be hosted on other servers or outside of the cluster • We’ve implemented dynamic ambassadors • http://coreos.com/blog/docker-dynamic-ambassador- powered-by-etcd/
  • 42. Cluster CoreOS 172.31.51.20 etcd Postgres systemd CoreOS 172.31.51.1 etcd io-container NXIO-0001 systemd … … … … Fleet postgres-amb Dynamic ambassador etcd ! /services/postgres/location:{host:172.31.51.20,port:49159}! https://github.com/nuxeo/etcd-netfw
  • 43. Cluster CoreOS 172.31.51.20 etcd Postgres systemd CoreOS 172.31.51.1 etcd io-container NXIO-0001 systemd … … … … Fleet postgres-amb Dynamic ambassador etcd ! /services/postgres/location:{host:172.31.51.20,port:49159}! https://github.com/nuxeo/etcd-netfw
  • 44. Cluster CoreOS 172.31.51.20 etcd Postgres systemd CoreOS 172.31.51.1 etcd io-container NXIO-0001 systemd … … … … Fleet postgres-amb Dynamic ambassador etcd ! /services/postgres/location:{host:172.31.51.20,port:49159}! https://github.com/nuxeo/etcd-netfw
  • 45. Dynamic ambassador • Very easy to start ! • Uses https://github.com/nuxeo/etcd-netfw docker run --rm --name postgres-amb -P nuxeo/service-amb postgres-service
  • 46. Launching commands • A manager web app now launches commands : • sets up basic etcd values (like domain to service resolution) • launches fleet commands (start & stop apps) • polls statuses
  • 47. Challenge status • Build a scalable infrastructure • Quick provisioning (less than a minute) • Easy setup (no admin needed) • Easy upgrade process • Automated healing • Easy to test
  • 49. From nuxeo.io… • We’ve built a cluster infrastructure that : • knows how to deploy Docker containers (with fleet) • Expose those containers thru domain name resolution (with gogeta) • Link dependent services to the container (with etcd-netfw)
  • 50. From nuxeo.io… • Given that a container should represent an application • Running this apps means : • Launch it with runtime parameters • Link it to its dependent services • Then we can build a metamodel around it
  • 51. … name: io-container! version: 1.0! params:! - env_tech_id: ! - description: The id of the environment to run! - type:string! - clid:! - description: a registered connect client ID! - type:string! ! buildimage:! path: "./docker"! ! dependencies:! - postgres.service! - s3.service! !
  • 52. … to arken • to be able to run any application ! • it will take care of : • Submitting the model to the cluster • Building the image if needed • Launching the container • Registering everything in etcd arkenctl start iocontainer.ark —domain myapp.mycluster.io
  • 53. Arken status • We are extracting things from http://github.com/nuxeo/nuxeo.io • http://github.com/arkenio • home of gogeta and etcd • provide cluster init instruction • arken and arkenctl tools needs to be done • new opensource projects (not in nuxeo repo) • pull requests are welcome
  • 54. Conclusion • Docker helped us to enter the micro containers world • Lots of new features : multi-repo, automatic /etc/hosts etc… • CoreOS provided us cluster deployment • Arken wants to provide a higher deployment model