SlideShare a Scribd company logo
1 of 32
Building and Deploying MediaSalsa
A Digital Asset Management System as a Service
Kris Buytaert
@krisbuytaert
Slides by Michel van de Ven and
Julien Pivotto
Kris BuytaertKris Buytaert
● I used to be a Dev,I used to be a Dev,
● Then Became an OpThen Became an Op
● Chief Trolling Officer and Open SourceChief Trolling Officer and Open Source
Consultant @inuits.euConsultant @inuits.eu
● Everything is an effing DNS ProblemEverything is an effing DNS Problem
● Building Clouds since before the bookstoreBuilding Clouds since before the bookstore
● Some books, some papers, some blogsSome books, some papers, some blogs
● Evangelizing devopsEvangelizing devops
MediaMosa
● Drupal-based Digital Asset Management system
● Commissioned by SURFnet and Kennisnet
● Open Source (GPLv2), Open Standards
● Webservice oriented: REST
● Store & retrieve assets
● Manage metadata using open standards
– Dublin Core, Qualified DC, IEEE/LOM, CZP
– OAI-PMH, BagIt
● Transcode video, audio, images, PDF &c, stream content
● Users: Kennisnet, NIBG, UGent, UvA, TiU, RUG, Radboud, UOslo,
Avans, PolitieAcademie, Acquia (NBC Sport), Cineca &c
● http://mediamosa.org
=
MediaMosa as a Service
MediaSalsa infrastructure (simplified)
● For each environment (OTAP)
– Backends: Core service (MediaMosa)
– Frontends: Optional
– Web servers
– Database servers
– Solr servers
– Transcoding servers
● Instance @ RUG DC for NL .edu
● Instances @ Commercial DCs for non-edu
Culture
Automation
Measurement
Sharing
Inuits
● Inuits is an Open Source company
– We contribute back
● +40 people in 3 countries (BE, NL, UA)
● One language: English
● We offer
– Consulting
● Development
● System Administration
– and a niche Saas Platform
Ideal world vs budget and reality
→ pragmatic approach
Distributed team
Communication = HELL
How to fix it?
Distributed team
● Daily virtual stand-up over XMPP / Hangout
● Redmine project management
– Issues, tasks, sprints
– Repositories
– Documentation (wiki)
● Internal mailing lists
● Jenkins notifications by mail and XMPP
● Internal training
Dev and Ops
● Dev
● Architecture
● Develop new features
● Write unit tests (!)
● Ops
● Infrastructure (puppet)
● CD and CI (jenkins)
→ Ops teach dev about monitoring and
distributed services
→ Dev teach ops about required libraries and
testing, dependencies
Culture
Automation
Measurement
Sharing
Puppet
Puppet automates all the things
→ mcollective orchestrates all the things
CD
● Continuous Delivery vs Continuous Deployment
– “Continuous Delivery doesn't mean every change is deployed to production
ASAP. It means every change is proven to be deployable at any time”
(@ccaum)
● Puppet code
– Deployed to dev environment
– Same puppet code for each environment
– User-triggered deployments to UAT & Prod
– Feature flags in Puppet code per environment (switchable architecture)
● Application code
– Continuous integration in dev
– User-triggered deployments to UAT
– Deployment to prod is a business decision
Testing
● Developers test a lot, but
– The tests don’t work
– It works on my machine™
– Wrong platform
– Wrong PHP version
Fixed now, thanks to Jenkins!
and Vagrant...
Version Control
● Git
● Code is under revision control
– Prefer small commits
– Local features branches
● Infrastructure as code → git / hiera
Using OS packaging system
● Consistency, security, dependencies
● Uniquely identify where files are coming from
● Source repo may not be reacheable
● Little overhead when you automate
● Configuration does not belong in a package
Pipelines
● A collection of jobs
● Run in sequence
● Start on checkout, end on deployment
● From the developers’ side:
→ Git push
← Mail with changes + link to deploy
Our current pipelines
● Puppetized
● Puppet code and Application code (backend and
frontends)
● 1 pipeline / project / customer
Defining a pipeline:
salsajobs::pipeline::frontend {
’qwerty-inc’:
git_repository => ’ssh://git@redmine/qwerty.git’,
dashboard_view => ’mds-frontends’,
target_urls => {
’uat’ => ’http://qwerty.uat.mediasalsa.eu’,
’prod’ => ’http://qwerty.mediasalsa.eu’,
},
require => Salsajobs::Views::Dashboard[’mds-frontends’],
}
Pipelines steps
● Checkout
● Syntax: php -l
● Style: Drupal Coder
● Package: FPM
● Deploy to dev environment: mcollective
● Tests in dev environment: drush run-tests
● Publish package and promote: mcollective
Promotion
● At the end of the pipeline
● Send a simple email
● A link to the promotion page
● The changelog
● Promotion page contains one button per environment
● Must promote to UAT before Production
Tools used with Jenkins
● Pulp to manage RPM repositories
● Mcollective to update packages, run drush
Same Pipelines, Tools, Patterns
are used by both devs and ops
Culture
Automation
Measurement
Sharing
Logstash
● Collect all the logs
– Drupal logs
– Apache logs
– Deployment logs
– System logs
● Interprete, filter and correlate them
● Logstash, ElasticSearch, Kibana, statsd, Graphite
Icinga
● Monitor everything
– vhosts
– databases
– cronjobs
– unit test suites
Graphite + gdash
● Collectd
● Monitor platform usage
● FFmpeg usage
● Number of accounts
● Pipelined !
Culture
Automation
Measurement
Sharing
Open Source
● Mediamosa is Fully Open Source
● Lots of the PuppetCode to deploy it is
● Our passwords etc aren't
Experiences
We have just shared our experience...
But also at the NLUUG Fall Conference, DrupalCon Prague
ContactContact
Kris BuytaertKris Buytaert Kris.Buytaert@inuits.beKris.Buytaert@inuits.be
Further ReadingFurther Reading
@krisbuytaert@krisbuytaert
http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/
http://www.inuits.be/http://www.inuits.be/
Inuits HQInuits HQ
Duboistraat 50Duboistraat 50
2060 Antwerpen2060 Antwerpen
BelgiumBelgium
891.514.231891.514.231
+32 475 961221+32 475 961221

More Related Content

What's hot

Building Resilient Cloud Native Apps in GKE
Building Resilient Cloud Native Apps in GKEBuilding Resilient Cloud Native Apps in GKE
Building Resilient Cloud Native Apps in GKEJerry Jalava
 
Vincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostusta
Vincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostustaVincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostusta
Vincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostustaVincitOy
 
Safe deployments with Blue-Green and Spinnaker
Safe deployments with Blue-Green and SpinnakerSafe deployments with Blue-Green and Spinnaker
Safe deployments with Blue-Green and SpinnakerMihnea Dobrescu-Balaur
 
Kubernetes and the Rise of Application-centric Computing
Kubernetes and the Rise of Application-centric ComputingKubernetes and the Rise of Application-centric Computing
Kubernetes and the Rise of Application-centric ComputingBitnami
 
Git for folk who like GUIs
Git for folk who like GUIsGit for folk who like GUIs
Git for folk who like GUIsTim Osborn
 
Containers not just for production nov8
Containers not just for production nov8Containers not just for production nov8
Containers not just for production nov8HARITHA HARI
 
Ansible @ Red Hat | December 2015 Ansible Meetup in Melbourne
Ansible @ Red Hat | December 2015 Ansible Meetup in MelbourneAnsible @ Red Hat | December 2015 Ansible Meetup in Melbourne
Ansible @ Red Hat | December 2015 Ansible Meetup in MelbourneKen Thompson
 
Embracing Continuous Integration
Embracing Continuous IntegrationEmbracing Continuous Integration
Embracing Continuous IntegrationIT MegaMeet
 
On the Importance of Infrastructure as Code
On the Importance of Infrastructure as CodeOn the Importance of Infrastructure as Code
On the Importance of Infrastructure as CodeKris Buytaert
 
Silverstripe CMS 4.9.0 release and beyound
Silverstripe CMS 4.9.0 release and beyoundSilverstripe CMS 4.9.0 release and beyound
Silverstripe CMS 4.9.0 release and beyoundMaximeRainville1
 
Building CI/CD Pipelines with Jenkins and Kubernetes
Building CI/CD Pipelines with Jenkins and KubernetesBuilding CI/CD Pipelines with Jenkins and Kubernetes
Building CI/CD Pipelines with Jenkins and KubernetesJanakiram MSV
 
Windows containers on Kubernetes
Windows containers on KubernetesWindows containers on Kubernetes
Windows containers on KubernetesCraig Peters
 
We Need to Talk: How Communication Helps Code
We Need to Talk: How Communication Helps CodeWe Need to Talk: How Communication Helps Code
We Need to Talk: How Communication Helps CodeDocker, Inc.
 
Languages don't matter anymore!
Languages don't matter anymore!Languages don't matter anymore!
Languages don't matter anymore!Soluto
 
Atlassian Bamboo Feature Overview
Atlassian Bamboo Feature OverviewAtlassian Bamboo Feature Overview
Atlassian Bamboo Feature OverviewJim Bethancourt
 

What's hot (20)

Building Resilient Cloud Native Apps in GKE
Building Resilient Cloud Native Apps in GKEBuilding Resilient Cloud Native Apps in GKE
Building Resilient Cloud Native Apps in GKE
 
Vincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostusta
Vincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostustaVincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostusta
Vincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostusta
 
Safe deployments with Blue-Green and Spinnaker
Safe deployments with Blue-Green and SpinnakerSafe deployments with Blue-Green and Spinnaker
Safe deployments with Blue-Green and Spinnaker
 
Kubernetes and the Rise of Application-centric Computing
Kubernetes and the Rise of Application-centric ComputingKubernetes and the Rise of Application-centric Computing
Kubernetes and the Rise of Application-centric Computing
 
Git for folk who like GUIs
Git for folk who like GUIsGit for folk who like GUIs
Git for folk who like GUIs
 
Containers not just for production nov8
Containers not just for production nov8Containers not just for production nov8
Containers not just for production nov8
 
Keptn Meetup Interoperable ci/cd ecosystem
Keptn Meetup Interoperable ci/cd ecosystemKeptn Meetup Interoperable ci/cd ecosystem
Keptn Meetup Interoperable ci/cd ecosystem
 
Intro to Gitflow
Intro to GitflowIntro to Gitflow
Intro to Gitflow
 
Ansible @ Red Hat | December 2015 Ansible Meetup in Melbourne
Ansible @ Red Hat | December 2015 Ansible Meetup in MelbourneAnsible @ Red Hat | December 2015 Ansible Meetup in Melbourne
Ansible @ Red Hat | December 2015 Ansible Meetup in Melbourne
 
Embracing Continuous Integration
Embracing Continuous IntegrationEmbracing Continuous Integration
Embracing Continuous Integration
 
On the Importance of Infrastructure as Code
On the Importance of Infrastructure as CodeOn the Importance of Infrastructure as Code
On the Importance of Infrastructure as Code
 
Silverstripe CMS 4.9.0 release and beyound
Silverstripe CMS 4.9.0 release and beyoundSilverstripe CMS 4.9.0 release and beyound
Silverstripe CMS 4.9.0 release and beyound
 
Ci & proServer
Ci & proServerCi & proServer
Ci & proServer
 
Building CI/CD Pipelines with Jenkins and Kubernetes
Building CI/CD Pipelines with Jenkins and KubernetesBuilding CI/CD Pipelines with Jenkins and Kubernetes
Building CI/CD Pipelines with Jenkins and Kubernetes
 
Kenzan Spinnaker Meetup
Kenzan Spinnaker MeetupKenzan Spinnaker Meetup
Kenzan Spinnaker Meetup
 
Windows containers on Kubernetes
Windows containers on KubernetesWindows containers on Kubernetes
Windows containers on Kubernetes
 
We Need to Talk: How Communication Helps Code
We Need to Talk: How Communication Helps CodeWe Need to Talk: How Communication Helps Code
We Need to Talk: How Communication Helps Code
 
Languages don't matter anymore!
Languages don't matter anymore!Languages don't matter anymore!
Languages don't matter anymore!
 
Stripe con 2021 UI stack
Stripe con 2021 UI stackStripe con 2021 UI stack
Stripe con 2021 UI stack
 
Atlassian Bamboo Feature Overview
Atlassian Bamboo Feature OverviewAtlassian Bamboo Feature Overview
Atlassian Bamboo Feature Overview
 

Viewers also liked

Package all the things, from #ihatepackaging to #packaginglove
Package all the things, from #ihatepackaging to #packaginglovePackage all the things, from #ihatepackaging to #packaginglove
Package all the things, from #ihatepackaging to #packagingloveKris Buytaert
 
Continuous Delivery of Puppet Manifests
Continuous Delivery of Puppet ManifestsContinuous Delivery of Puppet Manifests
Continuous Delivery of Puppet ManifestsKris Buytaert
 
State of devops and drupal 2014
State of devops  and drupal 2014State of devops  and drupal 2014
State of devops and drupal 2014Kris Buytaert
 
Puppet and Icinga 101
Puppet and Icinga 101Puppet and Icinga 101
Puppet and Icinga 101Kris Buytaert
 
Monitoring (with) puppet
Monitoring  (with) puppetMonitoring  (with) puppet
Monitoring (with) puppetKris Buytaert
 
Devops, The future is here, it's just not evenly distributed
Devops, The future is here, it's just not evenly distributedDevops, The future is here, it's just not evenly distributed
Devops, The future is here, it's just not evenly distributedKris Buytaert
 
Looking back at 6.5 years of #devopsdays
Looking back at 6.5 years of #devopsdaysLooking back at 6.5 years of #devopsdays
Looking back at 6.5 years of #devopsdaysKris Buytaert
 
7 Years of Puppet Horror Stories
7 Years of Puppet Horror Stories7 Years of Puppet Horror Stories
7 Years of Puppet Horror StoriesKris Buytaert
 
When traditional configuration management is to slow for your needs
When traditional configuration management is to slow for your needsWhen traditional configuration management is to slow for your needs
When traditional configuration management is to slow for your needsKris Buytaert
 
Looking back at 7 years of #devopsdays
Looking back at 7 years of #devopsdaysLooking back at 7 years of #devopsdays
Looking back at 7 years of #devopsdaysKris Buytaert
 
Devops 101 QA with Kris Buytaert
Devops 101 QA  with Kris BuytaertDevops 101 QA  with Kris Buytaert
Devops 101 QA with Kris BuytaertKris Buytaert
 
Open Source Monitoring in 2014, from #monitoringssucks to #monitoringlove and...
Open Source Monitoring in 2014, from #monitoringssucks to #monitoringlove and...Open Source Monitoring in 2014, from #monitoringssucks to #monitoringlove and...
Open Source Monitoring in 2014, from #monitoringssucks to #monitoringlove and...Kris Buytaert
 
The Future of System Administration
The Future of System AdministrationThe Future of System Administration
The Future of System AdministrationKris Buytaert
 
From Config Management Sucks to #cfgmgmtlove
From Config Management Sucks to #cfgmgmtlove From Config Management Sucks to #cfgmgmtlove
From Config Management Sucks to #cfgmgmtlove Kris Buytaert
 
No, we can't do continuous delivery
No, we can't do continuous deliveryNo, we can't do continuous delivery
No, we can't do continuous deliveryKris Buytaert
 
Open Source Monitoring in 2015
Open Source Monitoring in 2015Open Source Monitoring in 2015
Open Source Monitoring in 2015Kris Buytaert
 
Closing the gap between Distros(devs) and their Users(ops)
Closing the gap between Distros(devs) and their Users(ops)Closing the gap between Distros(devs) and their Users(ops)
Closing the gap between Distros(devs) and their Users(ops)Kris Buytaert
 
Dev secops opsec, devsec, devops ?
Dev secops opsec, devsec, devops ?Dev secops opsec, devsec, devops ?
Dev secops opsec, devsec, devops ?Kris Buytaert
 
Monitoring with ElasticSearch
Monitoring with ElasticSearch Monitoring with ElasticSearch
Monitoring with ElasticSearch Kris Buytaert
 
Adopting Devops , Stories from the trenches
Adopting Devops , Stories from the trenchesAdopting Devops , Stories from the trenches
Adopting Devops , Stories from the trenchesKris Buytaert
 

Viewers also liked (20)

Package all the things, from #ihatepackaging to #packaginglove
Package all the things, from #ihatepackaging to #packaginglovePackage all the things, from #ihatepackaging to #packaginglove
Package all the things, from #ihatepackaging to #packaginglove
 
Continuous Delivery of Puppet Manifests
Continuous Delivery of Puppet ManifestsContinuous Delivery of Puppet Manifests
Continuous Delivery of Puppet Manifests
 
State of devops and drupal 2014
State of devops  and drupal 2014State of devops  and drupal 2014
State of devops and drupal 2014
 
Puppet and Icinga 101
Puppet and Icinga 101Puppet and Icinga 101
Puppet and Icinga 101
 
Monitoring (with) puppet
Monitoring  (with) puppetMonitoring  (with) puppet
Monitoring (with) puppet
 
Devops, The future is here, it's just not evenly distributed
Devops, The future is here, it's just not evenly distributedDevops, The future is here, it's just not evenly distributed
Devops, The future is here, it's just not evenly distributed
 
Looking back at 6.5 years of #devopsdays
Looking back at 6.5 years of #devopsdaysLooking back at 6.5 years of #devopsdays
Looking back at 6.5 years of #devopsdays
 
7 Years of Puppet Horror Stories
7 Years of Puppet Horror Stories7 Years of Puppet Horror Stories
7 Years of Puppet Horror Stories
 
When traditional configuration management is to slow for your needs
When traditional configuration management is to slow for your needsWhen traditional configuration management is to slow for your needs
When traditional configuration management is to slow for your needs
 
Looking back at 7 years of #devopsdays
Looking back at 7 years of #devopsdaysLooking back at 7 years of #devopsdays
Looking back at 7 years of #devopsdays
 
Devops 101 QA with Kris Buytaert
Devops 101 QA  with Kris BuytaertDevops 101 QA  with Kris Buytaert
Devops 101 QA with Kris Buytaert
 
Open Source Monitoring in 2014, from #monitoringssucks to #monitoringlove and...
Open Source Monitoring in 2014, from #monitoringssucks to #monitoringlove and...Open Source Monitoring in 2014, from #monitoringssucks to #monitoringlove and...
Open Source Monitoring in 2014, from #monitoringssucks to #monitoringlove and...
 
The Future of System Administration
The Future of System AdministrationThe Future of System Administration
The Future of System Administration
 
From Config Management Sucks to #cfgmgmtlove
From Config Management Sucks to #cfgmgmtlove From Config Management Sucks to #cfgmgmtlove
From Config Management Sucks to #cfgmgmtlove
 
No, we can't do continuous delivery
No, we can't do continuous deliveryNo, we can't do continuous delivery
No, we can't do continuous delivery
 
Open Source Monitoring in 2015
Open Source Monitoring in 2015Open Source Monitoring in 2015
Open Source Monitoring in 2015
 
Closing the gap between Distros(devs) and their Users(ops)
Closing the gap between Distros(devs) and their Users(ops)Closing the gap between Distros(devs) and their Users(ops)
Closing the gap between Distros(devs) and their Users(ops)
 
Dev secops opsec, devsec, devops ?
Dev secops opsec, devsec, devops ?Dev secops opsec, devsec, devops ?
Dev secops opsec, devsec, devops ?
 
Monitoring with ElasticSearch
Monitoring with ElasticSearch Monitoring with ElasticSearch
Monitoring with ElasticSearch
 
Adopting Devops , Stories from the trenches
Adopting Devops , Stories from the trenchesAdopting Devops , Stories from the trenches
Adopting Devops , Stories from the trenches
 

Similar to Building and Deploying MediaSalsa, an Open Source DAM as Saas platform

Deploying an Open Source DAM in SAAS Mode (European Drupal Days 2015)
Deploying an Open Source DAM in SAAS Mode (European Drupal Days 2015)Deploying an Open Source DAM in SAAS Mode (European Drupal Days 2015)
Deploying an Open Source DAM in SAAS Mode (European Drupal Days 2015)Eugenio Minardi
 
Deploying your SaaS stack OnPrem
Deploying your SaaS stack OnPremDeploying your SaaS stack OnPrem
Deploying your SaaS stack OnPremKris Buytaert
 
Building and Deploying MediaSalsa, a drupal-based DAM as a Service
Building and Deploying MediaSalsa, a drupal-based DAM as a ServiceBuilding and Deploying MediaSalsa, a drupal-based DAM as a Service
Building and Deploying MediaSalsa, a drupal-based DAM as a ServiceJulien Pivotto
 
Fluo CICD OpenStack Summit
Fluo CICD OpenStack SummitFluo CICD OpenStack Summit
Fluo CICD OpenStack SummitMiguel Zuniga
 
Drupal 8 DevOps . Profile and SQL flows.
Drupal 8 DevOps . Profile and SQL flows.Drupal 8 DevOps . Profile and SQL flows.
Drupal 8 DevOps . Profile and SQL flows.Andrii Podanenko
 
Sonian, Open Source and Sensu
Sonian, Open Source and SensuSonian, Open Source and Sensu
Sonian, Open Source and SensuPete Cheslock
 
Docker 1 0 1 0 1: a Docker introduction, actualized for the stable release of...
Docker 1 0 1 0 1: a Docker introduction, actualized for the stable release of...Docker 1 0 1 0 1: a Docker introduction, actualized for the stable release of...
Docker 1 0 1 0 1: a Docker introduction, actualized for the stable release of...Jérôme Petazzoni
 
Devops with Python by Yaniv Cohen DevopShift
Devops with Python by Yaniv Cohen DevopShiftDevops with Python by Yaniv Cohen DevopShift
Devops with Python by Yaniv Cohen DevopShiftYaniv cohen
 
Devops is (not ) a buzzword
Devops is (not ) a buzzwordDevops is (not ) a buzzword
Devops is (not ) a buzzwordMiguel Fonseca
 
Pipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as CodePipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as CodeKris Buytaert
 
OSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris Buytaert
OSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris BuytaertOSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris Buytaert
OSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris BuytaertNETWAYS
 
GitLab for CI/CD process
GitLab for CI/CD processGitLab for CI/CD process
GitLab for CI/CD processHYS Enterprise
 
Drupal Continuous Integration Workflow
Drupal Continuous Integration WorkflowDrupal Continuous Integration Workflow
Drupal Continuous Integration WorkflowAndrii Podanenko
 
SCM Puppet: from an intro to the scaling
SCM Puppet: from an intro to the scalingSCM Puppet: from an intro to the scaling
SCM Puppet: from an intro to the scalingStanislav Osipov
 
Introduction to Docker at the Azure Meet-up in New York
Introduction to Docker at the Azure Meet-up in New YorkIntroduction to Docker at the Azure Meet-up in New York
Introduction to Docker at the Azure Meet-up in New YorkJérôme Petazzoni
 
Deploying software at Scale
Deploying software at ScaleDeploying software at Scale
Deploying software at ScaleKris Buytaert
 
OSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris Buytaert
OSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris BuytaertOSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris Buytaert
OSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris BuytaertNETWAYS
 
Netflix Open Source: Building a Distributed and Automated Open Source Program
Netflix Open Source:  Building a Distributed and Automated Open Source ProgramNetflix Open Source:  Building a Distributed and Automated Open Source Program
Netflix Open Source: Building a Distributed and Automated Open Source Programaspyker
 
Building a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at NetflixBuilding a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at NetflixAll Things Open
 
Free GitOps Workshop
Free GitOps WorkshopFree GitOps Workshop
Free GitOps WorkshopWeaveworks
 

Similar to Building and Deploying MediaSalsa, an Open Source DAM as Saas platform (20)

Deploying an Open Source DAM in SAAS Mode (European Drupal Days 2015)
Deploying an Open Source DAM in SAAS Mode (European Drupal Days 2015)Deploying an Open Source DAM in SAAS Mode (European Drupal Days 2015)
Deploying an Open Source DAM in SAAS Mode (European Drupal Days 2015)
 
Deploying your SaaS stack OnPrem
Deploying your SaaS stack OnPremDeploying your SaaS stack OnPrem
Deploying your SaaS stack OnPrem
 
Building and Deploying MediaSalsa, a drupal-based DAM as a Service
Building and Deploying MediaSalsa, a drupal-based DAM as a ServiceBuilding and Deploying MediaSalsa, a drupal-based DAM as a Service
Building and Deploying MediaSalsa, a drupal-based DAM as a Service
 
Fluo CICD OpenStack Summit
Fluo CICD OpenStack SummitFluo CICD OpenStack Summit
Fluo CICD OpenStack Summit
 
Drupal 8 DevOps . Profile and SQL flows.
Drupal 8 DevOps . Profile and SQL flows.Drupal 8 DevOps . Profile and SQL flows.
Drupal 8 DevOps . Profile and SQL flows.
 
Sonian, Open Source and Sensu
Sonian, Open Source and SensuSonian, Open Source and Sensu
Sonian, Open Source and Sensu
 
Docker 1 0 1 0 1: a Docker introduction, actualized for the stable release of...
Docker 1 0 1 0 1: a Docker introduction, actualized for the stable release of...Docker 1 0 1 0 1: a Docker introduction, actualized for the stable release of...
Docker 1 0 1 0 1: a Docker introduction, actualized for the stable release of...
 
Devops with Python by Yaniv Cohen DevopShift
Devops with Python by Yaniv Cohen DevopShiftDevops with Python by Yaniv Cohen DevopShift
Devops with Python by Yaniv Cohen DevopShift
 
Devops is (not ) a buzzword
Devops is (not ) a buzzwordDevops is (not ) a buzzword
Devops is (not ) a buzzword
 
Pipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as CodePipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as Code
 
OSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris Buytaert
OSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris BuytaertOSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris Buytaert
OSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris Buytaert
 
GitLab for CI/CD process
GitLab for CI/CD processGitLab for CI/CD process
GitLab for CI/CD process
 
Drupal Continuous Integration Workflow
Drupal Continuous Integration WorkflowDrupal Continuous Integration Workflow
Drupal Continuous Integration Workflow
 
SCM Puppet: from an intro to the scaling
SCM Puppet: from an intro to the scalingSCM Puppet: from an intro to the scaling
SCM Puppet: from an intro to the scaling
 
Introduction to Docker at the Azure Meet-up in New York
Introduction to Docker at the Azure Meet-up in New YorkIntroduction to Docker at the Azure Meet-up in New York
Introduction to Docker at the Azure Meet-up in New York
 
Deploying software at Scale
Deploying software at ScaleDeploying software at Scale
Deploying software at Scale
 
OSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris Buytaert
OSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris BuytaertOSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris Buytaert
OSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris Buytaert
 
Netflix Open Source: Building a Distributed and Automated Open Source Program
Netflix Open Source:  Building a Distributed and Automated Open Source ProgramNetflix Open Source:  Building a Distributed and Automated Open Source Program
Netflix Open Source: Building a Distributed and Automated Open Source Program
 
Building a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at NetflixBuilding a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at Netflix
 
Free GitOps Workshop
Free GitOps WorkshopFree GitOps Workshop
Free GitOps Workshop
 

More from Kris Buytaert

Years of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoopsYears of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoopsKris Buytaert
 
Observability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,IgniteObservability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,IgniteKris Buytaert
 
Infrastructure as Code Patterns
Infrastructure as Code PatternsInfrastructure as Code Patterns
Infrastructure as Code PatternsKris Buytaert
 
From devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learningFrom devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learningKris Buytaert
 
Devops is Dead, Long live Devops
Devops is Dead, Long live DevopsDevops is Dead, Long live Devops
Devops is Dead, Long live DevopsKris Buytaert
 
10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ? 10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ? Kris Buytaert
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure FirstKris Buytaert
 
Is there a Future for devops ?
Is there a Future for devops   ? Is there a Future for devops   ?
Is there a Future for devops ? Kris Buytaert
 
10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdnessKris Buytaert
 
ADDO 2019: Looking back at over 10 years of Devops
ADDO 2019:    Looking back at over 10 years of DevopsADDO 2019:    Looking back at over 10 years of Devops
ADDO 2019: Looking back at over 10 years of DevopsKris Buytaert
 
Can we fix dev-oops ?
Can we fix dev-oops ?Can we fix dev-oops ?
Can we fix dev-oops ?Kris Buytaert
 
Continuous Infrastructure First Ignite Edition
Continuous Infrastructure First  Ignite EditionContinuous Infrastructure First  Ignite Edition
Continuous Infrastructure First Ignite EditionKris Buytaert
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure FirstKris Buytaert
 
Open Source Monitoring in 2019
Open Source Monitoring in 2019 Open Source Monitoring in 2019
Open Source Monitoring in 2019 Kris Buytaert
 
Migrating to Puppet 5
Migrating to Puppet 5Migrating to Puppet 5
Migrating to Puppet 5Kris Buytaert
 
Repositories as Code
Repositories as CodeRepositories as Code
Repositories as CodeKris Buytaert
 
Devops is a Security Requirement
Devops is a Security RequirementDevops is a Security Requirement
Devops is a Security RequirementKris Buytaert
 
Is there a future for devops ?
Is there a future for devops ?Is there a future for devops ?
Is there a future for devops ?Kris Buytaert
 
Looking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcampLooking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcampKris Buytaert
 
Automating MySQL operations with Puppet
Automating MySQL operations with PuppetAutomating MySQL operations with Puppet
Automating MySQL operations with PuppetKris Buytaert
 

More from Kris Buytaert (20)

Years of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoopsYears of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoops
 
Observability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,IgniteObservability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,Ignite
 
Infrastructure as Code Patterns
Infrastructure as Code PatternsInfrastructure as Code Patterns
Infrastructure as Code Patterns
 
From devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learningFrom devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learning
 
Devops is Dead, Long live Devops
Devops is Dead, Long live DevopsDevops is Dead, Long live Devops
Devops is Dead, Long live Devops
 
10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ? 10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ?
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure First
 
Is there a Future for devops ?
Is there a Future for devops   ? Is there a Future for devops   ?
Is there a Future for devops ?
 
10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness
 
ADDO 2019: Looking back at over 10 years of Devops
ADDO 2019:    Looking back at over 10 years of DevopsADDO 2019:    Looking back at over 10 years of Devops
ADDO 2019: Looking back at over 10 years of Devops
 
Can we fix dev-oops ?
Can we fix dev-oops ?Can we fix dev-oops ?
Can we fix dev-oops ?
 
Continuous Infrastructure First Ignite Edition
Continuous Infrastructure First  Ignite EditionContinuous Infrastructure First  Ignite Edition
Continuous Infrastructure First Ignite Edition
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure First
 
Open Source Monitoring in 2019
Open Source Monitoring in 2019 Open Source Monitoring in 2019
Open Source Monitoring in 2019
 
Migrating to Puppet 5
Migrating to Puppet 5Migrating to Puppet 5
Migrating to Puppet 5
 
Repositories as Code
Repositories as CodeRepositories as Code
Repositories as Code
 
Devops is a Security Requirement
Devops is a Security RequirementDevops is a Security Requirement
Devops is a Security Requirement
 
Is there a future for devops ?
Is there a future for devops ?Is there a future for devops ?
Is there a future for devops ?
 
Looking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcampLooking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcamp
 
Automating MySQL operations with Puppet
Automating MySQL operations with PuppetAutomating MySQL operations with Puppet
Automating MySQL operations with Puppet
 

Recently uploaded

Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandIES VE
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctBrainSell Technologies
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTopCSSGallery
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfFIDO Alliance
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Patrick Viafore
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024Lorenzo Miniero
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxFIDO Alliance
 
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The InsideCollecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The InsideStefan Dietze
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxFIDO Alliance
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxFIDO Alliance
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...FIDO Alliance
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxFIDO Alliance
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsLeah Henrickson
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfFIDO Alliance
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfFIDO Alliance
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingScyllaDB
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...ScyllaDB
 

Recently uploaded (20)

Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The InsideCollecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptx
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 

Building and Deploying MediaSalsa, an Open Source DAM as Saas platform

  • 1. Building and Deploying MediaSalsa A Digital Asset Management System as a Service Kris Buytaert @krisbuytaert Slides by Michel van de Ven and Julien Pivotto
  • 2. Kris BuytaertKris Buytaert ● I used to be a Dev,I used to be a Dev, ● Then Became an OpThen Became an Op ● Chief Trolling Officer and Open SourceChief Trolling Officer and Open Source Consultant @inuits.euConsultant @inuits.eu ● Everything is an effing DNS ProblemEverything is an effing DNS Problem ● Building Clouds since before the bookstoreBuilding Clouds since before the bookstore ● Some books, some papers, some blogsSome books, some papers, some blogs ● Evangelizing devopsEvangelizing devops
  • 3. MediaMosa ● Drupal-based Digital Asset Management system ● Commissioned by SURFnet and Kennisnet ● Open Source (GPLv2), Open Standards ● Webservice oriented: REST ● Store & retrieve assets ● Manage metadata using open standards – Dublin Core, Qualified DC, IEEE/LOM, CZP – OAI-PMH, BagIt ● Transcode video, audio, images, PDF &c, stream content ● Users: Kennisnet, NIBG, UGent, UvA, TiU, RUG, Radboud, UOslo, Avans, PolitieAcademie, Acquia (NBC Sport), Cineca &c ● http://mediamosa.org
  • 4.
  • 6. MediaSalsa infrastructure (simplified) ● For each environment (OTAP) – Backends: Core service (MediaMosa) – Frontends: Optional – Web servers – Database servers – Solr servers – Transcoding servers ● Instance @ RUG DC for NL .edu ● Instances @ Commercial DCs for non-edu
  • 8. Inuits ● Inuits is an Open Source company – We contribute back ● +40 people in 3 countries (BE, NL, UA) ● One language: English ● We offer – Consulting ● Development ● System Administration – and a niche Saas Platform
  • 9. Ideal world vs budget and reality → pragmatic approach
  • 10. Distributed team Communication = HELL How to fix it?
  • 11. Distributed team ● Daily virtual stand-up over XMPP / Hangout ● Redmine project management – Issues, tasks, sprints – Repositories – Documentation (wiki) ● Internal mailing lists ● Jenkins notifications by mail and XMPP ● Internal training
  • 12. Dev and Ops ● Dev ● Architecture ● Develop new features ● Write unit tests (!) ● Ops ● Infrastructure (puppet) ● CD and CI (jenkins) → Ops teach dev about monitoring and distributed services → Dev teach ops about required libraries and testing, dependencies
  • 14. Puppet Puppet automates all the things → mcollective orchestrates all the things
  • 15. CD ● Continuous Delivery vs Continuous Deployment – “Continuous Delivery doesn't mean every change is deployed to production ASAP. It means every change is proven to be deployable at any time” (@ccaum) ● Puppet code – Deployed to dev environment – Same puppet code for each environment – User-triggered deployments to UAT & Prod – Feature flags in Puppet code per environment (switchable architecture) ● Application code – Continuous integration in dev – User-triggered deployments to UAT – Deployment to prod is a business decision
  • 16. Testing ● Developers test a lot, but – The tests don’t work – It works on my machine™ – Wrong platform – Wrong PHP version Fixed now, thanks to Jenkins! and Vagrant...
  • 17. Version Control ● Git ● Code is under revision control – Prefer small commits – Local features branches ● Infrastructure as code → git / hiera
  • 18. Using OS packaging system ● Consistency, security, dependencies ● Uniquely identify where files are coming from ● Source repo may not be reacheable ● Little overhead when you automate ● Configuration does not belong in a package
  • 19. Pipelines ● A collection of jobs ● Run in sequence ● Start on checkout, end on deployment ● From the developers’ side: → Git push ← Mail with changes + link to deploy
  • 20. Our current pipelines ● Puppetized ● Puppet code and Application code (backend and frontends) ● 1 pipeline / project / customer Defining a pipeline: salsajobs::pipeline::frontend { ’qwerty-inc’: git_repository => ’ssh://git@redmine/qwerty.git’, dashboard_view => ’mds-frontends’, target_urls => { ’uat’ => ’http://qwerty.uat.mediasalsa.eu’, ’prod’ => ’http://qwerty.mediasalsa.eu’, }, require => Salsajobs::Views::Dashboard[’mds-frontends’], }
  • 21. Pipelines steps ● Checkout ● Syntax: php -l ● Style: Drupal Coder ● Package: FPM ● Deploy to dev environment: mcollective ● Tests in dev environment: drush run-tests ● Publish package and promote: mcollective
  • 22. Promotion ● At the end of the pipeline ● Send a simple email ● A link to the promotion page ● The changelog ● Promotion page contains one button per environment ● Must promote to UAT before Production
  • 23. Tools used with Jenkins ● Pulp to manage RPM repositories ● Mcollective to update packages, run drush
  • 24. Same Pipelines, Tools, Patterns are used by both devs and ops
  • 26. Logstash ● Collect all the logs – Drupal logs – Apache logs – Deployment logs – System logs ● Interprete, filter and correlate them ● Logstash, ElasticSearch, Kibana, statsd, Graphite
  • 27. Icinga ● Monitor everything – vhosts – databases – cronjobs – unit test suites
  • 28. Graphite + gdash ● Collectd ● Monitor platform usage ● FFmpeg usage ● Number of accounts ● Pipelined !
  • 30. Open Source ● Mediamosa is Fully Open Source ● Lots of the PuppetCode to deploy it is ● Our passwords etc aren't
  • 31. Experiences We have just shared our experience... But also at the NLUUG Fall Conference, DrupalCon Prague
  • 32. ContactContact Kris BuytaertKris Buytaert Kris.Buytaert@inuits.beKris.Buytaert@inuits.be Further ReadingFurther Reading @krisbuytaert@krisbuytaert http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/ http://www.inuits.be/http://www.inuits.be/ Inuits HQInuits HQ Duboistraat 50Duboistraat 50 2060 Antwerpen2060 Antwerpen BelgiumBelgium 891.514.231891.514.231 +32 475 961221+32 475 961221