;
Building and Deploying MediasalsaBuilding and Deploying Mediasalsa
A Drupal Digital Asset Management system as a Service...
;
MediaMosa
• Drupal-based Digital Asset Management systemDrupal-based Digital Asset Management system
• Open-sourceOpen-s...
;
MediaSalsa
==
MediaMosa as a ServiceMediaMosa as a Service
Julien Pivotto Building and Deploying Mediasalsa
;
MediaSalsa infrastructure (simplified)
• Backend: Core service (MediaMosa)Backend: Core service (MediaMosa)
• Frontends: ...
;
Culture
Automation
Measurement
Sharing
Julien Pivotto Building and Deploying Mediasalsa
;
Inuits
• Inuits is an Open-Source companyInuits is an Open-Source company
⇒ We contribute backWe contribute back
• ± 40 ...
;
Inuits
What do we do?
• Consulting and Internal projectsConsulting and Internal projects
• Development and System admini...
;
Inuits
Ideal world vs budget and realityIdeal world vs budget and reality
⇒ pragmatic approach⇒ pragmatic approach
Julie...
;
Distributed team
Communication = hell
How to fix it?
• Daily virtual stand-up over XMPP/HangoutDaily virtual stand-up ove...
;
Dev and Ops
DevDev
• Develop new featuresDevelop new features
• Write testsWrite tests
OpsOps
• Infrastructure (puppet)I...
;
Culture
Automation
Measurement
Sharing
Julien Pivotto Building and Deploying Mediasalsa
;
Puppet
Puppet automates all the things
⇒ mcollective orchestrates all the things
Julien Pivotto Building and Deploying M...
;
CD
Continuous Delivery vs Continuous DeploymentContinuous Delivery vs Continuous Deployment
• Puppet codePuppet code
• D...
;
Tests
Developers test a lot
. . . The tests don’t workThe tests don’t work
. . . It works on my machineIt works on my ma...
;
SCM
Code is under revision control (git)Code is under revision control (git)
• Prefer small commitsPrefer small commits
...
;
Using OS packaging system
• Consistency, security, dependenciesConsistency, security, dependencies
• Uniquely identify w...
;
Pipelines
• A collection of jobsA collection of jobs
• Run one after anotherRun one after another
• Start on checkout, e...
;
Our pipelines
• PuppetizedPuppetized
• Puppet code and Application code (backend and frontends)Puppet code and Applicati...
;
Pipelines steps
• CheckoutCheckout
• Syntax: php -lSyntax: php -l
• Style: Drupal CoderStyle: Drupal Coder
• Package: FP...
;
Promotion
• At the end of the pipelineAt the end of the pipeline
• Send a simple emailSend a simple email
• A link to th...
;
Tools used with jenkins
• Pulp to manage RPM repositoriesPulp to manage RPM repositories
• Mcollective to update package...
;
Culture
Automation
Measurement
Sharing
Julien Pivotto Building and Deploying Mediasalsa
;
Logstash
Collect all the logs
• Drupal logsDrupal logs
• Apache logsApache logs
• Deployment logsDeployment logs
• Syste...
;
Icinga
Monitor everything
• "Basics""Basics"
+ vhostsvhosts
+ databasesdatabases
+ cronjobscronjobs
Julien Pivotto Build...
;
Graphite + gdash
• CollectdCollectd
• Monitor platform usageMonitor platform usage
• FFmpeg usageFFmpeg usage
• Number o...
;
FFmpeg
Julien Pivotto Building and Deploying Mediasalsa
;
Managed assets
Julien Pivotto Building and Deploying Mediasalsa
;
Number of backend ID’s
Julien Pivotto Building and Deploying Mediasalsa
;
Culture
Automation
Measurement
Sharing
Julien Pivotto Building and Deploying Mediasalsa
;
Sharing
We have just shared our experience. . .We have just shared our experience. . .
Any question?Any question?
Julien...
;
Contact
Julien PivottoJulien Pivotto
julien@inuits.eujulien@inuits.eu
@roidelapluie@roidelapluie
INUITS bvbaINUITS bvba
...
THANK YOU!
WHAT DID YOU THINK?
Locate this session at the
DrupalCon Prague website:
http://prague2013.drupal.org/schedule
...
Upcoming SlideShare
Loading in...5
×

Building and Deploying MediaSalsa, a drupal-based DAM as a Service

3,127

Published on

Conference given a Drupalcon Prague 2013

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
3,127
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Building and Deploying MediaSalsa, a drupal-based DAM as a Service"

  1. 1. ; Building and Deploying MediasalsaBuilding and Deploying Mediasalsa A Drupal Digital Asset Management system as a ServiceA Drupal Digital Asset Management system as a Service Julien PivottoJulien Pivotto DrupalCon PragueDrupalCon Prague September 26, 2013September 26, 2013
  2. 2. ; MediaMosa • Drupal-based Digital Asset Management systemDrupal-based Digital Asset Management system • Open-sourceOpen-source • Store assetsStore assets • Trancode videosTrancode videos • Create, extract and manage metadata using open standards:Create, extract and manage metadata using open standards: Dublin Core, Qualified DC, IEEE/LOM, CZPDublin Core, Qualified DC, IEEE/LOM, CZP • Webservice orientedWebservice oriented Julien Pivotto Building and Deploying Mediasalsa
  3. 3. ; MediaSalsa == MediaMosa as a ServiceMediaMosa as a Service Julien Pivotto Building and Deploying Mediasalsa
  4. 4. ; MediaSalsa infrastructure (simplified) • Backend: Core service (MediaMosa)Backend: Core service (MediaMosa) • Frontends: OptionalFrontends: Optional • Web serversWeb servers • Database serverDatabase server • Solr serverSolr server • Transcoding serversTranscoding servers    for each environmentfor each environment Julien Pivotto Building and Deploying Mediasalsa
  5. 5. ; Culture Automation Measurement Sharing Julien Pivotto Building and Deploying Mediasalsa
  6. 6. ; Inuits • Inuits is an Open-Source companyInuits is an Open-Source company ⇒ We contribute backWe contribute back • ± 40 people in 3 countries± 40 people in 3 countries • One language: EnglishOne language: English Julien Pivotto Building and Deploying Mediasalsa
  7. 7. ; Inuits What do we do? • Consulting and Internal projectsConsulting and Internal projects • Development and System administrationDevelopment and System administration Julien Pivotto Building and Deploying Mediasalsa
  8. 8. ; Inuits Ideal world vs budget and realityIdeal world vs budget and reality ⇒ pragmatic approach⇒ pragmatic approach Julien Pivotto Building and Deploying Mediasalsa
  9. 9. ; Distributed team Communication = hell How to fix it? • Daily virtual stand-up over XMPP/HangoutDaily virtual stand-up over XMPP/Hangout • Redmine project managmentRedmine project managment • TasksTasks • RepositoriesRepositories • Documentation (wiki)Documentation (wiki) • Internal mailing listsInternal mailing lists • Jenkins notifications by mail and XMPPJenkins notifications by mail and XMPP • Internal trainingInternal training Julien Pivotto Building and Deploying Mediasalsa
  10. 10. ; Dev and Ops DevDev • Develop new featuresDevelop new features • Write testsWrite tests OpsOps • Infrastructure (puppet)Infrastructure (puppet) • CD and CI (jenkins)CD and CI (jenkins) Ops teach dev to think about monitoring and distributed serviceOps teach dev to think about monitoring and distributed service Dev teach to ops about required libraries and testsDev teach to ops about required libraries and tests Julien Pivotto Building and Deploying Mediasalsa
  11. 11. ; Culture Automation Measurement Sharing Julien Pivotto Building and Deploying Mediasalsa
  12. 12. ; Puppet Puppet automates all the things ⇒ mcollective orchestrates all the things Julien Pivotto Building and Deploying Mediasalsa
  13. 13. ; CD Continuous Delivery vs Continuous DeploymentContinuous Delivery vs Continuous Deployment • Puppet codePuppet code • Deployed to dev environmentDeployed to dev environment • Same puppet code for each environmentSame puppet code for each environment • User-triggered deployments to UAT and prodUser-triggered deployments to UAT and prod • Feature flags in Puppet code per environmentFeature flags in Puppet code per environment • Application codeApplication code • Continuous integration in devContinuous integration in dev • User-triggered deployments to UATUser-triggered deployments to UAT • Deployment to prod is a business decisionDeployment to prod is a business decision Julien Pivotto Building and Deploying Mediasalsa
  14. 14. ; Tests Developers test a lot . . . The tests don’t workThe tests don’t work . . . It works on my machineIt works on my machine . . . Wrong platformWrong platform . . . Wrong PHP versionWrong PHP version ⇒ Fixed now, thanks to Jenkins!Fixed now, thanks to Jenkins! Julien Pivotto Building and Deploying Mediasalsa
  15. 15. ; SCM Code is under revision control (git)Code is under revision control (git) • Prefer small commitsPrefer small commits • Local features branchesLocal features branches • PackagesPackages Julien Pivotto Building and Deploying Mediasalsa
  16. 16. ; Using OS packaging system • Consistency, security, dependenciesConsistency, security, dependencies • Uniquely identify where files are coming fromUniquely identify where files are coming from • Source repo may not be reacheableSource repo may not be reacheable • Little overhead when you automateLittle overhead when you automate • Configuration does not belong to a packageConfiguration does not belong to a package Julien Pivotto Building and Deploying Mediasalsa
  17. 17. ; Pipelines • A collection of jobsA collection of jobs • Run one after anotherRun one after another • Start on checkout, end on deploymentStart on checkout, end on deployment • From the developers’ side:From the developers’ side: ⇒ Git pushGit push ⇐ Mail with changes + link to deployMail with changes + link to deploy Julien Pivotto Building and Deploying Mediasalsa
  18. 18. ; Our pipelines • PuppetizedPuppetized • Puppet code and Application code (backend and frontends)Puppet code and Application code (backend and frontends) • 1 pipeline/project/customer1 pipeline/project/customer Defining a pipeline salsajobs::pipeline::frontend { ’qwerty-inc’: git_repository => ’ssh://git@redmine/qwerty.git’, dashboard_view => ’mds-frontends’, target_urls => { ’uat’ => ’http://qwerty.uat.mediasalsa.eu’, ’prod’ => ’http://qwerty.mediasalsa.eu’, }, require => Salsajobs::Views::Dashboard[’mds-frontends’], } Julien Pivotto Building and Deploying Mediasalsa
  19. 19. ; Pipelines steps • CheckoutCheckout • Syntax: php -lSyntax: php -l • Style: Drupal CoderStyle: Drupal Coder • Package: FPMPackage: FPM • Deploy to dev environment: McoDeploy to dev environment: Mco • Tests in dev environment: drush run-testsTests in dev environment: drush run-tests • Publish package and promote: McoPublish package and promote: Mco Julien Pivotto Building and Deploying Mediasalsa
  20. 20. ; Promotion • At the end of the pipelineAt the end of the pipeline • Send a simple emailSend a simple email • A link to the promotion pageA link to the promotion page • The changelogThe changelog • Promotion page contains one button per environmentPromotion page contains one button per environment • Must promote to UAT before ProductionMust promote to UAT before Production Julien Pivotto Building and Deploying Mediasalsa
  21. 21. ; Tools used with jenkins • Pulp to manage RPM repositoriesPulp to manage RPM repositories • Mcollective to update packages, run drushMcollective to update packages, run drush Julien Pivotto Building and Deploying Mediasalsa
  22. 22. ; Culture Automation Measurement Sharing Julien Pivotto Building and Deploying Mediasalsa
  23. 23. ; Logstash Collect all the logs • Drupal logsDrupal logs • Apache logsApache logs • Deployment logsDeployment logs • System logsSystem logs ⇒ Logstash, ES, kibana, statsd, graphiteLogstash, ES, kibana, statsd, graphite Julien Pivotto Building and Deploying Mediasalsa
  24. 24. ; Icinga Monitor everything • "Basics""Basics" + vhostsvhosts + databasesdatabases + cronjobscronjobs Julien Pivotto Building and Deploying Mediasalsa
  25. 25. ; Graphite + gdash • CollectdCollectd • Monitor platform usageMonitor platform usage • FFmpeg usageFFmpeg usage • Number of accounts (backend ID’s)Number of accounts (backend ID’s) Julien Pivotto Building and Deploying Mediasalsa
  26. 26. ; FFmpeg Julien Pivotto Building and Deploying Mediasalsa
  27. 27. ; Managed assets Julien Pivotto Building and Deploying Mediasalsa
  28. 28. ; Number of backend ID’s Julien Pivotto Building and Deploying Mediasalsa
  29. 29. ; Culture Automation Measurement Sharing Julien Pivotto Building and Deploying Mediasalsa
  30. 30. ; Sharing We have just shared our experience. . .We have just shared our experience. . . Any question?Any question? Julien Pivotto Building and Deploying Mediasalsa
  31. 31. ; Contact Julien PivottoJulien Pivotto julien@inuits.eujulien@inuits.eu @roidelapluie@roidelapluie INUITS bvbaINUITS bvba Duboisstraat 50Duboisstraat 50 2060 Antwerp2060 Antwerp BelgiumBelgium +32 473 441 636+32 473 441 636 https://inuits.euhttps://inuits.eu Julien Pivotto Building and Deploying Mediasalsa
  32. 32. THANK YOU! WHAT DID YOU THINK? Locate this session at the DrupalCon Prague website: http://prague2013.drupal.org/schedule Click the “Take the survey” link
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×