• Like
  • Save
Building and Deploying MediaSalsa, an Open Source DAM as Saas platform
Upcoming SlideShare
Loading in...5
×
 

Building and Deploying MediaSalsa, an Open Source DAM as Saas platform

on

  • 1,328 views

Building and Deploying MediaSalsa an Open source Digital Asset Management platform as a Service.

Building and Deploying MediaSalsa an Open source Digital Asset Management platform as a Service.

My FlossUK 2014 talk

Statistics

Views

Total Views
1,328
Views on SlideShare
1,328
Embed Views
0

Actions

Likes
1
Downloads
13
Comments
1

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Building and Deploying MediaSalsa, an Open Source DAM as Saas platform Building and Deploying MediaSalsa, an Open Source DAM as Saas platform Presentation Transcript

    • Building and Deploying MediaSalsa A Digital Asset Management System as a Service Kris Buytaert @krisbuytaert Slides by Michel van de Ven and Julien Pivotto
    • Kris BuytaertKris Buytaert ● I used to be a Dev,I used to be a Dev, ● Then Became an OpThen Became an Op ● Chief Trolling Officer and Open SourceChief Trolling Officer and Open Source Consultant @inuits.euConsultant @inuits.eu ● Everything is an effing DNS ProblemEverything is an effing DNS Problem ● Building Clouds since before the bookstoreBuilding Clouds since before the bookstore ● Some books, some papers, some blogsSome books, some papers, some blogs ● Evangelizing devopsEvangelizing devops
    • MediaMosa ● Drupal-based Digital Asset Management system ● Commissioned by SURFnet and Kennisnet ● Open Source (GPLv2), Open Standards ● Webservice oriented: REST ● Store & retrieve assets ● Manage metadata using open standards – Dublin Core, Qualified DC, IEEE/LOM, CZP – OAI-PMH, BagIt ● Transcode video, audio, images, PDF &c, stream content ● Users: Kennisnet, NIBG, UGent, UvA, TiU, RUG, Radboud, UOslo, Avans, PolitieAcademie, Acquia (NBC Sport), Cineca &c ● http://mediamosa.org
    • = MediaMosa as a Service
    • MediaSalsa infrastructure (simplified) ● For each environment (OTAP) – Backends: Core service (MediaMosa) – Frontends: Optional – Web servers – Database servers – Solr servers – Transcoding servers ● Instance @ RUG DC for NL .edu ● Instances @ Commercial DCs for non-edu
    • Culture Automation Measurement Sharing
    • Inuits ● Inuits is an Open Source company – We contribute back ● +40 people in 3 countries (BE, NL, UA) ● One language: English ● We offer – Consulting ● Development ● System Administration – and a niche Saas Platform
    • Ideal world vs budget and reality → pragmatic approach
    • Distributed team Communication = HELL How to fix it?
    • Distributed team ● Daily virtual stand-up over XMPP / Hangout ● Redmine project management – Issues, tasks, sprints – Repositories – Documentation (wiki) ● Internal mailing lists ● Jenkins notifications by mail and XMPP ● Internal training
    • Dev and Ops ● Dev ● Architecture ● Develop new features ● Write unit tests (!) ● Ops ● Infrastructure (puppet) ● CD and CI (jenkins) → Ops teach dev about monitoring and distributed services → Dev teach ops about required libraries and testing, dependencies
    • Culture Automation Measurement Sharing
    • Puppet Puppet automates all the things → mcollective orchestrates all the things
    • CD ● Continuous Delivery vs Continuous Deployment – “Continuous Delivery doesn't mean every change is deployed to production ASAP. It means every change is proven to be deployable at any time” (@ccaum) ● Puppet code – Deployed to dev environment – Same puppet code for each environment – User-triggered deployments to UAT & Prod – Feature flags in Puppet code per environment (switchable architecture) ● Application code – Continuous integration in dev – User-triggered deployments to UAT – Deployment to prod is a business decision
    • Testing ● Developers test a lot, but – The tests don’t work – It works on my machine™ – Wrong platform – Wrong PHP version Fixed now, thanks to Jenkins! and Vagrant...
    • Version Control ● Git ● Code is under revision control – Prefer small commits – Local features branches ● Infrastructure as code → git / hiera
    • Using OS packaging system ● Consistency, security, dependencies ● Uniquely identify where files are coming from ● Source repo may not be reacheable ● Little overhead when you automate ● Configuration does not belong in a package
    • Pipelines ● A collection of jobs ● Run in sequence ● Start on checkout, end on deployment ● From the developers’ side: → Git push ← Mail with changes + link to deploy
    • Our current pipelines ● Puppetized ● Puppet code and Application code (backend and frontends) ● 1 pipeline / project / customer Defining a pipeline: salsajobs::pipeline::frontend { ’qwerty-inc’: git_repository => ’ssh://git@redmine/qwerty.git’, dashboard_view => ’mds-frontends’, target_urls => { ’uat’ => ’http://qwerty.uat.mediasalsa.eu’, ’prod’ => ’http://qwerty.mediasalsa.eu’, }, require => Salsajobs::Views::Dashboard[’mds-frontends’], }
    • Pipelines steps ● Checkout ● Syntax: php -l ● Style: Drupal Coder ● Package: FPM ● Deploy to dev environment: mcollective ● Tests in dev environment: drush run-tests ● Publish package and promote: mcollective
    • Promotion ● At the end of the pipeline ● Send a simple email ● A link to the promotion page ● The changelog ● Promotion page contains one button per environment ● Must promote to UAT before Production
    • Tools used with Jenkins ● Pulp to manage RPM repositories ● Mcollective to update packages, run drush
    • Same Pipelines, Tools, Patterns are used by both devs and ops
    • Culture Automation Measurement Sharing
    • Logstash ● Collect all the logs – Drupal logs – Apache logs – Deployment logs – System logs ● Interprete, filter and correlate them ● Logstash, ElasticSearch, Kibana, statsd, Graphite
    • Icinga ● Monitor everything – vhosts – databases – cronjobs – unit test suites
    • Graphite + gdash ● Collectd ● Monitor platform usage ● FFmpeg usage ● Number of accounts ● Pipelined !
    • Culture Automation Measurement Sharing
    • Open Source ● Mediamosa is Fully Open Source ● Lots of the PuppetCode to deploy it is ● Our passwords etc aren't
    • Experiences We have just shared our experience... But also at the NLUUG Fall Conference, DrupalCon Prague
    • ContactContact Kris BuytaertKris Buytaert Kris.Buytaert@inuits.beKris.Buytaert@inuits.be Further ReadingFurther Reading @krisbuytaert@krisbuytaert http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/ http://www.inuits.be/http://www.inuits.be/ Inuits HQInuits HQ Duboistraat 50Duboistraat 50 2060 Antwerpen2060 Antwerpen BelgiumBelgium 891.514.231891.514.231 +32 475 961221+32 475 961221