Devops
  and Drupal
 Current State
Kris Buytaert, march 2012
Kris Buytaert
•   I used to be a Dev,
•   Then Became an Op
•   Chief Trolling Officer and Open Source
    Consultant @inuits.eu
•   Everything is an effing DNS Problem
•   Building Clouds since before the bookstore
•   Some books, some papers, some blogs
•   But mostly, trying to be good at my job
Devop, definition
●   30 something
●   Senior Infrastructure guy
●   Development background
●   Open Source Expcerience
●   Mostly European (.be / .uk)
●   Likes Belgian Beer
●   Likes Sushi
What's this Devops thing really about ?
World , 200X-2009
Patrick Debois, Gildas Le Nadan, Andrew Clay Shafer, Kris Buytaert, Jezz
Humble, Lindsay Holmwood, John Willis, Chris Read, Julian Simpson, and
                            lots of others ..

                    Gent , October 2009
                Mountain View , June 2010
                 Hamburg , October 2010
                     Boston, March 2011
                Mountain View, June 2011
                  Bangalore, Melbourne,
                 Goteborg , October 2011
●   Devops is a growing movement
●   We don't have all the answers yet
●   We are reaching out to different communities
●   We will point out problems we see..
●   Only the name is new




         While we are still working out the solutions
What's the problem ?
The community of developers whose work you
see on the Web, who probably don’t know what
ADO or UML or JPA even stand for, deploy better
systems at less cost in less time at lower risk
than we see in the Enterprise. This is true even
when you factor in the greater flexibility and
velocity of startups.


Tim Bray , on his blog January 2010
The real problem :
●   Friday evening at 16:59
    “Put this Code Live, here's a tarball”
    NOW!
●   Backups ?
●   What database ?
●   Security ?
●   High Availability ?
●   Scalability ?
●   Who is on Call ?
“devops is a cultural and
professional movement”
                    Adam Jacob
CAMS
●   Culture
●   Automation
●   Measurement
●   Sharing


              Damon Edwards and John Willis
Why a survey ?
•   Gut feeling vs Reality


•   The sad state of our customers practices
•   Java folks GET the Problem !
•   Twittersphere says it doesn't matter for PHP
•   Lies , Damn Lies and Statistics
Audience
●   200+
●   Schizophrenic
●   Experienced (6y
    average)
●   Webdeveloping
●   System
    Administrators
Organisation Type
●   In small to midsized
    companies
●   Good internal
    Communication


●   Vs Large Enterprises
    with Silos ..
What's devops to them ?

•   A buzzword
•   Developers who also do IT operations, or visa versa.
•   The people in charge of the build/release cycle and planning.
•   system administrators with a development culture.
•   Someone who mixes both a sysop and dev duties
•   the combination of developer and operations into one overall
    functionality
What's devops to them ?
    #




                            NG
•   A buzzword
•   Developers who also do IT operations, or visa versa.
•



•
                 RO
    The people in charge of the build/release cycle and planning.

    system administrators with a development culture.
•   Someone who mixes both a sysop and dev duties
        W
•   the combination of developer and operations into one overall
    functionality
What's devops to them ?
•   Breaking the wall between dev and ops in the same way agile
    breaks the wall between business and dev e.g. coming to terms
    with changing requirements, iterative cycles
•   Sysadmin best-practise, using configuration as code, and
    facilitating communication between sysadmins and developers,
    with each understanding and participating in the activities of the
    other.
•   Devops is both the process of developers and system operators
    working closer together, as well as people who know (or who
    have worked in) both development and system operations.
•   Removing barriers to communication and efficiency through
    shared vocabulary, ideals, and business objectives to to deliver
    value.
•   A set of principles and good practices to improve the interactions
    between Operations and Development.
Practice devops ?
●   Lack of Clear
    Definition
●   33% does
●   24% wants
●   22% is in trouble
Know Continuous
   Delivery ?
Nirvana
An “ecosystem” that supports continuous delivery, from
infrastructure, data and configuration management to
business.
Through automation of the build, deployment, and testing
process, and improved collaboration between developers,
testers, and operations, delivery teams can get changes
released in a matter of hours — sometimes even minutes–no
matter what the size of a project or the complexity of its code
base.
                            Continuous Delivery , Jez Humble
Site up to date ?
●   70% reads Drupal
    Security News .
●   Sysadmins Update
●   Other = Developers
•Really ?
●   Not all components
    however :(
Critical Bugfixes ?
Typical Enviroments
            For Devs                   For Ops
●   Scrum                   ●   Kanban
●   Version Control         ●   Version Control
●   Automated Build         ●   Automated Build
●   Bugtracking             ●   Bugtracking
●   Continous integration   ●   Continous integration
●   Integrated testing      ●   Integrated testing
●   Automated               ●   Automated
    deployment                  deployment
What about the
 Drupalistas ?
Version Control
Continuous Integration
Testing
●   No frameworks used
            vs
●   No exploratory Testing
Where do you develop ?
●   It works on my
    machine :(




●   What other platforms
    do you use :
Looking for ?
“As a system administrator, I can tell when software
vendors hate me. It shows in their products.”

“DON'T make the administrative interface a GUI. System
administrators need a command-line tool for constructing
repeatable processes. Procedures are best documented by
providing commands that we can copy and paste from the
procedure document to the command line. We cannot
achieve the same repeatability when the instructions are:
"Checkmark the 3rd and 5th options, but not the 2nd
option, then click OK." Sysadmins do not want a GUI that
requires 25 clicks for each new user.”
                        Thomas A. Limoncelli in ACM Queue December 2010

                               http://queue.acm.org/detail.cfm?id=1921361
Deployment
●   Database Imports are
    EVIL
●   Manual Installations
    are
         ●   Error Prone       How do you deploy ?

         ●   Non
             Reproducible


●   61% does it WRONG
                            Do you Test Deployments ?
Deployment Alternatives
●   Aegir (security ?)
●   Fabric
●   Svn git checkouts
    (connectivity ?)
                         How do you deploy ?
●   Capistrano
●   Custom Tools
Deploying a Drupal Site
●   Drush make
●   Installation Profiles
●   Features ,strongarm , Custom code
          ●   Configuration as Code
●   What with external apps and libraries?
          ●   Solr – BI software
●   Test before you drive! (CI)
How do you configure modules ?
●   Not reproducable
●   Not automated
●   50% can improve :)


●   D8 Configuration
    Mgmt Efforts !
Can you fully reproduce the last site you've
             build from code ?
Challenges
●   What about the data ?
●   Content vs Config
          ●   Drupal provides no clear distinction (yet)
●   e.g a billing application
          ●   Table Creation
          ●   Provisioning (e.g. Rates, Call plan types)
          ●   Actual Data
Content Editing
●   Non technical people
●   In production
●   Plenty of UGC

                           Who edits content

●   Frequent Updated
    Sites
●   Upgrade troubles
    ahead

                              Where do you edit content ?
Upgrading a site

●   Drush
         ●   pm-updatecode
         ●   Updatedb
         ●   Features-update
●   Once again, test before you drive!
●   But ... what about userdata ?
Scalability
Proxy ?
WebServer ?
Caching / Scale / Deployment


●   MySQL   ●   NoSQL   ●   MemCache
Disaster Recovery ?
12 days into operations
The future
●   Don't forget
         ●   Lies, Damn Lies and Statistics
         ●   These people CARED to fill in the survey
•D8 Configuration management initiative
•DrupalCon Munich infrastructure/devops track !
Conclusions
●   Drupal gets is and is supportive
●   Conversation happens
●   Long Journey ahead
●   Heading in the right direction
It's not about the tools
  It's about change
 It's about the people
Contact
Kris Buytaert
Kris.Buytaert@inuits.eu

Further Reading
@krisbuytaert
http://krisbuytaert.be/blog/
http://www.inuits.eu/




                               Inuits
                               Duboistraat 50
                               2060 Antwerpen
                               Belgium
                               891.514.231

                               +32 475 961221
Deploy Where ?
Search

Drupal and Devops , the Survey Results

  • 1.
    Devops andDrupal Current State Kris Buytaert, march 2012
  • 2.
    Kris Buytaert • I used to be a Dev, • Then Became an Op • Chief Trolling Officer and Open Source Consultant @inuits.eu • Everything is an effing DNS Problem • Building Clouds since before the bookstore • Some books, some papers, some blogs • But mostly, trying to be good at my job
  • 3.
    Devop, definition ● 30 something ● Senior Infrastructure guy ● Development background ● Open Source Expcerience ● Mostly European (.be / .uk) ● Likes Belgian Beer ● Likes Sushi
  • 4.
    What's this Devopsthing really about ?
  • 5.
    World , 200X-2009 PatrickDebois, Gildas Le Nadan, Andrew Clay Shafer, Kris Buytaert, Jezz Humble, Lindsay Holmwood, John Willis, Chris Read, Julian Simpson, and lots of others .. Gent , October 2009 Mountain View , June 2010 Hamburg , October 2010 Boston, March 2011 Mountain View, June 2011 Bangalore, Melbourne, Goteborg , October 2011
  • 6.
    Devops is a growing movement ● We don't have all the answers yet ● We are reaching out to different communities ● We will point out problems we see.. ● Only the name is new While we are still working out the solutions
  • 7.
    What's the problem? The community of developers whose work you see on the Web, who probably don’t know what ADO or UML or JPA even stand for, deploy better systems at less cost in less time at lower risk than we see in the Enterprise. This is true even when you factor in the greater flexibility and velocity of startups. Tim Bray , on his blog January 2010
  • 8.
    The real problem: ● Friday evening at 16:59 “Put this Code Live, here's a tarball” NOW! ● Backups ? ● What database ? ● Security ? ● High Availability ? ● Scalability ? ● Who is on Call ?
  • 9.
    “devops is acultural and professional movement” Adam Jacob
  • 10.
    CAMS ● Culture ● Automation ● Measurement ● Sharing Damon Edwards and John Willis
  • 11.
    Why a survey? • Gut feeling vs Reality • The sad state of our customers practices • Java folks GET the Problem ! • Twittersphere says it doesn't matter for PHP • Lies , Damn Lies and Statistics
  • 12.
    Audience ● 200+ ● Schizophrenic ● Experienced (6y average) ● Webdeveloping ● System Administrators
  • 13.
    Organisation Type ● In small to midsized companies ● Good internal Communication ● Vs Large Enterprises with Silos ..
  • 14.
    What's devops tothem ? • A buzzword • Developers who also do IT operations, or visa versa. • The people in charge of the build/release cycle and planning. • system administrators with a development culture. • Someone who mixes both a sysop and dev duties • the combination of developer and operations into one overall functionality
  • 15.
    What's devops tothem ? # NG • A buzzword • Developers who also do IT operations, or visa versa. • • RO The people in charge of the build/release cycle and planning. system administrators with a development culture. • Someone who mixes both a sysop and dev duties W • the combination of developer and operations into one overall functionality
  • 16.
    What's devops tothem ? • Breaking the wall between dev and ops in the same way agile breaks the wall between business and dev e.g. coming to terms with changing requirements, iterative cycles • Sysadmin best-practise, using configuration as code, and facilitating communication between sysadmins and developers, with each understanding and participating in the activities of the other. • Devops is both the process of developers and system operators working closer together, as well as people who know (or who have worked in) both development and system operations. • Removing barriers to communication and efficiency through shared vocabulary, ideals, and business objectives to to deliver value. • A set of principles and good practices to improve the interactions between Operations and Development.
  • 17.
    Practice devops ? ● Lack of Clear Definition ● 33% does ● 24% wants ● 22% is in trouble
  • 18.
    Know Continuous Delivery ?
  • 19.
    Nirvana An “ecosystem” thatsupports continuous delivery, from infrastructure, data and configuration management to business. Through automation of the build, deployment, and testing process, and improved collaboration between developers, testers, and operations, delivery teams can get changes released in a matter of hours — sometimes even minutes–no matter what the size of a project or the complexity of its code base. Continuous Delivery , Jez Humble
  • 20.
    Site up todate ? ● 70% reads Drupal Security News . ● Sysadmins Update ● Other = Developers •Really ? ● Not all components however :(
  • 21.
  • 22.
    Typical Enviroments For Devs For Ops ● Scrum ● Kanban ● Version Control ● Version Control ● Automated Build ● Automated Build ● Bugtracking ● Bugtracking ● Continous integration ● Continous integration ● Integrated testing ● Integrated testing ● Automated ● Automated deployment deployment
  • 23.
    What about the Drupalistas ?
  • 24.
  • 25.
  • 26.
    Testing ● No frameworks used vs ● No exploratory Testing
  • 28.
    Where do youdevelop ? ● It works on my machine :( ● What other platforms do you use :
  • 29.
    Looking for ? “Asa system administrator, I can tell when software vendors hate me. It shows in their products.” “DON'T make the administrative interface a GUI. System administrators need a command-line tool for constructing repeatable processes. Procedures are best documented by providing commands that we can copy and paste from the procedure document to the command line. We cannot achieve the same repeatability when the instructions are: "Checkmark the 3rd and 5th options, but not the 2nd option, then click OK." Sysadmins do not want a GUI that requires 25 clicks for each new user.” Thomas A. Limoncelli in ACM Queue December 2010 http://queue.acm.org/detail.cfm?id=1921361
  • 30.
    Deployment ● Database Imports are EVIL ● Manual Installations are ● Error Prone How do you deploy ? ● Non Reproducible ● 61% does it WRONG Do you Test Deployments ?
  • 31.
    Deployment Alternatives ● Aegir (security ?) ● Fabric ● Svn git checkouts (connectivity ?) How do you deploy ? ● Capistrano ● Custom Tools
  • 32.
    Deploying a DrupalSite ● Drush make ● Installation Profiles ● Features ,strongarm , Custom code ● Configuration as Code ● What with external apps and libraries? ● Solr – BI software ● Test before you drive! (CI)
  • 33.
    How do youconfigure modules ? ● Not reproducable ● Not automated ● 50% can improve :) ● D8 Configuration Mgmt Efforts !
  • 34.
    Can you fullyreproduce the last site you've build from code ?
  • 35.
    Challenges ● What about the data ? ● Content vs Config ● Drupal provides no clear distinction (yet) ● e.g a billing application ● Table Creation ● Provisioning (e.g. Rates, Call plan types) ● Actual Data
  • 36.
    Content Editing ● Non technical people ● In production ● Plenty of UGC Who edits content ● Frequent Updated Sites ● Upgrade troubles ahead Where do you edit content ?
  • 37.
    Upgrading a site ● Drush ● pm-updatecode ● Updatedb ● Features-update ● Once again, test before you drive! ● But ... what about userdata ?
  • 38.
  • 39.
  • 40.
  • 41.
    Caching / Scale/ Deployment ● MySQL ● NoSQL ● MemCache
  • 42.
  • 43.
    12 days intooperations
  • 44.
    The future ● Don't forget ● Lies, Damn Lies and Statistics ● These people CARED to fill in the survey •D8 Configuration management initiative •DrupalCon Munich infrastructure/devops track !
  • 45.
    Conclusions ● Drupal gets is and is supportive ● Conversation happens ● Long Journey ahead ● Heading in the right direction
  • 46.
    It's not aboutthe tools It's about change It's about the people
  • 47.
  • 48.
  • 49.