Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Migraine Drupal - syncing your staging and live sites

16,089 views

Published on

This is the presentation to be delivered on 31st jan '09 at drupal camp by Mukesh agarwal.

Published in: Technology
  • DOWNLOAD THAT BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download Full EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download Full doc Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download doc Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book that can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer that is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story That Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money That the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths that Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Migraine Drupal - syncing your staging and live sites

  1. 1. Migraine Maintaining Live Production Sites Mukesh Agarwal iLoveBolly.com
  2. 2. AGENDA <ul><li>Discuss the development methodology </li></ul><ul><ul><li>Development – Staging - Production </li></ul></ul><ul><li>Synchronize our live websites with the current development </li></ul><ul><ul><li>Migraine – Synchronize Databases </li></ul></ul><ul><ul><li>SVN – Synchronize file system </li></ul></ul>
  3. 3. What we need? <ul><li>Add new features to the production site without affecting the content and of course with minimum downtime. </li></ul><ul><li>How does FACEBOOK or ORKUT or any other social networking site do it?? Do we ever notice these sites down for development?? NO!! Yet, there are so many new features so very often!!! That’s what we need to implement in Drupal!!! </li></ul>
  4. 4. Possibilities in Drupal <ul><li>Unfortunately, there are not too many ways to proceed with migrating your development to the production. </li></ul><ul><li>Why is it difficult in Drupal?? </li></ul><ul><ul><li>Drupal stores a lot of its configuration in database , and there is no module or tool to draw a line between your configuration and content. </li></ul></ul><ul><li>Disappointment for all those who thought it’s a 1-click feature </li></ul>
  5. 5. What then?? <ul><li>Don’t worry!! We are not in a no man’s land. </li></ul><ul><li>Big Shots might be using some rocket science, but then here comes MIGRAINE to aid of Drupal developers and make their life easy working with it. </li></ul>
  6. 6. Set the environment <ul><li>Lets understand the generic 4-tiers here. </li></ul><ul><li>Development </li></ul><ul><ul><li>Working Environment for individuals or small teams </li></ul></ul><ul><ul><li>Radical changes can be tried without adversely affecting the rest of development </li></ul></ul><ul><ul><li>Optional </li></ul></ul><ul><li>Integration </li></ul><ul><ul><li>Common Environment for developers to commit code changes – combine and validate work of entire team </li></ul></ul><ul><ul><li>Subset of data to test boundary conditions </li></ul></ul><ul><ul><li>Refresh/Update the subset frequently </li></ul></ul>
  7. 7. Cont… <ul><li>Staging </li></ul><ul><ul><li>To test your new features with the existing content </li></ul></ul><ul><ul><li>Simulate as much of production environment as possible [including software configurations] </li></ul></ul><ul><ul><li>Independent copy of production database to provide a true basis for QA testing </li></ul></ul><ul><li>Production </li></ul><ul><ul><li>The hosted website </li></ul></ul>
  8. 8. SVN <ul><li>Subversion (SVN) is a version control system, thus we can use commits and rollbacks. </li></ul><ul><li>Create a SVN repository at the development site </li></ul><ul><li>The development team takes a checkout of this repository </li></ul><ul><li>Staging can be at the same host as production [Pro: Identical server configuration for simulation] </li></ul><ul><li>Staging is also a a checkout of the repository. </li></ul>
  9. 9. Points to note <ul><li>Developers only commit only the changes that are tested on their local box. </li></ul><ul><li>Developers should work on different files . If they work on same file, then one commits, the other updates and then commit. </li></ul><ul><li>At no point is change made in staging or production environments. </li></ul>
  10. 10. Synchronize files <ul><li>Drupal is very much extensible. We can customize our website completely without changing anything in the core. Make all your modules and themes available in sites/all dir </li></ul><ul><li>Now, we need to commit/update this and this directory only. </li></ul><ul><li>Developers commit from their local boxes when the change works fine for them. </li></ul>
  11. 11. Cont… <ul><li>Integration site updates all these changes made by the developers and commits them if they work fine along with other changes made by the developers. </li></ul><ul><li>The staging site now looks for an update, checks the changes that are coming in, conforms with those made at the integration and then updates accordingly. </li></ul>
  12. 12. What with database? <ul><li>Files are synchronized. Yes!! Not the database. We need to bring the configuration of integration environment to the staging and test it with the production content. </li></ul><ul><li>Let us now understand MIGRAINE </li></ul>
  13. 13. MIGRAINE <ul><li>It is a python script that allows you to migrate content/configuration [acc to needs] </li></ul><ul><li>Drupal doesn’t have a strong concept of separation between configuration and content . The script attempts to provide one. </li></ul><ul><li>Freely available under the GNU General Public License </li></ul>
  14. 14. Requirements <ul><li>Python 2.4 or higher – to run the python script </li></ul><ul><li>Shell Access – to integrate with your database </li></ul><ul><li>Most important, understanding of your drupal database – Its you who have to categorize your tables. Don’t worry you don’t need to categorize everything. [Tables for additional modules and custom content types need to be classified] </li></ul>
  15. 15. Migraine unveiled <ul><li>So what’s the mystery behind the script? There are just 2 files that need to be manipulated </li></ul><ul><li>migraine.py – The python script that does all the work </li></ul><ul><li>migraine.ini – The configuration file given as an input to the python script </li></ul>
  16. 16. Test-Prod Analogy <ul><li>Test site is the one which has the configuration required to migrate to production </li></ul><ul><li>Prod site is the one which has the content </li></ul><ul><li>Dev to staging – dev becomes test and staging prod </li></ul><ul><li>Staging to prod – staging becomes test and prod is prod </li></ul>
  17. 17. Settings file <ul><li>Specifies the hosts ip/domain-name and the database name for test and prod servers </li></ul><ul><li>Also specifies the directories where the temporary database backups are stored </li></ul><ul><li>Optionally specifies the database prefixes – useful in case of multisite migration. </li></ul>
  18. 18. The python script <ul><li>Defines six categories </li></ul><ul><ul><li>Config – the configuration tables, e.g. variable </li></ul></ul><ul><ul><li>Content – the content tables, e.g. node, users </li></ul></ul><ul><ul><li>Temp – temporary tables, e.g. watchdog, sessions </li></ul></ul><ul><ul><li>Cache – cache tables, e.g. cache, cache_content </li></ul></ul><ul><ul><li>Sequence – the sequence table </li></ul></ul><ul><ul><li>Ignore – tables that you don’t want to include </li></ul></ul><ul><li>How to identify which tables go in which category?? Will look at it in the demonstration!! </li></ul>
  19. 19. Workflow <ul><li>Takes a backup of test and prod databases before proceeding so that the databases can be restored if things do not work. The backup contains 7 sql files , one for each category except content which has 2 sql files, one for the content schema and the other for content data. </li></ul><ul><li>Compares the content schema of both databases using a diff command. Tricky!! Does not make any change if the schemas do not match and show the errors. </li></ul>
  20. 20. Cont… <ul><li>Clears cache , dumps schema of temp tables. </li></ul><ul><li>If schemas match, replaces the configuration/content according to our need. </li></ul><ul><li>Also, updates the sequences table . [This is cool, as it updates rows according to the command we give] </li></ul>
  21. 21. Migraine commands <ul><li>dump-test, dump-prod – take backup </li></ul><ul><li>migrate-to-test – migrate configuration from test to prod </li></ul><ul><li>migrate-to-prod – migrate content from prod to test </li></ul><ul><li>restore-prod, restore-test – restore databases in case of any complication </li></ul>
  22. 22. Demonstration <ul><li>Lets go through the series of steps involved via a demonstration. </li></ul><ul><li>Few things – the demonstration assumes all the three dev-staging-production are on the same host, i.e. my localhost  However, this works well with all of them being on different sites. </li></ul>
  23. 23. Limitations <ul><li>Almost everything on earth comes with an exception, so does migraine. </li></ul><ul><li>Adding/removing/editing cck field definitions alters the content schema which is reported as an error and the script proceeds no further. </li></ul><ul><li>If there is an action that changes both the configuration and the content, there is no way to migrate it, e.g. Lightweight FeedAPI </li></ul>
  24. 24. Workarounds <ul><li>CCK field definitions – Only workaround make same changes in dev and production. </li></ul><ul><li>Lightweight FeedAPI – bring content on dev/staging before migrating and simultaneously take the action on both places so that they are associated with the same content. Will also require to migrate and content of production to development at the end to avoid future conflicts. </li></ul>
  25. 25. Conclusion <ul><li>Downtime is pretty low , optionally you can serve your site through another instance while migration is in the process. </li></ul><ul><li>Looking for a social networking site?? You will need to update features on regular basis to stay at par with your competitors!! Migraine provides answers to your ‘How … ??’ questions. </li></ul>
  26. 26. <ul><li>Thank You </li></ul><ul><li>For the Patience </li></ul><ul><li>Happy Drupaling !! </li></ul><ul><li>Do have a look at iLoveBolly.com  </li></ul>

×