University Shares Tips for  Migrating Thousands      of Sites With    One Install Profile                     Tyler Struyk...
What am I presentingMy goal of this presentation is to connect with other schools and maybe, ata later date, collaborate t...
About MeUsing Drupal since 2007FreelancerUniversity of Waterloo(6 Months)
How many sites do we have   and where are they      7 hosts:      wms-aux.uwaterloo.ca           2      wms-dev.uwaterloo....
HistoryDreamweaver templatesDe-centralizedDrupal 6 InstallationsDifferent ServersNo centralizationSometimes take a month t...
uWaterloo Content Management System(WCMS)One Install ProfileAll on the same serverUsed by all Faculties oncampusAll runnin...
HomepageLaunchedAugust 26th, 2012Trimmed downVersion of WCMS
Organization Structure•   (8) Content Migration and Training Team•   (5) Implementation Team•   (1) Accessibility Guy•   (...
Content Migration and Training TeamMade up of:• 2 Full time employees   • 6 Co-ops                          • new coops ev...
Process of Migration your site•   Request in Request Tracker System (RT)•   Schedule a review meeting     • Go over requir...
Bad Candidates?Athletics•   Sells Tickets•   Sports and Teams•   Sport Scores•   Advertisement•   Sponsorships•   Custom L...
TrainingMandatory• Content Maintainer Course• Site Manager Course• Webforms CourseAdditional Resources• Drop in Lab (All-d...
Camtasia Studio 8          http://stevehoward999.wordpress.com/2012/06/24/techsmith-camtasia-studio-8/
CommunicationMethods• Mailing List• Web Resources•   http://uwaterloo.ca/web-resources/Includes• Courses• New Features• Co...
Implementation Team• One Install Profile for  Waterloo  Content  Management  System• Version 1.0• Released:  November 1st,...
(Today)• Version 1.5.8• All Configuration  featurized
Servers and WebsitesWebsites• Request Tracker (RT) for bug fixes        bug tracking system already used campus wide• Agil...
Test Servers    Local       Sandbox       Dev        Profile      Pilots      Production                                  ...
Typical Bug fixCreate a ticket in RTImplementation Team- working on it on either local dev,   sandbox, dev/profilesAfter f...
Committing to Trunk2-3 Step ProcessNew & Update Modules                         Update make fileCommit to trunk           ...
What is amake file?Grocery ListGrabs• Theme• Libraries• Modules
Creating a new release2-3 Step Process                                     Update make fileFeatures                       ...
Pushing out a New ReleaseLocal      Dev       Profile             Pilots    Production        • Push ->        • We do cre...
Creating a new siteLocal                   Other than LocalInstall profile         Login into special server and run scrip...
Agile ProjectManagementWebsite1. Make feature request2. Become user stories3. Sprints on User stories• Stakeholders• User ...
User Priority Modulehttp://drupal.org/project/user_priority_rankingFlag Content (I want to be a stakeholder of X)Using dra...
Why we use AgileProject Management• Only way to keep your sanity• Prevents your project from become a    “Maintenance only...
System Administration
Apache Bits: defining sitesMethod 1: Symbolic links in Drupal Root directoryPros:   Easy to do - scriptable.Cons:   uwater...
Apaches for Dummies Explanation of Method#2Include /etc/apache/hosts/vhosts/*• environment.conf• physics.conf• applied-hea...
Pound/Varnish/Apache•There is content that isnt in Drupal – stored on a Network Applianceslice.•We have a Google Search Ap...
What is Pound?
HTTPS and/or VarnishVarnish and HTTPS dont play well together (okay, at all)• use Pound to terminate HTTPS.• SSL informati...
Legacy – The dreaded Dreamweaver templates•   800 legacy sites•   Getting templates from•   uWaterloo.ca/css•   uWaterloo....
Legacy HostsProblem: One weird App that you cannot migratePeople want to have the old links continue to work.There is no p...
Refactoring settings.phpStandard settings.php doesnt scale well to hundreds of sites when youhave to make changes.We have ...
Settings.phpExample
Whats coming• Research Proper High Availabilty for mySQL and Net App File  Storeage.• Research Possible use of Nginx to re...
SearchGoogle Search Appliance (GSA) vs Apache Solr
Additional InformationNo e-commerce integrationSpecial server on campus behind strict firewall settings
Questions & Further Discussion             My goal of this presentation was to connect with            other schools and m...
University Shares Tips for Migrating Thousands of Sites With One Install Profile
Upcoming SlideShare
Loading in...5
×

University Shares Tips for Migrating Thousands of Sites With One Install Profile

894

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
894
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Push and Pull
  • ADOAWCAG
  • Speak about why uWaterloo is big into coops/interns
  • Request in RTSchedule a review meeting with 2 full-time staffGo over any new feature requestTraining. May book them right awayIf it’s a good candidate for migrationAssigned a co-op student hep them out.Co-op student might do everything or just a small portionCreate a site on pilots.uwaterloo.caSet a launch date10 days of testingWe specify 10 days to encourge them to take time out of the busy schedule to test the siteThey have our full attentionQ/A – wave accessiblity
  • Fortunate to have a drop-in lab just for us Drop in Lab is for extra support, or for those tech savy individuals that don’t need to take the site manager coarse, or were playing on their Blackberry too muchFirst time we get their coops to attend training
  • 1.0 Basic Features, News Event1.5.8 Same Template, Photogallery and Slideshows, Webforms, Social Media Widgets, Live streaming
  • 1.0 Basic Features, News Event1.5.8 Same Template, Photogallery and Slideshows, Webforms, Social Media Widgets, Live streaming
  • LocalDev has whatever on it- Might or might not use aegirSandbox has lastest Trunk Everyone has there own siteWhos everyone, well a lot Developers across campus have their own accountRebuild every nightJenkins, rebuild every 5 minutesProfiles and DevSame profileLatest FDSU versionAgile websitePilotsUpcoming websiteSites with contentProductionCaching
  • Local dev – is it a core bug, module bug, what?Sandbox – hey have we fixed this by another commit, will it work with the next realesDev - trying to make it would with the current release
  • 2 Things you are doingTheres committing to trunkThenstheres creating a tag version
  • Local dev,
  • University Shares Tips for Migrating Thousands of Sites With One Install Profile

    1. 1. University Shares Tips for Migrating Thousands of Sites With One Install Profile Tyler StruykUniversity of Waterloo - Drupal Implementation Team Decemeber 5th , 2012
    2. 2. What am I presentingMy goal of this presentation is to connect with other schools and maybe, ata later date, collaborate together. Feel free to contact me if you have anyquestions. tyler.struyk@uwaterloo.ca
    3. 3. About MeUsing Drupal since 2007FreelancerUniversity of Waterloo(6 Months)
    4. 4. How many sites do we have and where are they 7 hosts: wms-aux.uwaterloo.ca 2 wms-dev.uwaterloo.ca 11 wms-profiles.uwaterloo.ca 6 wms-sand7.uwaterloo.ca 52 uwaterloo.ca 178 pilots.uwaterloo.ca 62 wcms-training.uwaterloo.ca 91 total 402
    5. 5. HistoryDreamweaver templatesDe-centralizedDrupal 6 InstallationsDifferent ServersNo centralizationSometimes take a month toChange the image in header
    6. 6. uWaterloo Content Management System(WCMS)One Install ProfileAll on the same serverUsed by all Faculties oncampusAll running the same features
    7. 7. HomepageLaunchedAugust 26th, 2012Trimmed downVersion of WCMS
    8. 8. Organization Structure• (8) Content Migration and Training Team• (5) Implementation Team• (1) Accessibility Guy• (1) System Administrator
    9. 9. Content Migration and Training TeamMade up of:• 2 Full time employees • 6 Co-ops • new coops every term – 4 monthsDuties• Migration Meetings• Q/A Duties• Training • 4 Migrating Content• Support • Migrating 1 site each at time• Communication • 2 Doing Q/A and drop in lab training
    10. 10. Process of Migration your site• Request in Request Tracker System (RT)• Schedule a review meeting • Go over requirements • Training • Determine if it’s a good candidate• Create a site on pilots.uwaterloo.ca• Assigned a co-op student help them out. • Co-op student might do everything or just a small portion• Set a launch date10 days prior to launch & Q/A – wave accessibility
    11. 11. Bad Candidates?Athletics• Sells Tickets• Sports and Teams• Sport Scores• Advertisement• Sponsorships• Custom Layout
    12. 12. TrainingMandatory• Content Maintainer Course• Site Manager Course• Webforms CourseAdditional Resources• Drop in Lab (All-day Twice a week)• Developer Drop in Lab (Once a month)• Training videos*• Advanced Webforms (regular expressions)• Writing for the Web• Writing Accessible Web Content*recorded Camtasia Studio 7 but recommend 8
    13. 13. Camtasia Studio 8 http://stevehoward999.wordpress.com/2012/06/24/techsmith-camtasia-studio-8/
    14. 14. CommunicationMethods• Mailing List• Web Resources• http://uwaterloo.ca/web-resources/Includes• Courses• New Features• Color Coding• Accessibility Guidelines• Term reminders (ie remove the list of coop students)
    15. 15. Implementation Team• One Install Profile for Waterloo Content Management System• Version 1.0• Released: November 1st, 2011
    16. 16. (Today)• Version 1.5.8• All Configuration featurized
    17. 17. Servers and WebsitesWebsites• Request Tracker (RT) for bug fixes bug tracking system already used campus wide• Agile Project management website for new features build in-house (more later…)Servers• Linux local development machines (test against standard profile and other)• Sandbox (test on trunk version of repo)• Profiles (test other profiles) ~ Agile PM website lives here• Dev (test on current tagged version of repo)• Pilots (staging)• Production (live)AdditionalCanary – testing production site
    18. 18. Test Servers Local Sandbox Dev Profile Pilots Production Don’t touch• Standard • Trunk • Current • Current One exception Profile • Rebuild Release Release Canary• Stuff that Nightly • Multiple will break Profiles • ie Kickstart • & CiviCRM
    19. 19. Typical Bug fixCreate a ticket in RTImplementation Team- working on it on either local dev, sandbox, dev/profilesAfter fixed:• if urgent, roll out new version• If not urgent, commit fix to trunk roll out next release
    20. 20. Committing to Trunk2-3 Step ProcessNew & Update Modules Update make fileCommit to trunk Features a simple commit it to trunkCreate new tagged version Modules – have to updateFeatureRoll out new feature and save locally*add style-sheet back in .info fileCommit it to trunkCreate a new tagged version 1.x.y- X is current version in main- Y is the incrementSo 1.24 becomes 1.24.1 -> 1.24.2 ….Attach RT Bug ticket to next release RT ticket
    21. 21. What is amake file?Grocery ListGrabs• Theme• Libraries• Modules
    22. 22. Creating a new release2-3 Step Process Update make fileFeatures Update every new moduleRetagged all of new features and featureAll 1.x.y retagged 1.[x+1].0So 1.24.2 becomes 1.25*Long process when rolling out a critical bug, or security update
    23. 23. Pushing out a New ReleaseLocal Dev Profile Pilots Production • Push -> • We do create beta & release candidates
    24. 24. Creating a new siteLocal Other than LocalInstall profile Login into special server and run script- .info- .install Does everything- .profile - Database- .make - Files system- rebuild.sh script - Specify Install profile - Specify which serverOne for trunkOne for every release Every run by DRUSH
    25. 25. Agile ProjectManagementWebsite1. Make feature request2. Become user stories3. Sprints on User stories• Stakeholders• User Priority Ranking System• Follow• Email communication on updates Dashboard
    26. 26. User Priority Modulehttp://drupal.org/project/user_priority_rankingFlag Content (I want to be a stakeholder of X)Using draggableviews rank your flagged contentFirst item get 10 point, Second 9 point, Third 8 points….
    27. 27. Why we use AgileProject Management• Only way to keep your sanity• Prevents your project from become a “Maintenance only”• Able to track new features http://www.flickr.com/photos/attefable/8184419537/
    28. 28. System Administration
    29. 29. Apache Bits: defining sitesMethod 1: Symbolic links in Drupal Root directoryPros: Easy to do - scriptable.Cons: uwaterloo.ca/site1/site2/site3/site1/ is valid. Makes for a very messy root directory. O(1) for lookups.Method 2: Defines and redirects in apache configPros: Also easy to do. uwaterloo/site1/site2 is not valid.Cons: O(n) for lookups.
    30. 30. Apaches for Dummies Explanation of Method#2Include /etc/apache/hosts/vhosts/*• environment.conf• physics.conf• applied-health-science.confWhen you create a new website you just add the a new configuration fileIf path is uwaterloo.ca/about- goto https://uwaterloo.ca/- /var/www/drupal/sites/ca.waterlooIf path is uwaterloo.ca/environment/about- goto https://uwaterloo.ca/environment- /var/www/drupal/sites/ca.environmentIf path is uwaterloo.ca/environment/ecology/about- goto https://uwaterloo.ca/environment/ecology- /var/www/drupal/sites/ca.ecology
    31. 31. Pound/Varnish/Apache•There is content that isnt in Drupal – stored on a Network Applianceslice.•We have a Google Search Appliance that is proxied intouwaterloo.ca/search•How to tell Apache were running on a secure channel post Varnish &Pound.•IPv6 support.
    32. 32. What is Pound?
    33. 33. HTTPS and/or VarnishVarnish and HTTPS dont play well together (okay, at all)• use Pound to terminate HTTPS.• SSL information gets passed though special HTTP headers inserted by Pound.Special rules in Varnish for private files.Special rules for Varnish to not strip SSSIDs.
    34. 34. Legacy – The dreaded Dreamweaver templates• 800 legacy sites• Getting templates from• uWaterloo.ca/css• uWaterloo.ca/images
    35. 35. Legacy HostsProblem: One weird App that you cannot migratePeople want to have the old links continue to work.There is no predefined relationship between old and new URLs.site.uwaterloo.ca/page1  uwaterloo.ca/site/page1Site.uwaterloo.ca/NaM3badly.html  uwaterloo.ca/sites/named-good
    36. 36. Refactoring settings.phpStandard settings.php doesnt scale well to hundreds of sites when youhave to make changes.We have minimal – site specific – data in settings.php. Everything elseis built from that.Can change to slave database server by updating one file per server.
    37. 37. Settings.phpExample
    38. 38. Whats coming• Research Proper High Availabilty for mySQL and Net App File Storeage.• Research Possible use of Nginx to replace Pound/Varnish/Apache.• Real support for https://uwaterloo.ca/ ALL THE TIME (with proper exceptions for legacy URLS).
    39. 39. SearchGoogle Search Appliance (GSA) vs Apache Solr
    40. 40. Additional InformationNo e-commerce integrationSpecial server on campus behind strict firewall settings
    41. 41. Questions & Further Discussion My goal of this presentation was to connect with other schools and maybe collaborate together at a later date. Feel free to contact me if you have any questions.My Contact InformationTwitter: @istrykerLinkedIn: http://www.linkedin.com/in/tylerstruykUniversity of Waterloo | http://uwaterloo.caEmail: tyler.struyk@uwaterloo.caStryker Enterprise | http://www.strykerenterprise.comEmail: tyler.struyk@strykerenterprise.com
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×