University Shares Tips for Migrating Thousands of Sites With One Install Profile

University Shares Tips for Migrating Thousands of Sites With One Install Profile Tyler StruykUniversity of Waterloo - Drupal Implementation Team Decemeber 5th , 2012

What am I presentingMy goal of this presentation is to connect with other schools and maybe, ata later date, collaborate together. Feel free to contact me if you have anyquestions. tyler.struyk@uwaterloo.ca

About MeUsing Drupal since 2007FreelancerUniversity of Waterloo(6 Months)

How many sites do we have and where are they 7 hosts: wms-aux.uwaterloo.ca 2 wms-dev.uwaterloo.ca 11 wms-profiles.uwaterloo.ca 6 wms-sand7.uwaterloo.ca 52 uwaterloo.ca 178 pilots.uwaterloo.ca 62 wcms-training.uwaterloo.ca 91 total 402

HistoryDreamweaver templatesDe-centralizedDrupal 6 InstallationsDifferent ServersNo centralizationSometimes take a month toChange the image in header

uWaterloo Content Management System(WCMS)One Install ProfileAll on the same serverUsed by all Faculties oncampusAll running the same features

HomepageLaunchedAugust 26th, 2012Trimmed downVersion of WCMS

Organization Structure• (8) Content Migration and Training Team• (5) Implementation Team• (1) Accessibility Guy• (1) System Administrator

Content Migration and Training TeamMade up of:• 2 Full time employees • 6 Co-ops • new coops every term – 4 monthsDuties• Migration Meetings• Q/A Duties• Training • 4 Migrating Content• Support • Migrating 1 site each at time• Communication • 2 Doing Q/A and drop in lab training

Process of Migration your site• Request in Request Tracker System (RT)• Schedule a review meeting • Go over requirements • Training • Determine if it’s a good candidate• Create a site on pilots.uwaterloo.ca• Assigned a co-op student help them out. • Co-op student might do everything or just a small portion• Set a launch date10 days prior to launch & Q/A – wave accessibility

Bad Candidates?Athletics• Sells Tickets• Sports and Teams• Sport Scores• Advertisement• Sponsorships• Custom Layout

TrainingMandatory• Content Maintainer Course• Site Manager Course• Webforms CourseAdditional Resources• Drop in Lab (All-day Twice a week)• Developer Drop in Lab (Once a month)• Training videos*• Advanced Webforms (regular expressions)• Writing for the Web• Writing Accessible Web Content*recorded Camtasia Studio 7 but recommend 8

Camtasia Studio 8 http://stevehoward999.wordpress.com/2012/06/24/techsmith-camtasia-studio-8/

CommunicationMethods• Mailing List• Web Resources• http://uwaterloo.ca/web-resources/Includes• Courses• New Features• Color Coding• Accessibility Guidelines• Term reminders (ie remove the list of coop students)

Implementation Team• One Install Profile for Waterloo Content Management System• Version 1.0• Released: November 1st, 2011

(Today)• Version 1.5.8• All Configuration featurized

Servers and WebsitesWebsites• Request Tracker (RT) for bug fixes bug tracking system already used campus wide• Agile Project management website for new features build in-house (more later…)Servers• Linux local development machines (test against standard profile and other)• Sandbox (test on trunk version of repo)• Profiles (test other profiles) ~ Agile PM website lives here• Dev (test on current tagged version of repo)• Pilots (staging)• Production (live)AdditionalCanary – testing production site

Test Servers Local Sandbox Dev Profile Pilots Production Don’t touch• Standard • Trunk • Current • Current One exception Profile • Rebuild Release Release Canary• Stuff that Nightly • Multiple will break Profiles • ie Kickstart • & CiviCRM

Typical Bug fixCreate a ticket in RTImplementation Team- working on it on either local dev, sandbox, dev/profilesAfter fixed:• if urgent, roll out new version• If not urgent, commit fix to trunk roll out next release

Committing to Trunk2-3 Step ProcessNew & Update Modules Update make fileCommit to trunk Features a simple commit it to trunkCreate new tagged version Modules – have to updateFeatureRoll out new feature and save locally*add style-sheet back in .info fileCommit it to trunkCreate a new tagged version 1.x.y- X is current version in main- Y is the incrementSo 1.24 becomes 1.24.1 -> 1.24.2 ….Attach RT Bug ticket to next release RT ticket

What is amake file?Grocery ListGrabs• Theme• Libraries• Modules

Creating a new release2-3 Step Process Update make fileFeatures Update every new moduleRetagged all of new features and featureAll 1.x.y retagged 1.[x+1].0So 1.24.2 becomes 1.25*Long process when rolling out a critical bug, or security update

Pushing out a New ReleaseLocal Dev Profile Pilots Production • Push -> • We do create beta & release candidates

Creating a new siteLocal Other than LocalInstall profile Login into special server and run script- .info- .install Does everything- .profile - Database- .make - Files system- rebuild.sh script - Specify Install profile - Specify which serverOne for trunkOne for every release Every run by DRUSH

Agile ProjectManagementWebsite1. Make feature request2. Become user stories3. Sprints on User stories• Stakeholders• User Priority Ranking System• Follow• Email communication on updates Dashboard

User Priority Modulehttp://drupal.org/project/user_priority_rankingFlag Content (I want to be a stakeholder of X)Using draggableviews rank your flagged contentFirst item get 10 point, Second 9 point, Third 8 points….

Why we use AgileProject Management• Only way to keep your sanity• Prevents your project from become a “Maintenance only”• Able to track new features http://www.flickr.com/photos/attefable/8184419537/

System Administration

Apache Bits: defining sitesMethod 1: Symbolic links in Drupal Root directoryPros: Easy to do - scriptable.Cons: uwaterloo.ca/site1/site2/site3/site1/ is valid. Makes for a very messy root directory. O(1) for lookups.Method 2: Defines and redirects in apache configPros: Also easy to do. uwaterloo/site1/site2 is not valid.Cons: O(n) for lookups.

Apaches for Dummies Explanation of Method#2Include /etc/apache/hosts/vhosts/*• environment.conf• physics.conf• applied-health-science.confWhen you create a new website you just add the a new configuration fileIf path is uwaterloo.ca/about- goto https://uwaterloo.ca/- /var/www/drupal/sites/ca.waterlooIf path is uwaterloo.ca/environment/about- goto https://uwaterloo.ca/environment- /var/www/drupal/sites/ca.environmentIf path is uwaterloo.ca/environment/ecology/about- goto https://uwaterloo.ca/environment/ecology- /var/www/drupal/sites/ca.ecology

Pound/Varnish/Apache•There is content that isnt in Drupal – stored on a Network Applianceslice.•We have a Google Search Appliance that is proxied intouwaterloo.ca/search•How to tell Apache were running on a secure channel post Varnish &Pound.•IPv6 support.

What is Pound?

HTTPS and/or VarnishVarnish and HTTPS dont play well together (okay, at all)• use Pound to terminate HTTPS.• SSL information gets passed though special HTTP headers inserted by Pound.Special rules in Varnish for private files.Special rules for Varnish to not strip SSSIDs.

Legacy – The dreaded Dreamweaver templates• 800 legacy sites• Getting templates from• uWaterloo.ca/css• uWaterloo.ca/images

Legacy HostsProblem: One weird App that you cannot migratePeople want to have the old links continue to work.There is no predefined relationship between old and new URLs.site.uwaterloo.ca/page1  uwaterloo.ca/site/page1Site.uwaterloo.ca/NaM3badly.html  uwaterloo.ca/sites/named-good

Refactoring settings.phpStandard settings.php doesnt scale well to hundreds of sites when youhave to make changes.We have minimal – site specific – data in settings.php. Everything elseis built from that.Can change to slave database server by updating one file per server.

Settings.phpExample

Whats coming• Research Proper High Availabilty for mySQL and Net App File Storeage.• Research Possible use of Nginx to replace Pound/Varnish/Apache.• Real support for https://uwaterloo.ca/ ALL THE TIME (with proper exceptions for legacy URLS).

SearchGoogle Search Appliance (GSA) vs Apache Solr

Additional InformationNo e-commerce integrationSpecial server on campus behind strict firewall settings

Questions & Further Discussion My goal of this presentation was to connect with other schools and maybe collaborate together at a later date. Feel free to contact me if you have any questions.My Contact InformationTwitter: @istrykerLinkedIn: http://www.linkedin.com/in/tylerstruykUniversity of Waterloo | http://uwaterloo.caEmail: tyler.struyk@uwaterloo.caStryker Enterprise | http://www.strykerenterprise.comEmail: tyler.struyk@strykerenterprise.com

University Shares Tips for Migrating Thousands of Sites With One Install Profile

by Acquia on Dec 06, 2012

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 72

Statistics

University Shares Tips for Migrating Thousands of Sites With One Install Profile Presentation Transcript

http://www.acquia.com	69
http://dev.local.acquia.com	3