Slides from a 4hr workshop at SXSW 2015 on Large Drupal Projects and how to manage them, and the special technical challenges.

1. Large Drupal Site Builds
SXSW Interactive, 2015

2. Who the Heck are We?
Rob Ristroph
● Technical Architect at Acquia, a digital
experience company based in Boston,
MA and with remote team members
worldwide.
● Based in Austin, TX!
● ATX Hacker Space Member
● Drupal/Open Source contributor

3. Who the Heck are We?
Jenn Sramek
● Sr. Engagement Manager at Acquia,
based in the San Francisco Bay Area
● Manage large-scale Drupal projects
for organizations and companies that
are often new to Drupal
● Formerly Dir. of Operations at
CivicActions
● Board President for the Haight
Ashbury Food Program

4. Who the Heck are We?
Alex Urevick-
Ackelsberg
● Co-Founder/CEO of Zivtech
● Extensive large Drupal project experience
● Wears many hats
Nick Lewis
● Site Architect at Zivtech
● Extensive Drupal site architecture
experience
● Sock Puppet Master

5. Who the Heck are You?
Drupal experience?
Large-scale web?
Open Source?
Have an active/current large-scale project?
Planning one?
Waterfall/Agile?
Local/from other states/countries?
Anything else interesting?

6. Logistics
Connectivity
● Power/Wireless
Biology
● Restrooms
● Food/Drink
Overall Schedule
● 9:30 am - 1:30 pm
Shared Notes and Resources https://pad.lullabot.
com/p/Large_Drupal_Sites_Workshop_SXSW_2015

7. Topic Overview
● Project Management - what’s different about big projects ?
Technical:
● Development workflow
● Custom APIs and the ecosystem of API consumers
● Scaling and performance at the enterprise level - configuring cache, finding bottlenecks
● Large-scale data migration
● Monitoring
Social:
● Open Source and corporate culture
● Training
● Larger development team dynamics - culture of quality
● Multiple teams and vendors
Tell us what to focus on

8. Defining “Large” Projects
SXSW Interactive, 2015

9. What is “Large”?
Customer has a lot of money ?
Site has a lot of traffic or data ?
Project build runs for a long time ?
Whatever is pushing past your comfort / experience zone in
some way.

10. What is “Large”
● More than one website
○ Platform and distribution builds
○ Another team uses your platform or distribution
● Multiple stakeholders
○ Company vs. Divisions or Franchisees
○ Technical vs. Marketing departments
● APIs and System interactions
○ mail delivery
○ SSO authentication, including roles and other metadata
○ Web metrics
● Workflows, editorial processes
● Significant migration of data

11. What is “Large”
● Content Staging
○ Workflow on steroids / between servers
○ Inflexible legal requirements
● Solutions
○ Content staging server and clone
○ Site Preview System (SPS)
○ Deploy suite of modules
● Challenging, Interesting and Fun
○ Inter-disciplinary development
○ Complex feature development
○ Cutting-edge interaction design

12. What is “Large”
The most common single feature of a “large” project is the
interaction with more team members and more
stakeholders.

13. Working with Large Teams
Working with large teams is the most likely source of
project failure...
● If you are new to big projects, interacting with and managing big teams is
probably the most unfamiliar area and the greatest area of risk
● Even large organizations can fail at large project implementation
repeatedly
● All the same technical issues that show up in large projects can show up in
smaller projects, but it is harder to maintain clarity, quality code,
expectations, and scope when dealing with large teams

14. Working with Large Teams
● You start with a client / customer
● Add a design / strategy vendor
● Add another developer team on the customer side
● Add other outside developer team(s), with sub-contractor relationship
● Add multiple other vendors, with no sub-contractor relationships
● Separate projects with out-of-sync timelines
All still working on the same project with ultimately the same goal for success...

15. Working with Large Teams
● Timeline differences
● Scope differences or gaps
● Resource gaps
● Process differences
● Deliverables differences or gaps that may be significant
● Coordinating vacations/time off
● Internal team mentorship, developer “ladders,” or other process for fully
on-boarding new team members (including client)

16. Working with Large Teams
Multiple parties make “DevOps” essential
Practices that just provide efficiency at the scale of one
team become absolutely necessary:
● Pull request workflows
● Peer code review
● Standardized development environments
● Automated testing
Tech demo ?

17. Working with Large Teams
The most rewarding parts of big projects
● Personal growth and learning
● Exposure to other teams is like having several jobs without switching
bosses
● Part of a larger community

18. Getting Large Projects
Pick the projects, don’t let them pick you
● Initiate contact with particular clients and projects that interest you
○ Talk with your whole team when going after something large
○ Long term projects should be something people find fulfilling
● Winning by the lowest bid is a red flag
● Spend much more time vetting/ do formal Discovery
● Be picky; build padding into quotes

19. Picking a Vendor
● Is the vendor interested enough to be committed to a long project ?
● Unrealistically low bids are a red flag
● Budget for discovery, and pay for it
● If the vendor has not done projects of your size before:
○ Allocate more to planning
○ Smallest MVP possible
○ A partnered bid may mitigate some risk
○ If you fall behind, cut scope instead of adding resources

20. Overall Process and Project Management
SXSW Interactive, 2015

21. Project (Program) Management
Overall...
● The basics become essential
○ JIRA or another tool
○ Discipline in using that tool
● Communications and Report Generation is key, and may involve “X-facing veation
● Reporting is part of the effort of a Project or Program Manager
● More project managers on different teams, coordination
● Often multiple JIRA instances and trackers in play
● On big projects, project managers can really shine and make an obvious difference,
and technical people who hate it are more likely to be able to avoid it (not necessarily
a good idea ?)

22. Discovery for Large Scale Drupal
Curiosity is a Skill...

23. Discovery for Large Scale Drupal
Discovery is really a very long list of questions to answer before you
start development:
- Create shared understanding of the work to be done and its
priority, and start technical requirements docs
- Align everyone on the process that will be used for development,
quality assurance, signoff/acceptance
- Surface any risks or potential issues that will require mitigation
planning
- Document any schedule or resource considerations
- “Right-size” the effort of development

24. Discovery for Large Scale Drupal
Technical Requirements:
- Content types and fields
- User roles/permissions
- Content and site management
workflows
- Integrated service requirements (CDN,
analytics, single-sign-on, Translation
Management Services, payment
processing, feeds,)
- Migration details (size and complexity of
database, content type/field mapping,
consistency of data)
Non-Technical Requirements:
- Resources including third-parties, dev team(s),
schedules, dependencies, internal technical
resources, training needs
- Internal timelines and schedule constraints
(vacations, major internal events, conferences,
product releases)
- Security/Governance workflow and approval
process (incl. clearance for those with data
access and secure systems for data
transmission and storage)
- Access to associated systems and tools

25. Planning for Large Scale Drupal
- Timeline
- Dependencies
- Risks and Mitigation Planning
- Resources (considering attrition of internal and external teams)
- Technical/Architectural planning (including POCs/rapid prototypes)
- Code management
- Security management
- Plan for addressing technical debt
- Governance (assignment and acceptance)
- Maintenance plan, development cycles, post-launch
- Pre-launch/Post-launch planning

26. Planning for Large Scale Drupal
● Involves LOTS of technical-specific planning (by the architect). A site
architect may do as much technical planning as the PM does project
planning (code management, maintenance, release planning, etc).
● Documenting risks protects everyone, and the project
● Interim maintenance and remediation
● Documentation needs may differ for large projects

27. Now that all the pieces are in
place...
● Waterfall?
OR
● Agile?

28. Working with the “MVP”

29. Tools for Large Scale Development
SXSW Interactive, 2015

30. Tools for Large Scale Development
Plan for tools to scale - know what happens to them when
the project is over
● Ticketing system
● Document repository
● Code Management tools
● UX/Design Management tools
● Communications Channel(s)
● Training tools

31. Tools for Large Scale Development
● JIRA/Rally/others - plan for tools at scale and know what happens to them
when the project is over.
● Code management tools - Github, BitBucket (more integrated with JIRA
for one PR per JIRA ticket)
○ Process more than product
○ Onboarding dev documentation.
● UX/Design management tools - differ by firm, many use BaseCamp
● Living document for coding/best practices/standard practices

32. Development
● Managing multi-team interactions
● Deployment tools and procedures will take up a lot of time
○ Delays between code and UAT
● Deployment and DevOps can have huge (%30 ?) impacts on efficiency
● Re-work is the biggest cost

33. Scalable Code Management
SXSW Interactive, 2015

34. Iterative Development
● Developers cannot JUST develop, have to document how they developed.
There is a much larger team depending on clarity.
○ Can effect technology choices
● Iterative can differ at the enterprise level
● There can be more than one effective Agile team
● Include maintenance rounds in iteration
● Plan for longer lags between dev and signoff
● Get sprint signoff outside of JIRA or other ticket system

35. Performance Planning
Plan for addressing performance issues early
● Collect realistic requirements with context
● Benchmark legacy site for reference
● Test early
● How do you review contributed modules/code for performance?
● Not intuition - always check
● What tools to use to benchmark?
○ Make sure they match your expected traffic
● How do you populate the site with enough content to test?
○ If benchmarking can be automated, just repeat as project proceeds

36. Scaling and Performance
● Scaling often matters more than Performance
● Editorial interactions, cache updating are more important
● Collect performance, scaling, time-to-refresh requirements just like any
other, and test for them
● Optimizing Drupal for performance is a specific skill that requires Drupal
experience
○ But if you can automatically test it can just like any other debuggin
Hard to designate the performance issues to one person or team; again
interactions between teams are essential

37. Scaling and Performance
● Scaling may mean more small sites not one big site
○ See multisite debates
● Enforcing Code quality and standards
○ Peer Reviews
○ Views Cache Bully and similar
○ Automated testing
○ Automated code sniffer
Automated testing is another thing you may not get to work on except on large
projects.

38. Deployment
For each deployment of code, it will be important to have a consistent and
documented process:
Preparing for Deployment: What do you need to do to get ready? Includes
reviewing all merged pull requests, updating release notes, and preparing and
validating any needed deployment steps.
QA Steps: What do you need to do to prepare for QA?
Includes tagging the release, cloning UAT database to QA environment, smoke
and unit tests.
Acceptance: What do you need to do to gain client acceptance?
Includes preparing a demo, regression testing, and preparing the client for UAT
testing.

39. Large-scale data migration and content
management
SXSW Interactive, 2015

40. Data Migration
● Review and get agreement on the process
● Consider data security
● Clean the data
● Do not “scope” it until you have mapped it
● Make sure you know what is all is!
● Dump and Map it! (content type by content type, field
by field)

41. Data Migration
● Re-Scoping migration after mapping
● Incremental migration
● Differential migration
● Special considerations for beta sites (where there may
be two versions of a “live” site)
● Plan for deprecated systems
● Plan for wrapping up with “manual” migration
(copy/paste or rewrite/edit)

42. Migrate Module
● A complex migration can be a large portion of
development effort
● Plan for “continuous migration”
○ Run a scheduled migration until “canonical DB
switchover”
○ Iterate like any other development
○ Make sure client can devote resources to checking
content throughout the process

43. Content Staging
SXSW Interactive, 2015

44. Content Staging
Often discussed as part of Discovery, because many enterprise companies have
structures that utilize a content staging environment.
With Drupal, this is most often not needed or can be fairly complex. Content
staging, if needed, may be one of the “interesting problems to solve” that
inspires your team to take on the project.

45. Content Staging
● Draft / published
● Workflows
● Workflows with SPS - complexities multiply
● Staging server, simple DB copy - limiting
● Staging server, with workflow, and deploy module
If editorial staff get involved late, this is often a late-in-project scope explosion.

46. Training
SXSW Interactive, 2015

47. Site Owner / Builder Training
● Talk about it during Discovery!!!
● Training needs to be built into the schedule
● Project Managers can become de-facto trainers
● Include teaching clients to do UAT (as requisite for end-
of-project training)
● Leverage available resources (Drupalize.me, Lynda.com,
Drupal.org, BuildAModule and others) for training
● Plan to train the trainer

48. Training Model
● Start with defined “needs”
● Decide on the format of training (in-person, training
materials on separate site or tool, formal training)
● Quantify trainees and role-specific tasks required of
them in order to plan and estimate
● Plan to train at least some roles incrementally (often
some training will be needed for UAT )
● Leverage QA and UAT testing steps for training
materials

49. Handoff and Transitions
SXSW Interactive, 2015

50. End Goals and Transition
Big projects will involve transitioning in some way:
● To other vendors
● To internal teams
● From active development to maintenance mode
● Re-acquiring the project later (in another phase)
Documentation and consistent development practices are essential

51. End Goals and Transition
● What happens next?
● Discuss at the beginning of the project what the end of it
looks like.
● Maintainability - avoid risk to support obligations
● Technical debt - document it
● Who will manage code after the “final” launch?
● Process and tools changes?

52. SXSW Interactive, 2015
Giving Back to the Open Source Community

53. Takin’ it to the Community
Contribution makes Drupal go!
● Discuss the value of and plans for contribution during
Discovery (even if you do not plan to contribute code
right away)
● Making “contributable code” a design principle
○ Reduce future risk / support costs
● Celebrate contributions (and give time to devs to do it as
part of the project schedule and effort)

54. Takin’ it to the Community
How to Contribute back to the community:
● Report Bugs!
● Contribute a patch if you fix the issue
● Design for Contribution
● Document how the module or code works
● Employ best practices
● Get a peer review

55. Takin’ it to the Community
Some examples of sponsored modules:
- https://www.drupal.org/project/media
- https://www.drupal.org/project/i18n
- https://www.drupal.org/project/commerce

56. Takin’ it to the Community
Now Available! Attributions for contributions on Drupal.
org! - https://www.drupal.org/node/2451283

57. SXSW Interactive, 2015
Questions for Us? Or for Each Other?

58. SXSW Interactive, 2015
Resources

59. Resources
Shared Resources and Notes document
● https://pad.lullabot.
com/p/Large_Drupal_Sites_Workshop_SXSW_2015

60. Contact Us
Rob Ristroph - @robgr - robert.ristroph@acquia.com
Jenn Sramek - @ideaseed - jenn.sramek@acquia.com

61. (we would love your)
feedback, please...
sxsw.feedogo.com/fdbk.do?sid=IAP42024

64. Questions Answered
1. What issues should receive special attention early in life time of a large project, that need to be set up in the first
few weeks to establish best practices and culture through the longer life of a big project?
2. How do you keep a larger team of developers working in close collaboration, especially as bigger projects are
more likely to have developers spread across time-zones and even different companies through multiple vendors
and sub-contractors?
3. How can I assure that project and team can handle people leaving and joining the team on a regular basis, given
that large Drupal projects can survive long enough to experience developer turnover, and often require
specialists to join the team for a shorter phase (such as a migration expert at the start)?
4. What special efforts might be required to make sure that enhancements to Drupal and other open source
projects can get outside of the organization and back to the parent project?
5. In terms of the "ecosystem" of a large enterprise drupal project, what special needs will surprise someone who
has only worked on smaller projects -- APIs and Documentation that need long term maintenance, automated
testing, large numbers of stakeholders and resulting approval bureaucracy?
6. What things are related to doing large projects that someone who had only done small/medium ones would not
know? Partner JIRA vs. Own JIRA - Tools differences? other diffs?

65. Iterative Development
● On large projects, devs spend a lot of time figuring out where a ticket is and whether or not it can
be moved forward. Another portion is if you do not document technical debt as it accumulates,
you cannot do maintenance sprints very efficiently, nor can you argue for a maintenance sprint
effectively. People can be hesitant to keeping a list of technical debt. By documenting it, you
protect yourself. Make it part of your process to document when you make technical decisions
that create technical debt or find technical debt on code review, and get signoff as part of sprint
planning.
● Estimate work only after the details of the story have been approved (so there is not scope creep
on each item as scope is added that client expects the same estimate still applies).