Mahara UK 2011 - Building Mahara for Multiple Institutions


Published on

My presentation on our experience scaling Mahara for a large number of institutions (123)

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Mahara UK 2011 - Building Mahara for Multiple Institutions

  1. 1. Mahara UK 201122nd June 2011<br />
  2. 2. Andrew Nicols<br />Scaling Mahara<br />Multiple Institutions<br />
  3. 3. Scaling Mahara<br /><ul><li>Architecture Design
  4. 4. Cron Job Processing
  5. 5. Feature - VERP for E-mail and bounce handling
  6. 6. Feature - Isolated Institutions
  7. 7. Feature - Institution Themes
  8. 8. Management and MNet
  9. 9. Issues and Future Improvement
  10. 10. Q&A</li></li></ul><li>Architecture Design<br /><ul><li>Using an existing Moodle Architecture already hosting over 1,000 moodles and over 250,000 users
  11. 11. Existing architecture had grown organically over many years
  12. 12. Hardware has changed considerably over this time
  13. 13. Initially expecting about 120 institutions
  14. 14. Results of our pilot suggested that we should run one mahara with multiple institutions, rather than 120 separate institutions
  15. 15. Requirement for restrictions between these institutions</li></li></ul><li>Architecture Design<br />
  16. 16. Architecture Design<br /><ul><li>High Resilience (geographically isolated server farms)
  17. 17. Could afford to ‘lose’ 4 frontends, and a load balancer in an unplanned scenario with no service loss
  18. 18. Possible to migrate all databases between database servers for planned maintenance with no service loss
  19. 19. Possible to migrate data between file stores with short outage
  20. 20. Horizontally Scalable at frontends and cron (quick)
  21. 21. Possibility for fragmentation to provide further expansion (database and filestorage)
  22. 22. Separation of concerns between different components</li></li></ul><li>Architecture Design<br /><ul><li>Use existing architecture design and hardware
  23. 23. Replace aging hardware
  24. 24. New database servers
  25. 25. Add connection pooling for Mahara databases
  26. 26. Improve connection speed</li></li></ul><li>Database Design<br /><ul><li>Existing Moodles using MySQL
  27. 27. Decided to use Postgres for Mahara
  28. 28. Better community support
  29. 29. It cares a lot more about your data!
  30. 30. Postgres 9.0 supports streaming replication
  31. 31. Read-only requests can be performed on slave reducing overheads to master
  32. 32. Pooling engines like pgpool and pgbouncer reduce connection overhead too</li></li></ul><li>Cron Job Processing<br />Some problems with cron<br />Our solution<br />It steals processes and CPU time from end users if run through the web server<br />It has a potential for high e-mail generation load which increases system load<br />MTAs typically stop delivering mail under high load – poor perception of performance for end users<br />Runs on a separate physical system – reduce direct impact on end users<br />Calls php directly (no web server overhead)<br />Can be parallelised (especially from Mahara 1.4)<br />Ensures timely delivery of e-mail<br />
  33. 33. New Feature – E-mail and bounce handling - VERP<br />The Problem<br />Our solution<br />Many Schools don’t give real e-mail addresses to pupils<br />Mahara requires an e-mail address<br />Sending lots of e-mail to fake users (or users with small inboxes) which are returned<br />Increases load on servers<br />E-mailed messages are marked as read so users don’t see them<br />Implement Variable Envelope Return Path (VERP) for Mahara<br />Catches e-mail which has been returned to sender<br />Disables e-mail for those users who have a high ratio of sent mail to bounced mail<br />Messages left unread in Mahara<br />
  34. 34. New Feature – E-mail and bounce handling - VERP<br />How it works<br />Outbound e-mails’ return path changed to identify user:<br /><prefix>-<type><string with checksum>@<bouncedomain><br /><br />Contains:<br /><ul><li>E-mail Address
  35. 35. User ID
  36. 36. Site Specific Identifiers</li></li></ul><li>New Feature – E-mail and bounce handling - VERP<br />What it does now<br />What it could do<br />Disables sending of e-mail for users with invalid e-mail addresses<br />Allows messages to be left unread in Mahara inbox<br />Allow users to reply to forum posts and messages via e-mail<br />Something similar could also be used to allow users to write journal entries via e-mail (thanks to Mark’s keynote for this idea)<br />
  37. 37. New Feature – Isolated Institutions<br />See Ruslan’s Presentation for more information<br /><ul><li>Complete Separation of Institutions
  38. 38. Per-Institution settings
  39. 39. Institutions can enable/disable plugins for their users
  40. 40. Still allows integration of institutions
  41. 41. Opt-out
  42. 42. Selected integration</li></li></ul><li>New Feature – Per-Institution Themes<br /><ul><li>Institutions want their own theme
  43. 43. They want lots of themes
  44. 44. They don’t want to see other institution’s themes or allow others to use their theme</li></ul>$theme->institutions = array(<br />‘schoolname’,<br />‘school2name’,<br />);<br />
  45. 45. Management and MNet<br /><ul><li>Integrating one or two moodles with Mahara isn’t too bad
  46. 46. More than 10 is too time intensive and problematic
  47. 47. Extended our existing management architecture to automate this:
  48. 48. Initial MNet set up on both Moodle and Mahara
  49. 49. Log in Moodleadmins to Mahara, promote them to Institution admins on Mahara
  50. 50. Fix MNet certificate issues</li></li></ul><li>Issues and Future Improvements<br /><ul><li>Improve File Storage – Solaris driver issues with hardware
  51. 51. Sparc architecture is slow!
  52. 52. Lots of (slow) threads was a feature five years ago
  53. 53. Intel and AMD hardware has caught up
  54. 54. Additional Ideas:
  55. 55. Alternative web servers
  56. 56. Varnish
  57. 57. Apache Traffic Server
  58. 58. Offload SSL to hardware accelerator</li></li></ul><li>Q&A<br />