My presentation on our experience scaling Mahara for a large number of institutions (123)

1. Mahara UK 201122nd June 2011

2. Andrew Nicols Scaling Mahara Multiple Institutions

4. Cron Job Processing

5. Feature - VERP for E-mail and bounce handling

6. Feature - Isolated Institutions

7. Feature - Institution Themes

8. Management and MNet

9. Issues and Future Improvement

11. Existing architecture had grown organically over many years

12. Hardware has changed considerably over this time

13. Initially expecting about 120 institutions

14. Results of our pilot suggested that we should run one mahara with multiple institutions, rather than 120 separate institutions

17. Could afford to ‘lose’ 4 frontends, and a load balancer in an unplanned scenario with no service loss

18. Possible to migrate all databases between database servers for planned maintenance with no service loss

19. Possible to migrate data between file stores with short outage

20. Horizontally Scalable at frontends and cron (quick)

21. Possibility for fragmentation to provide further expansion (database and filestorage)

23. Replace aging hardware

24. New database servers

25. Add connection pooling for Mahara databases

27. Decided to use Postgres for Mahara

28. Better community support

29. It cares a lot more about your data!

30. Postgres 9.0 supports streaming replication

31. Read-only requests can be performed on slave reducing overheads to master

33. New Feature – E-mail and bounce handling - VERP The Problem Our solution Many Schools don’t give real e-mail addresses to pupils Mahara requires an e-mail address Sending lots of e-mail to fake users (or users with small inboxes) which are returned Increases load on servers E-mailed messages are marked as read so users don’t see them Implement Variable Envelope Return Path (VERP) for Mahara Catches e-mail which has been returned to sender Disables e-mail for those users who have a high ratio of sent mail to bounced mail Messages left unread in Mahara

35. User ID

38. Per-Institution settings

39. Institutions can enable/disable plugins for their users

40. Still allows integration of institutions

41. Opt-out

43. They want lots of themes

44. They don’t want to see other institution’s themes or allow others to use their theme$theme->institutions = array( ‘schoolname’, ‘school2name’, );

46. More than 10 is too time intensive and problematic

47. Extended our existing management architecture to automate this:

48. Initial MNet set up on both Moodle and Mahara

49. Log in Moodleadmins to Mahara, promote them to Institution admins on Mahara

51. Sparc architecture is slow!

52. Lots of (slow) threads was a feature five years ago

53. Intel and AMD hardware has caught up

54. Additional Ideas:

55. Alternative web servers

56. Varnish

57. Apache Traffic Server