This talk is about how the Open University installs and manages the STACK mathematical online assessment system so that it works reliably for our students and teachers. We are probably one of the largest users of STACK, in terms of questions answered per year. The talk was give at The International STACK User Group Meeting in Edinburgh on Tuesday 30 April 2019.

1. A million questions per year:
Hosting STACK at the Open University
Tim Hunt
Senior Software Developer

2. Free as in puppy
2
Image: Designed by rawpixel.com / Freepik
STACK is ‘free’ …
… but it needs to be
properly cared for

3. Quiz at the OU

4. OU Moodle site
• 180,000 active users (students/staff)
• Around 450 module (courses) each year
• 40,000 logins per day
• 170,000 forum views per day
• 3,000 online tutorials per month (Adobe connect)
• 650,000 quiz attempts per year
• 6,700,000 questions attempted over last year

5. OU quiz usage over time
273,809
492,508
571,718
655,022 665,258 661,572
637,028 638,476 626,598
684,854
182,147
378,197
468,881
531,334 545,556
523,415
478,989 482,091 478,297
516,558
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
08/09 09/10 10/11 11/12 12/13 13/14 14/15 15/16 16/17 17/18
Testsserved
Academic year (September-August)
All informal quizzes and formal iCMAs on module websites by academic year
Orange: attempts started Blue: attempt submitted

6. [CELLRANGE], 0% [CELLRANGE], 1% [CELLRANGE], 4%
[CELLRANGE], 5%
[CELLRANGE], 3%
[CELLRANGE], 4%
[CELLRANGE], 6%
[CELLRANGE], 9%
[CELLRANGE], 37%
[CELLRANGE], 0%
[CELLRANGE], 0%
[CELLRANGE], 0%
[CELLRANGE], 0%
[CELLRANGE], 0%
[CELLRANGE], 1%
[CELLRANGE], 1%
[CELLRANGE], 3%
[CELLRANGE], 9%
[CELLRANGE], 18%
Selective (64%)
Constructive (28%)
Semi-constructive (8%)
1% = 55,000 attempts
Question attempts / year

7. Hosting STACK

8. STACK
STACK is a question type (plus other plugins)
for Moodle
Installed in the usual way
Also requires Maxima (computer algebra package)
installed on the server
8
Well, installed on
a
server

9. MaximaPool
Moodle MaximaPool
https://github.com/maths/stack_util_maximapool/
/usr/bin/ # Installed as part of RHEL.
gnuplot
maxima # maxima-optimised actually used, but this is present.
/usr/share/tomcat8/webapps/ or /var/lib/tomcat8/webapps/
MaximaPool.war # The compiled code. This is what we upload.
MaximaPool/ # Expanded version of the .war file. Mostly standard.
META-INF/
MANIFEST.MF
WEB-INF/
web.xml
classes/
servlet.conf # Configuration file: directory.root & admin.password.
... # All the compiled classes.
/usr/share/tomcat8/logs/ # The Tomcat logs.
/var/lib/maximapool/ # This is the path directory.root from servlet.conf.
pool.conf # Configuration for the whole pool.
2013073001/ # Directory name is a STACK Maxima code version number.
process.conf # Configuration settings for this version.
maxima-optimised # The pre-compiled Maxima executable.
maximalocal.mac # Extra file loaded on start-up.
maxima/ # Copy of question/type/stack/stack/maxima.
2013122500/ # ... can be mulitple versions present simultaniously.
...

10. Monitoring STACK

11. Built-in & Nagios
11
Nagios triggers
alerts for IT Ops
if values go out
of allowed range

12. Upgrading STACK

13. Not just Moodle plugins
Standard Moodle plugins
• Upgraded as usual
STACK-Maxima library code
• Build maxima-optimised
• Deploy to MaximaPool
To help deployment MaximaPool
can run multiple pools
13
/var/lib/maximapool/ # This is the path directory.root from servlet.conf.
pool.conf # Configuration for the whole pool.
2018080600/ # Directory name is a STACK Maxima code version number.
process.conf # Configuration settings for this version.
maxima-optimised # The pre-compiled Maxima executable.
maximalocal.mac # Extra file loaded on start-up.
maxima/ # Copy of question/type/stack/stack/maxima.
2019042600/
process.conf # Configuration settings for this version.
maxima-optimised # The pre-compiled Maxima executable.
maximalocal.mac # Extra file loaded on start-up.
maxima/ # Copy of question/type/stack/stack/maxima.
...
Deploy new maxima code
Test with test Moodle install
Upgrade live Moodle
Un-deploy old maxima code

14. Testing STACK

15. 1. Healthcheck page
2. Automated tests (PHPUnit & Behat)
3. Answer-tests and Input-tests scripts
4. Question tests
Lots of tools
15
1
1
3
3
4
Correct and expected
STACK-Maxima library
version being used
(2018080600).

16. Question tests save lives
Do all questions work with the latest
version of STACK?
16
4

17. Advertising feature:
Ad-hoc database queries
A useful tool for admins

18. Simple SQL reports
Often a STACK upgrade issue
affects several questions
18

19. It’s not all bad
SELECT COUNT(1)
FROM {question_attempts} qa
JOIN {question} q ON q.id = qa.questionid
JOIN {quiz_attempts} quiza
ON quiza.uniqueid = qa.questionusageid
WHERE quiza.preview = 0
AND q.qtype = 'stack'
AND qa.timemodified BETWEEN
%%STARTTIME%% AND %%ENDTIME%%
19

20. Supporting STACK

21. STACK: used by people
Powerful systems are complex
⇒ People need training
• Like Tim Lowe’s training
that you are missing now!
⇒ People need documentation
⇒ Worth sharing good practice
STACK is an essential part of teaching
learning & assessment
⇒ Users need to know about upgrades
⇒ Upgrades must to avoid key dates
21

22. Advertising feature:
Embedded questions
Having got STACK, make the most of it

23. The familiar

24. Embed questions
This is a
Moodle Book
resource

25. How to set up

26. Summary

27. Servers
Software
Monitoring
Upgrades
Testing
Support
People
Summary
Non-STACK Plugins advertised
https://moodle.org/plugins/report_customsql
https://moodle.org/plugins/filter_embedquestion
https://moodle.org/plugins/atto_embedquestion

28. 28
THANK YOU!
T.J.Hunt@open.ac.uk
@tim_hunt