1. Autograding with Docker containers
Student: Mihir M. Pandya, Advisor: David R. O’Hallaron
Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA
Introduction Architecture Implementation
Motivation Future Work
● Autolab is a course management system developed
at Carnegie Mellon University for autograding
submissions of programmatic assignments and
labs.
● Tango, the autograder used by Autolab, is a stand-
alone job management service.
● Each submission is graded in Virtual Machines
running on a set of cluster machines.
● Grading VMs must provide a homogeneous
environment for submissions.
● This project aims to replace VMs with Docker
containers.
● Tango is a stand-alone job management service.
Part of its architecture is outlined in below.
(1) Jobs get added to the job queue when clients call Tango’s
API. (2) Tango finds the next schedulable job in the job queue
and dispatches it. (3 to 7) The worker thread uses the VMMS
library for a given platform to manage grading VMs.
● Integrating Docker with Tango involves:
○ Implementing the VMMS library for Docker.
○ Bootstrapping host machines to run Docker
and be accessible over SSH.
Acknowledgements
initializeVM waitVM copyIn runJob copyOut
distDocker ec2SSH tashiSSH
● Docker containers provide isolation that can
ensure a homogeneous environment for jobs to
run in.
● Host machines that run Docker can be added and
removed live.
● Docker provides image support that allows for
more customized jobs to run with additional
software.
● Support for Docker exists on most major
platforms such as Ubuntu, Mac OS X and
Windows.
● Docker containers are light-weight and so they
initialize faster and can be discarded.
I would like to thank my research advisor, Professor
David R. O’Hallaron, for his guidance throughout
this project. I would also like to thank the Autolab
team for their support. Lastly, I thank CMU’s
Computing Services for the infrastructure support.
The Autolab Project | Meeting of the Minds | May 06, 2015
● Multiplex autograding platforms on the job-level.
The design is abstract enough to allow this.
● Host an internal repository of Docker images.
Instructors could then have immediate control of
the images used to run assessments.
● initializeVM: Assign a host machine for
container to run on. Use round-robin for assignment.
● waitVM: Ensure host machine is reachable by SSH.
● copyIn: Create a directory on host machine and
copy all input files to that directory.
● runJob: Start a Docker container with the given
Docker image and mount the previously created
directory as a volume on to that container. Then, run
the job and write the feedback file to that volume.
● copyOut: Copy feedback file from the volume to
Tango’s directory of feedback files.
● destroyVM: Destroy the Docker container (docker
rm CONTAINER) and its corresponding volume
directory on this host machine.
Results
● The implementation was ran on Greatwhite. Initally,
two t1.micro EC2 instances were used as host
machines. Following this, Tigershark was used as a
host machine.
● The first assessment from 15-213, Datalab, was used
to test the overall correctness.
● Results were similar to that of the existing system.