The document discusses the use of Kubernetes for scaling Open edX, an open-source platform developed by a coalition including MIT and Harvard. It covers Kubernetes concepts, architecture, and benefits such as scalability and resiliency, while also detailing technical aspects of Open edX deployment. The presentation highlights lessons learned and ongoing developments in leveraging Kubernetes for multi-tenant environments.

1. Scaling Open edX
with Kubernetes
DevOpsDays Boston
9.15.2015

2. Who we are
Nate Aune
Morgan Robertson

3. What we’ll cover
● Background -- Open edX
● Introducing Kubernetes
● Kubernetes concepts
● Scaling + resiliency
● Open edX on Kubernetes

4. Open edX background
● edX: non-profit founded by MIT and Harvard
● 500+ courses, 5M students learning on edX.org
● edX released Open edX in June 2013
● Stanford, MongoDB, Salesforce, Google, Microsoft,
McKinsey, Johnson & Johnson, Smithsonian

5. Open edX - a catalyst for innovation
212 Contributors
One of the fastest
growing open
source projects on
Github

6. Technical components
LMS/CMS (Django/Python)
Forum (Sinatra/Ruby)
User DB (MySQL)
Course DB (Mongo)
Tasks (Celery/RabbitMQ)
Caching (Memcache)
Proxy (Nginx)
Search (ElasticSearch)
Mapreduce (Hadoop)

7. Hosting infrastructure
S3 for serving:
● static assets
● grade downloads
● certificate downloads
● videos (for mobile)
● Load balancer
● Application server(s)
● Database server(s)
● Search server
● Utility server (tasks)
● Caching server
● Hadoop cluster

8. Typical scalable deployment of Open edX on AWS

9. Introducing Kubernetes
● Scheduling + orchestration layer for
containerized applications
● Abstracts your infrastructure
● Open source project by Google
● Production-ready as of July 2015

10. Kubernetes architecture

11. Kubernetes vs. the Docker triad
Kubernetes Swarm Compose Machine
Scheduling ✔ ✔
Service discovery ✔ ✅
Container scaling ✔ ✔
Machine provisioning ✅ ✔
Health checking ✔
Secret management ✔
Production-ready ✔

12. Kubernetes core concepts
● Pods
● Services
● Replication controllers

13. Pods
● Group of containers + volumes scheduled together
● Smallest deployable unit
● Containers share certain resources including network
stack

14. Services

15. Services
● Endpoint for a set of pods
● IP address, port, and label selectors
● Use round-robin routing to direct traffic to backend
pods

16. Services + Pods

17. Replication Controllers
● Manage pod lifecycles for a number of replicas
● Provide scaling + fault tolerance
● Use label selectors

18. Pods + Services + Replication Controllers

19. Scaling with Kubernetes
● Replication controllers scale pods
● Services provide a single endpoint for a group of
pods
● The Kubernetes master schedules pods across nodes

20. Resiliency with Kubernetes
● Replication controllers ensure a number of pods are
running
● Services provide load balancing
● Health checks allow bad pods to be ignored/removed

21. Open edX on Kubernetes
● Goals:
○ Multi-tenant
○ Scalable + resilient

22. The challenge

23. Architecture

24. Monitoring with Sysdig

25. Sysdig drill-down

26. Lessons learned
● Containers should be stateless
● Put initialization tasks into separate pods that run
once
● Services can be used to abstract non-containerized
components

27. Conclusion
● We’re still learning, but..
○ Kubernetes is a promising technology for
providing both scalability and resiliency

28. More info
Open edX - http://open.edx.org
Kubernetes - http://kubernetes.io
Google Container Engine - http://cloud.
google.com/container-engine

29. Thank you for your time!
Questions?
Slides: http://bit.ly/open-edx-kubernetes
nate@appsembler.com
morgan@appsembler.com
@appsembler