The document discusses deploying Apache Spark jobs on Kubernetes using Helm and the Spark Operator. It outlines the process of building, running, and deploying applications, highlighting the importance of Docker images and Helm templates. Additional resources and links for further information are provided, along with a call for feedback on the session.

2. Deploying Apache Spark Jobs on
Kubernetes with Helm and Spark
Operator
Tom Lous
Freelance Data Engineer @ Shell
@tomlous

3. Why?

4. Bad Idea?

5. 1. Build it!
2. Run it?

6. Challenge!

7. Solution?

8. Kubernetes!?

9. minikube
https://carbon.now.sh/5pwVel5DBKj0cO3ZNCRh

10. Application

11. Dependencies & App

12. Base Image

13. Dockerize

14. Deploy?

15. Spark Operator!

16. Helm Template

17. Helm Values

18. Chart Museum

19. Deploy!

20. Success!

21. Next Steps

22. Links
▪ HowTo: https://medium.com/@tomlous/deploying-apache-spark-
jobs-on-kubernetes-with-helm-and-spark-operator-eb1455930435
▪ SparkOperator: https://github.com/GoogleCloudPlatform/spark-on-
k8s-operator
▪ SparkOperator Helm:
https://github.com/helm/charts/tree/master/incubator/sparkoperat
or
▪ Code: https://github.com/TomLous/medium-spark-k8s
▪ Chart Museum: https://github.com/helm/chartmuseum

23. Questions?

24. Feedback
Your feedback is important to us.
Don’t forget to rate and
review the sessions.