This document summarizes an engineer's experience setting up and experimenting with Netflix's Spinnaker continuous delivery tool on a local Kubernetes cluster. The engineer and their team set up Kubernetes using Docker for Mac, installed Halyard to manage Spinnaker, and deployed Spinnaker services. They took a look around the Spinnaker and Kubernetes dashboards. An attempt was made to deploy sample applications but it initially failed due to incorrect assumptions about how Spinnaker works. After reviewing documentation, they were able to deploy applications a little better but more time is needed to fully understand Spinnaker. Key learnings included the importance of documentation and that Spinnaker works in a push rather than pull model.

1. Taking Spinnaker for a spin…
… a hackday worth of experience running Netflix’s Spinnaker on a local
Kubernetes ‘cluster’
+

2. The Gooflix Stack
https://github.com/gooflix/
aka

3. Agenda
• The Team
• A bit about Spinnaker and setting up Kube + Halyard
• Taking a peek
• Attempting to deploy an app, failing,“fixing” it
• A demo (or a video, depends)
• Things I/we learned

4. The team
• Alex Leonhardt (me)
• Systems Engineer @
Mendeley / Elsevier
• github.com/alex-leonhardt
• Filippo Mortari
• Java/NodeJS Dev @
Mendeley / Elsevier
• github.com/filippomortari
• Thomas Viaud
• DevOps Engineer @
Elsevier
• https://github.com/tviaud

5. TL;DR on Spinnaker
• Continuous Delivery for the Cloud/s (AWS, GCE,
Azure) + Kubernetes
• Spinnaker itself is a set of microservices
• Netflix OSS project
• "Cloud Native" & enables multi-cloud deployments
https://www.spinnaker.io/

6. Components
• kubernetes
• local docker4mac (edge) w/ kubernetes enabled
• halyard
• hal manages the spinnaker installation and its services incl. updates
and rolling out config changes
• spinnaker itself
• composed of several services

7. Setting up kube
• docker 4 mac (edge)
• give as much memory & cpu as possible
• enable kubernetes
• install heapster and kube dashboard
kubectl apply -f https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/rbac/heapster-rbac.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/standalone/heapster-controller.yaml
kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml

8. Setting up kube

9. Setting up Halyard
• deployed as a replicaset into kube
• persistent volume claim (pvc) to keep config/history
• config for this is on github - https://github.com/gooflix/config/
• tl;dr
• provider - cloud provider/s to use for deployments by spinnaker
(kubernetes, ecs/aws, gce, etc.)
• environment (type of spinnaker install: local, distributed, dev)
(distributed is what you want, probably)
• commands:
https://www.spinnaker.io/reference/halyard/commands

10. Deploy spinnaker
• connect to halyard pod
• use the script in configmap to “fly” through setup steps
• it basically, uses example config from spinnaker docs
• there was “trap” :)
https://github.com/spinnaker/spinnaker/issues/2813#issuecomment-391692923

11. It’s all a bit too much

12. But eventually

13. Taking a peek around
• Access kube dash
• http://localhost:8081/ui
http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/replicaset?namespace=spinnaker
• Access spinnaker
• deck (ui) http://localhost:9000/
• gate (api) http://localhost:8084/
• need to port-forward both to access spinnaker

14. Taking a peek around

15. Spinnaker architecture
Source: https://www.spinnaker.io/reference/architecture/

16. Other dependencies
• Minio (s3) deployment
(this is to save artifacts)
• this can be configured during setup steps, s3 or
equivalent would be the logical choice
• Redis
(job/history storage)

17. Demo apps
• build on commit via github -> trigger dockerhub
• nginx ingress
(https://github.com/gooflix/ingress-demo)
• a java app
(https://github.com/gooflix/backend-demo)
• DockerHub
• https://hub.docker.com/r/alexleonhardt/ingress-demo/
• https://hub.docker.com/r/alexleonhardt/backend-demo/

18. Assumptions
• spinnaker would pull kube manifest from Github for deployment
• manually triggered by human / trigger on push / poll for
changes
• manifest is authority / might overrides things like number of
pods set at runtime (manual/auto scaling)
• we could poll container registry to get notified of changes
• if changed, run deployment
• if not changed, do nothing

19. Setting up the Ingress app

20. Fail

21. What went wrong?
• my assumptions were garbage, spinnaker doesn’t
work the way I thought it would or should ;)
• just…. maybe RTFM ?
https://www.spinnaker.io/setup/other_config/
https://www.spinnaker.io/guides/user/kubernetes-v2/deploy-manifest/
https://www.spinnaker.io/guides/user/canary/
• not able to find the .yaml file in the “trigger” (but,
we didn’t even configure this… yet)

22. Making it work, well, kinda

23. Making it work, a little better

24. Making it work, a little better

25. Rinse & repeat for a java app

26. Rinse & repeat for a java app

27. Demo

28. What have we/I learned ?
• Things don’t work the way I think / expect
• RTFM
• We wrote some Kubernetes manifests :) - some previous knowledge was essential
• Spinnaker is push (not pull) orientated
• means you may need to expose it to public (github + docker registries + other event sources)
• Did not get to try hook into Jenkins
• BUT - I’d like to see more alternatives!
• Many (10+) services, that need managing, scaling, debugging, understanding
• New! some things are still ‘alpha’ (e.g. kube deployments from manifest)
• Needs more time! Get a PoC / trial set up for several weeks if you can, take it for a spin ;)
• Java services, which require a lot of memory by default

29. Links
• Some further reading, that might be useful
• https://blog.spinnaker.io/scaling-spinnaker-at-netflix-
part-1-8a5ae51ee6de
• https://blog.spinnaker.io/scaling-spinnaker-at-netflix-metrics-
and-more-dbc4910b74e3
• https://blog.spinnaker.io/codifying-your-spinnaker-pipelines-
ea8e9164998f
• https://kubernetes.io/docs/tutorials/kubernetes-basics/
• https://sysdig.com/blog/kubernetes-security-guide/

30. Thanks!
P.S. We’re hiring, I think ;)
https://github.com/gooflixhttps://www.sciencedirect.com https://www.mendeley.com