1. Netflix Open Source
Season 4, Episode 1
Feb 2016

2. Agenda
● Welcome
● Netflix Open Source Evolution
○ @aspyker
● Spinnaker
○ @ajordens, @mjduftler, @rcarunmsft, @dstengle
● Demo stations and reception

3. Open Source since 2010
● Architecture has shaped public cloud usage
○ Immutability, red/black deploys, chaos, regional
and worldwide high availability
● 102 Open Source Projects in Netflix org

4. Showing some age
● Across our projects
● Open issues
○ Astyanax (148), Asgard (106), Ice (74),
Total (1631)
● Open pull requests
○ Astyanax (44), Ice (23), Asgard (18),
Total (258)
● Some issues & PR’s have
been around for a while
Open Issues Per Project
Open PR’s Per Project

5. What are we doing to evolve?
● Establish ownership and group logically
○ Management and engineering aligned with business
● Classify projects into lifecycles
○ Clearly identify active vs. retired projects
○ Clearly identify experimental projects
● Track active projects
○ Actively work to improve project “health”

6. Establish Ownership and Grouping
● Consistency across
related projects
● Document how area
fits together
● Focus on OSS health
of functional area
OSS Area Shepherds

7. Already taking action
● Astyanax, Asgard, ICE are either not
used or being transitioned away
● Starting to document this better

8. ● OSSMETADATA tagging file
osslifecycle=
active | maintenance | archived
● You can see these today
○ Eventually badges in README.md
● Allows for a tool to automate lifecycle
verification
How do we automate this?

9. Our Dashboard: Netflix OSS Tracker
OSS Tracker
Nightly Jobs
Project Ownership
Database
Time Series
Project Statistics
OSS Tracker
Console
● Collects metrics on every project every day
● Allows shepherds to see combined stats for their area
● Allows project owners to see daily metrics
Whitepages
Service

10. github.com/Netflix/osstracker

11. OSS Tracker Future (Want to help?)
● Pluggable whitepages & database systems
● Collect more metrics from github
○ Time bucketed (last 60 days vs. forever)
○ Internal vs. external contributors
○ Tagging to allow for issues as features, etc.
● Create single consumable health score
● Emailing reports to shepherds and owners
● Better user interface

13. The Netflix Continuous Delivery Platform
Development began in 2014
Open sourced 11/2015
Spinnaker

14. Evolution of CD at Netflix
Asgard
Open Sourced in 2012
Was the primary deployment
tool until 2015
A better AWS Console
Codified Conventions
- Red/Black
- Naming
Pitfalls
Single Account
No first-class API
Mimir
Internal Netflix
Script Support (Jenkins)
Flexible Workflows
Used the “Asgard API”
Edge Center
Internal Netflix
Canaries
Automation
Spinnaker
Application-centric UI
Flexible Pipelines
RESTful API
Multi Account
Canaries (Netflix-only)
https://www.flickr.com/photos/spidermandragon5/2922128673

15. ● Intuitive UI (Self-service)
● Moving Beyond Simple Red/Black
● Out-of-the-box Best Practices
!! Community Participation !!
Goals

16. Abstraction
UI Components
Pipeline Stages

18. Let’s Take a Tour

22. Container
Support
PluggableComponents
UI and Services
Declarative
Continuous
Delivery
Performance
Documentation
Community Site /
Whitepapers /
FAQ

23. ● Netflix and Spinnaker principles align with
Google’s
○ Massive infrastructure, services at
scale
○ Immutable infrastructure
○ Continuous delivery
○ Traceability
○ Self-service tools
Why is Google investing in Spinnaker?

24. ● >30,000 developers in 40+ offices
● 13,000+ projects under active development
● 30k submissions per day (1 every 3 seconds)
● Single monolithic code tree with mixed language code
● Development on one branch - submissions at head
● All builds from source
● 30+ sustained code changes per minute with 90+ peaks
● 50% of code changes monthly
● 150+ million test cases / day, > 150 years of test / day
● Supports continuous deployment for all Google teams!
Internal Google Development

25. ● Developing out GCP toolset
● Our internal platform and tooling is container
oriented
● Spinnaker is a great starting point for VM
oriented tooling
● Build out functionality by contributing to OSS
that aligns with our practices and goals
○ Bazel, Kubernetes, ...
Google Cloud Platform

26. Let’s See it in Action

31. MAKE
NEXT
POSSIBLE

32. ● Professional Services
NetflixOSS
○ Microservices and Cloud
● Spinnaker Launch Contribs
○ Installation and VM Images
○ Docs
Kenzan & Spinnaker

34. ● Reduce Complexity of a true
E2E Spinnaker CD Setup
● Support AWS and GCP
Terraform Spinnaker

36. What you get
● Spinnaker
● Jenkins
● Apt repo
● Networking
● Example App
● Jobs and Pipelines

38. What’s Next
● Productionized Terraform
Deployment
● More sophisticated continuous
deployment pipeline examples

39. Demo Stations
● Spinnaker
○ Snag Spinnaker stickers
○ Netflix Spinnaker demos
○ See Google-Spinnaker integration
○ Meet the Azure team
● Kenzan
○ Spinnaker Terraform demo
○ Learn about their NetflixOSS consulting

40. Demo Stations
Come hear about Dynomite updates
● Dynamo layer to make a non-distributed
data store into distributed
○ Redis/memcache, HA scale-out, encrypted cross-region replication
● Quorum consistency
● Data warm up
● Speed improvements in latest release
○ 12 Dynomite nodes >1M OPS

41. Wrapup
● Thanks for attending!
● Join us for
○ Food and drinks
○ Demo stations