This document summarizes several Netflix open source projects discussed at a meeting, including: Fenzo, a generic scheduling library for Apache Mesos frameworks. It allows for heterogeneous resource scheduling with plugins for constraints and fitness. Falcor, a data modeling library that optimizes data access for web applications using caching, batching, and path optimization. Lemur, an x.509 certificate orchestration tool that provides pluggable CA support, private key management, and expiry monitoring.

1. Season 3 Episode 2
Oct 14, 2015

2. Welcome!

3. Agenda
NetflixOSS Website Relaunch @aspyker
Fenzo @podila
Vector @spiermar
Linux Java perf support @brendangregg
FIDO, Sleepy Puppy, Lemur @chanjbs
Falcor @jhusain

4. Website Relaunch
http://netflix.github.io

5. Goals of the Relaunch
● Show how the pieces fit together
○ Projects now discussed with each other in context
● OSS categories mirror internal teams
○ No artificial categories, focal points for each area
● Focus on projects that are core to Netflix
○ Projects mentioned are core and strategic
● Adding project-branded websites

6. High Level Categories
Big Data
Tools and services for (big) data
Build and Delivery Tools
Taking code from desktop to the cloud
Common Runtimes Service & Libraries
Runtime containers, libraries & services that power
microservices

7. High Level Categories
Data Persistence
Storing and serving data in the cloud
Insight, Reliability and Performance
Providing actionable insight at massive scale

8. High Level Categories
Security
Security for dynamic and distributed environments
User Interface
Libraries to help you build rich client applications

9. Fenzo
A generic, plug-ins based scheduling library for
Apache Mesos frameworks

10. Fenzo scheduling library
Heterogeneous
resources
Autoscaling
of cluster
Visibility of
scheduler
actions
Plugins for
Constraints, Fitness
High speed
Heterogeneous
task requests

11. Fenzo: scheduling model
Fitness
Pending
Assigned
Urgency

12. Fenzo: scheduling optimizations
Speed Accuracy
First fit assignment Optimal assignment
Real world tradeoffs
~ O (1) ~ O (N * M)1
1
Assuming tasks are not reassigned

13. Fenzo: fitness, constraints plugins
● Fitness value (0.0 - 1.0)
○ Degree of fitness - first fit, best fit, worst fit
○ Composable evaluators
○ e.g., bin packing
● Constraints
○ Hard constraints filter appropriate resources
○ Soft constraints specify preferences
○ e.g., zone balancing, instance type preferences

14. Fenzo: bin packing experiment
Bin pack tasks using Fenzo’s built-in CPU bin packer

15. Fenzo: cluster autoscaling
ASG/Cluster:
mantisagent
MinIdle: 8
MaxIdle: 20
CooldownSecs:
360
ASG/Cluster:
mantisagent
MinIdle: 8
MaxIdle: 20
CooldownSecs:
360
ASG/cluster:
computeCluster
MinIdle: 8
MaxIdle: 20
CooldownSecs: 360
Fenzo
ScaleUp
action:
Cluster, N
ScaleDown
action:
Cluster,
HostList

16. Fenzo: what’s next
● Task management SLAs
● Support for newer Mesos features
● Collaboration

18. Why?
● Easier way for users to troubleshoot
performance issues
● Access to low-level and specialized metrics
● Easier way to visualize and understand
● High-resolution data to detect anomalies
● Real-time and on-demand
● No additional overhead when not in use
● Something easier than SSH
● And simpler than full-fledged monitoring
solution

19. What?
● Is a Performance Monitoring tool
● Host-Level, On-Demand, High-Resolution Metrics (1 second)
● Client-side Application, User-friendly web UI
● Configurable dashboards and widgets
● Leverages Performance Co-Pilot (PCP)
● Stateless and Lightweight Metric Collection
● No persistence
● System Metrics: CPU, Memory, Network, Disk, ...
● Application Metrics*: Java, Memcached, C*, ElasticSearch, Apache
● Extensible. Custom metric agents and widgets.
* Agents are available, but not included by default.

22. What’s Next?
● Interface for different backends
● Better support for containers;
○ With container-specific dashboard and widgets.
● Native flame graph integration;
○ With our d3.js flame graph plugin.

23. CPU Flame Graphs

24. Java Mixed-Mode Flame Graphs
● Needs JDK8u60+ with
-XX:+PreserveFramePointer
○ May have some cost
● Lets Linux perf (perf_events)
see Java method frames
● Use with perf-map-agent for
symbols
● http://techblog.netflix.
com/2015/07/java-in-flames.
html
Java
Kernel
JVMGC

25. See all the things...

26. D3.js Flame Graph Plugin

27. Netflix Security OSS

28. FIDO - Security Response Orchestration
● Centralize alerts
● Enrich with data
○ User, machine
○ Threat
● Prioritize response
● Automate first
actions
Netflix's FIDO is not a part of or service of the FIDO Alliance

29. Cross-Site Scripting

30. Sleepy Puppy - XSS Testing Framework
● Visibility for non-
targeted vulnerable
apps
● Assessment
management over
time

31. Sleepy Puppy - Assessments and Payloads

32. TLS Certificate Management

33. Lemur - x.509 Certificate Orchestration
● Pluggable CA
support
● Private key
management and
distribution
● Expiry monitoring

34. Lemur Certificate Request

36. Every user wants to believe the entire cloud is
sitting right on their device.

37. Falcor let’s you code that way.

38. Let's talk about REST.

39. The Web used to be a place to get things.

40. Today, the Web is a place to do things.

41. Web pages use a small
number of large resources.

42. Web apps use large numbers of
small resources.

43. What is ?

44. Falcor is not a replacement for your Database,
MVC Framework, or your Web Server.

45. Falcor fits into your existing stack,
allowing the layers to communicate
more efficiently.

46. model.json

47. Demo

49. Falcor
● Designed for needs of Web Apps
● Model domain with JSON Graph
● Optimizes Data Access using...
○ caching
○ batching
○ path optimization

50. Falcor Roadmap
● netflix.github.io/falcor
● Java version of Router coming
● iOS client coming

51. Wrapup
● Thanks for attending!
● Join us in the courtyard for food and drinks