Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

NetflixOSS Meetup season 3 episode 2

1,561 views

Published on

Slides from Oct 14th NetflixOSS Meetup

Published in: Engineering
  • Be the first to comment

NetflixOSS Meetup season 3 episode 2

  1. 1. Season 3 Episode 2 Oct 14, 2015
  2. 2. Welcome!
  3. 3. Agenda NetflixOSS Website Relaunch @aspyker Fenzo @podila Vector @spiermar Linux Java perf support @brendangregg FIDO, Sleepy Puppy, Lemur @chanjbs Falcor @jhusain
  4. 4. Website Relaunch http://netflix.github.io
  5. 5. Goals of the Relaunch ● Show how the pieces fit together ○ Projects now discussed with each other in context ● OSS categories mirror internal teams ○ No artificial categories, focal points for each area ● Focus on projects that are core to Netflix ○ Projects mentioned are core and strategic ● Adding project-branded websites
  6. 6. High Level Categories Big Data Tools and services for (big) data Build and Delivery Tools Taking code from desktop to the cloud Common Runtimes Service & Libraries Runtime containers, libraries & services that power microservices
  7. 7. High Level Categories Data Persistence Storing and serving data in the cloud Insight, Reliability and Performance Providing actionable insight at massive scale
  8. 8. High Level Categories Security Security for dynamic and distributed environments User Interface Libraries to help you build rich client applications
  9. 9. Fenzo A generic, plug-ins based scheduling library for Apache Mesos frameworks
  10. 10. Fenzo scheduling library Heterogeneous resources Autoscaling of cluster Visibility of scheduler actions Plugins for Constraints, Fitness High speed Heterogeneous task requests
  11. 11. Fenzo: scheduling model Fitness Pending Assigned Urgency
  12. 12. Fenzo: scheduling optimizations Speed Accuracy First fit assignment Optimal assignment Real world tradeoffs ~ O (1) ~ O (N * M)1 1 Assuming tasks are not reassigned
  13. 13. Fenzo: fitness, constraints plugins ● Fitness value (0.0 - 1.0) ○ Degree of fitness - first fit, best fit, worst fit ○ Composable evaluators ○ e.g., bin packing ● Constraints ○ Hard constraints filter appropriate resources ○ Soft constraints specify preferences ○ e.g., zone balancing, instance type preferences
  14. 14. Fenzo: bin packing experiment Bin pack tasks using Fenzo’s built-in CPU bin packer
  15. 15. Fenzo: cluster autoscaling ASG/Cluster: mantisagent MinIdle: 8 MaxIdle: 20 CooldownSecs: 360 ASG/Cluster: mantisagent MinIdle: 8 MaxIdle: 20 CooldownSecs: 360 ASG/cluster: computeCluster MinIdle: 8 MaxIdle: 20 CooldownSecs: 360 Fenzo ScaleUp action: Cluster, N ScaleDown action: Cluster, HostList
  16. 16. Fenzo: what’s next ● Task management SLAs ● Support for newer Mesos features ● Collaboration
  17. 17. Why? ● Easier way for users to troubleshoot performance issues ● Access to low-level and specialized metrics ● Easier way to visualize and understand ● High-resolution data to detect anomalies ● Real-time and on-demand ● No additional overhead when not in use ● Something easier than SSH ● And simpler than full-fledged monitoring solution
  18. 18. What? ● Is a Performance Monitoring tool ● Host-Level, On-Demand, High-Resolution Metrics (1 second) ● Client-side Application, User-friendly web UI ● Configurable dashboards and widgets ● Leverages Performance Co-Pilot (PCP) ● Stateless and Lightweight Metric Collection ● No persistence ● System Metrics: CPU, Memory, Network, Disk, ... ● Application Metrics*: Java, Memcached, C*, ElasticSearch, Apache ● Extensible. Custom metric agents and widgets. * Agents are available, but not included by default.
  19. 19. What’s Next? ● Interface for different backends ● Better support for containers; ○ With container-specific dashboard and widgets. ● Native flame graph integration; ○ With our d3.js flame graph plugin.
  20. 20. CPU Flame Graphs
  21. 21. Java Mixed-Mode Flame Graphs ● Needs JDK8u60+ with -XX:+PreserveFramePointer ○ May have some cost ● Lets Linux perf (perf_events) see Java method frames ● Use with perf-map-agent for symbols ● http://techblog.netflix. com/2015/07/java-in-flames. html Java Kernel JVMGC
  22. 22. See all the things...
  23. 23. D3.js Flame Graph Plugin
  24. 24. Netflix Security OSS
  25. 25. FIDO - Security Response Orchestration ● Centralize alerts ● Enrich with data ○ User, machine ○ Threat ● Prioritize response ● Automate first actions Netflix's FIDO is not a part of or service of the FIDO Alliance
  26. 26. Cross-Site Scripting
  27. 27. Sleepy Puppy - XSS Testing Framework ● Visibility for non- targeted vulnerable apps ● Assessment management over time
  28. 28. Sleepy Puppy - Assessments and Payloads
  29. 29. TLS Certificate Management
  30. 30. Lemur - x.509 Certificate Orchestration ● Pluggable CA support ● Private key management and distribution ● Expiry monitoring
  31. 31. Lemur Certificate Request
  32. 32. Every user wants to believe the entire cloud is sitting right on their device.
  33. 33. Falcor let’s you code that way.
  34. 34. Let's talk about REST.
  35. 35. The Web used to be a place to get things.
  36. 36. Today, the Web is a place to do things.
  37. 37. Web pages use a small number of large resources.
  38. 38. Web apps use large numbers of small resources.
  39. 39. What is ?
  40. 40. Falcor is not a replacement for your Database, MVC Framework, or your Web Server.
  41. 41. Falcor fits into your existing stack, allowing the layers to communicate more efficiently.
  42. 42. model.json
  43. 43. Demo
  44. 44. Falcor ● Designed for needs of Web Apps ● Model domain with JSON Graph ● Optimizes Data Access using... ○ caching ○ batching ○ path optimization
  45. 45. Falcor Roadmap ● netflix.github.io/falcor ● Java version of Router coming ● iOS client coming
  46. 46. Wrapup ● Thanks for attending! ● Join us in the courtyard for food and drinks

×