Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Microservices reativos usando a stack do Netflix na AWS

952 views

Published on

Microservices reativos usando a stack do Netflix na AWS

Published in: Sports
  • Hi there! Get Your Professional Job-Winning Resume Here - Check our website! http://bit.ly/resumpro
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Microservices reativos usando a stack do Netflix na AWS

  1. 1. Microservices reativos usando a stack do Netflix na AWS Diego Pacheco Principal Software Architect at ilegra.com @diego_pacheco
  2. 2. www.ilegra.com
  3. 3. NetflixOSS Stack
  4. 4. Why Netflix?  Billions Requests Per Day  1/3 US internet bandwidth  ~10k EC2 Instances  Multi-Region  100s Microservices  Innovation + Solid Service  SOA, Microservices and DevOps Benchmark  Social Product  Social Network  Video  Docs  Apps  Chat Scalability Distributed Teams Could reach some Web Scale Netflix My Problem
  5. 5. AWS
  6. 6. Cloud Native
  7. 7. Principles  Stateless Services  Ephemeral Instances  Everything fails all the time  Auto Scaling / Down Scaling  Multi AZ and multi Region  No SPOF  Design for Failure (expected)  SOA  Microservices  No Central Database  NoSQL  Lightweight Serializable Objects  Latency tolerant protocols  DevOps Enabler  Immutable Infrastructure  Anti-Fragility
  8. 8. Right Set of Assumptons
  9. 9. Microservices
  10. 10. Reactive
  11. 11. Java Drivers X REST X
  12. 12. Simple View of the Architecture Zuul UI Microservice Cassandra Cluster
  13. 13. Stack
  14. 14. OSS
  15. 15. Zuul
  16. 16. Zuul
  17. 17. Karyon: Microbiology - Nucleus
  18. 18.  Reactive Extensions + Netty Server  Lower Latency under Heavy Load  Fewer Locks, Fewer Thread Migrations  Consumes Less CPU  Lower Object Allocation Rate RxNetty
  19. 19. Karyon: CODE
  20. 20. Karyon: Reactive
  21. 21. Karyon: Reactive
  22. 22. Eureka and Service Discovery http://microservices.io/patterns/server-side-discovery.html
  23. 23. Eureka  AWS Service Registry for Mid-tier  Load balancing and Failover  REST based  Karyon and Ribbon Integration
  24. 24. Eureka
  25. 25. Eureka and Service Discovery
  26. 26. Availability
  27. 27. Histryx
  28. 28.  IPC Library  Client Side Load Balancing  Multi-Protocol (HTTP, TCP, UDP)  Caching*  Batching  Reactive Ribbon
  29. 29. Ribbon CODE
  30. 30. Ribbon CODE
  31. 31.  Reactive Extension of the JVM  Async/Event based programming  Observer Pattern  Less 1mb  Heavy usage by Netflix OSS Stack RX-Java
  32. 32. Archaius  Configuration Management Solution  Dynamic and Typed Properties  High Throughtput and Thread Safety  Callbacks: Notifications of config changes  JMX Beans  Dynamic Config Sources: File, Db, DynamoDB, Zookeper  Based on Apache Commons Configuration
  33. 33. Archaius + Git MicroserviceMicroservice Slave Side Car Central Internal GIT Property Files File System MicroserviceMicroservice Slave Side Car File System MicroserviceMicroservice Slave Side Car File System
  34. 34. Asgard
  35. 35. Asgard
  36. 36. Packer JOB Create Bake/Provision Launch Deploys
  37. 37. Dynomite: Distributed Cache https://github.com/Netflix/dynomite
  38. 38. Dynomite  Implements the Amazon Dynamo Similar to Cassandra, Riak and DynamoDB Strong Consistency – Quorum-like – No Data Loss Pluggable Scalable Redis / Memcached Multi-Clients with Dyno Can use most of redis commands Integrated with Eureka via Prana
  39. 39. Dynomite: Internals
  40. 40. Oregon D1 Oregon D2 N California D3 Eureka Server Eureka Server Prana Prana Prana Multi-Region Cluster
  41. 41. Dynomite: CODE
  42. 42. Dynomite Contributions https://github.com/Netflix/dynomite https://github.com/Netflix/dynomite/pull/207 https://github.com/Netflix/dynomite/pull/200
  43. 43. Caos Engineering
  44. 44.  Isolate Failure – Avoid cascading  Redundancy – NO SPOF  Auto-Scaling  Fault Tolerance and Isolation  Recovery  Fallbacks and Degraded Experience  Protect Customer from failures – Don’t throw Failures -> Failures VS Errors Chaos / Failure
  45. 45. Gatling Stress Testing Tool Scala DSL Run on top of Akka Simple to use
  46. 46. Chaos Arch Zuul Microservice N1 Microservice N2 Cassandra Cluster Zuul Eureka ELB
  47. 47. Running…
  48. 48. Chaos Results and Learnings  Retry configuration and Timeouts in Ribbon  Right Class in Zuul 1.x (default retry only SocketException)  RequestSpecificRetryHandler (Httpclient Exceptions)  zuul.client.ribbon.MaxAutoRetries=1  zuul.client.ribbon.MaxAutoRetriesNextServer=1  zuul.client.ribbon.OkToRetryOnAllOperations=true Eureka Timeouts  It Works  Everything needs to have redudancy  ASG is your friend :-)  Stateless Service FTW
  49. 49. Microservice Producer Kafka / Storm :: Event System
  50. 50. Chaos Results and Learnings Before:  Data was not in Elastic Search  Producers was loosing data  After:  No Data Loss  It Works  Changes:  No logging on Microservice :( (Log was added)  Code that publish events on a try-catch  Retry config in kafka producer from 0 to 5
  51. 51. Main Challenges
  52. 52. Hacker Mindset
  53. 53. Next Steps  IPC  Spinnaker  Containers  Client side Aggregation  DevOps 2.0 -> Remediation / Skynet
  54. 54. Pocs https://github.com/diegopacheco/netflixoss-pocs http://diego-pacheco.blogspot.com.br/search/label/netflix?max-results=30
  55. 55. Microservices reativos usando a stack do Netflix na AWS Diego Pacheco Principal Software Architect at ilegra.com @diego_pacheco Obrigado!

×