Building Fault Tolerant Microservices
Report
Kristoffer ErlandssonFollow
Feb. 10, 2016•0 likes•878 views
Building Fault Tolerant Microservices
Feb. 10, 2016•0 likes•878 views
Download to read offline
Report
Software
Slides from presentation as held at Jfokus 2016. Despite all efforts to the contrary, faults will occur in our applications. When building an application we must prepare to minimize the faults' impact. This fact is especially evident in a microservices setting where the failure of one service must not be allowed to cascade to other, possibly more critical, services. This presentation will introduce you to stability patterns such as Bulkheads and Circuit Breakers. We will talk about how these patterns make your applications resilient to faults and how you implement them in a microservices architecture. Another important component to making your system resilient is the ability to detect and act on anomalies. Hence, we will also discuss monitoring guidelines for a distributed microservices system.
Kristoffer ErlandssonFollow
Recommended
Azure from scratch part 4Girish Kalamati
Choosing the Right EC2 Instance and Applicable Use Cases - AWS June 2016 Webi...Amazon Web Services
Web api scalability and performanceHimanshu Desai
Comet from JavaOne 2008Joe Walker
Enterprise Service Delivery from the AWS Cloud (ARC208) | AWS re:Invent 2013Amazon Web Services
Infrastructure-as-code: bridging the gap between Devs and OpsMykyta Protsenko
More Related Content
What's hot
JSR107 Come, Code, Cache, Compute! Payara
Lessons learned from writing over 300,000 lines of infrastructure codeYevgeniy Brikman
AWS와 Docker Swarm을 이용한 쉽고 빠른 컨테이너 오케스트레이션 - AWS Summit Seoul 2017Amazon Web Services Korea
Global Windows Azure Bootcamp : Cedric Derue playing with php on azure. (spon...MUG-Lyon Microsoft User Group
Spring Security PatternsVMware Tanzu
Running Vue Storefront in production (PWA Magento webshop)Vendic Magento, PWA & Marketing
Similar to Building Fault Tolerant Microservices
Scaling asp.net websites to millions of usersoazabir
App fabric introductionDennis van der Stelt
Virtual Stress-free Testing in the Cloudguest2e9c5f40
WebSockets in JEE 7Shahzad Badar
Pushing Datatothe Browserwith Comet Ajax Wrajivmordani
Refacoring vs Rewriting WixStoresDoron Rosenstock
Recently uploaded
What is Microsoft Power BI used for.pptxJohnCommuserv
Alliance Expedition BattleSilver Caprice
Winter 24 Highlights.pdfPatrickYANG48
Domain storytelling-one-size-fit-all processMichael Chen
Deckible POV - Point of ViewNick Kellet
MicroK8s 1.28 - MicroCeph on MicroK8s.pdfKonstantinos Tsakalozos
Building Fault Tolerant Microservices
- 1. Building Fault Tolerant Micro Services Kristoffer Erlandsson kristoffer.erlandsson@avanza.se @kerlandsson
- 4. Building Fault Tolerant Micro ServicesWhy? Failure modes Stability patterns Monitoring guidelines
- 6. 250+ services 1000+ service instances (JVMs) Largest actor on the Stockholm Stock Exchange
- 7. Acme Books
- 22. 0.99999 5 minutes per year
- 24. 0.999991000
- 25. 0.999991000 ≈ 0.99 87 hours per year
- 34. URL url = new URL("http://acme-books.com/special-offers"); URLConnection connection = url.openConnection(); connection.connect(); InputStream inputStream = connection.getInputStream(); // Read response from stream
- 36. URL url = new URL("http://acme-books.com/special-offers"); URLConnection connection = url.openConnection(); connection.setConnectTimeout(100); connection.setReadTimeout(500); connection.connect(); InputStream inputStream = connection.getInputStream(); // Read response from stream
- 41. Terrible response times Awful throughput
- 44. Web Application Thread pool Special Offers Throughput lower than number of incoming requests Many timeouts
- 45. Web Application Thread pool Special Offers Throughput lower than number of incoming requests Many timeouts
- 46. Web Application Thread pool Special Offers Throughput lower than number of incoming requests Growing queue! Many timeouts
- 47. Frequently called service Timeouts are not enough
- 48. Circuit BreakersCalls to broken services fail fast Offloads broken services
- 53. Single call Half open state Special Offers Web Application Error
- 55. Timeouts over threshold Unhandled errors over threshold Known irrecoverable error occurs
- 58. try { return specialOffers.getOffers(); } catch (Exception e) { return Offers.emptyOffers(); }
- 60. Terrible response times Awful throughput Again?!?
- 63. Web Application Thread pool Special Offers Throughput lower than number of incoming requests (again) Slow responses
- 64. Response time < timeout Timeouts and circuit breakers are not enough
- 66. Limit number of concurrent calls Upper bound on number of waiting threads
- 70. Special Offers Web Application Thread pool Error Error Bulkhead (size=2) - Fast page load - No special offers - Slow page load - Including special offers
- 72. Upper bound on number of waiting threads Protects very well against cascading failure …
- 73. … if bulkhead sizes are … … significantly smaller than request pool size
- 74. Peak load when healthy 40 requests per second (rps) 0.1 seconds response time
- 75. Suitable bulkhead size 40 rps x 0.1 seconds = 4 + breathing room = 7
- 76. Bonus: protects services from overload
- 77. Semaphore bulkhead = new Semaphore(2); Offers protectedGetOffers() { if (bulkhead.tryAcquire(0, TimeUnit.SECONDS)) { try { return specialOffers.getOffers(); } finally { bulkhead.release(); } } else { throw new RejectedByBulkheadException(); } }
- 79. Many threads are waiting Few available threads - low throughput All service calls are rejected!
- 83. Broken Client Library More protection required
- 84. Thread Pool HandoversCalling threads can always walk away Generic timeouts
- 92. Web Application Error Request thread pool Service thread pool Service call timeouts still required Service
- 94. ExecutorService executor = new ThreadPoolExecutor(3, 3, 1, TimeUnit.MINUTES, new SynchronousQueue<>()); Offers protectedGetOffers() { try { Future<Offers> future = executor.submit(specialOffers::getOffers); return future.get(1, TimeUnit.SECONDS); } catch (RejectedExecutionException e) { throw new RejectedByBulkheadException(); } catch (TimeoutException e) { throw new ServiceCallTimeoutException(); } }
- 95. Thread pool handovers are very powerful! What about performance?
- 98. Timeout rate Rejected call rate Short circuit rate Failure/success rate Response times
- 103. ”All this seems like a lot of work!”
- 105. class GetOffersCommand extends HystrixCommand<Offers> { public GetOffersCommand() { super(HystrixCommandGroupKey .Factory.asKey("SpecialOffers")); } @Override protected Offers run() throws Exception { return specialOffers.getOffers(); } } public Offers getOffers() { return new GetOffersCommand().execute(); }
- 106. class GetOffersCommand extends HystrixCommand<Offers> { // ... @Override protected Offers getFallback() { return Offers.emptyOffers(); } }
- 109. Design for failure Use timeouts Circuit Breakers Bulkheads Monitor service calls
- 111. Image Attributions • Polycelis felina" by Eduard Solà - Own work. Licensed under CC BY-SA 3.0 via Commons - https://commons.wikimedia.org/wiki/File:Polycelis_felina.jpg#/media/File:Polycelis_felina.jpg • "Old book bindings" by Tom Murphy VII - Own work. Licensed under CC BY-SA 3.0 via Commons - https://commons.wikimedia.org/wiki/File:Old_book_bindings.jpg#/media/File:Old_book_bindings.jpg • "Cute Snail" by gniyuhs - Own work. Licensed under CC BY-SA 3.0 via deviantart - http://gniyuhs.deviantart.com/art/Cute- Snail-278597934 • ”Cash” by 401(K) 2012 – Own work. Licensed under CC BY-SA 2.0 via Flickr - https://www.flickr.com/photos/68751915@N05/6355816649 • "Circuit breakers at substation near Denver International Airport, Colorado" by Greg Goebel from Loveland CO, USA - Yipws_2bUploaded by PDTillman. Licensed under CC BY-SA 2.0 via Wikimedia Commons - https://commons.wikimedia.org/wiki/File:Circuit_breakers_at_substation_near_Denver_International_Airport,_Colorado.j pg#/media/File:Circuit_breakers_at_substation_near_Denver_International_Airport,_Colorado.jpg • ”The control room of the nuclear ship NS Savannah, Baltimore, Maryland, USA” - Own work. Licensed under CC BY-SA 3.0 via Commons - https://en.wikipedia.org/wiki/File:NS_Savannah_control_room_MD1.jpg#/media/File:NS_Savannah_control_room_MD1.j pg
- 112. Thank you! - Questions? Kristoffer Erlandsson kristoffer.erlandsson@avanza.se @kerlandsson