Downloaded 19 times
![@chbatey
All on the tomcat thread
13:47:20.200 [http-8080-exec-1] INFO info.batey.examples.Resource -
Resource later: I wonder which thread I am on!
13:47:20.200 [http-8080-exec-1] INFO
info.batey.examples.ScaryDependency - Scary Dependency: I wonder which
thread I am on! Tomcats?](https://image.slidesharecdn.com/faulttolerantmicroservices-ljc-151128140833-lva1-app6892/85/LJC-Microservices-in-the-real-world-40-320.jpg)
![@chbatey
What an annotation can do...
13:51:21.513 [http-8080-exec-1] INFO info.batey.examples.Resource
- Resource later: I wonder which thread I am on!
13:51:21.614 [hystrix-ScaryDependency-1] INFO
info.batey.examples.ScaryDependency - Scary Dependency: I wonder
which thread I am on! Tomcats? :P](https://image.slidesharecdn.com/faulttolerantmicroservices-ljc-151128140833-lva1-app6892/85/LJC-Microservices-in-the-real-world-42-320.jpg)
Uploaded byChristopher Batey
LJC: Microservices in the real world
The document discusses techniques for building fault tolerant microservices. It covers: 1) Using timeouts to prevent services from hanging for long periods of time 2) Implementing queues and limiting thread pools to avoid overloading services and failing fast when they are overloaded 3) Employing circuit breakers and fallbacks to avoid hitting failing dependent services repeatedly 4) Including kill switches to gracefully take down broken parts of systems Code examples and links to relevant libraries are provided.
More Related Content
![[212] large scale backend service develpment](https://cdn.slidesharecdn.com/ss_thumbnails/221largescalebackendservicedevelpment-150915001346-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to LJC: Microservices in the real world
LJC: Microservices in the real world
- 1. @chbatey Microservices in thereal world Christopher Batey Freelancer / The Last Pickle @chbatey
- 2. @chbatey S!*t happens whenyou put a network between your classes Christopher Batey Freelancer / The Last Pickle @chbatey
- 3. @chbatey Who am I? •Freelance consultant: Docker, Kubernetes, Cassandra, JVM • Work on a OS testing library for Cassandra • Twitter: @chbatey Heavily recruiting!!
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10. @chbatey Small horizontal scalableservices • Move to small services independently deployed - Login service - Device service - etc • Move to a horizontally scalable Database that can run active active in multiple data centres
- 11.
- 12. @chbatey So... what dosystems look like now?
- 13. @chbatey Tech used • Allexamples are on github • Technologies used: - Dropwizard - Wiremock - Hystrix - Saboteur
- 14. @chbatey Java microservice definition? Amicroservice in Java == no dependency injection framework
- 15. @chbatey Testing microservices • Youdon’t know a service is fault tolerant if you don’t test faults
- 16. @chbatey Isolated service tests Prime Movieservice Play Movie Test Real HTTP/TCP Test
- 17. @chbatey Fault tolerance 1.Don’t takeforever - Timeouts 2.Don’t try if you can’t succeed 3.Don’t whack a dead horse 4.Turn broken stuff off
- 18. @chbatey 1 - Don’ttake forever • If at first you don’t succeed, don’t take forever to tell someone • Timeout and fail fast
- 19. @chbatey Which timeouts? • Socketconnection timeout • Socket read timeout
- 20. @chbatey Your service hungfor 30 seconds :( Customer You :(
- 21. @chbatey Which timeouts? • Socketconnection timeout • Socket read timeout • Resource acquisition
- 22. @chbatey Your service hungfor 10 minutes :(
- 23.
- 24. @chbatey Adding an automatedtest • Vagrant - launches + provisions localVMs • Saboteur - uses tc, iptables to simulate network issues • Wiremock - used to mock HTTP dependencies
- 25. @chbatey I can writean automated test for that? Wiremock: •User Service •Device Service •Pin Service S a b o t e u r Vagrant + Virtual box Test meAcceptance prime to drop traffic reset
- 26.
- 27.
- 28.
- 29.
- 30. @chbatey A little moredetail
- 31.
- 32.
- 33.
- 34.
- 35.
- 36. @chbatey Implementing reliable timeouts •Protect the container thread! • Homemade:Worker Queue + Thread pool (executor)
- 37. @chbatey Implementing reliable timeouts •Protect the container thread! • Homemade:Worker Queue + Thread pool (executor) • Hystrix • Spring cloud Netflix
- 38. @chbatey A simple SpringRestController @RestController public class Resource { private static final Logger LOGGER = LoggerFactory.getLogger(Resource.class); @Autowired private ScaryDependency scaryDependency; @RequestMapping("/scary") public String callTheScaryDependency() { LOGGER.info("Resource later: I wonder which thread I am on!"); return scaryDependency.getScaryString(); } }
- 39. @chbatey Scary dependency @Component public classScaryDependency { private static final Logger LOGGER = LoggerFactory.getLogger(ScaryDependency.class); public String getScaryString() { LOGGER.info("Scary Dependency: I wonder which thread I am on! Tomcats?”); if (System.currentTimeMillis() % 2 == 0) { return "Scary String"; } else { Thread.sleep(5000) return “Slow Scary String"; } } }
- 40. @chbatey All on thetomcat thread 13:47:20.200 [http-8080-exec-1] INFO info.batey.examples.Resource - Resource later: I wonder which thread I am on! 13:47:20.200 [http-8080-exec-1] INFO info.batey.examples.ScaryDependency - Scary Dependency: I wonder which thread I am on! Tomcats?
- 41. @chbatey Scary dependency @Component public classScaryDependency { private static final Logger LOGGER = LoggerFactory.getLogger(ScaryDependency.class); @HystrixCommand() public String getScaryString() { LOGGER.info("Scary Dependency: I wonder which thread I am on! Tomcats?”); if (System.currentTimeMillis() % 2 == 0) { return "Scary String"; } else { Thread.sleep(5000) return “Slow Scary String"; } } }
- 42. @chbatey What an annotationcan do... 13:51:21.513 [http-8080-exec-1] INFO info.batey.examples.Resource - Resource later: I wonder which thread I am on! 13:51:21.614 [hystrix-ScaryDependency-1] INFO info.batey.examples.ScaryDependency - Scary Dependency: I wonder which thread I am on! Tomcats? :P
- 43. @chbatey Async libraries areyour friend • DataStax Java Driver - Guava ListenableFuture
- 44. @chbatey Drive this viarequirements • Reliable timeout • Throttling • Monitoring of failures / successes
- 45. @chbatey Timeouts take home •You can’t use network level timeouts for SLAs • Test your SLAs - if someone says you can’t, hit them with a stick • Scary things happen without network issues
- 46. @chbatey Fault tolerance 1.Don’t takeforever - Timeouts 2.Don’t try if you can’t succeed 3.Don’t whack a dead horse 4.Turn broken stuff off
- 47. @chbatey 2 - Don’ttry if you can’t succeed
- 48. @chbatey Complexity “When an applicationgrows in complexity it will eventually start sending emails”
- 49. @chbatey Complexity “When an applicationgrows in complexity it will eventually start using queues and thread pools”
- 50. @chbatey Don’t try ifyou can’t succeed
- 51. @chbatey Don’t try ifyou can’t succeed • Executor Unbounded queues :( - newFixedThreadPool - newSingleThreadExecutor - newThreadCachedThreadPool • Bound your queues and threads • Fail quickly when the queue / maxPoolSize is met • Know your drivers
- 52. @chbatey This is afunctional requirement • Set the timeout very high • Use Wiremock to add a large delay to the requests • Set queue size and thread pool size to 1 • Send in 2 requests to use the thread and fill the queue • What happens on the 3rd request?
- 53. @chbatey Fault tolerance 1.Don’t takeforever - Timeouts 2.Don’t try if you can’t succeed 3.Don’t whack a dead horse 4.Turn broken stuff off
- 54. @chbatey 5 - Don’twhack a dead horse Movie Player User Service Device Service Pin Service
- 55. @chbatey What to do… •Yes this will happen… • Mandatory dependency - fail *really* fast • Throttling • Fallbacks
- 56. @chbatey Circuit breaker pattern (Picturefrom Akka docs)
- 57. @chbatey Fault tolerance 1.Don’t takeforever - Timeouts 2.Don’t try if you can’t succeed 3.Don’t whack a dead horse 4.Turn broken stuff off
- 58. @chbatey 6 - Turnoff broken stuff • The kill switch
- 59. @chbatey To recap 1.Don’t takeforever - Timeouts 2.Don’t try if you can’t succeed 3.Don’t whack a dead horse 4.Turn broken stuff off
- 60. @chbatey Links • Examples: - https://github.com/chbatey/spring-cloud-example -https://github.com/chbatey/dropwizard-hystrix - https://github.com/chbatey/vagrant-wiremock-saboteur • Tech: - https://github.com/Netflix/Hystrix - https://www.vagrantup.com/ - http://wiremock.org/ - https://github.com/tomakehurst/saboteur
- 61. @chbatey Questions? Thanks for listening! Questionslater? @chbatey christopher.batey@gmail.com http://christopher-batey.blogspot.co.uk/