Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SoundCloud's Toolbox for Microservices

1,154 views

Published on

At SoundCloud we managed to break away from the monolith while delivering key business features. Our journey towards a microservices architecture has not been a straightforward one. We experimented a lot to reach the set of tools and technologies that we use today. We changed how we build our applications. We introduced specific apis for our mobile and web clients. We call them BFFs (backend for the frontend). They became the central piece of SoundCloud’s architecture. We rethought how we monitor our services. We created a service registry for knowledge sharing. While making all these changes, we benefited from the learnings of our peer companies. This talk will share our learnings from this journey: what worked for us and what we moved away from.

Published in: Software
  • Be the first to comment

SoundCloud's Toolbox for Microservices

  1. 1. SoundCloud’s Toolbox for Microservices
  2. 2. SoundCloud is the largest online audio distribution platform
  3. 3. 12 HOURS OF AUDIO every minute
  4. 4. Bora Tunca Joined SoundCloud 2.5 years ago soundcloud.com front-end client specific apis core-engineering @grandbora
  5. 5. Microservices 0 20 40 60 80 100 120 2013 2014 2015 2016
  6. 6. Architecture Productivity Monitoring
  7. 7. FINAGLE
  8. 8. https://twitter.github.io/finagle/
  9. 9. Finagle Future Services Filters https://monkey.org/~marius/funsrv.pdf Your Server as a Function
  10. 10. Service Req Future[Rep] Req => Future[Rep] Service
  11. 11. Service Req Future[Rep]ReqA Future[RepA] (ReqA, Service) => Future[RepA] Filter Filter
  12. 12. Service Req Future[Rep]ReqA Future[RepA] (ReqA, Service) => Future[RepA] Filter Filter + Service
  13. 13. val service = new ExceptionFilter andThen new HeaderFilter andThen new HttpTimeoutFilter(Duration(1, TimeUnit.SECONDS)) andThen new RestService()
  14. 14. Toolbox Finagle
  15. 15. CLIENT SPECIFIC APIS
  16. 16. MOTHERSHIP Public API messages likes
  17. 17. Api-v2 Api-mobile Api-embedded Public API
  18. 18. Api-v2 Api-mobile Api-embedded Public API messages MOTHERSHIP likes search stats images
  19. 19. Backend For Fronted
  20. 20. Toolbox Client Specific Apis Finagle
  21. 21. STRANGLER PATTERN
  22. 22. Api-v2 Api-mobile Api-embedded Public API messages MOTHERSHIP likes search stats images
  23. 23. Api-v2 Api-mobile Api-embedded Public API messages MOTHERSHIP likes search stats images
  24. 24. Api-v2 Api-mobile Api-embedded Public API messages MOTHERSHIP likes search stats images
  25. 25. MOTHERSHIP Legacy Public API
  26. 26. MOTHERSHIP Legacy Public API Public API Strangler
  27. 27. MOTHERSHIP Legacy Public API Public API Strangler messages likes search
  28. 28. MOTHERSHIP Legacy Public API Public API Strangler messages likes search stats images
  29. 29. MOTHERSHIP Legacy Public API Public API Strangler messages likes search stats images
  30. 30. Toolbox Strangler Pattern Client Specific Apis Finagle
  31. 31. CRITICAL PATH
  32. 32. BFF
  33. 33. BFF authentication geo location features 1
  34. 34. BFF messages MOTHERSHIP likes search stats images 2 authentication geo location features 1
  35. 35. messages MOTHERSHIP BFF likes search stats images authentication geo location features
  36. 36. messages MOTHERSHIP BFF likes search stats images authentication geo location features
  37. 37. messages MOTHERSHIP BFF likes search stats images authentication geo location features
  38. 38. messages MOTHERSHIP BFF likes search stats images authentication geo location features
  39. 39. messages MOTHERSHIP BFF likes search stats images authentication geo location features
  40. 40. BFF messages MOTHERSHIP likes search stats images 2 authentication geo location features 1
  41. 41. Microservices
  42. 42. BFFs Microservices
  43. 43. Client ApplicationsBFFs Microservices
  44. 44. messages MOTHERSHIP BFF likes search stats images authentication geo location features BFF BFFBFF BFFBFF
  45. 45. messages MOTHERSHIP BFF likes search stats images authentication geo location features BFF BFFBFF BFFBFF Critical Path
  46. 46. On critical path…
  47. 47. Handle failures gracefully On critical path…
  48. 48. Handle failures gracefully Be ready for traffic bursts On critical path…
  49. 49. Toolbox Critical Path Client Specific Apis Strangler Pattern Finagle
  50. 50. SERVICE LAYERING
  51. 51. Public API messages likes MOTHERSHIP
  52. 52. messages MOTHERSHIP BFF likes search stats images
  53. 53. messages MOTHERSHIP BFF likes search stats images BFF
  54. 54. messages MOTHERSHIP BFF likes search stats images BFF
  55. 55. messages MOTHERSHIP BFF likes search stats images tracks BFF
  56. 56. images userstracks Foundation Layer
  57. 57. images userstracks Foundation Layer playlistsstream profiles Value Added Layer
  58. 58. images userstracks Foundation Layer playlistsstream profiles Value Added Layer
  59. 59. images userstracks Foundation Layer playlistsstream profiles Value Added Layer
  60. 60. images userstracks Foundation Layer playlistsstream profiles Value Added Layer BFF BFF Edge Layer
  61. 61. images userstracks Foundation Layer playlistsstream profiles Value Added Layer BFF BFF Edge Layer
  62. 62. images userstracks Foundation Layer playlistsstream profiles Value Added Layer MOTHERSHIP BFF BFF Edge Layer
  63. 63. Toolbox Service Layering Client Specific Apis Strangler Pattern Critical Path Finagle
  64. 64. HUMANE REGISTRY
  65. 65. Microservices 0 20 40 60 80 100 120 2013 2014 2015 2016
  66. 66. http://martinfowler.com/bliki/HumaneRegistry.html
  67. 67. Humane Registry
  68. 68. Oriented around people Humane Registry
  69. 69. Oriented around people Don't rely on people to stay up to date Humane Registry
  70. 70. Oriented around people Don't rely on people to stay up to date Expect humans to read and contribute Humane Registry
  71. 71. SERVICES DIRECTORY
  72. 72. Github PagerDuty Distributed Tracing Prometheus Deployment Pipeline Eng-doc Live Configs
  73. 73. Toolbox Humane Registry Client Specific Apis Strangler Pattern Critical Path Service Layering Finagle
  74. 74. PROMETHEUS
  75. 75. Prometheus Scalable data collection Operational simplicity Multi-dimensional data Powerful query language https://developers.soundcloud.com/blog/prometheus-monitoring-at-soundcloud http://prometheus.io/
  76. 76. Prometheus My Service
  77. 77. Prometheus My Service http://instance-url/metrics
  78. 78. Prometheus My Service http://instance-url/metrics jvm client
  79. 79. Prometheus My Service jvm client My Service jvm client My Service jvm client My Service jvm client My Service jvm client My Service jvm client My Service jvm client My Service jvm client My Service jvm client My Service jvm client
  80. 80. val requestCount = telemetry.counter("outgoing_http_requests_total", "A counter for the HTTP requests", "service", "method", "status", "statusClass") send requestCount .labels( serviceName, request.getMethod, response.getStatus, response.getStatusClass ).inc()
  81. 81. val requestCount = telemetry.counter("outgoing_http_requests_total", "A counter for the HTTP requests", "service", "method", "status", "statusClass") send requestCount .labels( serviceName, request.getMethod, response.getStatus, response.getStatusClass ).inc() query SUM(rate(outgoing_http_requests_total{statusClass="5xx"}[10m])) by (service, method, status)
  82. 82. Toolbox Client Specific Apis Strangler Pattern Critical Path Service Layering Humane Registry Prometheus Finagle
  83. 83. SERVICE LEVEL OBJECTIVES
  84. 84. ERROR_RATE=0.0001
  85. 85. ERROR_RATE=0.0001 P99_LATENCY=0.500
  86. 86. ERROR_RATE=0.0001 P99_LATENCY=0.500 AVAILABILITY_TARGET=0.99999
  87. 87. ERROR_RATE=0.0001 P99_LATENCY=0.500 AVAILABILITY_TARGET=0.99999
  88. 88. Toolbox Service Level Objectives Prometheus Client Specific Apis Strangler Pattern Critical Path Service Layering Humane Registry Finagle
  89. 89. RECAP
  90. 90. Toolbox Service Level Objectives Prometheus Client Specific Apis Strangler Pattern Critical Path Service Layering Humane Registry Finagle
  91. 91. backstage.soundcloud.com @pcalcado - How we ended up with microservices @starcoffe - Why SoundCloud stopped using its own public API InfoQ - Moving from a Monolith to Microservices at SoundCloud @SoundCloudDev InfoQ - Microservices Evolution at Soundcloud Sam Newman - Backends for Frontends
  92. 92. bora@soundcloud.com https://soundcloud.com/jobs Thank you! @grandbora

×