Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

of

Architecture Sustaining LINE Sticker services Slide 1 Architecture Sustaining LINE Sticker services Slide 2 Architecture Sustaining LINE Sticker services Slide 3 Architecture Sustaining LINE Sticker services Slide 4 Architecture Sustaining LINE Sticker services Slide 5 Architecture Sustaining LINE Sticker services Slide 6 Architecture Sustaining LINE Sticker services Slide 7 Architecture Sustaining LINE Sticker services Slide 8 Architecture Sustaining LINE Sticker services Slide 9 Architecture Sustaining LINE Sticker services Slide 10 Architecture Sustaining LINE Sticker services Slide 11 Architecture Sustaining LINE Sticker services Slide 12 Architecture Sustaining LINE Sticker services Slide 13 Architecture Sustaining LINE Sticker services Slide 14 Architecture Sustaining LINE Sticker services Slide 15 Architecture Sustaining LINE Sticker services Slide 16 Architecture Sustaining LINE Sticker services Slide 17 Architecture Sustaining LINE Sticker services Slide 18 Architecture Sustaining LINE Sticker services Slide 19 Architecture Sustaining LINE Sticker services Slide 20
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

6 Likes

Share

Download to read offline

Architecture Sustaining LINE Sticker services

Download to read offline

Masahiro Ide (LINE Corporation)
LINE Vietnam Opening Day, March 31st 2018

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Architecture Sustaining LINE Sticker services

  1. 1. Architecture Sustaining LINE Sticker Services Masahiro IDE, LINE CORP
  2. 2. About me • Masahiro Ide • Software Engineer of LINE server development • Especially StickerShop backend server • Joined: Oct, 2015 • Contribute to Armeria • Our open sourced asynchronous RPC/REST library • https://github.com/line/armeria
  3. 3. Agenda • Sticker Shop Server • History • Our Architecture • Implementations
  4. 4. What is Sticker Shop?
  5. 5. Sticker Shop … in 2015 One of the biggest monolithic Java application in LINE • Purchasing, Serving, Ranking stickers in 1 app • Many pains, troubles • Takes more than 3 hours to restart app
  6. 6. Sticker Shop in 2018 Now shift to microservice way for scalability • 1 million stickers • 30 services • Purchasing, Serving, Ranking stickers, etc. • 70K requests/sec in peek time
  7. 7. StickerShop Architecture Overview (Simplified) Central Dogma – Configuration Repository Service Talk server ElasticSearch Client ReverseProxy Thrift/HTTP2 Thrift/HTTP2 Redis MySQL Sticker capability server Shop server Search FE Thrift/HTTP2
  8. 8. Implementation • What we need is… • Performance • Prepared to faults • Observability • Stable feature rollout
  9. 9. Central Dogma • Repository service for textual configurations • JSON, YAML, XML … • Highly available • multi-master, eventually consistency • Version controlled • https://line.github.io/centraldogma/
  10. 10. Armeria - Our RPC layer • Asynchronous RPC/REST library • built on top of Java 8, Netty, HTTP/2, Thrift and gRPC • Take care common functionality for microservice 1. Load balancing 2. Circuit Breaker / Retry / Throttling 3. Tracing (Zipkin) / Monitoring integration • etc. https://line.github.io/armeria/
  11. 11. 1. Client-side load balancing • No proxy load balancers • Proxy would be inefficient when considering request heavy services • CentralDogma as a service discovery • All services register a name and a list of machines in CentralDogma • All clients monitor CentralDogma and balance requests across machines LB Client Server Client Server Server Proxy load balancing
  12. 12. 1. Client-side load balancing • No proxy load balancers • Proxy would be inefficient when considering request heavy services • CentralDogma as a service discovery • All services register a name and a list of machines in CentralDogma • All clients monitor CentralDogma and balance requests across machines Central Dogma Client Server Client Server Server
  13. 13. 2. Circuit Breaker - Avoid cascading failure • Server die suddenly • e.g. Hardware failure • Upstream cannot proceed part of requests, in worst case • Blocks all of application thread • LINE die Service A Service C Service B
  14. 14. 2. Circuit Breaker - Avoid cascading failure • Circuit Breaker pattern* 1. Monitors failures 2. Return error immediately without RPC once the failures reach a threshold Service A Service C Service B CircuitBreaker CircuitBreaker * https://martinfowler.com/bliki/CircuitBreaker.html
  15. 15. 3. Tracing • Hard to point out the slowness/bug in microservice • No one provides complete picture of system performance Talk server ElasticSearch Reverse Proxy Redis MySQL Shop server Search FE T=start T=end
  16. 16. 3. Tracing • Solution: Distributed tracing with Zipkin
  17. 17. Rollout features using CentralDogma (CD) • Rollout to production within 10 minutes of being changed • Deploy a YAML file to all our production servers via CD • Record changes in CD commit log
  18. 18. Rollout features using CentralDogma • Rollout to production within 10 minutes of being changed • Deploy a YAML file to all our production servers via CD • Record changes in CD commit log 30%CacheV2 rollout: Central Dogma Service Service Service commit YAMLPull & Reload config Developer
  19. 19. Conclusion • Our system is built on top of our OSSs • Armeria, CentralDogma, and more • … are still evolving • Trouble may happen • Make a well trouble-prepared system • Let's do a smooth release
  20. 20. Q & A
  • KarenGarcia404420

    Nov. 28, 2021
  • ManhPhan23

    Jun. 26, 2021
  • eniton

    Mar. 17, 2019
  • taitrantuan

    Apr. 7, 2018
  • tucq88

    Apr. 2, 2018
  • luanxitrum

    Apr. 2, 2018

Masahiro Ide (LINE Corporation) LINE Vietnam Opening Day, March 31st 2018

Views

Total views

1,425

On Slideshare

0

From embeds

0

Number of embeds

2

Actions

Downloads

34

Shares

0

Comments

0

Likes

6

×