Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Go, memcached,
microservices
2016/09/27 cache_night
@mosa_siru
@mosa_siru ( )
•
• 2
@mosa_siru as engineer
• DeNA
• Gunosy
• CTO
1.
2. cache
3. cache
4. microservices
Cache is hard
• Cache Invalidation
• There are only two hard things in Computer Science: cache
invalidation and naming things. -- Phil K...
•
•
• DB
Cache Invalidation
Cache Invalidation
• (DB)


Cache Invalidation
• DB
memcached
• ( API
memcached ) TTL
Cache Invalidation
• Invalidation
•
• 

ID
thundering herd
Thundering Herd
• expire
(DB)
• DB
•


• 2016/06 KDDI 

•
•
•
•
•
•
•
•
•
• Push 

•
•
• 2
get
(1)
1. local memd 2.
(2)
1. local memd
2.
remote
4. local set
3.
(3)
1. local memd
2.
remote
6. local set
4.
3. DB
5. remote set
• TTL (5s-60s) 

TTL (60s-1h) invalidation
updatedelete
• local memd
• DB thundering herd
• local memd TTL invalidation


local memd
•
insert invalidation
insertdelete
• memcached
• redis
• go process cache
• go groupcache
• File DB
Redis
•
•
• 

•
Go process cache
•
•
• local, remote
• remote
Go groupcache
• google memd
local/remote
•
• expire …
File DB
•
File DB(LevelDB )
•
•
•
memcached
•
•
• AWS Elasticache
• JSON msgpack
go memcached client
go memcached client
•
1. bradfitz/gomemcache
2. douban/golibmc
1. gomemcache Pros
• (600star)
• go
1. gomemcache Cons
• 1 issue, PR
• memcached
• SetMulti,
DeleteMulti
1. gomemcache Cons
• TCP 2
(2 )
• 2 configurable PR
•
memcached
•
2. golibmc Pros
• C
• SetMulti
• ”noreply"
(TCP 1 )
• issue PR 2
2. golibmc Cons
• Python libmc (320start)
golibmc
• C
2. golibmc Cons
• goroutine 1Client 1
•
• Client
gomemcache
• golibmc
•
• issue PR 2
• SetMulti
•
•
go
1. local memd
2.
remote
6. local set
4.
3. DB
5. remote set
1
get multi
get multi
select …

where id in
…
set multi
set multi
ID( ) 

get, set 1
• https://github.com/mosasiru/memd
•
• GetOrSet
• GetOrSetMulti
GetOrSet
• local/remote memd
GetOrSetMulti
• local/remote memd
GetMulti/SetMulti
mosasiru/memd
• 

ver0.1 … patch is welcome!!!!
microservices 

microservices
•
• API
• DB
• AWS security group/IAM
•
microservices
• API
• API
microservices
• cache
• invalidation
invalidation
• (DB) write read 

• cache invalidation
write read
invalidation
• cache read/write 1
• cache key 1
• consistent hash 1
write
DELETE /hoge/cache
delete
Worker API
…
CDN
• CDN
• URL invalidation 

https://...?t=1474969378
Cache is hard…
Gunosy
@mosa_siru
Go, memcached, microservices
Go, memcached, microservices
Go, memcached, microservices
Upcoming SlideShare
Loading in …5
×

Go, memcached, microservices

8,320 views

Published on

2016/09/27 Cache Night http://gunosy-beer.connpass.com/event/40945/
で発表した、 Goとmemcachedとマイクロサービスの話です。キャッシュ戦略やライブラリ選定、紹介の話がメインです。

Published in: Technology
  • Be the first to comment

Go, memcached, microservices

  1. 1. Go, memcached, microservices 2016/09/27 cache_night @mosa_siru
  2. 2. @mosa_siru ( ) • • 2
  3. 3. @mosa_siru as engineer • DeNA • Gunosy • CTO
  4. 4. 1. 2. cache 3. cache 4. microservices
  5. 5. Cache is hard
  6. 6. • Cache Invalidation • There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton • • TTL • • thundering herd
  7. 7. • • • DB
  8. 8. Cache Invalidation
  9. 9. Cache Invalidation • (DB) 

  10. 10. Cache Invalidation • DB memcached • ( API memcached ) TTL
  11. 11. Cache Invalidation • Invalidation • • 
 ID
  12. 12. thundering herd
  13. 13. Thundering Herd • expire (DB) • DB •
  14. 14.
  15. 15. • 2016/06 KDDI 
 • •
  16. 16. • • • • • •
  17. 17. • • Push 
 • •
  18. 18. • 2 get
  19. 19. (1) 1. local memd 2.
  20. 20. (2) 1. local memd 2. remote 4. local set 3.
  21. 21. (3) 1. local memd 2. remote 6. local set 4. 3. DB 5. remote set
  22. 22. • TTL (5s-60s) 
 TTL (60s-1h) invalidation updatedelete
  23. 23. • local memd • DB thundering herd • local memd TTL invalidation 
 local memd
  24. 24. • insert invalidation insertdelete
  25. 25. • memcached • redis • go process cache • go groupcache • File DB
  26. 26. Redis • • • 
 •
  27. 27. Go process cache • • • local, remote • remote
  28. 28. Go groupcache • google memd local/remote • • expire …
  29. 29. File DB • File DB(LevelDB ) • • •
  30. 30. memcached • • • AWS Elasticache • JSON msgpack
  31. 31. go memcached client
  32. 32. go memcached client • 1. bradfitz/gomemcache 2. douban/golibmc
  33. 33. 1. gomemcache Pros • (600star) • go
  34. 34. 1. gomemcache Cons • 1 issue, PR • memcached • SetMulti, DeleteMulti
  35. 35. 1. gomemcache Cons • TCP 2 (2 ) • 2 configurable PR • memcached •
  36. 36. 2. golibmc Pros • C • SetMulti • ”noreply" (TCP 1 ) • issue PR 2
  37. 37. 2. golibmc Cons • Python libmc (320start) golibmc • C
  38. 38. 2. golibmc Cons • goroutine 1Client 1 • • Client gomemcache
  39. 39. • golibmc • • issue PR 2 • SetMulti • •
  40. 40. go
  41. 41. 1. local memd 2. remote 6. local set 4. 3. DB 5. remote set
  42. 42. 1 get multi get multi select …
 where id in … set multi set multi ID( ) 
 get, set 1
  43. 43. • https://github.com/mosasiru/memd • • GetOrSet • GetOrSetMulti
  44. 44. GetOrSet • local/remote memd
  45. 45. GetOrSetMulti • local/remote memd GetMulti/SetMulti
  46. 46. mosasiru/memd • 
 ver0.1 … patch is welcome!!!!
  47. 47. microservices 

  48. 48. microservices • • API • DB • AWS security group/IAM •
  49. 49. microservices • API • API
  50. 50. microservices • cache • invalidation
  51. 51. invalidation • (DB) write read 
 • cache invalidation write read
  52. 52. invalidation • cache read/write 1 • cache key 1 • consistent hash 1 write DELETE /hoge/cache delete Worker API …
  53. 53. CDN • CDN • URL invalidation 
 https://...?t=1474969378
  54. 54. Cache is hard…
  55. 55. Gunosy @mosa_siru

×