This document discusses using caching to improve performance for web applications. It provides three key points: 1. Cache stores data to serve future requests faster by avoiding accessing the database. It is commonly used for things like login information, page content, and API responses. 2. There are different cache architectures like memcached and Redis that support storing data in-memory for fast retrieval. Factors like data size, update frequency, and consistency requirements determine the appropriate caching strategy. 3. Real-world examples show how companies like Facebook, Twitter, and Wonga use caching extensively to handle high volumes of traffic and database requests. Caching is critical to scaling applications in a cost-effective way.

1. 초보자를 위한 분산 캐시 이야기
charsyam@naver.com
http://charsyam.wordpress.com

2. 잉여서버개발자 In NHN
관심분야!!!
Cloud, Big Data
특기!!!
발표 날로 먹기!!!

3. What is Cache?

4. Cache is a component
that transparently stores data so
that future requests for that
data can be served faster.
In Wikipedia

5. Cache 는 나중에 요청올 결과를
미리 저장해두었다가 빠르게
서비스해 주는 것

6. Lots of
Data

7. Cache

8. Lots of
Data
Cache

9. CPU Cache

10. Browser Cache

11. Why is Cache?

13. Use Case: Login

14. Use Case: Login
Common Case

15. Use Case: Login
Common Case
Read From DB
Select * from Users where id=‘charsyam’;

16. 일반적인 DB구성
Master
REPLICATION/FailOver
Slave

17. 일반적인 DB구성
모든 Traffic은 Master 가 처리 Slave는 장애 대비
Master
REPLICATION/FailOver
Slave

18. 이러니깐 부하가!!!

19. 선택의 기로!!!

20. Scale UP
Vs
Scale OUT

21. Scale
UP

22. 초당 1000 TPS

23. 초당 3000 TPS
3배 처리 가능한 서버를 투입

24. Scale
OUT

25. 초당 1000 TPS

26. 초당 2000 TPS

27. 초당 3000 TPS

28. Scale Out을 선택!!!

29. Scale Out을 선택!!!
돈만 많으면 Scale Up도 좋습니다.

30. Request 분석
읽기 70%, 쓰기 30%

31. 분석 결론
읽기 70%,
읽기를 분산하자!!!

32. One Write
Master
+
Multi Read Slave

33. Client
ONLY
WRITE Only READ
Master
REPLICATION
Slave Slave Slave

34. Eventual
Consistency

35. Ex) Replication
Master
REPLICATION
Slave에 부하가 있거나, 다른 이유로, 복제가
느려질 수 있다. 그러나 언젠가는 같아진다.
Slave

36. Login 정보 중에 잠시
라도 서로 다르면 안되
는 경우는?

37. Login 정보 중에 잠시
라도 서로 다르면 안되
는 경우는?
=> 다시 처음으로!!!

38. 초반에는 행복했습니다.

39. Client
Master
REPLICATION
Slave Slave Slave Slave Slave

40. 부하가 더 커지니!!!

41. Client
Master REPLICATION
Slave Slave Slave Slave Slave Slave
Slave Slave Slave Slave Slave Slave

42. 성능 향상이 미비함!!!
WHY?

43. 머신의 I/O는 Zero섬

46. Partitioning

47. Scalable Partitioning
Client
PART 1 PART 2
Web Server Web Server
DBMS DBMS

48. Paritioning
성능

49. Paritioning
성능
관리이슈

50. Paritioning
성능
관리이슈
비용

51. 간단한 해결책

52. DB 서버 Disk는 SSD
메모리도 데이터보다 많이

53. DB 서버 Disk는 SSD
메모리도 데이터보다 많이
=> 돈돈돈!!!

54. Why is Cache?

55. Use Case: Login

56. Use Case: Login
Read From Cache
Get charsyam

57. Use Case: Login
Apply Cache For Read

58. General Cache Layer
Storage Layer
Cache
Application READ
WRITE
Server UPDATE
WRITE
DBMS

59. Type 1: 1 Key – N Items
KEY Value
Profile:User_ID Profile
- LastLoginTime
- UserName
- Host
- Name

60. Type 2: 1 Key – 1 Item
KEY Value
name:User_ID
UserName
LastLoginTime:User_ID LastLoginTime

61. Pros And Cons
Type 1: 1 Key – N Items
Pros: Just 1 get.

62. Pros And Cons
Type 1: 1 Key – N Items
Pros: Just 1 get.
Cons: if 1 item is
changed. Need Cache
Update
And Race Condition

63. Pros And Cons
Type 2: 1 Key – 1 Item
Pros: if 1 item is
changed, just change
that item.

64. Pros And Cons
Type 2: 1 Key – 1 Item
Pros: if 1 item is
changed, just change
that item.
Cons: Some Items can
be removed

65. 변화하는 데이터
변화하지 않는 데이터

66. 변화하는 데이터
Divide
변화하지 않는 데이터

67. Don’t Try Update After didn’t Read DB
Value
Profile
- LastLoginTime
- UserName
- Host
- Name

68. Don’t Try Update After didn’t Read DB
Value EVENT: Update Only LastLoginTime
Profile
- LastLoginTime
- UserName
- Host
- Name

69. Don’t Try Update After didn’t Read DB
Value EVENT: Update Only LastLoginTime
Profile Read Cache: User Profile
- LastLoginTime
- UserName
- Host
- Name

70. Don’t Try Update After didn’t Read DB
Value EVENT: Update Only LastLoginTime
Profile Read Cache: User Profile
- LastLoginTime
- UserName
Update Data & DB
- Host
- Name
Just Save Cache

71. Don’t Try Update After didn’t Read DB
Value EVENT: Update Only LastLoginTime
Profile Read Cache: User Profile
- LastLoginTime
- UserName
Update Data & DB
- Host
- Name
Just Save Cache
It Makes Race Condition

72. Need
Global Lock

73. Need
Global Lock
=> 오늘은 PASS!!!

74. How to Test

75. Using Real Data
100,000 Request
In 25 million User ID
Reproduct From Log

76. Test Result
Memcache VS Mysql
136 vs 1613
seconds

77. Result of
Applying Cache

78. Select Query

79. CPU utility

80. 균등한 속도!!!
부하 감소!!!

81. What is Hard?

82. Key Value – SYNC Easy
and
Fail!!! 1. Update To DB
2. Fail To Transaction

83. Key Value – SYNC HARD
and
1. Update To DB
Fail!!! 2. Update to Cache

84. Key Value – How
and
1. Update To DB
Fail!!! 2. Update to Cache
RETRY BATCH DELETE

85. Data!
The most important thing

86. If data is not important
Cache Updating is not important
Login Count
Last Login Time
ETC

87. BUT!

88. If data is important
Cache Updating is important
Server Address
Data Path
ETC

89. HOW!

90. RETRY!
Retry, Retry, Retry
Solve Over 9x%.
Or Delete Cache!!!

91. Batch!
Queuing Service

92. Error Log
Queue Batch
Error Log Processor
Error Log
Error Log Cache
Error Log Server

93. Error Log
Queue Batch
Error Log Processor
Error Log Error Log
Error Log Cache
Server

94. Error Log
Queue Batch UPDATE
Error Log Processor
Error Log
Error Log Cache
Server
Error Log

95. Caches
Memcache
Redis

96. Memcache
Atomic Operation

97. Memcache
Atomic Operation
Key:Value

98. Memcache
Atomic Operation
Key:Value
Single Thread

99. Memcache
Processing
Over 100,000 TPS

100. Memcache
Max size of item
1 MB

101. Memcache
LRU,
ExpireTime

102. Redis
Key:Value

103. Redis
ExpireTime
Replication
Snapshot

104. Redis
Key:Value
Collection
Hash List Sorted Set

105. 주의 사항

106. Item Size
1~XXX KB,
Item 사이즈는 적을수록
좋다.

107. Cache
In
Global Service

108. Memcached In Facebook
• Facebook and Google and Many Companies
• Facebook
– 하루 Login 5억명(한달에 8억명)(최신, 밑에는 작년 2010/04자료)
– 활성 사용자 7,000만
– 사용자 증가 비율 4일에 100만명
– Web 서버 10,000 대, Web Request 초당 2000만번
– Memcached 서버 805대 -> 15TB, HitRate: 95%
– Mysq server 1,800 대 Master/Slave(각각, 900대)
• Mem: 25TB, SQL Query 초당 50만번

109. Cache In Twitter(Old)
API WEB
Page Cache
DB DB

110. Cache In Twitter(NEW)
API WEB
Page Cache
Fragment Cache
Row Cache
Vector Cache
DB DB DB

111. Redis In Wonga
Using Redis for DataStore
Write/Read

112. Redis In Wonga
Flash Client
Ruby Backend

113. NoCache In Wonga
1 million daily Users
200 million daily HTTP Requests

114. NoCache In Wonga
1 million daily Users
200 million daily HTTP Requests
100,000 DB Operation Per Sec
40,000 DB update Per Sec

115. NoCache In Wonga

116. First Scale Out

117. First Setting – 3 Month

118. More Traffic

119. MySQL hiccups – DB Problems Began

120. Analysis About DBMS
ActiveRecord’s status Check caused
20% extra DB
60% of All Updates were done
on ‘tiles’ table

121. Applying Sharding 2xD

122. Result of Applying Sharding 2xD

123. Doubling MySQL

124. Result of Doubling MySQL

125. 50,000 TPS on EC2

126. Wonga Choose Redis!
But some failure
=> 오늘은 PASS!!!

128. Result!!!

129. Consistent Hashing

130. Origin
K = 10000 N=5
Server
Server
User Request Proxy Server
Server
Server

131. FAIL : Redistribution about 2000 Users
K = 10000 N=4
Server
Server
User Request Proxy Server
Server
Server

132. RECOVER: Redistribution about 2500 Users
K = 10000 N=5
Server
Server
User Request Proxy Server
Server
Server

133. Add A,B,C Server
A

134. Add A,B,C Server
A
B

135. Add A,B,C Server
A
B
C

136. Add Item 1
A
1
B
C

137. Add Item 2
A
1
B
2
C

138. Add Item 3,4,5
A
3
1
4
B
5
2
C

139. Fail!! B Server
A
3
4
B
5
2
C

140. Add Item 1 Again -> Allocated C Server
A
3
1
4
B
5
2
C

141. Recover B Server -> Add Item 1
A
3
1
4
B
1
5
2
C

142. Thank You!

143. Real Implementation
A
C+2 C+3
B+3 A+1
A+4 B
B+2
C+1 A+2
B+1 A+3
C