Sina Weibo is the most popular Microblogging platform in China. It has more than 100 million user and tens of millions of daily updates. This slide explains the performance challenges in Weibo platform.

1. Build High Performance
Weibo System
@TimYang

2. •
• 140
•
•

3. • QCon 2010
• WDC 2010

4. Agenda
3 4
2 1

5. Part 1

6. ( )

7. MySQL
•
•

8. • hash
•
• user_timeline index

9. •
•
•
• 5
•

10. MySQL + cache
•
•
• cache
•
•

11. NoSQL
“Databases are specializing – the “one
size fits all” approach no longer applies.”
MongoDB

12. NoSQL?
•
• Redis
• MongoDB
•
• Cassandra

14. • MongoDB
• Redis
• HBase
• Cassandra

15. Redis -
•
• snapshot -
• vm -
• diskstore -
• aof -

16. Redis -
• string: key value redisObject 16 bytes/item
• list: 40 bytes / item
• hash: zipmap(<64)
• set/sorted set

17. Redis - Replication
• rdb
•

18. Redis -

19. Redis -
•
• failover
• list/set (optional)

20. • MySQL
• NoSQL MySQL cache

21. Part 2

22. ( )

23. → →

24. Web cache
cache
•

25. “Web json cache
json
( ) 2~5K
, xml 10k , protobuf
500 ”

26. RDBMS→
Key value (JSON)→
Protocol buffers(binary)

27. JSON
• DB
• Cache
• Message Queue
• API

28. PB
• Numeric: varint, from 1 byte
•
• Java, C++, Python...

29. •
• cache
• “192.168.0.1” → “0xc0a80000” → varint

30. Benchmark
Text
Text
( http://code.google.com/p/thrift-protobuf-compare/wiki/Benchmarking)

31. “We would like to provide public APIs that
accept protocol buffers as well as XML, both
because it is more efficient and because we're
just going to convert that XML to protocol
buffers on our end anyway.”
- Google

33. •
•
•

34. “ ”

35. • LAMP
• 1ms
•

36. • MQ stat
• MQ Processor stat

37. ?
“1 ”

38. • Timeline
•
•
• cache

39. 1.
2.
3.
4. cache master
5. replication

41. • 5

42. • 5
•

43. • 5
•
•

44. • 5
•
•
•

45. • RAM is the new disk
•

46. “Percona Server now both SQL and NOSQL
HandlerSocket,
100 rps 12cores/24threads and
380GB of RAM, mysql
Redis NoSQL
”

47. @jackbillow “ 1. PK lookup 2. cache
”
@kobe “ innodb
”
@TimYang “
InnoDB Adaptive Hash Indexes
hash ”

48. • NoSQL
• binlog Redis
• jbinlog https://github.com/tangfl/jbinlog

49. Part 3

50. app

51. •
•
• spam
•
•

52. Text

53. •
• GZIP
•

54. Part 4

56. “@ Zheng:“
” //@ :
@bian " " http://t.cn/h0k4r
(49)| (15)”
2 14 10:36 (22) | (10)

57. •
•

58. •
•
•
•

59. •
•
•
• cache
•

60. •
•

61. Timeline
Memcached MySQL
Redis

62. •
• RAM
•

63. Q&A
@TimYang