cache
 @TimYang
Agenda

•   Feed
•          Cache
•
•
•   ,   status feed

•


•
Feed

•                 feed

    • Push(       )

    • Pull(   )

    •
Push
•     feed

    • Inbox:
    • Outbox:
•                    inbox(   )

•                Inbox( )

• Offline computati...
Push

•
•


•
Pull


•                 outbox( )

•                       Inbox(   )

• Online computation
Pull


•
•
•
•


•
Cache
memory is the new disk,
and disk is the new tape.
for "real-time" web applications,
and systems that require massive...
Cache
cache
          Weibo cache arch

  Inbox       hot cache




Outbox       Vector cache    Archive cache




 Social      ...
Inbox

•       “         ”

• cache      id list

•                    (hot cache)

•                    get set

• redis ...
Outbox

•
•     id list(e.g. 100)

•   id LRU
Social Graph cache
• Following ids
• Followers
  •
  •
  •           delete all

•           followers list
Content cache

•           cache

•
    •
•
    • Open API      xml, json...
Cache


•
•
Update Workflow


                  Update status




Content cache    Hot Inbox Vector   Outbox vector



Content cache
 r...
•   outbox hot vector

•   followers list

•   inbox
feed
                Home timeline Workflow


                     home_timeline


                      aggregator



    ...
feed

•   inbox cache

•
•         ,   following

•   id list         feed
cache

•
• hot keys
•
• mutex
•                     Content cache

• multi get n feed(n = items/ , e.g. 50)
• cache = n * (feed +                    ,
 ...
• 1,000    800Mbps

•1        8Gbps
•       1G       300~400Mbps

•
    •        local cache

    •
    •
hot keys
• content cache of
• create local cache
  1. get user_yaochen_local
  2. get user_yaochen
       1. set user_yaoc...
cache
•                      key
      memcache

    •
    • LRU cache,
    • Memory storage
    •
mutex
•       content item   cache

    •           db

    •                          db

• key_mutex                    ...
•
    •
    •   less is more

    •
•
    • code review
    •
• Q&A
• weihua2@staff.sina.com.cn
Upcoming SlideShare
Loading in …5
×

Timyang新浪微博设计谈

1,572
-1

Published on

Published in: Technology

Timyang新浪微博设计谈

  1. 1. cache @TimYang
  2. 2. Agenda • Feed • Cache •
  3. 3. • • , status feed • •
  4. 4. Feed • feed • Push( ) • Pull( ) •
  5. 5. Push • feed • Inbox: • Outbox: • inbox( ) • Inbox( ) • Offline computation
  6. 6. Push • • •
  7. 7. Pull • outbox( ) • Inbox( ) • Online computation
  8. 8. Pull • •
  9. 9. • • •
  10. 10. Cache memory is the new disk, and disk is the new tape. for "real-time" web applications, and systems that require massive scalability - Jim Gray
  11. 11. Cache
  12. 12. cache Weibo cache arch Inbox hot cache Outbox Vector cache Archive cache Social Following Followers users Graph Content Hot cache Total
  13. 13. Inbox • “ ” • cache id list • (hot cache) • get set • redis list/set
  14. 14. Outbox • • id list(e.g. 100) • id LRU
  15. 15. Social Graph cache • Following ids • Followers • • • delete all • followers list
  16. 16. Content cache • cache • • • • Open API xml, json...
  17. 17. Cache • •
  18. 18. Update Workflow Update status Content cache Hot Inbox Vector Outbox vector Content cache replication
  19. 19. • outbox hot vector • followers list • inbox
  20. 20. feed Home timeline Workflow home_timeline aggregator Content hot Inbox cache Outbox Vector cache Inbox archive Inbox archive Content cache
  21. 21. feed • inbox cache • • , following • id list feed
  22. 22. cache • • hot keys • • mutex
  23. 23. • Content cache • multi get n feed(n = items/ , e.g. 50) • cache = n * (feed + , e.g. 2k) • 1,000 / • = 50 * 2k * 1,000 / sec = 100MB
  24. 24. • 1,000 800Mbps •1 8Gbps
  25. 25. • 1G 300~400Mbps • • local cache • •
  26. 26. hot keys • content cache of • create local cache 1. get user_yaochen_local 2. get user_yaochen 1. set user_yaochen_local:value 3. delete all
  27. 27. cache • key memcache • • LRU cache, • Memory storage •
  28. 28. mutex • content item cache • db • db • key_mutex key • mutex • mutex
  29. 29. • • • less is more •
  30. 30. • • code review •
  31. 31. • Q&A • weihua2@staff.sina.com.cn

×