• Like

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

微博架构与平台安全

  • 47,035 views
Uploaded on

新浪微博平台与安全架构

新浪微博平台与安全架构

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Write a comment...
    Are you sure you want to
    Your message goes here
  • 新浪微博至今发言或评论要刷新点按多次才能发出,有时weibo.com都无法访问,这些都是scalable扩展性做得不是非常线性的现象所在。
    Are you sure you want to
    Your message goes here
  • 55
    Are you sure you want to
    Your message goes here
  • 为什么下来的是key
    Are you sure you want to
    Your message goes here
  • 新浪微博平台与安全架构
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
47,035
On Slideshare
0
From Embeds
0
Number of Embeds
18

Actions

Shares
Downloads
4,166
Comments
16
Likes
254

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide



  • 1 week

  • MPSS 大系统普遍原则
    一台服务器多个小的服务,而不是一台服务器一个大服务
    最高层次,虚拟化
  • MPSS 大系统普遍原则
    一台服务器多个小的服务,而不是一台服务器一个大服务
    最高层次,虚拟化
  • MPSS 大系统普遍原则
    一台服务器多个小的服务,而不是一台服务器一个大服务
    最高层次,虚拟化

  • 锁表:同时评论,加关注增多




  • 解决方法:
    1. 合并所有表(不是最佳)
    2. 二次索引,
  • 异步原因:发表需要入库,索引,统计,后台,计数等,周期较长
  • todo: 改进思路,增加伪代码


























































Transcript

  • 1. @TimYang
  • 2. 0 ~ 50,000,000 3
  • 3. 1 PHP MySQL Apache Linux
  • 4. • / • 1 / insert / select
  • 5. • LAMP • MySQL , MyISAM • MPSS (Multi-Port Single Server)
  • 6. • • • • •
  • 7. 2
  • 8. 2
  • 9. 2
  • 10. 2
  • 11. • • • key-value (NoSQL) •
  • 12. • • • MemcacheQ • stats queue
  • 13. • InnoDB • PHP libmemcached memcache •
  • 14. • • “ ” • • • MySQL •
  • 15. • • • CDN • •
  • 16. • Web • • API • • •
  • 17. “Break large complex systems down into many services... google.com search touches 100s of services (ads, web search, books, news, spelling correction...)” - Jeff Dean, Google Fellow
  • 18. → →
  • 19. 3
  • 20. 3
  • 21. 3
  • 22. 3
  • 23. 3
  • 24. • • feed cache • •
  • 25. • DB • • IDC
  • 26. • 50,000,000 • 3,000 / • 3,689,713 11 10
  • 27. • • • •
  • 28. Web cache
  • 29. • • • • 100 10 • 1000 100 •
  • 30. CPU IO
  • 31. “CPU L1 L2 L3 ” - @ ecug 2010 Erlang
  • 32. cache
  • 33. cache Inbox (home)
  • 34. cache Inbox (home) Outbox (Myfeed)
  • 35. cache Inbox (home) Outbox (Myfeed) Content
  • 36. • • • Amazon S3: 99.9% • Amazon EC2: 99.95% • Facebook: n/a
  • 37. • • • admission control... • , 7x24 • ,
  • 38. • curl / • / access log • non-200 / / exceptions •
  • 39. “Many services are written to alert operations on failure and to depend upon human intervention for recovery, about 20% of the time they will make mistakes. Designing for automation.” - James Hamilton,VP of Amazon
  • 40. • • • • •
  • 41. “System administration at Google usually have 1 week of "on call" duty, and the other 5 weeks are spent making improvements to make the on call portion more optimized, automated, and trouble- free” - Tom Limoncelli @ Everything Sysadmin Lumeta Corporation
  • 42. • • IDC •
  • 43. CDN
  • 44. • • • •
  • 45. • Master / Slave • master • Multi-Master • • • Paxos
  • 46. Multi-Master Web
  • 47. Yahoo! Message Broker
  • 48. “We use YMB for replication for 2 reasons. 1.YMB ensure msgs are not lost before they are applied to the db. 2.YMB is designed for wide-area replication.This isolates individual PNUTS clusters from dealing with update between regions” PNUTS:Yahoo!’s Hosted Data Serving Platform
  • 49. • API • Rest API • •
  • 50. • (Stream API) • • • •
  • 51. • 1 •
  • 52. Weibo Data Msg MQ processor Queue Push DB push deamon Other Stream Server consumer
  • 53. • • KISS Keep It Simple and Stupid • • batch insert • cursor read
  • 54. Remote cache (Social Graph, nicks) Local Cache {uid:nick} Push {uid: Followers} data Receiver Social Stream Graph Service Stream Engine buffer resume continuation Weibo Jetty Continuation Servlet http://server/req?uid=1&since_id=2 Stream apps
  • 55. • Stream Buffer • •
  • 56. • • • •
  • 57. Freq Realtime Post processing Content Offline Access log log analysis Activity log Abnormal Data Multi time monitor dimension data Weibo anti SPAM System
  • 58. apps API requests • Auth url mapping • AppKey Auth Rate limit & Permission • OAuth Service method • • Output formatter(xml,json) Service
  • 59. Join us! @TimYang