Building Zingme News Feed           System               Chau Nguyen Nhat Thanh                Senior Software Manager    ...
Agenda1) Introduction to News Feed2) ZingMe News Feed system history3) ZingMe News Feed system4) Some statistics5) Bonus6)...
Introduction to News Feed
Introduction to News Feed●   Update friends info●   Update Biz info●   Update VIP info●   Interaction to them by comment, ...
Introduction to News Feed
Introduction to News Feed
Introduction to News Feed
Introduction to News Feed●   Terms    ●   Social graph: Users in most social networking        sites are describable in te...
Introduction to News Feed●   What do we need?    ●   Someone does actions, his friend will see these        action in his ...
Introduction to News Feed●   Push model    ●   This method involves denormalizing the users activity        data and pushi...
ZingMe News Feed system history
ZingMe News Feed system history●   First version    ●   Using PHP for worker    ●   Using MySQL for feed item    ●   Using...
ZingMe News Feed system history●   Second version    ●   Still using PHP for worker    ●   Using Cassandra for feed item  ...
ZingMe News Feed system history●   Third version    ●   Moving to Java for better performance    ●   Still using Cassandra...
ZingMe News Feed system    (The current one :))
ZingMe News Feed system
ZingMe News Feed system●   Still using push model because of twitter public some    info related to this●   Not enough tec...
ZingMe News Feed system●   Feed Item    ●   UserId, ObjectId, Created date...    ●   Storage: home build based on Kyoto Ca...
ZingMe News Feed system●   Rate limit    ●   Prefilter Spam or auto tool based on rate of write request    ●   When hit li...
ZingMe News Feed system●   Gearman feed storage queue    ●   Very fast    ●   Support multi language client    ●   Some ti...
ZingMe News Feed system●   Feed Sync center    ●   Sync the new feed to the others such as:        –   Spam detection     ...
ZingMe News Feed system●   Feed Render worker    ●   The main and heavy job:        –   Get the feed item        –   Extra...
ZingMe News Feed system●   Feed Aggregate    ●   Get the feed index    ●   Get the rendered cache    ●   Return to the fro...
Some statistics
Some statistics●   ~15M actions / day●   10% Spam    ●   Gift receive    ●   Meaningless status●   Cache hit 98%●   ~80M r...
Bonus●   Twemcache    ●   From twitter    ●   Solve most problems with memcached    ●   Twemcache proxy●   Redis    ●   Re...
Question and Answer
Q&A●   What is the problem with followers?    ●   Do that with the trick    ●   Cheating the owner ;)
We are hiring!!!!!!!
Q&AContact info:         Chau Nguyen Nhat Thanh       thanhcnn@vng.com.vn       me.zing.vn/thanhcnn2000
Building zing me news feed system
Upcoming SlideShare
Loading in …5
×

Building zing me news feed system

2,147 views
2,019 views

Published on

Speaker: Châu Nguyễn Nhật Thanh
Work at: VNG

Published in: Technology
0 Comments
15 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,147
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
84
Comments
0
Likes
15
Embeds 0
No embeds

No notes for slide

Building zing me news feed system

  1. 1. Building Zingme News Feed System Chau Nguyen Nhat Thanh Senior Software Manager User Platform Division - VNG
  2. 2. Agenda1) Introduction to News Feed2) ZingMe News Feed system history3) ZingMe News Feed system4) Some statistics5) Bonus6) Q&A
  3. 3. Introduction to News Feed
  4. 4. Introduction to News Feed● Update friends info● Update Biz info● Update VIP info● Interaction to them by comment, like ...
  5. 5. Introduction to News Feed
  6. 6. Introduction to News Feed
  7. 7. Introduction to News Feed
  8. 8. Introduction to News Feed● Terms ● Social graph: Users in most social networking sites are describable in terms of a social graph. The relationships between users are represented by adjacency lists. If Jack and Jill are friends, they are said to be adjacent. This is known as an "edge" in the graph. (from Quora) ● Not only Friends ● but also Followers …
  9. 9. Introduction to News Feed● What do we need? ● Someone does actions, his friend will see these action in his home as soon as possible● What will we solve the problems? ● Solution 1: Push model (fan out on write) ● Solution 2: Pull model (fan out on read) ● Solution 3: Mixing push and pull (Yahoo paper)
  10. 10. Introduction to News Feed● Push model ● This method involves denormalizing the users activity data and pushing the meta data to all the users friends at the time it occurs. (from Quora)● Pull model ● This method involves keeping all recent activity data in memory and pulling in (or fanning out) that data at the time a user loads their home page. Data doesnt need to be pushed out to all subscribers as soon as it happens, so no back-log and no disk seeks (from Quora)● Mix model ● Active user using push model ● Non active user using pull
  11. 11. ZingMe News Feed system history
  12. 12. ZingMe News Feed system history● First version ● Using PHP for worker ● Using MySQL for feed item ● Using MySQL for feed indexing ● Having full feature: feed type filtering, ignoring users .. ● Restarting DB and other services are the favorite jobs at that time :) ● Lesson learn: – Relation DB may not be fit for this kind of project
  13. 13. ZingMe News Feed system history● Second version ● Still using PHP for worker ● Using Cassandra for feed item ● Using home build list id service for feed Indexing ● Using Memcached for caching item ● Removing all deluxe features :) (stupid features due to our limited technique) ● Restarting Cassandra, and waiting for compaction is our favorite jobs :) :) ● Headache with changing avatar ● Lesson learn: believe only ourself
  14. 14. ZingMe News Feed system history● Third version ● Moving to Java for better performance ● Still using Cassandra for feed item ● Trying to uses redis in lab ● Keep only simple feature (KISS) ● Cannot control memcache – The new one expired before the old one ??? – Memcached is wrong ??? ● Cannot believe to Cassandra from this time ● Lesson learn: memcached is not the “thuốc tiên” :)
  15. 15. ZingMe News Feed system (The current one :))
  16. 16. ZingMe News Feed system
  17. 17. ZingMe News Feed system● Still using push model because of twitter public some info related to this● Not enough technical when choosing pull model● Begin to understand a little bit about how to keep it scaling● Do not use Cassandra any more for such kind of this system → do not believe to anyone, learn from what they do and try our best
  18. 18. ZingMe News Feed system● Feed Item ● UserId, ObjectId, Created date... ● Storage: home build based on Kyoto Cabinet ● Fast recovery when crash● Feed Index ● UserId → [feedId1,feedId2...] ● Storage: home build ● Fast recovery when crash
  19. 19. ZingMe News Feed system● Rate limit ● Prefilter Spam or auto tool based on rate of write request ● When hit limit, block that user for amount of time● Feed writer ● Receive the write command ● Get the next Id from Generator ● Push the item to queue ● Return the feedId for future reference
  20. 20. ZingMe News Feed system● Gearman feed storage queue ● Very fast ● Support multi language client ● Some time block the all workers when network unstable :) ● Solve most of our heavy jobs
  21. 21. ZingMe News Feed system● Feed Sync center ● Sync the new feed to the others such as: – Spam detection – Feed ranking system – Logging system ● Feed replication function for future use
  22. 22. ZingMe News Feed system● Feed Render worker ● The main and heavy job: – Get the feed item – Extract the template id – Get user info – Render the feed based on them ● Put rendered feed in to appropriate cache ● Mobile and Desktop are totally different
  23. 23. ZingMe News Feed system● Feed Aggregate ● Get the feed index ● Get the rendered cache ● Return to the front-end ● Some cheat: – If the cached items less than 5, in stead of returning the data return the JavaScript to reload that list – At the same time push a task to warm-up the rendered cache ● Auto fail-over when a cache service die
  24. 24. Some statistics
  25. 25. Some statistics● ~15M actions / day● 10% Spam ● Gift receive ● Meaningless status● Cache hit 98%● ~80M registered users● ~3M active users / days
  26. 26. Bonus● Twemcache ● From twitter ● Solve most problems with memcached ● Twemcache proxy● Redis ● Replacement for home build when you have not enough time ● Set is default supported ● Supported cluster ● Persistence
  27. 27. Question and Answer
  28. 28. Q&A● What is the problem with followers? ● Do that with the trick ● Cheating the owner ;)
  29. 29. We are hiring!!!!!!!
  30. 30. Q&AContact info: Chau Nguyen Nhat Thanh thanhcnn@vng.com.vn me.zing.vn/thanhcnn2000

×