Breaking The Monolith: Fast Distributed Web Services Using Sets (Feb13, NUS)


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Breaking The Monolith: Fast Distributed Web Services Using Sets (Feb13, NUS)

  1. 1. Breaking the Monolith Fast Distributed Web Services Using SetsCristobal Viedma
  2. 2. What is Viki?
  3. 3. Cross-culturalcommunication andunderstanding
  4. 4. Technology
  5. 5. Web,,,,,
  6. 6. MobileAndroid,iOS,Blackberry,Windows Phone,Samsung Bada,Kindle Fire...
  7. 7. TVGoogle TV, Samsung SmartTV, Roku...
  8. 8. Top 5 cities: Singapore, Santiago, La Victoria,Jakarta, New York Data: Oct12
  9. 9. ~22M users, 2500% mobile growth in 2012
  10. 10. ...viewers start leaving if video doesnt play in 2 seconds...viewers start leaving ifvideo doesnt play in 2seconds...and every second ofadditional delay about 6%more viewers jumping ship! Video Stream Quality Impacts Viewer Behavior: Inferring Causality Using Quasi-Experimental Designs. S. Shunmuga Krishnan, Ramesh K. Sitaraman, 2012
  11. 11. Large ecosystemGlobal audience User growth Performance
  12. 12. Breaking themonolith
  13. 13. Platform Scalability Availability Performance
  14. 14. Performance Network time Generation time Render time
  15. 15. Essence Read heavy (writes can wait) Everything fits in memory Goal: 25ms uncached (10-100x better)
  16. 16. Hyperion(platform cluster)
  17. 17. Data structure
  18. 18. Bitmapsgenre:1 -> 010101001genre:2 -> 010010011type:music -> 011000001intersect -> 010000001 (ids: 2 and 9)Good: speed for intersect & memory efficiencyBad: get the ids and the real data. sorting!,paging...
  19. 19. Everything is a setgenre:1 = [1, 2, 4]type:1 = [1, 2, 5]type:2 = [3, 4]Good: Sparse (too many 0s with bitmaps)Complexity O(n+m) (m is sets, n elements inthe smallest set)genre:1 -> 10 elementsvideos -> 100K elementscomplexity: O(10)
  20. 20. Building our own indexesData storedKeeping track of the indexesHow do we find, my_set, BY, v:*->created_at,desc, LIMIT, offset, count, GET, v:*->details)
  21. 21. HoldbacksOld system: not in (id, id, id)first attempt: hash with a list of rulespermutations (too many countries in the world!)CAP -> 10GB. meh~alias matching permutations: 800mb ;)Redis 32bit, even better!CAP is just another set! (well, but a DIFF!)
  22. 22. Hacking RedisVfind: Building our own Redis functionSetlets: Pre-calculated sorted listsMost requests 18~20ms, some cases 100ms(depends on the bigger set)Vfind only gets content to fill 1 page: 15msPaging just showing more: 9msSerialization of jsons: 5msEnough.. for now!
  23. 23. ListsA list is just a sorted set. E.g. a list ofsubscriptions, list of featured content...Is a set!You can apply holdbacks or any other filter.
  24. 24. The platform
  25. 25. Many web services Each vertical is a source of truth Logical and operational reasons Everything routed through Oceanus Activities Aphrodite Gaia (Videos) (Behaviour) (Community) (Users)
  26. 26. Queue Centralized queue for events and messages Event-driven web services Messages must be idempotent Message / Events Queue
  27. 27. PrometheusCentral logging system Visualizations Tasseo / Kibana CubismGraphics, stats & logs Backend ElasticSearch Graphite Improve visibility Collectors Logstash Stastd Prometheus Logs Stats (Logs)
  28. 28. KnowledgeContent Baboon Subtitling Segmenting Prometheus Analytics (CMS) tool tool (Logs) Oceanus Subber Aphrodite Gaia (Videos) (Subs,segs) (Community) (Users)Platform Message / Events Queue Hyperion Hyperion Hyperion (platform (platform (platform cluster) cluster) cluster) Devices Partners Third partyClients developers (distributed (Android, (Samsung, thin-client) iPhone, Youtube, (Open API) TV...) Renren...)
  29. 29. Nothing new... Web services and APIs Centralized queue for messages
  30. 30. ... just having fun and learning. :)
  31. 31. btw, were hiring!