Scaling for Web - Architecting HootSuite for 5+ Million Users

529 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
529
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
12
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Creating Shared ValueSustainable InnovationNatural Selection
  • Deep Listening vs Real-Time AnalysisAppropriate and Scalable EngagementActionable Insight and Empowered Employees
  • Scaling for Web - Architecting HootSuite for 5+ Million Users

    1. 1. Scaling for WebArchitecting HootSuite for 5 Million+ Users
    2. 2. About UsBeier CaiChief Architectbeier.cai@hootsuite.com@beiercaiBill MonkmanLead Developer, Webbill.monkman@hootsuite.com@bmonkman
    3. 3. Hoo’s HootSuite
    4. 4. Some Numbers• 5+ million Members• 10+ million Social Networks• 25+ million Streams• 350+ million Messages Scheduled• 1.3+ Billion Messages Sent• 2+ Billion Page Views per month• 2,000 Web Requests/sec
    5. 5. TechnologiesPHPMySQL
    6. 6. TechnologiesPHPMemcachedMySQL
    7. 7. TechnologiesPHPGearmanMemcachedMySQL
    8. 8. TechnologiesPHPGearmanMemcachedMySQL MongoDB
    9. 9. TechnologiesPHPGearman ZeroMQMemcachedMySQL MongoDB
    10. 10. Technology Stack• LAMP (Linux, Apache, MySQL, PHP)• Scala• Akka• Python• Clojure• MongoDB• Redis• Memcached• Gearman• ZeroMQ• RabbitMQ/AMQP• jQuery/Backbone• EC2 Cloud / CloudStack• Jenkins / CruiseControl• CloudFront CDN• HAProxy/Nginx/Varnish• Zend• Statsd/Graphite• LogStash• ElasticSearch/Lucene• Nagios/Munin/Sensu• ZooKeeper
    11. 11. Optimization• Push processing up to the front-end, make your users’ computers into adistributed processing network. e.g. stream rendering• Cache as much as possible, invalidate only when necessary.• Sometimes features have to take a back-seat to performance/scalability.• Look at profiler data.• Watch query execution time, optimize when necessary with indexes,partitions, de-normalization, etc.• Know what your weak spots are and keep a close eye on them.
    12. 12. Optimization
    13. 13. Optimization Case Study
    14. 14. Optimization Case Study
    15. 15. Scaling Dev Team• Passion• Commitment• Dedication• Responsibility• General AwesomenessBesides technical competency, we also value
    16. 16. Scaling Dev Team• Passion• Commitment• Dedication• Responsibility• General AwesomenessBesides technical competency, we also value
    17. 17. Scaling Dev Team
    18. 18. Team Work• Entrepreneurial• Communication• Respect• Trust
    19. 19. Coding Practice• Clean and Readable• Maintainable• Reusable• Scalable• CommentedHave a well defined coding standard within your dev team, andfollow it!
    20. 20. Monitoring, Profiling & Logging
    21. 21. Monitoring, Profiling & Logging
    22. 22. Case Study
    23. 23. Case Study
    24. 24. Thank You!Beier Caibeier.cai@hootsuite.comtwitter.com/beiercaiBill Monkmanbill.monkman@hootsuite.comtwitter.com/bmonkman

    ×