Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Evgeniy Karelin. Mongo DB integration example solving performance and high load problems. DrupalCamp Kyiv 2011


Published on

This presentation is about real life example of using MongoDB on our not specific project for Drupal which supports more than 25m pageviews per day, more than 500k registered users with page load time less than 1sec.
It will give an understanding how MongoDB can be easily used to increase performance of web-site.
My presentation will be as easy as possible with simple examples and schemas but it will require at least intermediate level of developers.

- project and tasks overview. - quiz game site. There are a lot of dynamic info, users (registered and anonymous), groups and their different game statistis, user statuses etc.
- problems while using MySQL
- server optimization attempts. Memcache+Varnish. MySQL replication. Using game as separate script and AJAX blocks with "light" bootstrap.
- MongoDB overview and it's benefits on current project.
- PHP and MongoDB
- project's MongoDB architecture overview
- nodes and MongoDB
- users/groups, their statistics and MongoDB
- switching MySQL to MongoDB in Views.
- indexing problems and statistic calculations.
- multilingual support
- scalability and using MongoDB replica set.
- totals

Этот доклад о применении MongoDB в одном из наших реальных проектов, который на данный момент обслуживает более 25млн показов страниц в день, более 500тыс зарегистрированных пользователей с скоростью загрузки страниц менее 1сек.
Он позволит понять каким образом можно использовать MongoDB для увеличения производительности сайта.
Мой доклад будет на столько простым на сколько это возможно с несложными схемами и примерами, но он требует как минимум среднего уровня разработчиков для полного понимания.

- краткое описание проекта и поставленных задач. - игра-викторина. много динамических данных, группы, игроки (зарегистрированные и анонимусы) и их статистики по разным параметрам, статусы игроков и т.д.
- возникшие проблемы с работой MySQL.
- попытки серверной оптимизации. Memcache+Varnish. Репликация MySQL. Перенос игры в отдельный скрипт и AJAX блоки с использованием "легкого" бутстрапа.
- краткое описание MongoDB и приимущества его применения в текущем проекте.
- PHP и MongoDB
- общее описание архитектуры MongoDB на проекте.
- работа с нодами в MongoDB.
- работа с юзерами/группами и их статистиками.
- переход с MySQL на MongoDB в Views.
- проблемы с индексами, пересчеты статистик.
- поддержка многоязычности.
- масштабируемость MongoDB. Использование реплики.
- итоги

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Evgeniy Karelin. Mongo DB integration example solving performance and high load problems. DrupalCamp Kyiv 2011

  1. 1. MongoDB Integration ExampleSolving Performance And High Load Problems
  2. 2. Gold Sponsor ofDrupalCamp Kyiv 2011
  3. 3. Silver Sponsors ofDrupalCamp Kyiv 2011
  4. 4. Freerice v2 Easy questions and categories management Users registration Facebook/Twitter integration Friendship system Groups Detailed game statistics per user/group/total, rating system, user statuses
  5. 5. Round 1 Questions = Nodes. Data imported. Categories = Nodes + Taxonomy User profiles = Content profile module Groups = Organic groups module Friendship system = Flag module Game logic, statistics = custom Blog, comments… = Core modules
  6. 6. Beta test – OKLive test - Fail
  7. 7. MySQL is slow Views node_load() user_load() Organic group = SLOW QUERIES Game data processing …
  8. 8. What can help? Pressflow Memcache Varnish + AJAX Block Game as separate script MySQL replication Code optimization
  9. 9. AJAX Blocks AJAX blocks request -> index.php New module for managing AJAX blocks Set of blocks on page with needed bootstrap level
  10. 10. Game as separate script
  11. 11. MySQL is slow
  12. 12. What is MongoDB? Fast INSERTs Fast SELECTs Pretty fast UPDATEs No JOINs
  13. 13. PHP and MongoDB PECL extension for MongoDB Very easy to use Redundancy of documentation and examples on both MongoDB and sites
  14. 14. Code examples
  15. 15. Code example
  16. 16. What is 1 right answer? 10 rice 1 update into user tables 1 update into group tables 1 update into totals tables 1 update in user status tables …
  17. 17. 1.5-4.5 millions right answers per day
  18. 18. statistics.grains collection Every right answer "uid": 1, "cid": 17555, = 1 insert "gid": 0, Statistics "amt": 10, "dat": 1305496800, recalculation every "lvl": 1 night
  19. 19. Nodes collections node.question, node.caterogy collection hook_nodeapi() for insert, update, delete Full Question and Category nodes into MongoDB Questions and Categories SELECTs millions time often than INSERTs/UPDATEs Nothing changed for site administration
  20. 20. statistics.user_totals collection
  21. 21. Other collections statistic.group_totals statistic.totals statistic.useractivity
  22. 22. Views Bunch of views used on site Views produce a lot of SELECTs into MySQL It is not possible to cache them It is not possible to replace them
  23. 23. sview module
  24. 24. Sview disadvantages Hard to support No page/block view “from a box”, custom code needed
  25. 25. Multilingual support MySQL data is translated as usual Nothing to translate from users, groups collections Questions and Categories nodes are fully saved in MongoDB -> easy to translate
  26. 26. MongoDB replication
  27. 27. Why ReplicaSet Data redundancy Automated failover Distribute read load Simplify maintenance Recommended for sharding
  28. 28. How to use ReplicaSet
  29. 29. Freerice today 560k registered users 17k groups 25m page views per day 9.3b rice gained = 1m meals 3 DB servers and 2 Front end servers
  30. 30. If you have any questions http:/ mail: skype: smart_here