Zing Me Configuration System Architecture Nguyen Quang Nam Zing Me Team
Content Why Introduction Designs 1 3 2 Q&A 4
Introduction
Some info: - Servers: ~700 - ZiDB: ~115 instances - Memcached: ~80 instances - MySQL: ~20 instances - Cassandra: ~8 instan...
Why
- Too many services to keep in your head   - Auto synchronize configuration changes - C/C++/Java/PHP (or any) apps can sha...
Designs
System design
Modules design
<ul><li>- Written in C++ </li></ul><ul><li>- NonblockingServer with Thrift interface </li></ul><ul><li>- Cache: hash table...
- Written in C++ - Agent is scheduled to read only changed config from ServicesMap - SHM ConfSet is on Shared Memory - SHM...
ConfSet Extension - Written in C & C++ - Provide APIs for PHP app to access SHM ConfSet
APC? - Access APC from PHP: 120K reads per second - Access SHM ConfSet using ConfSet Extension from PHP: 250K reads per se...
PHP to ServicesMap? - Direct access through a TCP connection: <10K reads per second - ServicesMap is dead, nothing can wor...
Agents Monitoring - ServicesMap manages all agents - Each agent in ServicesMap has 2 state Active or Dead - Each agent in ...
Q & A Contact: Nguyễn Quang Nam [email_address] http://me.zing.vn/nam.nq
Upcoming SlideShare
Loading in …5
×

Php day 2011 - Zing me configuration system arch

1,384 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,384
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Php day 2011 - Zing me configuration system arch

  1. 1. Zing Me Configuration System Architecture Nguyen Quang Nam Zing Me Team
  2. 2. Content Why Introduction Designs 1 3 2 Q&A 4
  3. 3. Introduction
  4. 4. Some info: - Servers: ~700 - ZiDB: ~115 instances - Memcached: ~80 instances - MySQL: ~20 instances - Cassandra: ~8 instances - Others: Gearman, Scribe, HBase, ZiHttpd, ZiProxy, ..: many many How to manage?
  5. 5. Why
  6. 6. - Too many services to keep in your head - Auto synchronize configuration changes - C/C++/Java/PHP (or any) apps can share same configurations
  7. 7. Designs
  8. 8. System design
  9. 9. Modules design
  10. 10. <ul><li>- Written in C++ </li></ul><ul><li>- NonblockingServer with Thrift interface </li></ul><ul><li>- Cache: hash table data structure with zallocator is designed to reduce memory fragmentation </li></ul><ul><li>- Persistent DB: ZiDB core </li></ul><ul><li>- Very high performance: 180K reads per second </li></ul>ServicesMap
  11. 11. - Written in C++ - Agent is scheduled to read only changed config from ServicesMap - SHM ConfSet is on Shared Memory - SHM ConfSet contains double buffer to store both old & new conf Agent & SHM ConfSet
  12. 12. ConfSet Extension - Written in C & C++ - Provide APIs for PHP app to access SHM ConfSet
  13. 13. APC? - Access APC from PHP: 120K reads per second - Access SHM ConfSet using ConfSet Extension from PHP: 250K reads per second - So hard to build my own module to access APC - APC can be accessed from any PHP application
  14. 14. PHP to ServicesMap? - Direct access through a TCP connection: <10K reads per second - ServicesMap is dead, nothing can works - Network is slow, anything can works?
  15. 15. Agents Monitoring - ServicesMap manages all agents - Each agent in ServicesMap has 2 state Active or Dead - Each agent in ServicesMap has a last sync time field, and an expiration time duration - ServicesMap exposes APIs for Admin module getting all agents and states
  16. 16. Q & A Contact: Nguyễn Quang Nam [email_address] http://me.zing.vn/nam.nq

×