0
  MongoDB @ Fliptop 2011/12/10
Agenda <ul><ul><li>Fliptop </li></ul></ul><ul><ul><ul><li>infrastructure </li></ul></ul></ul><ul><ul><li>MongoDB </li></ul...
What is Fliptop? <ul><ul><li>Social profiles lookup </li></ul></ul><ul><ul><ul><li>facebook, twitter, linkedin </li></ul><...
Fliptop Infrastructure <ul><ul><li>Infrastructure </li></ul></ul><ul><ul><ul><li>Amazon EC2 </li></ul></ul></ul><ul><li>  ...
Fliptop DataBases  <ul><ul><li>Fliptop Data </li></ul></ul><ul><ul><ul><li>~50MM records </li></ul></ul></ul><ul><ul><li>w...
From Solr to MongoDB <ul><ul><li>Our Storage Requirement </li></ul></ul><ul><ul><ul><li>auto sharding </li></ul></ul></ul>...
MongoDB Features <ul><ul><li>Auto-Sharding </li></ul></ul><ul><ul><ul><li>scale out to 1000 nodes  </li></ul></ul></ul><ul...
MongoDB Servers  
MongoDB Shardings <ul><ul><li>Automatic balancing for changes in load and data distribution </li></ul></ul><ul><ul><li>Eas...
MongoDB Replication <ul><ul><li>master/slave </li></ul></ul><ul><ul><ul><li>easy setup </li></ul></ul></ul><ul><ul><ul><li...
MongoDB Failover <ul><ul><li>Voting algorithm (replication set) </li></ul></ul><ul><ul><ul><li>  floor(all nodes/current n...
Fliptop MongoDB Infrastructures <ul><li>  </li></ul><ul><ul><li>Data </li></ul></ul><ul><ul><ul><li>10MM/replication set <...
MongoDB and AWS EC2 <ul><ul><li>Instances type </li></ul></ul><ul><ul><ul><li>m2.xlarge </li></ul></ul></ul><ul><ul><ul><u...
MongoDB Sharding Strategy <ul><ul><li>Sharding Key Strategy </li></ul></ul><ul><ul><ul><li>Ascending shard key </li></ul><...
From timestamp to uuid <ul><ul><li>Why timestamp? </li></ul></ul><ul><ul><ul><li>same sharding key with our solr </li></ul...
MongoDB Balancer <ul><ul><li>if number of chunks are not evenly distributed, balancer can fix it </li></ul></ul><ul><ul><u...
MongoDB Schema <ul><ul><li>Document oriented </li></ul></ul><ul><ul><ul><li>json </li></ul></ul></ul><ul><ul><li>Schema Fr...
MongoDB Schema and Size <ul><ul><li>Size matters </li></ul></ul><ul><ul><ul><li>simple schema is better </li></ul></ul></u...
MongoDB Queries 1) COLUMN = VALUE 2) COLUMN in RANGE 3) boolean operators AND, OR, NOT 4) pagination (start, rows) 5) sort...
MongoDB Index <ul><ul><li>Tree structure Index </li></ul></ul><ul><ul><li>At most 64 indexes per collection(table) </li></...
MongoDB Index Types <ul><ul><li>Basic Index </li></ul></ul><ul><ul><ul><li>  db.persons.ensureIndex({name:1}); </li></ul><...
MongoDB Index Limits <ul><ul><li>negations operation </li></ul></ul><ul><ul><ul><li>  $ne, $not </li></ul></ul></ul><ul><u...
MongoDB Index Optimization <ul><ul><li>simple data type </li></ul></ul><ul><ul><ul><li>ex. int is faster than string </li>...
MongoDB Miscellaneous <ul><ul><li>Monitoring </li></ul></ul><ul><ul><ul><li>CPU </li></ul></ul></ul><ul><ul><ul><ul><li>if...
What's Next? <ul><ul><li>Further Data and Index weight lose </li></ul></ul><ul><ul><ul><li>target: 20MM/instance </li></ul...
Q & A Robbie Cheng Lead Software Engineer [email_address]
We're Hiring <ul><ul><li>please mail to jobs@fliptop.com </li></ul></ul>
Thank you!
Upcoming SlideShare
Loading in...5
×

MongoDB @ fliptop

3,400

Published on

tech talk about how fliptop leverage mongodb in its infrastructure for better scalability @ twjug

Published in: Technology
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,400
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
76
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • http://blog.jayway.com/2009/10/22/google-collections/ http://www.slideshare.net/gosain20/google-collections -api-an-introduction http://www.discursive.com/books/cjcook/reference/gua va -collect http://www.tfnico.com/presentations/google-guava
  • Transcript of "MongoDB @ fliptop"

    1. 1.   MongoDB @ Fliptop 2011/12/10
    2. 2. Agenda <ul><ul><li>Fliptop </li></ul></ul><ul><ul><ul><li>infrastructure </li></ul></ul></ul><ul><ul><li>MongoDB </li></ul></ul><ul><ul><ul><li>architecture </li></ul></ul></ul><ul><ul><ul><li>sharding strategy </li></ul></ul></ul><ul><ul><ul><li>data schema </li></ul></ul></ul><ul><ul><ul><li>index and query </li></ul></ul></ul><ul><ul><ul><li>miscellaneous </li></ul></ul></ul>
    3. 3. What is Fliptop? <ul><ul><li>Social profiles lookup </li></ul></ul><ul><ul><ul><li>facebook, twitter, linkedin </li></ul></ul></ul><ul><ul><ul><li>campaign analysis </li></ul></ul></ul><ul><ul><ul><li>api lookup </li></ul></ul></ul><ul><ul><li>  Our problems </li></ul></ul><ul><ul><ul><li>scalability </li></ul></ul></ul><ul><ul><ul><ul><li>Data </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>~ 7 billion data </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>Infrastructure </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>~ 1MM lookup/day </li></ul></ul></ul></ul></ul>
    4. 4. Fliptop Infrastructure <ul><ul><li>Infrastructure </li></ul></ul><ul><ul><ul><li>Amazon EC2 </li></ul></ul></ul><ul><li>  </li></ul><ul><ul><li>NoSQL Database </li></ul></ul><ul><ul><ul><li>MongoDB </li></ul></ul></ul><ul><ul><li>Indexing and full-text search </li></ul></ul><ul><ul><ul><li>Apache SOLR </li></ul></ul></ul><ul><ul><li>Distributed computing </li></ul></ul><ul><ul><ul><li>AWS Elastic MapReduce (Hadoop) </li></ul></ul></ul>
    5. 5. Fliptop DataBases  <ul><ul><li>Fliptop Data </li></ul></ul><ul><ul><ul><li>~50MM records </li></ul></ul></ul><ul><ul><li>w/t MongoDB </li></ul></ul><ul><ul><ul><li>MySQL </li></ul></ul></ul><ul><ul><ul><ul><li>AWS RDS x1 </li></ul></ul></ul></ul><ul><ul><ul><li>Solr </li></ul></ul></ul><ul><ul><ul><ul><li>AWS EC2 m1.large x 10 </li></ul></ul></ul></ul><ul><ul><li>w MongoDB </li></ul></ul><ul><ul><ul><li>MySQL </li></ul></ul></ul><ul><ul><ul><ul><li>AWS RDS x1 </li></ul></ul></ul></ul><ul><ul><ul><li>Solr </li></ul></ul></ul><ul><ul><ul><ul><li>AWS EC2 m1.large x 2 (master/slave) </li></ul></ul></ul></ul><ul><ul><ul><li>MongoDB </li></ul></ul></ul><ul><ul><ul><ul><li>AWS EC2 m2.large x 10 (replication set) </li></ul></ul></ul></ul>
    6. 6. From Solr to MongoDB <ul><ul><li>Our Storage Requirement </li></ul></ul><ul><ul><ul><li>auto sharding </li></ul></ul></ul><ul><ul><ul><li>richness of queries </li></ul></ul></ul><ul><ul><ul><li>short insert latency </li></ul></ul></ul><ul><ul><li>Other Reasons </li></ul></ul><ul><ul><ul><li>documentation </li></ul></ul></ul><ul><ul><ul><li>active community </li></ul></ul></ul><ul><ul><ul><li>word of mouth </li></ul></ul></ul><ul><ul><li>  Migration Efforts </li></ul></ul><ul><ul><ul><li>queries </li></ul></ul></ul><ul><ul><ul><li>db driver </li></ul></ul></ul><ul><ul><ul><li>performance tuning </li></ul></ul></ul>
    7. 7. MongoDB Features <ul><ul><li>Auto-Sharding </li></ul></ul><ul><ul><ul><li>scale out to 1000 nodes  </li></ul></ul></ul><ul><ul><li>Replication & High Availability </li></ul></ul><ul><ul><ul><li>master/slave and replication set  </li></ul></ul></ul><ul><ul><li>Querying </li></ul></ul><ul><ul><ul><li>most SQL syntax </li></ul></ul></ul><ul><ul><li>Document-oriented storage </li></ul></ul><ul><ul><ul><li>json, schema-free </li></ul></ul></ul><ul><ul><li>Full Index Support </li></ul></ul><ul><ul><ul><li>inde any field </li></ul></ul></ul><ul><ul><li>Map/Reduce </li></ul></ul><ul><ul><ul><li>javascript at server side </li></ul></ul></ul>
    8. 8. MongoDB Servers  
    9. 9. MongoDB Shardings <ul><ul><li>Automatic balancing for changes in load and data distribution </li></ul></ul><ul><ul><li>Easy addition of new machines </li></ul></ul><ul><ul><li>Scaling out to one thousand nodes </li></ul></ul><ul><ul><li>No single points of failure </li></ul></ul><ul><ul><li>Automatic failover </li></ul></ul>
    10. 10. MongoDB Replication <ul><ul><li>master/slave </li></ul></ul><ul><ul><ul><li>easy setup </li></ul></ul></ul><ul><ul><ul><li>manually fail-over  </li></ul></ul></ul><ul><ul><li>replication set </li></ul></ul><ul><ul><ul><li>bit complex setup </li></ul></ul></ul><ul><ul><ul><li>automatic fail-over </li></ul></ul></ul><ul><ul><ul><li>minimun nodes: 3 (1 abriter) </li></ul></ul></ul><ul><ul><ul><li>maximun nodes :12  </li></ul></ul></ul>
    11. 11. MongoDB Failover <ul><ul><li>Voting algorithm (replication set) </li></ul></ul><ul><ul><ul><li>  floor(all nodes/current nodes)+1 </li></ul></ul></ul><ul><ul><li>Priority </li></ul></ul><ul><ul><ul><li>if 0, never becomes primary </li></ul></ul></ul><ul><ul><ul><ul><li>backup with small machine </li></ul></ul></ul></ul>
    12. 12. Fliptop MongoDB Infrastructures <ul><li>  </li></ul><ul><ul><li>Data </li></ul></ul><ul><ul><ul><li>10MM/replication set </li></ul></ul></ul><ul><ul><li>MongoDB servers </li></ul></ul><ul><ul><ul><li>router x 1 </li></ul></ul></ul><ul><ul><ul><li>config server x1 </li></ul></ul></ul><ul><ul><ul><li>shards servers x 10 </li></ul></ul></ul><ul><ul><ul><ul><li>5 primary </li></ul></ul></ul></ul><ul><ul><ul><ul><li>5 secondary </li></ul></ul></ul></ul><ul><ul><ul><li>abriter servers x 5 </li></ul></ul></ul><ul><ul><li>AWS EC2 Instances </li></ul></ul><ul><ul><ul><li>m2.large x 10 </li></ul></ul></ul>
    13. 13. MongoDB and AWS EC2 <ul><ul><li>Instances type </li></ul></ul><ul><ul><ul><li>m2.xlarge </li></ul></ul></ul><ul><ul><ul><ul><li>  17.1 GB of memory </li></ul></ul></ul></ul><ul><ul><ul><ul><li>  6.5 EC2 Compute Units </li></ul></ul></ul></ul><ul><ul><li>  Storage </li></ul></ul><ul><ul><ul><li>  Local Drive </li></ul></ul></ul><ul><ul><ul><ul><li>faster i/o </li></ul></ul></ul></ul><ul><ul><ul><ul><li>not portable </li></ul></ul></ul></ul><ul><ul><ul><li>EBS </li></ul></ul></ul><ul><ul><ul><ul><li>i/o = network + disk i/o </li></ul></ul></ul></ul><ul><ul><ul><ul><li>portable </li></ul></ul></ul></ul><ul><ul><ul><ul><li>easy backup </li></ul></ul></ul></ul><ul><ul><ul><ul><li>raid 1/0  </li></ul></ul></ul></ul>
    14. 14. MongoDB Sharding Strategy <ul><ul><li>Sharding Key Strategy </li></ul></ul><ul><ul><ul><li>Ascending shard key </li></ul></ul></ul><ul><ul><ul><ul><li>data locality </li></ul></ul></ul></ul><ul><ul><ul><ul><li>hotspot for read/write </li></ul></ul></ul></ul><ul><ul><ul><ul><li>ex. timestamp, auto-incement PK </li></ul></ul></ul></ul><ul><ul><ul><li>Random sharding key  </li></ul></ul></ul><ul><ul><ul><ul><li>evenly distribute read/write </li></ul></ul></ul></ul><ul><ul><ul><ul><li>no data locality </li></ul></ul></ul></ul><ul><ul><ul><ul><li>ex. UUID, md5 </li></ul></ul></ul></ul><ul><ul><ul><li>Hybrid sharding key </li></ul></ul></ul><ul><ul><ul><ul><li>  ascending  </li></ul></ul></ul></ul><ul><ul><ul><ul><li>  evenly distribute </li></ul></ul></ul></ul><ul><ul><ul><ul><li>ex. timestamp + uuid </li></ul></ul></ul></ul>
    15. 15. From timestamp to uuid <ul><ul><li>Why timestamp? </li></ul></ul><ul><ul><ul><li>same sharding key with our solr </li></ul></ul></ul><ul><ul><ul><li>issues </li></ul></ul></ul><ul><ul><ul><ul><li>slowness of count (traverse) query </li></ul></ul></ul></ul><ul><ul><ul><ul><li>maintenance headache </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>add node more frequently </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>duplication of uuids </li></ul></ul></ul></ul><ul><ul><li>From timestamp to uuid </li></ul></ul><ul><ul><ul><li>  performance gain with cout </li></ul></ul></ul><ul><ul><ul><ul><li>2x faster </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>ex. count 1MM, from 10s ~ 5s. </li></ul></ul></ul></ul></ul><ul><ul><ul><li>less maintenance </li></ul></ul></ul><ul><ul><ul><ul><li>enable multiple nodes at the same time </li></ul></ul></ul></ul><ul><ul><ul><li>dedup </li></ul></ul></ul><ul><ul><ul><ul><li>uniqueness of uuid is guarantee local only </li></ul></ul></ul></ul>
    16. 16. MongoDB Balancer <ul><ul><li>if number of chunks are not evenly distributed, balancer can fix it </li></ul></ul><ul><ul><ul><li>stop criteria </li></ul></ul></ul><ul><ul><ul><ul><li>until diff between each nodes is <=2 </li></ul></ul></ul></ul><ul><ul><ul><li>balancer window </li></ul></ul></ul><ul><ul><ul><ul><li>active time window </li></ul></ul></ul></ul><ul><ul><ul><li>blocking if moving massive data </li></ul></ul></ul><ul><ul><ul><ul><li>while add brand new node </li></ul></ul></ul></ul>
    17. 17. MongoDB Schema <ul><ul><li>Document oriented </li></ul></ul><ul><ul><ul><li>json </li></ul></ul></ul><ul><ul><li>Schema Free </li></ul></ul><ul><ul><ul><li>pros </li></ul></ul></ul><ul><ul><ul><ul><li>no predefined schema is required </li></ul></ul></ul></ul><ul><ul><ul><ul><li>save 'as is' </li></ul></ul></ul></ul><ul><ul><ul><li>cons </li></ul></ul></ul><ul><ul><ul><ul><li>overhead of headers </li></ul></ul></ul></ul><ul><ul><ul><ul><li>low sensitivity of broken data </li></ul></ul></ul></ul>
    18. 18. MongoDB Schema and Size <ul><ul><li>Size matters </li></ul></ul><ul><ul><ul><li>simple schema is better </li></ul></ul></ul><ul><ul><ul><ul><li>payment:[{&quot;publisher_id&quot;: 176, &quot;paid&quot;:true}] </li></ul></ul></ul></ul><ul><ul><ul><ul><li>payment:[176_1] </li></ul></ul></ul></ul><ul><ul><ul><li>abbreviation of headers </li></ul></ul></ul><ul><ul><ul><ul><li>payment:[176_1] </li></ul></ul></ul></ul><ul><ul><ul><ul><li>pm:[176_1] </li></ul></ul></ul></ul>
    19. 19. MongoDB Queries 1) COLUMN = VALUE 2) COLUMN in RANGE 3) boolean operators AND, OR, NOT 4) pagination (start, rows) 5) sort 6) count (of query result) 7) COLUMN is non-existent 8) multiValued fields 9) dynamic fields 10) dynamic multiValued fields 11) stats queries (min, max) 12) faceted queries (aggregation of specific fields) 13) free text search (regular expression)
    20. 20. MongoDB Index <ul><ul><li>Tree structure Index </li></ul></ul><ul><ul><li>At most 64 indexes per collection(table) </li></ul></ul><ul><ul><li>A query only leverages 1 index unless using $or query </li></ul></ul><ul><ul><li>Index entails addition work on insert, delete, update  </li></ul></ul>
    21. 21. MongoDB Index Types <ul><ul><li>Basic Index </li></ul></ul><ul><ul><ul><li>  db.persons.ensureIndex({name:1}); </li></ul></ul></ul><ul><ul><li>Embedded Index </li></ul></ul><ul><ul><ul><li>  db.pesons.ensureIndex({location.city:1}) </li></ul></ul></ul><ul><ul><li>Compound Index </li></ul></ul><ul><ul><ul><li>  db.persins.ensureIndex({name:1, location.city:1}) </li></ul></ul></ul><ul><ul><li>Sparse Index </li></ul></ul><ul><ul><ul><li>  db.persons.ensureIndex({job:1}, {sparse: true}) </li></ul></ul></ul>
    22. 22. MongoDB Index Limits <ul><ul><li>negations operation </li></ul></ul><ul><ul><ul><li>  $ne, $not </li></ul></ul></ul><ul><ul><ul><li>  ex. db.things.find( { x : { $ne : 3 } } ); </li></ul></ul></ul><ul><ul><li>arithmetic operations  </li></ul></ul><ul><ul><ul><li>$mod </li></ul></ul></ul><ul><ul><ul><li>ex. db.things.find( &quot;this.a % 10 == 1&quot;) </li></ul></ul></ul><ul><ul><li>most regular expression </li></ul></ul><ul><ul><ul><li>yes </li></ul></ul></ul><ul><ul><ul><ul><li>db.persons.find({/^robbie/}) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>db.persons.find({/^robbie.*/}) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>db.persons.find({/^robbie.*/i}) </li></ul></ul></ul></ul><ul><ul><ul><li>no </li></ul></ul></ul><ul><ul><ul><ul><li>db.persons.find({/robbie}}) </li></ul></ul></ul></ul><ul><ul><li>$where </li></ul></ul>
    23. 23. MongoDB Index Optimization <ul><ul><li>simple data type </li></ul></ul><ul><ul><ul><li>ex. int is faster than string </li></ul></ul></ul><ul><ul><li>simple data schema </li></ul></ul><ul><ul><ul><li>ex. {payment: &quot;176_1&quot;} </li></ul></ul></ul><ul><ul><li>sparse index </li></ul></ul><ul><ul><ul><li>if optional fields </li></ul></ul></ul>
    24. 24. MongoDB Miscellaneous <ul><ul><li>Monitoring </li></ul></ul><ul><ul><ul><li>CPU </li></ul></ul></ul><ul><ul><ul><ul><li>if high which implies index is broken </li></ul></ul></ul></ul><ul><ul><ul><li>Driver Size </li></ul></ul></ul><ul><ul><ul><ul><li>time to add new instance </li></ul></ul></ul></ul><ul><ul><li>Backup </li></ul></ul><ul><ul><ul><li>EBS: snapshot </li></ul></ul></ul><ul><ul><ul><li>mongo import/export tool </li></ul></ul></ul><ul><ul><ul><ul><li>mongodump/mongoimport </li></ul></ul></ul></ul><ul><ul><li>Auto Deployment </li></ul></ul><ul><ul><ul><li>Hudson + fabric (python) </li></ul></ul></ul>
    25. 25. What's Next? <ul><ul><li>Further Data and Index weight lose </li></ul></ul><ul><ul><ul><li>target: 20MM/instance </li></ul></ul></ul><ul><ul><li>introduce Java POJO/DAO </li></ul></ul><ul><ul><ul><li>Morphia </li></ul></ul></ul><ul><ul><ul><li>Spring mongodb </li></ul></ul></ul><ul><ul><li>Watchdog mechanism </li></ul></ul><ul><ul><ul><li>restart server automatically </li></ul></ul></ul>
    26. 26. Q & A Robbie Cheng Lead Software Engineer [email_address]
    27. 27. We're Hiring <ul><ul><li>please mail to jobs@fliptop.com </li></ul></ul>
    28. 28. Thank you!
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×