Your SlideShare is downloading. ×
0
Powered by
                          MongoDB & no-SQL

                              Mongo Berlin
                        ...
/ME

      • Developer (backend) and software-analyst
      • Strong background in Python, Zope and Plone
      • Former Z...
The Zope and Plone Expert Network


                    • German based full-Service partner network
                     •...
Agenda



              • What is BRAINREPUBLIC?
              • „no-SQL“ techologies used in the project
              • ...
Montag, 4. Oktober 2010
Montag, 4. Oktober 2010
Montag, 4. Oktober 2010
Montag, 4. Oktober 2010
Montag, 4. Oktober 2010
Montag, 4. Oktober 2010
Montag, 4. Oktober 2010
Montag, 4. Oktober 2010
Montag, 4. Oktober 2010
Montag, 4. Oktober 2010
Montag, 4. Oktober 2010
Montag, 4. Oktober 2010
Choosing a database and tools for
                                  BRAINREPUBLIC

                           • Criteria:
...
Powered by
                           repoze.bfg




Montag, 4. Oktober 2010
repoze.bfg
             •      BFG is a "pay only for what you eat" Python web framework

             •      based on WSG...
•      fulltext search-engine based on Apache Lucene

               •      REST-style API for HTTP (XML/JSON)

          ...
•       AMPQ (Advanced Message Queuing Protocol) based message queue

     •       Open-Source (VMWare)

     •       impl...
Possible database candidates


                          • ZODB (Zope Object DataBase)
                          • RDBMS
 ...
Evaluation of key-value storages


      • breaking more complex data structures into key-value
              pairs is a p...
Evaluation of document-oriented storages

                          • schema-less databases are nice
                     ...
MongoDB              CouchDB

             very fast (>10K ops/second)        pretty slow

                    native driv...
So why MongoDB (and not CouchDB)?

 • Performance, performance, performance
 • implementing a fast system on top of HTTP-b...
BRAINREPUBLIC Architecture
                          Varnish            HA             Varnish
                           ...
Lessons learned/Looking back

     • MongoDB is kind of the „swiss knife of the no-SQL“ DBs
     • very fast and reliable
...
My wish list...

   • Poor replication performance (Master-Slave: 2.5-3 MB/sec)
   • Indexes should fit completely into me...
www.brainrepublic.com
                          www.zopyxgroup.com




Montag, 4. Oktober 2010
Upcoming SlideShare
Loading in...5
×

BRAINREPUBLIC - Powered by no-SQL

2,064

Published on

Talk given at Mongo Berlin 2010 by Andreas Jung

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,064
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
17
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "BRAINREPUBLIC - Powered by no-SQL"

  1. 1. Powered by MongoDB & no-SQL Mongo Berlin October, 4th 2010 Andreas Jung www.zopyxgroup.com Montag, 4. Oktober 2010
  2. 2. /ME • Developer (backend) and software-analyst • Strong background in Python, Zope and Plone • Former Zope 2 release manager • Co-founder and chairman of the German Speaking Zope User Group (DZUG) • Director of the Zope Foundation • Member of Plone Foundation • Author of tons of add-ons for Python, Zope and Plone • Head of ZOPYX Montag, 4. Oktober 2010
  3. 3. The Zope and Plone Expert Network • German based full-Service partner network • ZOPYX (Tübingen) • Veit Schiele (Berlin) • Zetwork (Oldenburg) • Banality (Essen) • Python, Zope, Plone & other cool stuff Montag, 4. Oktober 2010
  4. 4. Agenda • What is BRAINREPUBLIC? • „no-SQL“ techologies used in the project • Evaluation of technologies • My view on MongoDB - pros and cons • BRAINREPUBLIC architecture Montag, 4. Oktober 2010
  5. 5. Montag, 4. Oktober 2010
  6. 6. Montag, 4. Oktober 2010
  7. 7. Montag, 4. Oktober 2010
  8. 8. Montag, 4. Oktober 2010
  9. 9. Montag, 4. Oktober 2010
  10. 10. Montag, 4. Oktober 2010
  11. 11. Montag, 4. Oktober 2010
  12. 12. Montag, 4. Oktober 2010
  13. 13. Montag, 4. Oktober 2010
  14. 14. Montag, 4. Oktober 2010
  15. 15. Montag, 4. Oktober 2010
  16. 16. Montag, 4. Oktober 2010
  17. 17. Choosing a database and tools for BRAINREPUBLIC • Criteria: • fast • scalable • distributed • Special requirement: • having fun :-) Montag, 4. Oktober 2010
  18. 18. Powered by repoze.bfg Montag, 4. Oktober 2010
  19. 19. repoze.bfg • BFG is a "pay only for what you eat" Python web framework • based on WSGI (Web Service Gateway Interface) • What makes BFG special: • It‘s tested • Simplicity • Minimalism • Documentation • Speed Montag, 4. Oktober 2010
  20. 20. • fulltext search-engine based on Apache Lucene • REST-style API for HTTP (XML/JSON) • flexible field-based configuration through XML • many plugins • fast • scales up/vertically (data partitioning) • scales out/horizontally (clustering) Montag, 4. Oktober 2010
  21. 21. • AMPQ (Advanced Message Queuing Protocol) based message queue • Open-Source (VMWare) • implemented in Erlang • very fast (7500 messages/second) • very stable • flexible routing mechanisms for messages • support for clustering • implements producer & consumer pattern Montag, 4. Oktober 2010
  22. 22. Possible database candidates • ZODB (Zope Object DataBase) • RDBMS • „no-SQL“ databases • key-value stores • document-oriented databases: • MongoDB • Couchdb Montag, 4. Oktober 2010
  23. 23. Evaluation of key-value storages • breaking more complex data structures into key-value pairs is a pain • Map-reduce is brainfuck • implementations do not provide a „traditional“ query API Montag, 4. Oktober 2010
  24. 24. Evaluation of document-oriented storages • schema-less databases are nice • easy to deal w/ requirement changes • JSON suitable for complex data structures Montag, 4. Oktober 2010
  25. 25. MongoDB CouchDB very fast (>10K ops/second) pretty slow native drivers (TCP/IP) REST/HTTP API Map-Reduce Map-Reduce rich query API Master-Slave Replica set easy replication Sharding Montag, 4. Oktober 2010
  26. 26. So why MongoDB (and not CouchDB)? • Performance, performance, performance • implementing a fast system on top of HTTP-based web- services/APIs is a bad idea • Rich query API (the world needs more than pure M-R) • JSON-like queries are not my thing (better syntax needed?) Montag, 4. Oktober 2010
  27. 27. BRAINREPUBLIC Architecture Varnish HA Varnish Heartbeat HAProxy HAProxy Payment App App App App App App Billing MongoDB RabbitMQ SOLR Montag, 4. Oktober 2010
  28. 28. Lessons learned/Looking back • MongoDB is kind of the „swiss knife of the no-SQL“ DBs • very fast and reliable • very low entry-barrier • easy programming • offers more than Map-Reduce • 10gen seems to have ambitious goals with MongoDB • good documentation (update website, books upcoming) • very good community support (IRC, mailing list) Montag, 4. Oktober 2010
  29. 29. My wish list... • Poor replication performance (Master-Slave: 2.5-3 MB/sec) • Indexes should fit completely into memory? • A more fine-grained authentication model? • Parallel map-reduce? • Better usage of existing indexes (vs. compound indexes)? • An alternative query API (not based on JSON) possible? Montag, 4. Oktober 2010
  30. 30. www.brainrepublic.com www.zopyxgroup.com Montag, 4. Oktober 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×