BRAINREPUBLIC - Powered by no-SQL

  • 1,973 views
Uploaded on

Talk given at Mongo Berlin 2010 by Andreas Jung

Talk given at Mongo Berlin 2010 by Andreas Jung

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,973
On Slideshare
0
From Embeds
0
Number of Embeds
5

Actions

Shares
Downloads
16
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Powered by MongoDB & no-SQL Mongo Berlin October, 4th 2010 Andreas Jung www.zopyxgroup.com Montag, 4. Oktober 2010
  • 2. /ME • Developer (backend) and software-analyst • Strong background in Python, Zope and Plone • Former Zope 2 release manager • Co-founder and chairman of the German Speaking Zope User Group (DZUG) • Director of the Zope Foundation • Member of Plone Foundation • Author of tons of add-ons for Python, Zope and Plone • Head of ZOPYX Montag, 4. Oktober 2010
  • 3. The Zope and Plone Expert Network • German based full-Service partner network • ZOPYX (Tübingen) • Veit Schiele (Berlin) • Zetwork (Oldenburg) • Banality (Essen) • Python, Zope, Plone & other cool stuff Montag, 4. Oktober 2010
  • 4. Agenda • What is BRAINREPUBLIC? • „no-SQL“ techologies used in the project • Evaluation of technologies • My view on MongoDB - pros and cons • BRAINREPUBLIC architecture Montag, 4. Oktober 2010
  • 5. Montag, 4. Oktober 2010
  • 6. Montag, 4. Oktober 2010
  • 7. Montag, 4. Oktober 2010
  • 8. Montag, 4. Oktober 2010
  • 9. Montag, 4. Oktober 2010
  • 10. Montag, 4. Oktober 2010
  • 11. Montag, 4. Oktober 2010
  • 12. Montag, 4. Oktober 2010
  • 13. Montag, 4. Oktober 2010
  • 14. Montag, 4. Oktober 2010
  • 15. Montag, 4. Oktober 2010
  • 16. Montag, 4. Oktober 2010
  • 17. Choosing a database and tools for BRAINREPUBLIC • Criteria: • fast • scalable • distributed • Special requirement: • having fun :-) Montag, 4. Oktober 2010
  • 18. Powered by repoze.bfg Montag, 4. Oktober 2010
  • 19. repoze.bfg • BFG is a "pay only for what you eat" Python web framework • based on WSGI (Web Service Gateway Interface) • What makes BFG special: • It‘s tested • Simplicity • Minimalism • Documentation • Speed Montag, 4. Oktober 2010
  • 20. • fulltext search-engine based on Apache Lucene • REST-style API for HTTP (XML/JSON) • flexible field-based configuration through XML • many plugins • fast • scales up/vertically (data partitioning) • scales out/horizontally (clustering) Montag, 4. Oktober 2010
  • 21. • AMPQ (Advanced Message Queuing Protocol) based message queue • Open-Source (VMWare) • implemented in Erlang • very fast (7500 messages/second) • very stable • flexible routing mechanisms for messages • support for clustering • implements producer & consumer pattern Montag, 4. Oktober 2010
  • 22. Possible database candidates • ZODB (Zope Object DataBase) • RDBMS • „no-SQL“ databases • key-value stores • document-oriented databases: • MongoDB • Couchdb Montag, 4. Oktober 2010
  • 23. Evaluation of key-value storages • breaking more complex data structures into key-value pairs is a pain • Map-reduce is brainfuck • implementations do not provide a „traditional“ query API Montag, 4. Oktober 2010
  • 24. Evaluation of document-oriented storages • schema-less databases are nice • easy to deal w/ requirement changes • JSON suitable for complex data structures Montag, 4. Oktober 2010
  • 25. MongoDB CouchDB very fast (>10K ops/second) pretty slow native drivers (TCP/IP) REST/HTTP API Map-Reduce Map-Reduce rich query API Master-Slave Replica set easy replication Sharding Montag, 4. Oktober 2010
  • 26. So why MongoDB (and not CouchDB)? • Performance, performance, performance • implementing a fast system on top of HTTP-based web- services/APIs is a bad idea • Rich query API (the world needs more than pure M-R) • JSON-like queries are not my thing (better syntax needed?) Montag, 4. Oktober 2010
  • 27. BRAINREPUBLIC Architecture Varnish HA Varnish Heartbeat HAProxy HAProxy Payment App App App App App App Billing MongoDB RabbitMQ SOLR Montag, 4. Oktober 2010
  • 28. Lessons learned/Looking back • MongoDB is kind of the „swiss knife of the no-SQL“ DBs • very fast and reliable • very low entry-barrier • easy programming • offers more than Map-Reduce • 10gen seems to have ambitious goals with MongoDB • good documentation (update website, books upcoming) • very good community support (IRC, mailing list) Montag, 4. Oktober 2010
  • 29. My wish list... • Poor replication performance (Master-Slave: 2.5-3 MB/sec) • Indexes should fit completely into memory? • A more fine-grained authentication model? • Parallel map-reduce? • Better usage of existing indexes (vs. compound indexes)? • An alternative query API (not based on JSON) possible? Montag, 4. Oktober 2010
  • 30. www.brainrepublic.com www.zopyxgroup.com Montag, 4. Oktober 2010