Scaling the server side of occasionally-connected, mobile systems with MongoDB


Published on

Howto scale the process of replacting data between the smartphone(all important platforms, iphone, android, symbian...) and the server.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Scaling the server side of occasionally-connected, mobile systems with MongoDB

  1. 1. Scaling the server side of occasionally-connected, mobile systems with MongoDB Thomas Huber - Fertl EDV-Systeme MongoBerlin October 4th source:
  2. 2. Introduction Occasionally Connected Systems • For e. g. Exchange Server & Outlook • Every user has an Inbox, Calendar ….. • If you get a new Laptop – install Outlook and synch your Exchange account (synch takes time) • After synching it is possible to go offline and read all mails offline • booorrriing -> what about DB replication?
  3. 3. Definitions around Replication • Lazy / Optimistic Replication <-> Eager Replication Eager Replication – all nodes online are in synch, the transaction-steps are atomic Optimistic Replication – changes are logged, if the node is online, all changes are synched with other nodes, if not online, the node will synch later on. Conflicts are possible. • Master-Slave <-> Peer-to-Peer One master, one or more slaves, not symmetric, all synchronization through the master (no Slave-to-Slave-Synch) Peer-to-Peer – all instances are talking with each other
  4. 4. Here is our solution • Lazy / Optimistic Replication Smartphones can be out of coverage – offline functionality for mobile apps robust in terms of going out of coverage and coming back lost data package detection • Master-Slave One webserver / webportal, the user is able to change data on the smartphone and in the webportal, data will be synched • One Security-Credential for Web-Portal & Smartphone Authentication (Login/Pwd) and Authorization (Permissions) are shared • But keep in mind -> Exchange Server & Outlook Data in the „inbox“ is synched whith the smartphone not all data of all users • It is possible to connect 990 different mobile devices to one user account (mixed scenarios - Symbian, iPhone, Android)
  5. 5. Mobile-Assistance-Framework
  6. 6. WTF – WHERE IS MONGODB????!!!!!! • To scale this replication-scenario you need cheap power • 1996 - Microsoft published a scientific paper about big replication scenarios – one among many „The dangers of replication and a solution” ... a ten-fold increase in nodes and traffic gives a thousand fold increase in deadlocks or reconciliations ... • It is not that ugly ;) • But mobile apps will change the characteristics in typical load scenarios – think about more writes – think about longer transactions – because of GPRS
  7. 7. Mongo helps in two Dimensions • Think about 6-channel-ECG – 200Hz 16bit resolution 35MByte per person per day -> 12GB per year • 10.000 participants one year -> 120TB • First dimension – the size -> with every new shard MongoDB is able save more data • Second dimension – ops per seconds scales with every new shard • if the problem is equally distributed over the machines the number of ops per second scales (it is very important to chose the right parameter)
  8. 8. Custom Development – maximizing the things covered in the framework
  9. 9. Hybrid – MongoDB and SQL
  10. 10. Behind the scenes We are experimenting with the ratio WebServer______________ MongoMachines
  11. 11. Put things on the right track • The portal is interlinked with the SQL Server (stored procedures) • The important functions have good caching mechanisms • Even 5 Billion User Credentials is not a real problem for a good SQL Server • If Data grows per User per day, MongoDB helps
  12. 12. Easiest NoSQL-Engine (as far as I know) to port from SQL to MongoDB • But there are no joins – so we had to implement it in our App-Server • Compared to CassandraDB - MongoDB is much easier to query but not that easy to up- scale – but this is not a problem for us • Easy to query
  13. 13. Thank you for your attention! More informations here: I‘m presenting at the CloudConf November 18th