MongoDB and MongoMK Source Event
•Download as PPTX, PDF•
1 like•1,258 views
The document discusses MongoDB and MongoMK. It provides an overview of MongoDB including its document-oriented data model, clustering using replica sets and sharding, and resilience. It also discusses MongoMK which exposes the Oak microkernel API and implements a document store using MongoDB for persistence. The document recommends various best practices for hardening, backup, monitoring, and sizing the oplog when using MongoDB in production.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to MongoDB and MongoMK Source Event
Similar to MongoDB and MongoMK Source Event (20)
Recently uploaded
Recently uploaded (20)
MongoDB and MongoMK Source Event
- 1. • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 1 MongoDB & MongoMK Mark Puddick (MongoDB) and Yuval Ararat (Adobe)
- 2. Agenda • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 2 MongoDB? • Enterprise on my mind • Moving In! • Practice of the Best!
- 3. • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 3
- 4. What is MongoDB? • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 4 Document Orientated Database • Open source • High performance • Horizontally scalable • Full featured
- 5. Document Model • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 5 JSON • Binary = BSON • Index-able
- 6. Clustering • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 6 Replica Set • Sharding
- 7. Replica Set • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 7 Replica Set – two or more copies • Self-healing shard • Addresses availability considerations • High Availability • Disaster Recovery • Maintenance • Deployment Flexibility • Data locality to users • Workload isolation: operational & analytics
- 8. Sharding • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 8 • Three types of sharding: hash-based, range-based, tag-aware • Increase or decrease capacity as you go • Automatic balancing
- 9. Sharding? • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 9 No sharding required But you can still shard • Easy to shard a later Date – Good shard key choice essential
- 10. Resilience • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 10 • Replica sets require uneven number. West Coast DC Central DC East Coast DC Secondary priority = 5 Primary priority = 10 Secondary Priority = 5 Secondary priority = 10 Arbiter
- 11. • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 11 MongoDB Management Service (MMS) The Easiest Way to Run MongoDB
- 12. • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 12 What MMS can do Deploy Upgrade Scale Continuous Backup Point-in-Time Recovery Performance Alerts
- 13. • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 13 How MMS Works
- 14. Common Tasks, Performed in Minutes • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 14 Deploy – any size, most topologies • Upgrade/Downgrade – with no downtime • Scale – add/remove shards or replicas, with no downtime • Resize Oplog – with no downtime • Specify users, roles, custom roles • Provision AWS instances and optimize for MongoDB
- 15. MMS – Monitor and Backup • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 15 • Monitor multiple MongoDB metrics • Point in Time
- 16. • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 16 OAK
- 17. OAK - Goals • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 17 Support Big repositories • Distributed repository • Improved write throughput • Support for many child nodes • Support for many ACLs • Built in clustering. Instead of built on top • OSGi friendly
- 18. • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 18 OAK Architecture
- 19. Available MK Impl • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 19 MongoMK Built over MongoDB as persistance layer. – Build for rapid sync between nodes • TarMK – Built over Tar file system – Build for speed. – Can be resiliant with 1.0.8 release.
- 20. MongoMK • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 20 Exposes Micro kernel API • Implements a DocumentStore – DocumentMK and DocumentNodeStore • MVCC Approach • GIT/SVN-inspired DAG-based versioning model [0] [0] http://wiki.apache.org/jackrabbit/RepositoryMicroKernel?action=AttachFile&do=view&target=MicroKernel+Revision+Model.pdf
- 21. MVCC • In JCR 2 sessions always reflects the latest state of the repository. • In Oak a session reflects a stable view of the repository from the time the session was Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 21 acquired. Counter = 1 Session 1 Counter = 0 Session 2 Counter = 0 getCounter()? returns 0 Counter+1 = 1 Time Counter = 1 Counter+1 = 1 JCR Counter = 0
- 22. Scaling • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 22 Advantages of MVCC: Session cannot contain references to nodes that are not accessible anymore – The MVCC model makes it possible to scale the repository significantly better • Disadvantage of MVCC: – Session may work on data that has been updated in another session. • avoiding long-running sessions is recommended. • If two sessions perform modifications relying on one session seeing the other session’s changes, call Session.refresh(). – Write skew: in two non-conflicting commits on NodeStore level can lead to JCR level constraints being violated • exception handling on saving/committing the session was required before so keep it there.
- 23. Data Structure Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 23 • simple JSON-inspired data model: nodes and properties • properties are name/value pairs • supported property types: string, number, boolean, array • a property value is stored and used as an opaque, unparsed character sequence
- 24. • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 24 PRACTICE OF THE BEST
- 25. Hardening • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 25 Only traffic from trusted sources can reach mongod instances • mongod instances can only connect to trusted outputs. • In Addition: – The “nohttpinterface” setting for mongod instances disables the “home” status page, which would run on port 28017 by default. Disable this option for production deployments – The REST setting for mongod enables a fully interactive administrative REST interface, which is disabled by default. Disable this option for production deployments. – bind_ip setting for mongod and mongos instances limits the network interfaces on which MongoDB programs will listen for incoming connections. You can also specify a number of interfaces by passing bind_ip a comma separated list of IP addresses.
- 26. Backup • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 26 Use MongoDB tools to create a backup of the repository db. • Preferred sequence: – Create replica of MongoDB primary (if not already available) – Optional: create checkpoint on MongoDB to handle merged conflicts – Stop replication – Perform backup – Restart replication • MongoDB Managed Service can be used to create a backup.
- 27. General guidance • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 27 Be sure you have the latest stable release. • Always use 64-bit builds for production. • Linux kernel version 2.6.36 or later. • Ext4 and XFS file systems are preffered. • Arbiter to run on their own server. • Tune connection pool size to avoid flooding the instance • Consider separating data, journal, and logs to different storage devices • On AWS it is recommended to use Ephermal SSD or SSD Backed EBS.
- 28. Monitoring • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 28 SaaS options MongoDB Management Service – Scout – Server Density • On premise options – Nagios – Munin – MongoDB Management Service
- 29. oplog • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 29 Idempotent log of operations • Same size on all members of a replica set. • By default it will use 5% of available disk space • Should be at least 72hrs to a weeks of operations.
- 30. • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 30
- 31. • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level 31