• Save
MongoDB - Who, What & Where!
Upcoming SlideShare
Loading in...5
×
 

MongoDB - Who, What & Where!

on

  • 1,862 views

Introductory presentation to MongoDB - who, what, where & usage in the cloud :) Yes, the #cloud!!!

Introductory presentation to MongoDB - who, what, where & usage in the cloud :) Yes, the #cloud!!!

Brown Bag session with UCD.

Statistics

Views

Total Views
1,862
Views on SlideShare
1,861
Embed Views
1

Actions

Likes
2
Downloads
0
Comments
0

1 Embed 1

http://us-w1.rockmelt.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    MongoDB - Who, What & Where! MongoDB - Who, What & Where! Presentation Transcript

    • Who, what, how & in the cloud :) Mark  Hillick mark@10gen.com http://www.10gen.com  &  http://www.mongodb.orgTuesday 3 July 2012
    • AgendaTuesday 3 July 2012
    • Agenda • Who • What • Example Deployments • EC2 Notes • EC2 Best Practices • Further TuningTuesday 3 July 2012
    • Who, what & where!Tuesday 3 July 2012
    • Who? • 10gen -> company behind MongoDB • Created by Dwight & Eliot • MongoDB is open-source & community is key • Offices in California, NY, Dublin, London & Sydney • $73.4 million in VC fundingTuesday 3 July 2012
    • What? (1) • Powerful, flexible, scalable, fast data store • Document-oriented • Embedded docs & arrays • Scale out • Easy to start & develop withTuesday 3 July 2012
    • What? (2) • Indexing • Stored Javascript • Javascript Shell • Fixed-size collections • File StorageTuesday 3 July 2012
    • On What? • Linux • 10gen repos for Debian & Red Hat like distros • Windows • MAC OS (primarily dev) • Raspberry PI :)Tuesday 3 July 2012
    • Today’s ChallengesTuesday 3 July 2012
    • Where?Tuesday 3 July 2012
    • The Past & NowTuesday 3 July 2012
    • The Past & NowTuesday 3 July 2012
    • A bit deeperTuesday 3 July 2012
    • Getting Started • Document -> basic unit of data ~ a row in RBMS • Collection -> schema equivalent of a table • Single instance can have multiple dbs • JS Shell -> administration • Event document has special, unique key -> _idTuesday 3 July 2012
    • Collection > show collections file_tweets mini_tweets system.indexesTuesday 3 July 2012
    • Document { "_id" : 122, "user_id" : 123456, "total" : 100, "items" : [ { "item_name" : "my_item_one", "price" : 20 }, { "item_name" : "my_item_one", "price" : 20 }, { "item_name" : "my_item_three", "price" : 30 } ] }Tuesday 3 July 2012
    • The ShellTuesday 3 July 2012
    • Drivers • Python • Ruby • JavaScript [node.js] • PHP • Perl • Scala • And more...... • http://docs.mongodb.org/manual/applications/drivers (Wiki’s & Docs)Tuesday 3 July 2012
    • Indexes • Data structure collecting info • Enhance query performance • Optimiser sorts through & orders docs • db.file_tweets.ensureIndex({“user_id” : 1}) • http://www.mongodb.org/display/DOCS/IndexesTuesday 3 July 2012
    • Wire Protocol • Simple socket-based, request-response style protocol • MongoDB stores “BSON” documents, which are binary encoded serialisation of JSON documents • TCP /IP socket on port 27017 - fire & forget •http://www.mongodb.org/pages/viewpage.action?pageId=131351Tuesday 3 July 2012
    • DeploymentsTuesday 3 July 2012
    • Deployment Types • Master-Slave (not recommended) • Replica Sets • Shards • Some notes on EC2 deploymentsTuesday 3 July 2012
    • Replica Set Configurations Primary Primary Primary Secondary Secondary Secondary Arbiter Secondary Secondary (Minimum) (Typical) Secondary SecondaryTuesday 3 July 2012
    • Replica Set Info • Asynchronous replication (single primary) • Automatic failover • App-level definition of “write replication” • Secondary nodes can replicate with a slaveDelay • Secondary nodes can be hidden • Max of 12 nodes, with 7 votingTuesday 3 July 2012
    • Sharding config DB mongos mongos mongos mongos config DB config DB Primary Primary Primary Primary Secondary Secondary Secondary Secondary Secondary Secondary Secondary SecondaryTuesday 3 July 2012
    • Sharding Notes • Each “shard” usually a Replica Set (same options) • Copy of meta data stored in-memory by mongos • Data split into chunks, using range based shard key • Chunks may be migrated between shards • New chunks created by “splitting” old chunksTuesday 3 July 2012
    • Shard Server in EC2 (1) Category/Impact Low Medium High Disk Speed x Disk Capacity x RAM x CPU xTuesday 3 July 2012
    • Shard Server in EC2 (2) • MongoDB designed for OS defaults on 64 bit instance • Use standard virtual memory page size • Raise “nofiles” ulimit (20,000) • Use RAID10 & modern f/s -> ext4, xfs etc • Use “noatime” mount optionTuesday 3 July 2012
    • Server in EC2 (1) • kernel >= 2.6.23/2.6.25 respectively • Readahead: how much more to read than what you asked for • If too high => possible performance impact • Set to 0 on EBS devices • Set to desired value on RAID deviceTuesday 3 July 2012
    • Server in EC2 (2) • Turn off atime on filesystem (pre-2.6.30 especially) • RAID 10 is recommended everywhere • mitigates slow EBS volumes (fail the bad volume) • Do not use large VM pages • Do configure swap to prevent OOM KillerTuesday 3 July 2012
    • Config Server • Meta Data for shard stored in ConfigDB • Copy of meta data stored in-memory by mongos • Config DB cluster is *not* a replica set -> run 3!! • If config server goes down then: • no splits and migrates • new mongos cannot be started • running mongos can still use cache to route r/wTuesday 3 July 2012
    • Config Server in EC2 (1) Category/Impact Low Medium High Disk Speed x Disk Capacity x RAM x CPU xTuesday 3 July 2012
    • Config Server in EC2 (2) • Use Raid10 • Use 64 bit instance • Can run on shard serversTuesday 3 July 2012
    • Mongos in EC2 (1) Category/Impact Low Medium High Disk Speed Disk Capacity RAM x CPU xTuesday 3 July 2012
    • Mongos in EC2 (2) • Often run on application servers • 32 bit mongos ok with 64-bit mongodTuesday 3 July 2012
    • Arbiter in EC2 (1) Category/Impact Low Medium High Disk Speed x Disk Capacity x RAM x CPU xTuesday 3 July 2012
    • Arbiter in EC2 (2) • Can use micro instance • Elections may be slower • Can use instance store • Still want backups :)Tuesday 3 July 2012
    • HA in EC2 • Replica Sets • EC2 Availability ZonesTuesday 3 July 2012
    • DR in EC2 • Replica Sets • EC2 RegionsTuesday 3 July 2012
    • Security • Turn on authentication • Create a key between shards • EC2 Security Groups • Can reference other Security Groups • EC2 Regions • Follow SDLC in coding your appTuesday 3 July 2012
    • Monitoring • Links in with Cacti, Nagios, Munin-Node etc. • MMS - > it’s freeTuesday 3 July 2012
    • Mongo Ports • mongos, unsharded mongod : 27017 • sharded mongod : 27018 • config mongod : 27019 • http port : 28017Tuesday 3 July 2012
    • Some More EC2 NotesTuesday 3 July 2012
    • Instances Guidelines (1) • Use 64-bit only, 32-bit is not recommended • Primary/Secondary should be equal* • High CPU is not necessary • High Memory for large mongod instances • Network capacity is also IO capacity (EBS)Tuesday 3 July 2012
    • Instances Guidelines (2) • Note the trade-offs - memory/network • m1.large to m2.xlarge = 2x Mem, 0.5x Network • Do not use micro except for testing & config • m1.medium is usually sufficient for config DB • m1.small can be used for ArbitersTuesday 3 July 2012
    • Backups • EBS Snapshots - RAID complicates things • Single EBS volume, with journaling means: • No fsync & lock required • Similar applies to LVM snapshots • EC2 • GeneralTuesday 3 July 2012
    • EC2/MongoDB Best Practices • https://wiki.10gen.com/display/DOCS/Amazon+EC2 • https://wiki.10gen.com/display/DOCS/Production+NotesTuesday 3 July 2012
    • node.jsTuesday 3 July 2012
    • node.js • server-side written in javascript • orginally written for push web apps • created by Ryan DahlTuesday 3 July 2012
    • Sample node.js code var express = require(express), Db = require(mongodb).Db, Server = require(mongodb).Server, Connection = require(mongodb).Connection; var host = localhost; var port = Connection.DEFAULT_PORT; var db = new Db(node-mongo-examples, new Server(host, port, {}), {native_parser:false}); var app = express.createServer(); app.get(/, function(req, res){ res.send(Hello World); }); db.open(function(err, db) { if(err) throw err app.listen(8124); });Tuesday 3 July 2012
    • node.js with mongo • https://github.com/christkv/node-mongodb-native • http://www.nodebeginner.orgTuesday 3 July 2012
    • Need Help?Tuesday 3 July 2012
    • • MongoDB Google User Group • New MongoDB Docs & Old MongoDB Docs • Presentations • If you’re curious :) Image Source: http://www.cannotstartoutlook.com/wp-content/uploads/2012/06/outlook-problems-help.jpgTuesday 3 July 2012
    • Credits • Credit must go to @comerford, @jonnyeight & @mikefiedler as I borrowed some of the knowledgeable slides :)Tuesday 3 July 2012