MongoDB vs Mysql. A devops point of view
Upcoming SlideShare
Loading in...5
×
 

MongoDB vs Mysql. A devops point of view

on

  • 41,282 views

A fotopedia presentation made at the MongoDay 2012 in Paris at Xebia Office. ...

A fotopedia presentation made at the MongoDay 2012 in Paris at Xebia Office.

Talk by Pierre Baillet and Mathieu Poumeyrol.

French Article about the presentation:

http://www.touilleur-express.fr/2012/02/06/mongodb-retour-sur-experience-chez-fotopedia/

Video to come.

Statistics

Views

Total Views
41,282
Views on SlideShare
33,058
Embed Views
8,224

Actions

Likes
50
Downloads
378
Comments
3

36 Embeds 8,224

http://www.touilleur-express.fr 3979
http://nosql.mypopescu.com 2363
http://blog.xebia.fr 1184
http://data.story.lu 242
http://sdelamorena.me 107
http://assets.txmblr.com 41
http://www.newsblur.com 28
http://360kb.ednoland.net 24
http://flavors.me 23
http://www.stample.it 23
http://127.0.0.1 22
http://feeds.feedburner.com 20
http://a0.twimg.com 19
http://www.hanrss.com 19
https://www.google.fr 17
http://press.ednolab.info 15
http://localhost 15
http://kb.ednoland.net 14
http://www.stample.in 12
http://webcache.googleusercontent.com 11
https://twitter.com 9
http://www.shisoft.net 8
http://www.docshut.com 4
http://www.schoox.com 4
http://us-w1.rockmelt.com 4
http://pinterest.com 3
http://www.tumblr.com 3
http://core-qa.traackr.com 2
http://199.15.250.228 2
http://www.google.fr 1
http://cloud.feedly.com 1
http://feedproxy.google.com 1
http://translate.googleusercontent.com 1
http://www.bonvote.com 1
http://blog.xebia.fr&_=1332276296259 HTTP 1
http://www.google.nl 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • http://www.youtube.com/watch?v=w5qr4sx5Vt0
    Are you sure you want to
    Your message goes here
    Processing…
  • i will like u to tell me what u like and what u dislike, tell me everything i need to know about u i will also tell you about my self in the next mail if i eventually see your reply.please if you are interested write me back at (( alicedavids72@yahoo.com ))
    Are you sure you want to
    Your message goes here
    Processing…
  • Another presentation about MongoDB we made at Fotopedia:
    http://www.slideshare.net/kalizoy/mongodb-our-swiss-army-knife-database
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

MongoDB vs Mysql. A devops point of view MongoDB vs Mysql. A devops point of view Presentation Transcript

  • MongoDB vs MySQL A DevOps point of view.Pierre Baillet <oct@fotopedia.com> @octplaneMathieu Poumeyrol <kali@fotopedia.com>
  • Summary«The question is, which is to be master» Humpty Dumpty Who we are Context and constraints High availability and day-to-day operations Scalability
  • Who we areFotopedia (and not photopedia or fotolia)Created in 2006, Paris basedaround 20 people, some Apple ex-employeesPictures for humanity, cross-breed between flickr andwikipedia
  • What we doWebsite http://www.fotopedia.com (100% free website,become member and show us your best photos)2011 Crunchies award for Best Tablet Application
  • Some statistics 150 Millions photos views 1 MySQL database 4 MongoDB ‘clusters’ (spread on 4 servers) Around 500GB of structured data
  • Context and Constraints«La nuit ne peut quempirer mille fois» Roméo & Juliette 24/7 Website and web-services Continuous deployment Current Infrastructure What we expect from our NoSQL DBMS and our compromise
  • 24/7Several million of users around the world. Between 300and several thousands at once connected.Using either the website or one of the 7 available iOSapplications. Business CriticalWhen the website is down at the application level,everything starts to fail graduallyWe cannot stop the website completely. Ever.
  • Overall activity on the main HTTP entry point
  • Continuous deploymentGit-based development flow with several activebranchesdevelopment branch deployed every wednesdayan average of 3 minor hot-fixes every workdayagile: any developer can push its hot fixs in production,at any timeWe cannot easily schedule migrations. They should beas transparent as possible.
  • Our infrastructure Software stack Monitoring tools Hosting platform
  • Software StackRoR WebsiteMultiple OpenSource software used in the web stack:HAProxy, Nginx, Unicorn, mongo-resque, ...Some well known NoSQL tools: MySQL used to manage what was the core of our data MongoDB in production since September 2009, now managing more than 70% of our data
  • Monitoring tools Munin, Nagios Custom log feeder build around MongoDB (cf slideshare presentation: "mongodb as a log collector") MongoDB is also used to store slow transactions, exceptions and profiling traces for later inspection
  • Hosting Platform: 100% AWS Instances are not highly reliable but they are both abundant and disposable Disk is abundant and disposable too Use AWS RDS for MySQL hosting. Cheap and easy to setup but very shaky failover process (DNS based). We cannot rely too much on the hardware
  • What we expect from our NoSQL DBMS and ourcompromise No downtime: High availability No migration cost Easy to deploy, redeploy, replicate, reconfigure Quietly losing seconds of writes is preferable to weekly minutes-long maintenances periods minutes-long unscheduled downtime and manual failover in case of hardware failure
  • High Availability,Day to Day operations«Au fond de la cave, Paraît quil y a pas de sots métiers» Le poinçonneur de lilas Development environment Operations cycle Fit for the DevOps
  • Dev’ CycleData localityData migration Alter table Index creationData backup/restoration
  • Dev’ Data Locality In MongoDB, a collection will typically replace 2 or 3 SQL tables The physical proximity, locality, enables faster, simpler and more complete data retrieval from the application point of view. Less requests, more data.
  • Dev’ Data Migration: ALTERALTER TABLE is nightmarish leads to various forms of model abusing strategy: reuse of fields flag fields (binary encoded), blob fields (json/xml encoded), ...MongoDB solution: free form data storage, extensible.
  • Defensive strategy Application code aware of possible inconsistencies: gracefully failing view layer self-healing data access layer routine data checking and fixing batch
  • Dev’ Data Migration: INDICES Indices creation leads to table-wide lock in MySQL. Renders part of the Cluster unavailable MongoDB solution: Background indices creation, slows access a tiny bit, but do not lock !
  • Dev’ Backup/RestoreMongoDB ability to dump a db/collection empowersdeveloperPossible to restore part of the production datasetsimply on a development boxBackup a MongoDB by collections in S3, recover ondev’ platform in a matter of minutes
  • Ops CycleMongoDB, small is beautifulCornerstone: the Replica SetHigh availabilityBackup and data import/exportHardware migration
  • Ops, MongoDB, small and beautiful Young software, relatively compact (around 150,000 of C++ code) Builds out of the box on modern distributions Distros Package made by 10gen Drivers for most popular languages are also provided and maintained by 10gen staff. (although quality varies)
  • Ops, Replica SetA set of machine sharing the same dataOnly one Primary, several SecondariesAll writes go to Primary, routed to secondaries.Reads can be routed to primary or secondary at theapplication choiceWith the combination of AWS, Replica Set are verypowerful. MongoDB Loves the Cloud !
  • Ops, Master/Slave reloadedClient libraries are replica set aware connect to any node(s), the configuration and current layout is discoveredDatabase semantics are preservedIncredibly easy to setup Priority between nodes can be dynamically changed It’s possible to prevent a node from ever becoming master (slow-disk server used as a «hot backup»)
  • Ops, High Availability Strengths Primary step down can be triggered. Lead to election of a new Primary. a new Primary is picked when the Primary becomes unreachable clients will transparently connect to the new Primary MongoDB Arbiter ensure split brain will not happen Config. Server contains the sharding information. 1 or 3 config servers with internal failover mechanism
  • Ops, High Availability compromise Switch over will take 20 to 25 seconds Some queries in the interval may crash Some writes may reach a split primary
  • Ops, Backup and exportsStop the secondary and do whatever you need done.Easy to backup a single collection or a whole databaseAs a matter of fact, we just dumbly «mongodump»every collection of interest separately.
  • Ops, Hardware migrationOptionally possible to «preload» with a FS or block levelsnapshotAdd the brand new node to the replica setWait for synchroChange RS rules to get your new server primaryRemove the old hardware
  • Fit for the DevOps In the modern sense of DevOps, MongoDB provides the Agility and Ease of use required It provides working tools for developers And is much more confortable than MySQL in its daily usage Truly a DevOps-friendly tool.
  • Scalability«Accroche toi au pinceau, j’enlève le shell.» Entendu @fotopedia Cloud Limitations Sharding and Replica Set Performance Reading Writing Storage Scalable from the ground up
  • Cloud Limitations Virtual Hardware Neighbors can eat all you I/O No precise control and overview of this situation Largest VM cannot compare to largest Metal Hardware issue means zero-notice before instance retirement (Metal has same issue though). Need to be flexible Scaling-out is the way to scale on the Cloud
  • ShardingUse a business key to part your dataEach shard is typically a replica setAccess is provided via the MongoS serversConfiguration is stored and managed in the Configservers
  • ReadingWithout Sharding Reading is performed on a master by default to perserve read-your-own-writes. Can be programmatically allowed on a slave. To scale up reads, add Replica Set nodesWith Sharding Reading is performed in parallel across data nodes To scale up reads: ensure most queries will reach only one single shard
  • Writing Writes are always performed on the Primary node, so replica set does not help. Sharding distributes the write among the cluster
  • StorageReplica Set and shards can be moved on as manyservers as needed.To get more space scale up by migrating your Replica Set to bigger hardware scale out by sharding existing the collection
  • Scalable from the ground upMongoDB is scalable as soon as you need it to.No complex configuration for replicationBeautiful ability to handle replica set and shards out ofthe boxMongoS / Config Server / Shards allows more complexsetupCloud Friendly
  • Questions ? Oh, and btw, we hire in Paris !