Alicebbi will like u to tell me what u like and what u dislike, tell me everything i need to know about u i will also tell you about my self in the next mail if i eventually see your reply.please if you are interested write me back at (( alicedavids72@yahoo.com ))1 year ago
Are you sure you want to
Pierre Baillet, Server Shepherd at FotopediaAnother presentation about MongoDB we made at Fotopedia: http://www.slideshare.net/kalizoy/mongodb-our-swiss-army-knife-database1 year ago
MongoDB vs Mysql. A devops point of viewPresentation Transcript
MongoDB vs MySQL A DevOps point of view.Pierre Baillet <oct@fotopedia.com> @octplaneMathieu Poumeyrol <kali@fotopedia.com>
Summary«The question is, which is to be master» Humpty Dumpty Who we are Context and constraints High availability and day-to-day operations Scalability
Who we areFotopedia (and not photopedia or fotolia)Created in 2006, Paris basedaround 20 people, some Apple ex-employeesPictures for humanity, cross-breed between flickr andwikipedia
What we doWebsite http://www.fotopedia.com (100% free website,become member and show us your best photos)2011 Crunchies award for Best Tablet Application
Some statistics 150 Millions photos views 1 MySQL database 4 MongoDB ‘clusters’ (spread on 4 servers) Around 500GB of structured data
Context and Constraints«La nuit ne peut quempirer mille fois» Roméo & Juliette 24/7 Website and web-services Continuous deployment Current Infrastructure What we expect from our NoSQL DBMS and our compromise
24/7Several million of users around the world. Between 300and several thousands at once connected.Using either the website or one of the 7 available iOSapplications. Business CriticalWhen the website is down at the application level,everything starts to fail graduallyWe cannot stop the website completely. Ever.
Overall activity on the main HTTP entry point
Continuous deploymentGit-based development flow with several activebranchesdevelopment branch deployed every wednesdayan average of 3 minor hot-fixes every workdayagile: any developer can push its hot fixs in production,at any timeWe cannot easily schedule migrations. They should beas transparent as possible.
Software StackRoR WebsiteMultiple OpenSource software used in the web stack:HAProxy, Nginx, Unicorn, mongo-resque, ...Some well known NoSQL tools: MySQL used to manage what was the core of our data MongoDB in production since September 2009, now managing more than 70% of our data
Monitoring tools Munin, Nagios Custom log feeder build around MongoDB (cf slideshare presentation: "mongodb as a log collector") MongoDB is also used to store slow transactions, exceptions and profiling traces for later inspection
Hosting Platform: 100% AWS Instances are not highly reliable but they are both abundant and disposable Disk is abundant and disposable too Use AWS RDS for MySQL hosting. Cheap and easy to setup but very shaky failover process (DNS based). We cannot rely too much on the hardware
What we expect from our NoSQL DBMS and ourcompromise No downtime: High availability No migration cost Easy to deploy, redeploy, replicate, reconfigure Quietly losing seconds of writes is preferable to weekly minutes-long maintenances periods minutes-long unscheduled downtime and manual failover in case of hardware failure
High Availability,Day to Day operations«Au fond de la cave, Paraît quil y a pas de sots métiers» Le poinçonneur de lilas Development environment Operations cycle Fit for the DevOps
Dev’ CycleData localityData migration Alter table Index creationData backup/restoration
Dev’ Data Locality In MongoDB, a collection will typically replace 2 or 3 SQL tables The physical proximity, locality, enables faster, simpler and more complete data retrieval from the application point of view. Less requests, more data.
Dev’ Data Migration: ALTERALTER TABLE is nightmarish leads to various forms of model abusing strategy: reuse of fields flag fields (binary encoded), blob fields (json/xml encoded), ...MongoDB solution: free form data storage, extensible.
Defensive strategy Application code aware of possible inconsistencies: gracefully failing view layer self-healing data access layer routine data checking and fixing batch
Dev’ Data Migration: INDICES Indices creation leads to table-wide lock in MySQL. Renders part of the Cluster unavailable MongoDB solution: Background indices creation, slows access a tiny bit, but do not lock !
Dev’ Backup/RestoreMongoDB ability to dump a db/collection empowersdeveloperPossible to restore part of the production datasetsimply on a development boxBackup a MongoDB by collections in S3, recover ondev’ platform in a matter of minutes
Ops CycleMongoDB, small is beautifulCornerstone: the Replica SetHigh availabilityBackup and data import/exportHardware migration
Ops, MongoDB, small and beautiful Young software, relatively compact (around 150,000 of C++ code) Builds out of the box on modern distributions Distros Package made by 10gen Drivers for most popular languages are also provided and maintained by 10gen staff. (although quality varies)
Ops, Replica SetA set of machine sharing the same dataOnly one Primary, several SecondariesAll writes go to Primary, routed to secondaries.Reads can be routed to primary or secondary at theapplication choiceWith the combination of AWS, Replica Set are verypowerful. MongoDB Loves the Cloud !
Ops, Master/Slave reloadedClient libraries are replica set aware connect to any node(s), the configuration and current layout is discoveredDatabase semantics are preservedIncredibly easy to setup Priority between nodes can be dynamically changed It’s possible to prevent a node from ever becoming master (slow-disk server used as a «hot backup»)
Ops, High Availability Strengths Primary step down can be triggered. Lead to election of a new Primary. a new Primary is picked when the Primary becomes unreachable clients will transparently connect to the new Primary MongoDB Arbiter ensure split brain will not happen Config. Server contains the sharding information. 1 or 3 config servers with internal failover mechanism
Ops, High Availability compromise Switch over will take 20 to 25 seconds Some queries in the interval may crash Some writes may reach a split primary
Ops, Backup and exportsStop the secondary and do whatever you need done.Easy to backup a single collection or a whole databaseAs a matter of fact, we just dumbly «mongodump»every collection of interest separately.
Ops, Hardware migrationOptionally possible to «preload» with a FS or block levelsnapshotAdd the brand new node to the replica setWait for synchroChange RS rules to get your new server primaryRemove the old hardware
Fit for the DevOps In the modern sense of DevOps, MongoDB provides the Agility and Ease of use required It provides working tools for developers And is much more confortable than MySQL in its daily usage Truly a DevOps-friendly tool.
Scalability«Accroche toi au pinceau, j’enlève le shell.» Entendu @fotopedia Cloud Limitations Sharding and Replica Set Performance Reading Writing Storage Scalable from the ground up
Cloud Limitations Virtual Hardware Neighbors can eat all you I/O No precise control and overview of this situation Largest VM cannot compare to largest Metal Hardware issue means zero-notice before instance retirement (Metal has same issue though). Need to be flexible Scaling-out is the way to scale on the Cloud
ShardingUse a business key to part your dataEach shard is typically a replica setAccess is provided via the MongoS serversConfiguration is stored and managed in the Configservers
ReadingWithout Sharding Reading is performed on a master by default to perserve read-your-own-writes. Can be programmatically allowed on a slave. To scale up reads, add Replica Set nodesWith Sharding Reading is performed in parallel across data nodes To scale up reads: ensure most queries will reach only one single shard
Writing Writes are always performed on the Primary node, so replica set does not help. Sharding distributes the write among the cluster
StorageReplica Set and shards can be moved on as manyservers as needed.To get more space scale up by migrating your Replica Set to bigger hardware scale out by sharding existing the collection
Scalable from the ground upMongoDB is scalable as soon as you need it to.No complex configuration for replicationBeautiful ability to handle replica set and shards out ofthe boxMongoS / Config Server / Shards allows more complexsetupCloud Friendly
http://www.slideshare.net/kalizoy/mongodb-our-swiss-army-knife-database 1 year ago