NoSQL Infrastructure - Late 2013

  • 783 views
Uploaded on

NoSQL databases are often touted for their performance and whilst it's true that they usually offer great performance out of the box, it still really depends on how you deploy your infrastructure. …

NoSQL databases are often touted for their performance and whilst it's true that they usually offer great performance out of the box, it still really depends on how you deploy your infrastructure. Dedicated vs cloud? In memory vs on disk? Spindal vs SSD? Replication lag. Multi data centre deployment.

This talk considers all the infrastructure requirements of a successful high performance infrastructure with hints and tips that can be applied to any NoSQL technology. It includes things like OS tweaks, disk benchmarks, replication, monitoring and backups.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
783
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
12
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. NoSQL Infrastructure
  • 2. ~3333 write ops/s 0.07 - 0.05 ms response
  • 3. David Mytton Woop Japan!
  • 4. MongoDB at Server Density
  • 5. •27 nodes MongoDB at Server Density
  • 6. • June 2009 - 4yrs •27 nodes MongoDB at Server Density
  • 7. •MySQL -> MongoDB •27 nodes MongoDB at Server Density • June 2009 - 4yrs
  • 8. •MySQL -> MongoDB •25TB data per month •27 nodes MongoDB at Server Density • June 2009 - 4yrs
  • 9. Why?
  • 10. Why? • Replication
  • 11. Why? • Replication • Official drivers
  • 12. Why? • Replication • Official drivers • Easy deployment
  • 13. Why? • Replication • Official drivers • Easy deployment • Fast out of the box
  • 14. It’s a little different.
  • 15. Picture is unrelated! Mmm, ice cream. • Fast network Performance
  • 16. • Fast network Performance EC2 10 Gigabit Ethernet - Cluster Compute - High Memory Cluster - Cluster GPU - High I/O - High Storage
  • 17. • Fast network Performance Workload: Read/Write? Result set size What is being stored?
  • 18. • Fast network Performance Use Network Throughput Normal 0-100Mb/s Replication (Initial Sync) Burst +100Mb/s Replication (Oplog) 0-100Mb/s Backup Initial Sync + Oplog
  • 19. • Fast network Performance Inter-DC LAN
  • 20. • Fast network Performance Inter-DC LAN Cross USA Washington, DC - San Jose, CA
  • 21. • Fast network Performance Location Ping RTT Latency Within USA 40-80ms Trans-Atlantic 100ms Trans-Pacific 150ms Europe - Japan 300ms
  • 22. Failover •Replica sets
  • 23. Failover •Replica sets •Master/slave
  • 24. Failover •Replica sets •Min 3 nodes •Master/slave
  • 25. Failover •Replica sets •Min 3 nodes •Master/slave •Automatic failover
  • 26. •Replication lag Performance Location Ping RTT Latency Within USA 40-80ms Trans-Atlantic 100ms Trans-Pacific 150ms Europe - Japan 300ms
  • 27. Replication Lag 1. Reads: eventual consistency
  • 28. Replication Lag 1. Reads: eventual consistency 2. Failover: slave behind
  • 29. Eventual Consistency Stale data
  • 30. Eventual Consistency Stale data Inconsistent data
  • 31. Eventual Consistency Stale data Inconsistent data Changing data
  • 32. Eventual Consistency Use Case Needs consistency? Graphs No User profile Yes Statistics Depends Alert config Yes
  • 33. Replication Lag 1. Reads: eventual consistency 2. Failover: slave behind
  • 34. Slave behind Failover: out of date master
  • 35. • Safe by default >>> from pymongo import MongoClient >>> connection = MongoClient(w=int/str) Value Meaning 0 Unsafe 1 Primary 2 Primary + x1 secondary 3 Primary + x2 secondaries MongoDB WriteConcern
  • 36. Tags { _id : "someSet", members : [ {_id : 0, host : "A", tags : {"dc": "ny"}}, {_id : 1, host : "B", tags : {"dc": "ny"}}, {_id : 2, host : "C", tags : {"dc": "sf"}}, {_id : 3, host : "D", tags : {"dc": "sf"}}, {_id : 4, host : "E", tags : {"dc": "cloud"}} ] settings : { getLastErrorModes : { veryImportant : {"dc" : 3}, sortOfImportant : {"dc" : 2} } } } > db.foo.insert({x:1}) > db.runCommand({getLastError : 1, w : "veryImportant"})
  • 37. { _id : "someSet", members : [ {_id : 0, host : "A", tags : {"dc": "ny"}}, {_id : 1, host : "B", tags : {"dc": "ny"}}, {_id : 2, host : "C", tags : {"dc": "sf"}}, {_id : 3, host : "D", tags : {"dc": "sf"}}, {_id : 4, host : "E", tags : {"dc": "cloud"}} ] settings : { getLastErrorModes : { veryImportant : {"dc" : 3}, sortOfImportant : {"dc" : 2} } } } > db.foo.insert({x:1}) > db.runCommand({getLastError : 1, w : "veryImportant"}) (A or B) + (C or D) + E Tags
  • 38. { _id : "someSet", members : [ {_id : 0, host : "A", tags : {"dc": "ny"}}, {_id : 1, host : "B", tags : {"dc": "ny"}}, {_id : 2, host : "C", tags : {"dc": "sf"}}, {_id : 3, host : "D", tags : {"dc": "sf"}}, {_id : 4, host : "E", tags : {"dc": "cloud"}} ] settings : { getLastErrorModes : { veryImportant : {"dc" : 3}, sortOfImportant : {"dc" : 2} } } } > db.foo.insert({x:1}) > db.runCommand({getLastError : 1, w : "sortOfImportant"}) (A + C) or (D + E) ... Tags
  • 39. Picture is unrelated! Mmm, ice cream. • Fast network •More RAM Performance
  • 40. http://www.slideshare.net/jrosoff/mongodb-on-ec2-and-ebs
  • 41. http://blog.pythonisito.com/2011/12/mongodbs-write-lock.html
  • 42. http://blog.pythonisito.com/2011/12/mongodbs-write-lock.html
  • 43. More RAM = expensive Performance x2 4GB RAM 12 month Prices
  • 44. RAM SSDs Spinning disk Cost Speed
  • 45. Softlayer disk pricing Performance
  • 46. EC2 disk/RAM pricing Performance $2232/m $2520/m $43/m $295/m
  • 47. SSD vs Spinning Performance
  • 48. SSD vs Spinning Performance
  • 49. SSD vs Spinning Performance
  • 50. Tips: rand() •Field names
  • 51. Tips: rand() •Field names •Covered indexes
  • 52. Tips: rand() •Field names •Covered indexes •Collections / databases
  • 53. Backups What is the use case?
  • 54. Backups What is the use case? Fixing user errors? Point in time restore? Disaster recovery?
  • 55. Backups •Disaster recovery Offsite
  • 56. Backups •Disaster recovery Age Offsite
  • 57. Backups •Disaster recovery Age Offsite Restore time
  • 58. Backups Frequency Consistency Verification
  • 59. www.flickr.com/photos/daddo83/3406962115/ Monitoring •System Disk i/o Disk use
  • 60. www.flickr.com/photos/daddo83/3406962115/ Monitoring Disk i/o Disk use •System Swap
  • 61. www.flickr.com/photos/daddo83/3406962115/ Monitoring Optime State •Replication
  • 62. mongostat
  • 63. Monitoring tools Run yourself Ganglia
  • 64. Monitoring tools www.serverdensity.com
  • 65. David Mytton david@serverdensity.com @davidmytton Woop Japan! blog.serverdensity.com/mongodb