9/14/12	  
Route & Elevation data example   (Lost on the way to MongoSeattle)
Implementation Patterns•                 	  Standard	  Datastore	  -­‐	  3	  member	  replica	  set	  	  	  	  	  (small	 ...
Implementation Patterns•      	  In	  the	  cloud,	  tune	  the	  instance	  type	  to	  the	  mongo	         implementa:o...
Operational Automation( example of automated mongodb install via puppet )
Replica Set Expansion•    MongoDB	  is	  “replica:on	  made	  elegant”	  •    Ridiculously	  simple	  to	  add	  addi:onal...
Monitoring and Introspection•    	  MMS,	  10gens	  cloud-­‐based	  monitoring	  service	  (best	       available)	       ...
10gens MMS(the one-stop shop for mongdb metrics)
Mongo in Zabbix( Mikoomi Plugins: http://code.google.com/p/mikoomi )
mongostat( Very useful for real-time troubleshooting )
Operational Automation( example of automated mongodb restart action )
Security Considerations•    	  MongoDB	  provides	  authen:ca:on	  support	  and	  basic	       permissions	       	  •   ...
Network Security Automation## Puppet Pattern for Mongodb network securityclass iptables::public {      iptables::add_rule ...
Security Considerations•    	  Use	  the	  rule	  of	  least-­‐privilege	  to	  allow	  access	  to	  environments	  	    ...
Maintenance• 	  Far	  less	  maintenance	  required	  than	  tradi:onal	  RDMBS	  systems	  	  • 	  Regularly	  perform	  ...
Indexing Patterns or “Know Your App”•    Proper	  indexing	  cri:cal	  to	  performance	  at	  scale	       (monitor	  slo...
Capped Collections•  Use	  standard	  capped	  collec:ons	  for	  retaining	  a	  fixed	  amount	     of	  data.	  	  Uses	...
Lessons Learned•    	  Mongo	  2.2	  upgrade	  containing	  a	  capped	  collec:on	  created	  in	  1.8.4.	  	  This	  sev...
9/14/12	  
Upcoming SlideShare
Loading in …5
×

MongoDB at MapMyFitness

1,387 views

Published on

As one of our primary data stores, we utilize MongoDB heavily. Early last year our DevOps lead, Chris Merz, submitted some of our use cases to 10gen (http://www.10gen.com/events) as fodder for a presentation at the MongoDB conference in Boulder. The presentation went well enough at the Boulder conference that 10gen asked him to give it again at San Francisco, Seattle and again in Boulder.

Hopefully there are some nuggets in this deck that can help you in your quest to dominate MongoDB.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,387
On SlideShare
0
From Embeds
0
Number of Embeds
925
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

MongoDB at MapMyFitness

  1. 1. 9/14/12  
  2. 2. Route & Elevation data example (Lost on the way to MongoSeattle)
  3. 3. Implementation Patterns•   Standard  Datastore  -­‐  3  member  replica  set          (small  to  med  implementa:ons)    •   Big  Data  implementa:on  –  sharded  cluster  (TB+)    •   Buffering  Layer  -­‐  high  memory            (load  all  data  and  index  files  into  RAM)    •   Write  Heavy  -­‐  u:lize  sharding  to  op:mize  for  writes    •   Read  Heavy  -­‐  3+n  replica  set  configura:on  for  rapid  read  scaling          (up  to  12  nodes)  
  4. 4. Implementation Patterns•   In  the  cloud,  tune  the  instance  type  to  the  mongo   implementa:on    •   On  iron,  plan  carefully  and  dedicate  servers  completely  to  mongo   to  avoid  memory  map  conten:on    •   For  DR,  spin  up  a  delayed,  hidden  replica  node  (preferably  in  a   different  datacenter)    •   Aggrega:on  framework  can  be  used  in  myriad  ways,  including   bridging  the  gap  to  SQL  data  warehousing  via  ETL.    •   Automate  install  paYerns  for  rapid  development,  prototyping,   and  infrastructure  scaling.  
  5. 5. Operational Automation( example of automated mongodb install via puppet )
  6. 6. Replica Set Expansion•  MongoDB  is  “replica:on  made  elegant”  •  Ridiculously  simple  to  add  addi:onal  members  •  Be  sure  to  run  Ini:alSync  from  a  secondary!     rs.add(  “host”  :  “livetrack_db09”,  “ini:alSync”  :  {  “state”  :  2  }  )  •  Both  rs.add()  and  rs.remove()  can  be  scripted  and  connected  to   Monitoring  systems  for  autoscaling  
  7. 7. Monitoring and Introspection•   MMS,  10gens  cloud-­‐based  monitoring  service  (best   available)    •   Supported  by  Zabbix,  Nagios,  Munin,  Server  Density,  etc    •   mongostat,  mongotop,  REST  interface,  database  profiler    •   Monitoring  system  triggers  can  ini:ate  node  addi:ons,      removals,  service  restarts,  etc    •   In  addi:on  to  service-­‐level  monitoring,  use  more  advanced      tests  to  check  for  and  alert  on  query  latency  spikes      
  8. 8. 10gens MMS(the one-stop shop for mongdb metrics)
  9. 9. Mongo in Zabbix( Mikoomi Plugins: http://code.google.com/p/mikoomi )
  10. 10. mongostat( Very useful for real-time troubleshooting )
  11. 11. Operational Automation( example of automated mongodb restart action )
  12. 12. Security Considerations•   MongoDB  provides  authen:ca:on  support  and  basic   permissions    •   Auth  is  turned  off  by  default  to  allow  for  op:mal  performance      •   Always  run  databases  in  a  trusted  network  environment    •   Lock  down  host  based  firewalls  to  limit  access  to  required   clients      •   Automate  iptables  with  puppet  or  chef,  in  EC2  use  security   groups  
  13. 13. Network Security Automation## Puppet Pattern for Mongodb network securityclass iptables::public { iptables::add_rule { 001 MongoDB established: rule => -A RH-Firewall-1-INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT } iptables::add_rule { 002 MongoDB: rule => -A RH-Firewall-1-INPUT -i eth1 -p tcp -m tcp --dport 27017 -j ACCEPT } iptables::add_rule { 003 MongoDB MMF Phase II Network: rule => -A RH-Firewall-1-INPUT -i eth0 -s 172.16.16.0/20 -p tcp -m tcp --dport 27017 -j ACCEPT } iptables::add_rule { 004 MongoDB MMF Cloud Network: rule => -A RH-Firewall-1-INPUT -i eth0 -s 10.178.52.0/24 -p tcp -m tcp --dport 27017 -j ACCEPT } }
  14. 14. Security Considerations•   Use  the  rule  of  least-­‐privilege  to  allow  access  to  environments      •   Data  sensi:vity  should  determine  the  extent  of  security   measures    •   For  non-­‐sensi:ve  data,  good  network  security  can  be  sufficient      •   In  open  environments,  be  sure  experience  matches  access  level    •   Lack  of  granular  perms  allows  for  full  admin  access,  use   discre:on  
  15. 15. Maintenance•   Far  less  maintenance  required  than  tradi:onal  RDMBS  systems    •   Regularly  perform  query  profile  analysis  and  index  audi:ng    •   Rebuild  databases  to  reclaim  space  lost  due  to  fragmenta:on    •   Automate  checks  of  log  files  for  known  red-­‐flags    •   Regularly  review  data  throughput  rate,  storage  growth  rate,  and      overall  business  growth  graphs  to  inform  capacity  planning.    •   For  HA  tes:ng,  periodically  step-­‐down  the  primary  to  force  failover  
  16. 16. Indexing Patterns or “Know Your App”•  Proper  indexing  cri:cal  to  performance  at  scale   (monitor  slow  queries  to  catch  non-­‐performant  requests)  •  MongoDB  is  ul:mately  flexible,  being  schemaless   (mongo  gives  you  enough  rope  to  hang  yourself,  choose  wisely)  •  Avoid  un-­‐indexed  queries  at  all  costs     (its  quickest  way  to  crater  your  app...  consider  -­‐-­‐notablescan)  •  Onus  on  DevOps  to  match  applica:on  to  indexes   (know  your  query  profile,  never  assume)  •  Shoot  for  covered  queries  wherever  possible   (answer  can  be  obtained  from  indexes  only)  
  17. 17. Capped Collections•  Use  standard  capped  collec:ons  for  retaining  a  fixed  amount   of  data.    Uses  a  FIFO  strategy  for  pruning.   (based  on  data  size,  not  number  of  rows)    •  TTL  Collec:ons  (2.2)  age  out  data  based  on  a  reten:on  :me   configura:on.       (great  for  data  reten:on  requirements  of  all  types)     Gotcha!     Explicitly  create  the  capped  collec:on  before  any  data  is  put   into  the  system  to  avoid  auto-­‐crea:on  of  collec:on  
  18. 18. Lessons Learned•   Mongo  2.2  upgrade  containing  a  capped  collec:on  created  in  1.8.4.    This  severely   impacted  replica:on  (RC:  no  "_id"  index,    FIX:  add  "_id"  index)      •   Never  start  mongo  when  a  mount  point  is  missing  or  incorrectly  configured.  Mongo   may  decide  to  take  maYers  into  its  own  hands  and  resync  itself  with  the  replica  set.     Make  sure  your  devops  and  your  hos0ng  provider  admins  are  aware  of  this    •   Some  drivers  that  use  connec:on  pooling  can  freak  the  freaky  freak  when  the  primary   member  changes  (older  pymongo).    Kicking  the  applica:on  can  fix,  also:  upgrade  drivers    •   High  locked  %  is  a  big  red-­‐flag,  and  can  be  caused  by  a  large  number  of  simultaneous   dml  ac:ons  (high  insert  rate,  high  update  rate).  Consider  this  in  the  design  phase.    •   Be  wary  of  automa:on  that  can  change  the  state  of  a  node  during  maintenance  mode.     Disable  automa:on  agents  for  reduced  risk  during  cri:cal  administra:ve  opera:ons   (filesystem  maint,  etc)  
  19. 19. 9/14/12  

×