Your SlideShare is downloading. ×
Scaling my sql_in_3d
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Scaling my sql_in_3d

475

Published on

Different organizations mean different things when they talk about scaling. Sarah will offer some tips about a few different ways that this term is thrown around for MySQL databases. Each different …

Different organizations mean different things when they talk about scaling. Sarah will offer some tips about a few different ways that this term is thrown around for MySQL databases. Each different dimension – data volume, read volume, and write volume – present different challenges to the operations and development staff working with the system.

Published in: Technology, Sports
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
475
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Relational data bases are good at storing lots of small bits of information related to lots of other small bits of information. MySQL is particularly good at fast reads , can be made to do fast writes and is a lovely store for large datasets
  • API SHOULD KNOW THE DIFF proxy as an option if API not possible. scale writes. write own interfaces? check on it. btree index larger than memory -- writes get _slow_ Memcached – good for short lived data and caching
  • Consistently inno now. Myisam still has some good uses. Append only non volatile many reads. Volume manager
  • Large objects I personally say shouldn’t be in a database. They can be handled more gracefully in a key value store like a filesystem… to keep them in a database, keep them as a pointer to the filesystem. Lots of other options in the nosql space
  • Volume managers. Volume managers Volume managers
  • Developers and operations need to be able to try things. hard to justify test databases with all the storage Rapidly stand up a slave and try
  • operationally this is a great big cudgel, but often cheaper than lots of staff of consulting time
  • innodb evolving. myisam is not being developed. there are specific cases where myisam is faster… single row retrieval and full table scans in a single table… mark callahan had a good post about this back in march. recent innodb plugin has data packing
  • cpu tradeoff on compressing data, but worth it for io myisam has compression. innodb has it in the latest plugin retrieve only the number of rows you need.
  • innodb clusters the data around primary keys so retrieval is less expensive. decrease fragmentation by alter table tablename ENGINE=innodb will reorder data by primary key. alter tables can be expensive by locking.
  • If you’re worried about swapping, use huge page support for mysql (linux doesn’t swap out huge pages)
  • operationally this is a great big cudgel, but often cheaper than lots of staff of consulting time
  • journaled filesystmes and write back caching. make sure you’re optimizing for write speeds as well as recoverabilty. benchmark
  • The size of the table slows down the insertion of indexes by log N , assuming B-tree indexes. trx_commit -> still flush every second but there is risk of data loss cfq completely fair queueing deadline scheduler InnoDB read-ahead
  • •❑ disk•❑memory•❑up/down•❑cachehit -- baseline defiinition•❑replication •❑ mktablesync•❑mkchecksum
  • Transcript

    • 1. scaling MySQL in 3d sarah novotny – [email_address] open databases and LAMP services www .BlueGecko . net
    • 2.
      • large datasets
      • high volume reads
      • high volume writes
      www .BlueGecko . net http://www.flickr.com/photos/elbragon
    • 3.
      • things you’ve heard about scale
      • write 1 / read many
      • partitioning / sharding
      • multimaster / rings
      • memcached / nosql
      www .BlueGecko . net
    • 4.
      • storage choices
      • engine options
      • storage engine
      • filesystem
      • volume manager
      • hardware
      www .BlueGecko . net http://www.flickr.com/photos/shuttercat7
    • 5.
      • large datasets
      • large objects
      • many rows
      www .BlueGecko . net http://www.flickr.com/photos/olivander
    • 6.
      • storage flexibility, reliability, clone-ability
      www .BlueGecko . net http://www.flickr.com/photos/wwworks
    • 7. www .BlueGecko . net http://www.flickr.com/photos/alreadytaken
    • 8.
      • high volume reads
      • more memory
      • fast disks
      • more memory
      www .BlueGecko . net http://www.flickr.com/photos/teclasorg
    • 9. www .BlueGecko . net myisam vs innodb http://www.flickr.com/photos/redjar
    • 10. www .BlueGecko . net not to be obvious, but -- read less data! compress data (if you can) don’t use limit http://www.flickr.com/photos/rogersmith
    • 11.
      • use thoughtful primary keys
      www .BlueGecko . net
    • 12.
      • a
      • short
      • diversion
      • to swap or
      • not to swap
      • that is the
      • question
      www .BlueGecko . net
    • 13. www .BlueGecko . net http://www.flickr.com/photos/teclasorg
    • 14.
      • high volume writes
      • choose your filesystem well
      • understand how your filesystem and raid controller work together
      • tune them to work in concert
      www .BlueGecko . net
    • 15.
      • facebook game case:
      • highly concurrent writes
      • low risk of --
      • omg, i lost my most recent score!
      www .BlueGecko . net
    • 16.
      • shard data
      • innodb_log_flush_at_trx_commit=0
      • benchmark i/o schedulers
      www .BlueGecko . net
    • 17.
      • free tools
      • innotop
      • maatkit
      • MySQL proxy
      • monitoring/trending
      • cacti templates
      • $monitoring_server
      • – the one you know
      www .BlueGecko . net
    • 18. additional resources
      • irc.freenode.org
        • #mysql
        • #maatkit
      • mysql.com
      • HPM2e - Baron Schwartz, Peter Zaitsev, Vadim Tkachenko, and Jeremy Zawodny
      www .BlueGecko . net
    • 19. credits
      • swap image
        • http://www.vocw.edu.vn/content/m10106/latest/
      • special thanks to gabriel cain and mike hamrick for suggestions on content and slides
      www .BlueGecko . net
    • 20. Blue Gecko and contact info
      • [email_address]
      • [email_address]
      • @sarahnovotny
      • @bluegecko
      • senk on #mysql
      www .BlueGecko . net Blue Gecko provides Remote DBA services for companies around the world 7x24x365 support including monitoring, performance analysis, proactive maintenance and architectural guidance for small and large datasets.

    ×