Scaling my sql_in_3d
Upcoming SlideShare
Loading in...5
×
 

Scaling my sql_in_3d

on

  • 621 views

Different organizations mean different things when they talk about scaling. Sarah will offer some tips about a few different ways that this term is thrown around for MySQL databases. Each different ...

Different organizations mean different things when they talk about scaling. Sarah will offer some tips about a few different ways that this term is thrown around for MySQL databases. Each different dimension – data volume, read volume, and write volume – present different challenges to the operations and development staff working with the system.

Statistics

Views

Total Views
621
Views on SlideShare
613
Embed Views
8

Actions

Likes
0
Downloads
2
Comments
0

2 Embeds 8

http://www.linkedin.com 6
https://www.linkedin.com 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Relational data bases are good at storing lots of small bits of information related to lots of other small bits of information. MySQL is particularly good at fast reads , can be made to do fast writes and is a lovely store for large datasets
  • API SHOULD KNOW THE DIFF proxy as an option if API not possible. scale writes. write own interfaces? check on it. btree index larger than memory -- writes get _slow_ Memcached – good for short lived data and caching
  • Consistently inno now. Myisam still has some good uses. Append only non volatile many reads. Volume manager
  • Large objects I personally say shouldn’t be in a database. They can be handled more gracefully in a key value store like a filesystem… to keep them in a database, keep them as a pointer to the filesystem. Lots of other options in the nosql space
  • Volume managers. Volume managers Volume managers
  • Developers and operations need to be able to try things. hard to justify test databases with all the storage Rapidly stand up a slave and try
  • operationally this is a great big cudgel, but often cheaper than lots of staff of consulting time
  • innodb evolving. myisam is not being developed. there are specific cases where myisam is faster… single row retrieval and full table scans in a single table… mark callahan had a good post about this back in march. recent innodb plugin has data packing
  • cpu tradeoff on compressing data, but worth it for io myisam has compression. innodb has it in the latest plugin retrieve only the number of rows you need.
  • innodb clusters the data around primary keys so retrieval is less expensive. decrease fragmentation by alter table tablename ENGINE=innodb will reorder data by primary key. alter tables can be expensive by locking.
  • If you’re worried about swapping, use huge page support for mysql (linux doesn’t swap out huge pages)
  • operationally this is a great big cudgel, but often cheaper than lots of staff of consulting time
  • journaled filesystmes and write back caching. make sure you’re optimizing for write speeds as well as recoverabilty. benchmark
  • The size of the table slows down the insertion of indexes by log N , assuming B-tree indexes. trx_commit -> still flush every second but there is risk of data loss cfq completely fair queueing deadline scheduler InnoDB read-ahead
  • •❑ disk•❑memory•❑up/down•❑cachehit -- baseline defiinition•❑replication •❑ mktablesync•❑mkchecksum

Scaling my sql_in_3d Scaling my sql_in_3d Presentation Transcript

  • scaling MySQL in 3d sarah novotny – [email_address] open databases and LAMP services www .BlueGecko . net
    • large datasets
    • high volume reads
    • high volume writes
    www .BlueGecko . net http://www.flickr.com/photos/elbragon
    • things you’ve heard about scale
    • write 1 / read many
    • partitioning / sharding
    • multimaster / rings
    • memcached / nosql
    www .BlueGecko . net
    • storage choices
    • engine options
    • storage engine
    • filesystem
    • volume manager
    • hardware
    www .BlueGecko . net http://www.flickr.com/photos/shuttercat7
    • large datasets
    • large objects
    • many rows
    www .BlueGecko . net http://www.flickr.com/photos/olivander
    • storage flexibility, reliability, clone-ability
    www .BlueGecko . net http://www.flickr.com/photos/wwworks
  • www .BlueGecko . net http://www.flickr.com/photos/alreadytaken
    • high volume reads
    • more memory
    • fast disks
    • more memory
    www .BlueGecko . net http://www.flickr.com/photos/teclasorg
  • www .BlueGecko . net myisam vs innodb http://www.flickr.com/photos/redjar
  • www .BlueGecko . net not to be obvious, but -- read less data! compress data (if you can) don’t use limit http://www.flickr.com/photos/rogersmith
    • use thoughtful primary keys
    www .BlueGecko . net
    • a
    • short
    • diversion
    • to swap or
    • not to swap
    • that is the
    • question
    www .BlueGecko . net
  • www .BlueGecko . net http://www.flickr.com/photos/teclasorg
    • high volume writes
    • choose your filesystem well
    • understand how your filesystem and raid controller work together
    • tune them to work in concert
    www .BlueGecko . net
    • facebook game case:
    • highly concurrent writes
    • low risk of --
    • omg, i lost my most recent score!
    www .BlueGecko . net
    • shard data
    • innodb_log_flush_at_trx_commit=0
    • benchmark i/o schedulers
    www .BlueGecko . net
    • free tools
    • innotop
    • maatkit
    • MySQL proxy
    • monitoring/trending
    • cacti templates
    • $monitoring_server
    • – the one you know
    www .BlueGecko . net
  • additional resources
    • irc.freenode.org
      • #mysql
      • #maatkit
    • mysql.com
    • HPM2e - Baron Schwartz, Peter Zaitsev, Vadim Tkachenko, and Jeremy Zawodny
    www .BlueGecko . net
  • credits
    • swap image
      • http://www.vocw.edu.vn/content/m10106/latest/
    • special thanks to gabriel cain and mike hamrick for suggestions on content and slides
    www .BlueGecko . net
  • Blue Gecko and contact info
    • [email_address]
    • [email_address]
    • @sarahnovotny
    • @bluegecko
    • senk on #mysql
    www .BlueGecko . net Blue Gecko provides Remote DBA services for companies around the world 7x24x365 support including monitoring, performance analysis, proactive maintenance and architectural guidance for small and large datasets.