- The document provides basic performance tuning guidelines for MySQL databases, including checking for hardware/software issues, measuring performance at different system levels, changing one variable at a time, and tracking changes to enable rollback.
- It recommends starting with simple fixes like updates before optimizing the database configuration, and notes performance tuning is only needed to address identified problems rather than tuning for its own sake.
- Specific MySQL configuration variables are listed that could improve performance on systems with over 2GB RAM, such as increasing buffer sizes and adjusting logging settings.
DevOps Drive-In: Automate Database Deployments in Your Continuous Delivery Pi...
MYSQL_Basic_Performance_Tuning_Guidelines_-_V2
1. MYSQL Basic Performance Tuning Guidelines
Shelton Reese, DBA
Abstract
Oracle'sMySQL Database (MySQL) isconfiguredtorunon any systemthatmeetsitsminimumhardware
requirements. Asa result,the basicconfiguration providedmustbe tunedtoachieve optimumperformance
on systemsthatexceedthose requirements.
MySQL Tuninga systemcanbe a longand involved story. Volumes have beenwrittenandoftenhave beenon
tuningeachsystemlevel. Because the subjectiscomplex,nosingle papercanreallyteachyoueverything
abouttuninga MySQL systemforoptimumperformance. However,asingle papercancoverthe basics
neededtotune one level of asystemandthat isthe purpose of thispaper. It providesyouwithbasictuni ng
informationandaset of guidelinesthatcanbe usedto modifythe basicconfigurationof MySQLto improve
overall database performance.
Table of Contents
Overview...........................................................................................................................................1
Performance-Tuning Guidelines ..........................................................................................................2
1. Check for known hardware and software problems. .....................................................................2
2. Consider the whole system..........................................................................................................5
3. Measure and reconfigure by levels...............................................................................................5
4. Change one thing at a time..........................................................................................................6
5. Put tracking andfallback proceduresin place before you start.......................................................6
6. Do not tune just for the sake of tuning.........................................................................................7
7. Remember the law of diminishing returns....................................................................................8
End Notes:.....................................................................................................................................8
2. Xtivia On-line Banking
Shelton Reese Database Architecture release v1
Modified: 1/19/2015 11:32:00 PM Public Open Source Page 1 of 9
Overview
Withthe release of MySQL5.7, InnoDB hasbecome the defaultstorage engine.MyISAM,NDB (MySQLCluster),
Memory,Archive,etc.are includedaspluggable storage engines,givingusersthe flexibilitytochoose the
storage engine thatbestfitstheirspecificuse-case.
InnoDBis the mostlywidelyusedstorage engine forWeb/Web2.0,e-commerce,Financial Systems,
Telecommunications,HealthCare andRetail applicationsbuiltonMySQL.InnoDBprovideshighlyefficient
ACID-complianttransactional capabilitiesandincludesuniquearchitectural elementsthatdeliverhigh
performance andscalability. InnoDBisstructurallydesignedtohandle transactional applicationsthatrequire
crash recovery,referentialintegrity,highlevelsof userconcurrencyandfastresponse times.
While MyISAMand the otherstorage enginescontinue tobe readilyavailable,userscannow create
applicationsbuiltonInnoDBwithoutalteringdefaultconfigurationsettings. All of the capabilitiesof InnoDB
are nowdelivered“outof the box”withanyMySQL 5.5 deployment.
The ACID propertiesof InnoDBare configurable,andso userscan ensure ACID-complianceforthose workloads
demandingthe highestlevelsof dataintegrity,suchasecommerce,whilerelaxingACIDpropertieswhere
throughputismore important.
The purpose of thispaper isto tune one level of asystemfor performance with the latestInnoDB release
includedwithMySQL5.7 and MyISAM, usinga benchmarkthatis commonlyrunto measure MySQL
throughput.
MySQL can existinenvironmentsthatrange fromsimple standalone systemstocomplex combinationsof
serversand clientsrunningona wide varietyof platforms.And,regardlessof whichenvironmentisused,one
of the mostimportantaspectsfroma user’spointof view ishow well (orhow poorly) the database
applicationsperform. But,justwhatis“performance,”andhow do youknow if it isbad, and if itis bad,what
can be done toimprove it? In itssimplestterms,performance isthe wayacomputersystembehaveswhile it
isperformingagiventask. Performance istypicallymeasuredintermsof systemresponse time,throughput,
and availability. Eachof these metricscanbe affectedbyseveral factorsincluding:
1. Type of hardware beingused
2. System(and,inour case,the database) configurationused
3. Type of andnumberof usersworkingconcurrently orworkloadperformedbyeachuser’sapplication
3. Xtivia On-line Banking
Shelton Reese Database Architecture release v1
Modified: 1/19/2015 11:32:00 PM Public Open Source Page 2 of 9
Performance-Tuning Guidelines
Afteryouhave decidedthata systemisperformingpoorly,there are usuallyseveral thingsyoucando to tune
it. However,because awide varietyof tuningoptionsare available,tuningshouldalwaysbe done inan
organized,concise mannerwithaspecificgoal inmind. To be successful,the goal mustbe realistic,
quantitative,andmeasurable. Otherwise,performancetuning becomesahit-or-missexercise.
The followingperformance-tuningguidelinescanhelpyougetstarted:
1. Check for known hardware and software problems.
Some performance problemscanbe correctedsimplybyapplyingservice packstoyoursoftware,byupgrading
your hardware,orboth.Why spendthe time andeffortmonitoringandtuningotherpartsof yoursystem
whena simple servicemightresolve the problem? Withthatsaid,make sure youunderstandthe problem
before youupgrade yourhardware. If youdecide toadd anothernetworkinterface card(NIC) onlytodiscover
the systemneedsmore memory,youhave justmade acostlymistake thatdidnothingtoimprove
performance.
How to TroubleshootMySQLProblems
Start with the “status” command
SHOW [GLOBAL | SESSION] STATUS
[LIKE 'pattern' | WHERE expr]
SHOW STATUS providesserverstatusinformation.Thisinformationalsocanbe obtainedusingthe
mysqladminextended-statuscommand.The LIKE clause,if present,indicateswhichvariablenamesto
match. The WHERE clause can be givento selectrowsusingmore general conditions,asdiscussedinthe
resourcessection.Thisstatementdoesnotrequire anyprivilege.Itrequiresonlythe abilitytoconnecttothe
server.
The servermaintainsmanystatusvariablesthatprovide informationaboutitsoperation. Ingeneral,
troubleshootingrequiresthatyouisolate andidentifyaproblem, thenseekaresolution. Whatfollowswill
provide troubleshootinginformationrelatedtospecificfeaturesof MySQLInnoDBproducts.
As commonproblemsare identified,the findingswill be addedtothissectioninthe formof checklists. If the
checklistdoesnotleadyoutoa resolution, youcancollectadditional diagnosticdataandanalyze ityourself,
or submitthe data to XtiviaMySQL Supportforanalysis.
Thefollowingquestionsdirectyouto appropriatetroubleshootingtasks:
Haveyouapplied all knownupgrades?Ifnot, consider “ApplyingMySQLupgrades”
4. Xtivia On-line Banking
Shelton Reese Database Architecture release v1
Modified: 1/19/2015 11:32:00 PM Public Open Source Page 3 of 9
MySQL is evolvingquite rapidlyandhasa new upgrade every18 months. AlsoMySQL developmentteam
producesa newrelease whenevertheyhave new featuresthatothersalsoseemtohave aneedfor. Noone
isrequiredtodownloadanewrelease.The Release Noteshelpyoudetermine whetherthe new releasehas
somethingyoureallywant.
Symptom
Has it been18 monthsor 6 monthsyouappliedaquarterlyservice packor monthlyrapidupdate?Community
Serverreleasesare meantto appear2 to 3 timesperyear.
Stepsto take
End of Product Lifecycle. Active developmentforMySQLDatabase Serverversion5.1has ended.Oracle offers
varioussupportofferingswhichmaybe of interest.Fordetailsandmore information,see the MySQL sectionof
the Lifetime SupportPolicyforOracle TechnologyProducts(http://www.oracle.com/us/support/lifetime-
support/index.html).Pleaseconsiderupgradingtoarecentversion.
Does the problemoccurwhenyouare installingorupgradingdatabaseservers orclients?
There are some alternativestorunningthe mysql_install_dbprogramprovidedinthe MySQLdistribution:
Symptom
If you geta warningaboutnotbeingable tocreate testtables,thisindicatesapermissionsproblem.First
ensure thatthe user mysql will runashas the abilitytowrite tothe data directories.Note alsothatSELinux
and AppArmorcancause permissions problems,evenif the filepermissionswouldallowaccess.Thisisa
particularissue if youwantto run a serverina sandbox.
Stepsto take
If you wantthe initial privilegestobe differentfromthe standarddefaults,youcanmodifymysql_install_db
before yourunit. However,itispreferable touse GRANTandREVOKE to change the privilegesafterthe grant
tableshave beensetup.Inotherwords,youcan run mysql_install_db,andthenuse mysql -urootmysql to
connectto the serveras the MySQL root userso that you can issue the necessaryGRANTandREVOKE
statements.
If you wantto install MySQLon several machineswiththe same privileges,youcanputthe GRANTand
REVOKEstatementsina file andexecute the file asascript usingmysql afterrunningmysql_install_db.For
example:
shell>bin/mysql_install_db --user=mysql
5. Xtivia On-line Banking
Shelton Reese Database Architecture release v1
Modified: 1/19/2015 11:32:00 PM Public Open Source Page 4 of 9
shell>bin/mysql -uroot< your_script_file
By doingthis,youcan avoidhavingto issue the statementsmanuallyoneachmachine.
Checkthe documentationforhowtoconfigure these securitysystemstopermitmysql toaccessthe
directoriesinquestion.
Dose the Problemoccur when creating,dropping,downgrading,updatingormigratinga MySQL
database?
A binaryupgrade or downgrade isone thatinstallsone versionof MySQL“in place”overan existingversion,
withoutdumpingandreloadingtables:
1. Stop the serverforthe existingversionif itisrunning.
2. Install a differentversionof MySQL.Thisis an upgrade if the new versionishigherthanthe original
version,adowngrade if the versionislower.
3. Start the serverforthe newversion.
In manycases,the tablesfromthe previousversionof MySQLcan be usedwithoutproblembythe new
version. However,sometimeschangesoccurthat require tablesortable indexestobe rebuilt.
Symptom
Whenyoumove to a newdatadirectoryand are executingthe mysql_install_dbscript, youwill getanerror
message thatthe bin-log.index filesare notfound.
Stepsto take
Make sure youhave disabledthe bin-logging.Youcanre-enable bin-loggingafterrunningthe script.
Does the problemoccurwhenmovingdata usingSELECT,LOAD or MySQLdumpcommands?
The mysqldumpisa commandtool to create backupsfor MySQL. The word dumpisusedwhenwe transfer
data fromone place to another. Data movementfromadatabase tablestoa textfile orfrom a memorytoa
file.
Symptom
Slowresponse timescouldmeanyouare usingthe wrongtool.
Stepsto take
6. Xtivia On-line Banking
Shelton Reese Database Architecture release v1
Modified: 1/19/2015 11:32:00 PM Public Open Source Page 5 of 9
Use the SELECT statementforexportsof lessthan1000 rows.Otherwise use the mysqldumpcommandfor
largeramountsof data.
If your problemdoesnotfall intoone of these categories,basicdiagnosticdatamightstill be required.
Use the mysqlfrmutility, the command isdesignedasarecoverytool that reads.frmfilesandproduces
equivalentCREATEstatementsfromthe table definitiondatafoundinthe file. Inmost cases,the generated
CREATE statementisusable forrecreatingthe table onanotherserver,orforextendeddiagnostics. However,
some featuresare notsavedinthe .frmfilesandthereforewillbe omitted.
The exclusionsinclude butare notlimitedto:
• foreignkeyconstraints
• auto incrementnumbersequences
2. Consider the whole system.
Usually,tuningone aspectof a systemaffectsatleastone otherpart of thatsystem.Before youmake changes,
considerhowthose changesmightaffectthe systemasa whole.
Symptom
Applied the update andnowexperiencingmixedresults;notfasterandsomewhatslower.
Stepsto take
Use the commandmysqladmin –version.We cantake the statushere as a startingpointfor furtheractions.
3. Measure and reconfigure by levels.
Nevertune more thanone level of yoursystemata time. Evenif you are sure that all the changesyouplanto
make are beneficial,you have nowaytoevaluate how mucheachchange contributedtoany performance
improvementsthatare seen. However,if youare wrongand performance goesdowninsteadof up,youmay
have no wayof determiningwhichchange hadthe negative effect. In a MySQL database environment,the
followinglistof levelsina systemcanbe usedas a guide:
Hardware
Operatingsystem
Communicationssoftware
Database
SQL statements
Applications
7. Xtivia On-line Banking
Shelton Reese Database Architecture release v1
Modified: 1/19/2015 11:32:00 PM Public Open Source Page 6 of 9
4. Change one thing at a time.
For the same reasonsthat youshouldtune onlyone systemlevel ata time,youmustchange onlyone thingat
a time as youtune each systemlevel.
5. Put tracking and fallback procedures in place before you start.
Unfortunately,performance tuningisnotan exactscience,andsome changesthatyoumake are likelytohave
a negative effectonperformance. Whensucha situationoccurs,youcan avoidspendingtime tryingtoget
the systemback to an earlierstate if youhave a wayto back out of everychange made.Likewise,if youare
forcedto back outof several changes,be preparedtoreapplyeverychange made.
On-lineand Off-linesnapshots
One way tocreate a snapshotof the data in an existingmasterdatabase istouse the mysqldumptool to
create a dumpof all the databasesyouwantto replicate. Once the data dump hasbeencompleted,youthen
importthisdata intothe standbyserverbefore startingthe replicationprocess. Otherwise:
On-line snapshot - If yourMySQL data livesonan LVMvolume,usinganLVMsnapshotisa great low-impact
methodof gettinga point-in-time backupof yourentire database. The onlyreal gotchais that snapshotmust
be made while the datafilesare ina "safe"state (meaningthatall of the data has beenwrittentofile,andnot
onlypresentinmemory).Here are the basicstepsneededtogetabackup:
1. Connectto MySQL and flushyourtablestodiskand lockthemso nothingnew canbe writtentomemory
withFLUSH TABLES WITH READ LOCK.
2. In anotherterminal session,create the LVMsnapshotof the filesystemwhere yourMySQLdata resides.
Make sure to make thissnapshotbigenoughsothat there isenoughroomto accommodate new data coming
into the database aswe performthe backup.
3. Back at your openMySQL connection,release the table locksothatnormal database operationcanresume
UNLOCK TABLES.
4. Mount the LVMsnapshotsomewhere convenient.
5. Copythe entire directorytoa locationof yourchoice,usingthe methodof yourchoice.
6. Once the backup isdone,unmountandremove the snapshot ->DONE.
8. Xtivia On-line Banking
Shelton Reese Database Architecture release v1
Modified: 1/19/2015 11:32:00 PM Public Open Source Page 7 of 9
6. Do not tune just for the sake of tuning.
Performance tuningisonlytoresolve aproblemthathasbeenidentified. If youtune resourcesthatare not
directlyrelatedtothe primarycause of the problemthatyou are tryingto solve,theyhave nonoticeable effect
until the primaryproblemitself hasbeenresolved. Andinsome cases,suchactionscan actuallymake
subsequenttuningworkmuchmore difficult.
Symptom
Systemvariable valuescanbe setgloballyatserverstartupby usingoptionsonthe commandline orinan
optionfile. Whenyouuse a startupoptiontoset a variable thattakesa numericvalue,the value can be given
witha suffix of K,M, or G (eitheruppercase orlowercase) toindicate amultiplierof 1024, 10242 or 10243;
that is,unitsof kilobytes,megabytes,orgigabytes,respectively.Thus,the followingcommandstartsthe server
witha querycache size of 16 megabytesanda maximumpacketsize of one gigabyte:
mysqld --query_cache_size=16M --max_allowed_packet=1G
Withinanoptionfile,those variablesare setlike this:
[mysqld]
query_cache_size=16M
max_allowed_packet=1G
Stepsto Take
Withthat said, for systemswithHardware > 2GB of RAM:
--set-variable=query-cache-type=2
-read_buffer_size=128k
--read_rnd_buffer_size=1M
--sort_buffer_size=2M
--myisam_sort_buffer_size=8M
--tmp_table_size=32M
--max_heap_table_size=32M
--key_buffer_size=64M
--innodb_buffer_pool_size=750M
--bulk_insert_buffer_size=16M
--table_cache=750
--innodb_flush_log_at_trx_commit=0
--low-priority-updates..
9. Xtivia On-line Banking
Shelton Reese Database Architecture release v1
Modified: 1/19/2015 11:32:00 PM Public Open Source Page 8 of 9
7. Remember the law of diminishing returns.
Keepinmindthatthe greatestperformance tuninggainsusuallycome fromyourinitial efforts.Subsequent
tuningusuallyresultsinprogressivelysmallergainsandrequiresprogressivelygreateramountsof effort.
In sum,complete the followingtasks:
1. Review the MySQLconfigurationfile
2. Reviewproblemqueries usingthe queryanalyzer
3. Reviewschemaforredundantdataandcorrect data types
4. Reviewindexingfordatalocation, primarykeyand foreignkeyrelationships
5. Reviewstorage engine selection MyISAMfor small databasesandInnoDBfor largerdatabases
6. Reviewhighlevel architecture doesitmatch the userrequirements
7. Assistwith applyingrecommendationsin a testenvironment first
8 IdentifyotherfactorsthatmaynegativelyimpactMySQLperformance suchas upgradesand security
fixes.
End Notes:
Some extensionstoSHOWstatementsaccompanythe implementationof INFORMATION_SCHEMA:
• SHOW can be usedtoget informationaboutthe structure of INFORMATION_SCHEMA itself.
• Several SHOWstatementsacceptaWHERE clause that providesmore flexibilityinspecifyingwhich
rowsto display.See also Section 20.31, “Extensions to SHOW Statements”
“ApplyingMySQL upgrades” http://dev.mysql.com/doc/refman/5.0/en/mysql-upgrade.html
“Collectdata for installationproblems” http://dev.mysql.com/doc/refman/5.5/en/installing.html
“Upgradingand downgrading MySQL” http://dev.mysql.com/doc/refman/5.5/en/upgrading-
downgrading.html
“Collectdata for data movementproblems” http://zetcode.com/databases/mysqltutorial/exportimport/
“Installingfromsource code” http://howtolamp.com/lamp/mysql/5.6/installing#post-install
“CollectingSnapshotsusingthe file systemmethod”
https://www.lullabot.com/blog/article/mysql-backups-using-lvm-snapshots