Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Java MySQL Connector & Connection Pool Features & Optimization

5,352 views

Published on

This talk will give an overview of the different available Java MySQL connectors (Connector/J, MariaDB Java Connector) and connection pools (Commons-DBCP, C3P0, ...).
One of the things with the default configuration of these solutions is that they are very chatty. This can have a noticeable impact on application performance and database load. I've seen many environments where over 50% of the total amount of queries are caused by such behavior. This behavior will be explained through examples seen on production systems as well as recommendations on optimization will be made.
Load balancing and transparent failover solutions will be described for both MySQL's traditional asynchronous replication and Galera based replication (Percona XtraDB Cluster or MariaDB Galera Cluster).

Published in: Data & Analytics
  • Nice deck. I wanted to provide some corrections re: HikariCP. Slide#57, JDBC4 isValid() is the default (and in 2.2.4 support automatically detected). Slide#61, Startup Size = minimumIdle. Slide#76, that's a good thing! Slide#84, we'll take that as a feature request. :-)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Java MySQL Connector & Connection Pool Features & Optimization

  1. 1. Java MySQL Connector & Connection Pool Features & Optimization Kenny Gryp <kenny.gryp@percona.com> April 14, 2015 @gryp
  2. 2. DISCLAIMER Please excuse me for not being a Java developer 2
  3. 3. What I Don’t Like • Brussels Sprouts • Taxes • Calories • Java(’s chattiness) 3
  4. 4. MYSQL CONNECTORS CONNECTION POOLS 4
  5. 5. MYSQL CONNECTORS CONNECTION POOLS Connectors Configuring Connector Creating A Database Connection Prepared Statements Example Transaction 5
  6. 6. MySQL Connector/J & MariaDB Java Client • MySQL Connector/J – Oracle – 5.1.35 Latest – Compatible with • MySQL • Percona Server • MariaDB 6
  7. 7. MySQL Connector/J & MariaDB Java Client • MariaDB Java Client • MariaDB • Fork from Drizzle Connector • Latest 1.1.8 • Compatible with – MySQL – Percona Server – MariaDB 7
  8. 8. MySQL Connector/J Features • Enterprise Plugin: Query Analyzer • MySQL Fabric Integration • Load Balancing • Failover • Replication 8
  9. 9. MYSQL CONNECTORS CONNECTION POOLS Connectors Configuring Connector Creating A Database Connection Prepared Statements Example Transaction 9
  10. 10. Creating Connection Connection con = DriverManager.getConnection (“jdbc:mysql://node2/employees? user=connj&password=test"); Statement stmt = con.createStatement(); String query = "select * from employees where emp_no = 20000;"; ResultSet rs = stmt.executeQuery(query); ... MariaDB: jdbc:mariadb://node2/employees ?user=connj&password=test" 10
  11. 11. Creating Connection - Tomcat w. JDBC- Pool context.xml (local): <Resource name="jdbc/test" auth="Container" type="javax.sql.DataSource" username=“jdbc-pool" password="test" driverClassName="com.mysql.jdbc.Driver" url=“jdbc:mysql://node2:3306/employees” /> MariaDB: driverClassName="org.mariadb.jdbc.Driver" 11
  12. 12. Creating Connection - JDBC URL jdbc:mysql://node2:3306/employees? useServerPrepStmts=true&... 12
  13. 13. MYSQL CONNECTORS CONNECTION POOLS Connectors Configuring Connector Creating A Database Connection Prepared Statements Example Transaction 13
  14. 14. Connector/J - Creating Connection SHOW VARIABLES WHERE Variable_name ='language' OR… SELECT @@session.auto_increment_increment; SET NAMES latin1; SET character_set_results = NULL; SET autocommit=1; SET sql_mode= 'NO_ENGINE_SUBSTITUTION,STRICT_TRAN S_TABLES'; 14
  15. 15. MariaDB - Creating Connection set autocommit=1; USE employees; show variables like 'sql_mode'; 15
  16. 16. Creating Connection - Defaults • Connector/J: • MariaDB Java
 Client: 16
  17. 17. Connector/J & MariaDB Java Client - Verbosity • Connector/J is more verbose when starting a connection • Usually not a problem: – connection pools are commonly used
 (more coming soon…) – connections are reused – Actually I like but not too much. 17
  18. 18. Optimization • MariaDB Java Client vs MySQL Connector/J • Prepared Statements 18
  19. 19. Connector Performance - SELECT 1 localhost, single threaded 19 QPS 0 4000 8000 12000 16000 localhost ConnectorJ MariaDB 15.213 13.477
  20. 20. Connector Performance - MariaDB %faster 20 Faster% 5% 10% 15% 20% MariaDB Connector +Speed% ConnectorJ SELECT1 LO 13%
  21. 21. Connector Performance - MariaDB %faster 21 Faster% 5% 10% 15% 20% MariaDB Connector +Speed% ConnectorJ SELECT1 LO SELECT1 net pk range 0% 4%4% 13% Benefit is relative!
  22. 22. MYSQL CONNECTORS CONNECTION POOLS Connectors Configuring Connector Creating A Database Connection Prepared Statements Example Transaction 22
  23. 23. Client or Server Side Prepared Statements • Server side Prepared statements: – reduce network traffic – query is already optimized on server. • Support: – MariaDB Java client only supports client side – Connector/J default in client side 23
  24. 24. Server Side Prepared Statements PREPARE stmt1 FROM select * from employees where emp_no = ?; EXECUTE # API CALL select * from employees where emp_no = 20000; DEALLOCATE PREPARE stmt1; 24
  25. 25. Connector/J: Server Side Prepared Statements • useServerPrepStmts = false – disabled by default • Mind looking at: – cachePrepStmts = false • do PREPARE, EXECUTE, DEALLOCATE every time…, 3 round trips? – prepStmtCacheSize = 25 – prepStmtCacheSqlLimit = 256 • low defaults 25 https://bugs.mysql.com/bug.php?id=74932
  26. 26. Benchmark: Prepared Statements 26 QPS 900 1800 2700 3600 MariaDB CONN/J CONN/J SRVCONN/J SRV+Cache 3.506QPS 2.047QPS 3.342QPS3.400QPS select * from employees_alotofindexes where first_name='moss' and birth_date > "1954-06-14" and gender="M" and hire_date > "1998-01-01"G
  27. 27. Cracked!! 27
  28. 28. MYSQL CONNECTORS CONNECTION POOLS Connectors Configuring Connector Creating A Database Connection Prepared Statements Example Transaction 28
  29. 29. Connector/J Optimization + Default JDBC- Pool Connection con = ds.getConnection(); con.setTransactionIsolation (Connection.TRANSACTION_READ_COMMITTED); con.setAutoCommit(false); PreparedStatement stmt = con.prepareStatement("select * from employees where emp_no = ?"); stmt.setInt(1, 20000); ResultSet rs = stmt.executeQuery(); stmt.close(); rs.close(); con.commit(); con.close(); 29
  30. 30. Connector/J Optimization + Default JDBC- Pool SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED; SET autocommit=0; # administrator command: Prepare; select * from employees where emp_no = 20000; # administrator command: Close stmt; commit; 30 Taxes
  31. 31. Connector/J Optimization • useConfigs=maxPerformance – cachePrepStmts=true – cacheCallableStmts=true – cacheServerConfiguration=true – useLocalSessionState=true – elideSetAutoCommits=true – alwaysSendSetIsolation=false – enableQueryTimeouts=false 31
  32. 32. Connector/J Optimization • useLocalTransactionState=true
 commit() / rollback() 32
  33. 33. Connector/J Optimization - Tuned JDBC URL: useConfigs=maxPerformance& useServerPrepStmts=true: select * from employees where emp_no = 20000; commit; 33
  34. 34. MariaDB Java Client Optimization SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED; select * from employees where emp_no = 20000; COMMIT; 34
  35. 35. MariaDB Java Client Optimization - Code if ( con.getTransactionIsolation() != Connection.TRANSACTION_READ_COMMITTED ) { con.setTransactionIsolation (Connection.TRANSACTION_READ_COMMITTED); } 35
  36. 36. MariaDB Java Client Optimization - Interceptors SELECT @@tx_isolation; select * from employees where emp_no = 20000; COMMIT; Still @@tx_isolation. Now add JDBC Interceptor: <Resource name="jdbc/test" auth="Container" factory= "org.apache.tomcat.jdbc.pool.DataSourceFactory" jdbcInterceptors="ConnectionState" driverClassName="org.mariadb.jdbc.Driver" url="jdbc:mysql://node2:3306/employees"/> 36
  37. 37. MariaDB Java Client Optimization - Optimized! select * from employees where emp_no = 20000; COMMIT; 37
  38. 38. Benchmark - Connector Concurrency - SELECT 1 38 HikariCP-bench with JDBC Pool, 4 Threads, SELECT 1 (4,8,16,32 Pool Size) QPS 17.500 35.000 52.500 70.000 MariaDB CONN/J
  39. 39. Benchmark - Connector Concurrency - TRX 39 HikariCP-bench with JDBC Pool, 4 Threads, TRX (4,8,16,32 Pool Size) QPS 4.750 9.500 14.250 19.000 Conn/J MariaDB Conn/J Optim MariaDB Optim
  40. 40. MYSQL CONNECTORS CONNECTION POOLS 40
  41. 41. MYSQL CONNECTORS CONNECTION POOLS Connection Pools Issues Resetting Environment Testing Connectivity Pool Sizing Lingering Transactions Analysis Examples Graceful Failover Conn/J Extra Features 41
  42. 42. Java Connection Pools – The Usual: – C3P0 – Commons-DBCP (v1&v2) – JDBC Pool (fork commons-DBCP) – Out In The Wild: – Vibur-DBCP – HikariCP – BoneCP 42
  43. 43. Java Connection Pools – The Usual: – C3P0 – Commons-DBCP (v1&v2) – JDBC Pool (fork commons-DBCP) – Out In The Wild: – Vibur-DBCP – HikariCP – BoneCP 43
  44. 44. Connection Pool Key Points • Connection Management • Pool Sizing • Connection Testing • Avoid Lingering Transactions 44
  45. 45. MYSQL CONNECTORS CONNECTION POOLS Connection Pools Issues Resetting Environment Testing Connectivity Pool Sizing Lingering Transactions Analysis Examples Graceful Failover Conn/J Extra Features 45
  46. 46. Connection Pool Issues • Coming from DBA side, I do not like ‘chattiness’ 46
  47. 47. Connection Pool Issues 47
  48. 48. Connection Pool Issues 48
  49. 49. Connection Pool Issues 49
  50. 50. Connection Pool Issues 50
  51. 51. WHY DOES IT HAVE TO DO THAT? Because of ‘application bugs’ 51
  52. 52. Connection Pools - Why Chattiness Examples • *maybe* forgot to COMMIT / ROLLBACK • wanting AUTOCOMMIT=1
 but a previous TRX set it to 0 • Changing TRX Isolation Level • Is connection still working? 52
  53. 53. MYSQL CONNECTORS CONNECTION POOLS Connection Pools Issues Resetting Environment Testing Connectivity Pool Sizing Lingering Transactions Analysis Examples Graceful Failover Conn/J Extra Features 53
  54. 54. Connection Pool - Resetting Status 54 JDBC-Pool C3P0 DBCP2 HikariCP Rollback rollbackOnReturn=false autoCommitOnClose=false rollbackOnReturn
 =true Commit commitOnReturn=false autoCommitOnClose=false n/a n/a Avoid see above forceIgnoreUnresolvedTransa ctions=false see above Auto Commit Driver Driver enableAutoCommit
 OnReturn=true Driver
  55. 55. MYSQL CONNECTORS CONNECTION POOLS Connection Pools Issues Resetting Environment Testing Connectivity Pool Sizing Lingering Transactions Analysis Examples Graceful Failover Conn/J Extra Features 55
  56. 56. Connection Pool - Testing • Making sure the connection is still active • If not, maybe reopen a connection • Not recommended as DB • However, applications: • do not like errors • do not retry gracefully 56
  57. 57. Connection Pool - Testing • If connections REALLY need to be tested… • do not specify test query like: • SELECT 1 • SELECT * FROM DUAL • Leave default, all of the connection pools use:
 JDBC4 isValid(); 57
  58. 58. Connection Pool - Testing 58 JDBC-Pool C3P0 DBCP2 HikariCP Test Before testOnBorrow=false testConnectionOnCheckOut
 =false testOnBorrow=false n/a Test After testOnReturn=false testConnectionOnCheckIn
 =false testOnReturn=false n/a Test While Idle testWhileIdle=false idleConnectionTestPeriod
 =0 testWhileIdle=false n/a JDBC4 isValid() default default default jdbc4ConnectionTest
 =true (default) Query validationQuery
 (isValid) preferredTestQuery=null validationQuery
 (isValid) connectionTestQuery
 =none Interval? validationInterval=30000 n/a n/a n/a
  59. 59. Connection Pool - Testing • JDBC: validationInterval=30s
 WHY? It defeats the whole purpose! 59
  60. 60. MYSQL CONNECTORS CONNECTION POOLS Connection Pools Issues Resetting Environment Testing Connectivity Pool Sizing Lingering Transactions Analysis Examples Graceful Failover Conn/J Extra Features 60
  61. 61. Connection Pool - Pool Sizing • Funnelling on Application Level, is good • Smaller Number is Better • +- * CPU’s on DB • maybe a bit more (waiting on IO…) • all application servers combined • Response Time vs Throughput 61
  62. 62. Connection Pool - Pool Sizing 62 JDBC-Pool C3P0 DBCP2 HikariCP Amount of Connections maxActive=100 maxPoolSize=15 maxTotal=8 maximumPoolSize=10 Maximum Idle Connections maxIdle=100 maxIdleTime=0** maxIdle=8 n/a Minimum Idle Connections minIdle=10 minPoolSize=3 minIdle=0 minimumIdle=max Startup Size initialSize=10 initialPoolSize=3 initialSize=0 minimumIdle
  63. 63. MYSQL CONNECTORS CONNECTION POOLS Connection Pools Issues Resetting Environment Testing Connectivity Pool Sizing Lingering Transactions Analysis Examples Graceful Failover Conn/J Extra Features 63
  64. 64. Connection Pool - Avoid Lingering Transactions • Application forgets to return the connection • Statements that take longer than … • Avoid this! • Fix Application 64
  65. 65. Connection Pool - Avoid Lingering Transactions 65 KILL Warning JDBC-Pool removeAbandoned=false
 removeAbandonedTimeout=60
 abandonWhenPercentageFull=0 suspectTimeout=0 C3P0 unreturnedConnectionTimeout=0 n/a DBCP removeAbandoned=false
 removeAbandonedTimeout=300
 Only When: 
 getNumIdle() < 2 and getNumActive() > getMaxTotal() - 3) n/a HikariCP n/a leakDetectionThreshold=0
  66. 66. MYSQL CONNECTORS CONNECTION POOLS Connection Pools Issues Resetting Environment Testing Connectivity Pool Sizing Lingering Transactions Analysis Examples Graceful Failover Conn/J Extra Features 66
  67. 67. Connection Pools - How To Look At Workload? • Slow Query Log • tcpdump • pt-query-digest • Percona Cloud Tools 67
  68. 68. MYSQL CONNECTORS CONNECTION POOLS Connection Pools Issues Resetting Environment Testing Connectivity Pool Sizing Lingering Transactions Analysis Examples Graceful Failover Conn/J Extra Features 68
  69. 69. Connection Pool - Example Transaction Connection con = ds.getConnection(); con.setTransactionIsolation (Connection.TRANSACTION_READ_COMMITTED); con.setAutoCommit(false); PreparedStatement stmt = con.prepareStatement("select * from employees where emp_no = ?"); stmt.setInt(1, 20000); ResultSet rs = stmt.executeQuery(); stmt.close(); rs.close(); con.commit(); con.close(); 69
  70. 70. For Connectors - RECAP • MySQL Connector/J • useConfigs=maxPerformance • useServerPrepStmts=true • MariaDB Java Client • HikariCP: Built in • JDBC-Pool: jdbcInterceptors=“ConnectionState" • Other Pools: UNKNOWN 70
  71. 71. Connection Pool - TRX JDBC select * from employees where emp_no = 20000; commit; 71 200
  72. 72. Connection Pool - TRX C3P0 SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED; SET autocommit=0; select * from employees where emp_no = 20000; commit; SET autocommit=1; SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ; 72 600
  73. 73. Connection Pool - TRX C3P0 mysql> set global tx_isolation=“READ-COMMITTED”; forceIgnoreUnresolvedTransactions=true 73 200
  74. 74. Connection Pool - TRX DBCP SET autocommit=1; # administrator command: Ping; SET autocommit=0; select * from employees where emp_no = 20000; commit; rollback; SET autocommit=1; 74 700
  75. 75. Connection Pool - TRX DBCP testOnBorrow=false rollbackOnReturn=false enableAutoCommitOnReturn=false jdbcUrl: useLocalTransactionState=true 75 200
  76. 76. Connection Pool - TRX HikariCP SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED; SET autocommit=0; select * from employees where emp_no = 20000; commit; SET autocommit=1; SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ; 76 600
  77. 77. Connection Pool - TRX HikariCP mysql> set global tx_isolation=“READ-COMMITTED”; autoCommit=false 77 200
  78. 78. Connection Pools 78
  79. 79. MariaDB vs. Connector/J 79
  80. 80. MYSQL CONNECTORS CONNECTION POOLS Connection Pools Issues Resetting Environment Testing Connectivity Pool Sizing Lingering Transactions Analysis Examples Graceful Failover Conn/J Extra Features 80
  81. 81. Connection Pools - Graceful Failover 81
  82. 82. Connection Pools - Graceful Failover • HAProxy ‘stats socket’ /etc/haproxy/haproxy.cfg global . . . stats socket /tmp/haproxy.sock level admin • Disable Node # echo "disable server database/node1" 
 | socat stdio /tmp/haproxy.sock 82
  83. 83. Connection Pools - Graceful Failover 83
  84. 84. Connection Pools - Graceful Failover • During ‘maintenance’, what do we do? • KILL old connections? • Wait until connections are closed? (Define lifetimes?) • Ignore it? 84
  85. 85. Connection Pools - Graceful Failover • Some connection pools can close connections gracefully, when idle. • For ‘synchronous’ replication systems • using JMX • No Application Errors! 85 Method JDBC-Pool purgeOnReturn() C3P0 softResetAllUsers() DBCP n/a HikariCP softEvictConnections(), suspendPool(), resumePool() <—- ASYNC
  86. 86. Connection Pools - Graceful Failover 86
  87. 87. Connection Pools - Graceful Failover 87
  88. 88. Connection Pools - Graceful Failover 88
  89. 89. Connection Pools - Graceful Failover • 0 Application Errors • Completely seamless 89
  90. 90. MYSQL CONNECTORS CONNECTION POOLS Connection Pools Issues Resetting Environment Testing Connectivity Pool Sizing Lingering Transactions Analysis Examples Graceful Failover Conn/J Extra Features 90
  91. 91. Connector/J - Extra Features • Load Balancing • jdbcUrl: ”jdbc:mysql:loadbalance:// node1,node2/db? loadBalanceConnectionGroup=lb&
 loadBalanceEnableJMX=true” • loadBalanceStrategy 
 (random/bestResponseTime) • Failover • ReplicationDriver (setReadOnly) • Combining with Connection Pools is less useful • Fabric 91
  92. 92. Java MySQL Connector & 
 Connection Pool Optimization • http://dev.mysql.com/doc/connector-j/en • https://mariadb.com/kb/en/mariadb/client-libraries/mariadb-java- client/ • http://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html • http://www.mchange.com/projects/c3p0 • http://commons.apache.org/proper/commons-dbcp/ • https://github.com/brettwooldridge/HikariCP 92 MYSQL CONNECTORS CONNECTION POOLS Kenny Gryp <kenny.gryp@percona.com> November 4, 2014 @gryp

×