Your SlideShare is downloading. ×
High Performance Jdbc
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

High Performance Jdbc

2,554

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,554
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
57
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. High Performance JDBC Jorg Janke Consultant and Coach [email_address]
  • 2. Agenda
    • What I do
    • Persistence
    • Performance
    • JDBC
      • Connection
      • Query
      • Result
    Timing Statistics
  • 3. Shameless Self-Promotion
    • Consultant & Coach (available)
    • I talk business
      • Trained Accountant, MBA
      • VP Product Marketing
    • I talk tech
      • Started programming PL/I, SmallTalk, Oracle5
      • Director in Oracle Apps Development
      • Certified Oracle DBA, Sun Java Pr/Dev/Arch
  • 4. Compiere
    • The Open Source Business Solution
      • More than 240.000 downloads
      • Usually among Top 10 in SourceForce
    • Targeting Small-Medium Enterprise
      • ASP Solution
      • Accounting, Inventory, …
      • Worldwide (Language, Law, …)
    • J2EE compliant Application
      • Swing or HTML interface
  • 5. Agenda -- Persistence
    • What I do
    • Persistence
    • Performance
    • JDBC
      • Connection
      • Query
      • Result
  • 6. Persistence
    • Object meets Database
      • Put a square in a round
    • JDBC is not a Persistence Framework
    • Object-relational Mapping
      • Container Managed Persistence
      • Java Data Objects
  • 7. Persistence – Entity Bean CMP <enterprise-beans> <entity> <description>Models a music CD</description> <ejb-name>CDBean</ejb-name> <home>com.web_tomorrow.cd.CDHome</home> <remote>com.web_tomorrow.cd.CD</remote> <ejb-class>com.web_tomorrow.cd.CDBean</ejb-class> <persistence-type>Container</persistence-type> <prim-key-class> java.lang.String </prim-key-class> <reentrant>False</reentrant> <cmp-field><field-name> id </field-name></cmp-field> <cmp-field><field-name> title </field-name></cmp-field> <cmp-field><field-name> artist </field-name></cmp-field> <primkey-field> id </primkey-field> </entity> </entity-beans>
  • 8. Persistence – JDO
    • Java Data Objects
      • Transparent database access
      • Code that accesses the underlying data store, without using database-specific code
    • Use of JDO in J2EE
      • Persistent helper classes for Session Beans
      • Delegate classes for Bean or Container Managed Persistence Entity Beans
  • 9. JDBC Based Frameworks
    • Everyone (… and their Grandmother) writes JDBC based Persistence Libraries
      • Free or Commercial
      • Often with IDE integration
      • Examples
        • Borland
        • Oracle
        • Compiere (no IDE)
    30 % of typical IT project: DB programming & persistence mgt
  • 10. Agenda -- Performance
    • What I do
    • Persistence
    • Performance
    • JDBC
      • Connection
      • Query
      • Result
  • 11. Performance
    • Is a side effect of good OO design
      • Schema should be ≥ 3 rd Normal Form
      • That is want Optimizers recognize
    • For high-end databases (e.g. Oracle) there is no need for de-normalization
    • Performance Rules
      • #1 Clean design
      • #2 Tweak performance
    Don’t Design for Performance
  • 12. Performance Rules
    • Tune the Design
    • Tune the Application
    • Tune the OS
    • Tune Memory Structure
    • Tune Disk I/O Usage
    • Detect and Eliminate Resource Contention
  • 13. Performance Design Options
    • Data Intensive Routines in DB Server
      • Within DB Engine - Low Switching Cost
      • No Communication Overhead
      • PL/SQL, Java Stored Procedures, C++, …
    • “Automatic” data consistency (Helps the Optimizer)
      • Explicit Constraints
      • Trigger (in PL/SQL, Java, …)
  • 14. DB Programming Options
    • Set Processing outperforms everything else
      • Not your beginner SQL
        • Complex UPDATE statements
    • PL/SQL
      • Dependency Management (*)
      • Data intensive manipulations
    • Java Stored Procedures
      • Special VMs – Upload, Debugging
      • Special JDBC Driver “always connected”
  • 15. DB Programming Options
    • SQLJ
      • Java Language SQL Pre-compiler
      • High level SQLJ-Language
      • Converted to “normal” JDBC code
      • Supported by IBM, Oracle, …
  • 16. Question: Is SQL a Standard?
  • 17. DB Independence
    • Lowest common denominator
      • Use Access Functionality with Oracle ?
    • You have to design & program differently for the database / camps
      • Oracle, DB2, (PostgreSQL), …
      • Microsoft SQL Server, Sybase, …
      • MS Access, mySQL, …
  • 18. DB Incompatibilities (1)
    • SQL join
      • SELECT .. FROM a JOIN b ON (a.id=b.id);
      • SELECT .. FROM a LEFT OUTER JOIN b ON (a.id=b.id);
      • SELECT .. FROM a, b WHERE a.id=b.id(+);
      • SELECT .. FROM a, b WHERE a.id=*b.id;
    • Sub-Query
      • SELECT .. FROM a, (SELECT .. FROM b) AS b WHERE …
    • Pseudo-Columns
      • ROWNUM, SysDate
      • LIMIT
    • Reserved Words – Optional Syntax – Alias
  • 19. DB Incompatibilities (2)
    • Data Types
      • Not just naming (NUMBER vs. DECIMAL)
      • BLOB, CLOB, …
    • PL/SQL - Syntax/Functionality for Packages, Functions, Procedures, Triggers, …
      • Commit Strategy
      • Exception (raise & catch)
    • Data Dictionary
    • OLAP Support
      • Materialized Views, …
  • 20. DB Independence – cont.
    • DB Management Layer
      • Lowest Common SQL Denominator Alternative
      • Application Dependent
        • Uses Database specific Functionality
      • Examples:
        • SAP’s DB Access Layer
        • (CMP Products)
        • Compiere’s dbPort
    Oracle PG dbPort Application I am talking to Oracle DB/2 ?
  • 21. Performance Reality
    • You can’t have everything
      • High functionality & Ability to fine-tune
      • Database independence
        • Automatic Object Mapping
    • You need skill-mix
      • (Traditional) Database Design
      • Database Programming Environment
      • Object Design
      • Java Programming Environment
  • 22. The Test Environment
    • Windows 2000 SP 2 Server
      • Dual 450MHz CPU
      • Oracle 8.1.7 (parallel option not enabled)
      • JDBC driver for 9i
    • RedHat Linux 7.2
      • Single 500 MHz CPU
      • PostgreSQL 7.1.3
      • JDBC driver for 7.2
    • Java 1.4.0
    • SELECT * FROM AD_Column;
      • 4,800 Rows - 38 Columns – ca. 1,2 MB
    Testing DB Driver
  • 23. Agenda -- JDBC
    • What I do
    • Persistence
    • Performance
    • JDBC
      • Connection
      • Query
      • Result
  • 24. Connecting
    • Connection Options
      • java.sql.Driver & java.sql.DriverManager
      • javax.sql.DataSource
      • javax.sql.PooledConnection
      • Connection Cache
  • 25. JDBC Driver Types Type 4 Pure Java Type 3 Type 1 Type 2
  • 26. Connecting .. Driver
    • Type 1..4 Driver
      • Type 4: Pure Java Driver
      • DB/2 does not provide Type 4 driver
    • Interface java.sql.Driver
      • Explicitly
        • Class.forName(“org.postgresql.Driver&quot;);
        • new oracle.jdbc.OracleDriver();
      • System Properties
        • jdbc.drivers= oracle.jdbc.OracleDriver:org.postgresql.Driver
  • 27. Connecting DriverManager
    • Basic service for managing a set of JDBC drivers
    • JDBC Drivers register with the DriverManager
    • Source
      • Connection c = DriverManager.getConnection (&quot;jdbc:oracle:thin:@dev:1521:dev1&quot;);
    • Connection Strings
      • jdbc:oracle:oci8:@ -- (#1) Bequeath
      • jdbc:oracle:oci8:@ora -- (#3) Net8
      • jdbc:oracle:thin:@dev:1521:ora -- (#2) Type 4
      • jdbc:postgresql://dev:5432/compiere
  • 28. Connecting .. Data Source
    • Interface javax.sql.DataSource
      • Alternative to DriverManager
      • Based on JNDI
    • Server Source
      • DataSource ds = new OracleDataSource(); Context ctx = new Initial Context(); ctx.bind (“jdbc/myDB”, ds);
    • Client Source
      • Context ctx = new Initial Context(); DataSource ds = (DataSource)ctx.lookup(“jdbc/myDB”); Connection c = ds.getConnection();
  • 29. Connection .. Pool
    • Optional extension
    • One Physical Connection
    • Multiple Logical Connection
      • Connection.close() makes it available
    • Eliminates overhead of creating physical connection:
      • Connection c = pool.getConnection(); Statement stmt = c.createStatement(..); stmt.close(); c.close();
  • 30. Connection .. Cache
    • Optional Extension … no standard (!)
    • Multiple Physical Connections
    • We wrote own Connection Cache
      • Connection not closed in code
  • 31. Multi-Thread
    • Most JDBC implementations synchronize on Connection
      • You need multiple connections for multi-threaded applications
    • Compiere uses
      • 2 “read-only” never closed Cached Connections (w. health check)
      • 1 “update” never closed Connection
  • 32. Test Results (10 Threads)
  • 33. Test Results (10 Threads)
    • Multiple Pre-Created Connections
      • No xxx.getConnection()
      • Yield ‘surprise’
        • while (rs.next) yield();
        • (High) switching costs
        • Database has result in buffer
          • just transmitting
    • Connection Cache
      • Optimum Cache Size 2-4
  • 34. Test Results
    • Oracle
      • Use Thin Driver
        • jdbc:oracle:thin:@dev:1521:ora
      • Fastest Driver – Bequeath
        • &quot;jdbc:oracle:oci8:@“
        • 20-40% faster – Only works if only one instance installed
        • For Server Connections
      • Slowest Driver – Net8
        • jdbc:oracle:oci8:@ora
        • 10-20% slower
        • Net8 conversion Overhead
  • 35. Connection – Trx Level
    • Transaction Isolation Level
      • TRANSACTION_NONE
        • Not supported
      • TRANSACTION_READ_UNCOMMITTED
        • Get: dirty, non-repeatable, phantom reads
      • TRANSACTION_READ_COMMITTED
        • No: dirty reads (uncommitted data)
        • Get: non-repeatable, phantom reads
      • TRANSACTION_REPEATABLE_READ
        • No: dirty, non-repeatable (data changed) reads
        • Get: phantom reads
      • TRANSACTION_SERIALIZABLE
        • No: dirty reads, non-repeatable, phantom (WHERE changed) reads
  • 36. Connection
    • You probably want to get the latest data
      • conn.setTransactionIsolation (Connection.TRANSACTION_READ_COMMITTED);
      • Data updated by others & Triggers
      • Set explicitly
        • Connection is Interface
        • Default set by Vendor Implementation
    • Consider
      • Read Only – Hint to Driver/Database
        • conn.setReadOnly(true);
  • 37. Querying
    • Statement – conn.createStatement()
      • ResultSet.TYPE_
        • FORWARD_ONLY, SCROLL_INSENSITIVE, SCROLL_SENSITIVE
      • ResultSet.CONCUR_
        • READ_ONLY or _UPDATABLE
    • Prepared Statement – conn.prepareStatement()
      • Database dependent statement caching
      • “ UPDATE EMPLOYEES SET SALARY = ? WHERE ID = ?”
      • pstmt.setDataType (pos, value);
    • Callable Statement – conn.prepareCall()
      • “ {?= call myProcedure (?,?)}”
    • Batch [implementation not mandatory]
      • stmt.addBatch(“Insert ..”);
      • stmt.executeBatch();
  • 38. set DataType Conversion
  • 39. Result - ResultSet
    • Created by stmt.executeQuery()
    • Fetch Size
      • rs.setFetchSize(int);
      • stmt.setFetchSize(int);
      • VendorConnection.setFetchSize(int);
      • Driver hint
    • Non updateable RS
      • Forward only & read just once
    • Data Retrieval
      • get DataType (int pos);
      • get DataType (String columnName);
      • wasNull();
    No way to Cancel a Query
  • 40. get DataSet Conversion
  • 41. ResultSet
    • Navigate
      • absolute(int), relative(int) – first(), last(), next(), previous()
    • Update
      • updateDataType (pos, data); updateDataType (columnName, data);
      • updateRow(); – or – cancelRowUpdates();
    • Insert
      • moveToInsertRow(); updateDataType insertRow(); – or – cancelRowUpdates();
      • moveToCurrentRow();
    • Delete
      • deleteRow();
    • Refresh
      • refreshRow();
  • 42. Result - RowSet
    • JavaBean - Extends ResultSet
    • Types – Implementation Optional
      • Cached – serialized - disconnected - X-JVM - Synchronize
      • JDBC
      • Web
    • Code Example
      • RowSet rowset = new OracleCachedRowSet(); rowset.setUrl (CONNECTION); rowset.setUsername (UID); rowset.setPassword (PWD); rowset.setFetchSize (fetchSize); rowset. setCommand (SQL_STATEMENT); rowset. execute ();
    Client … RMI … no JDBC
  • 43. Meta Data Interfaces
    • DatabaseMetaData
      • Driver JDBC compatibility
      • Data Dictionary Queries
    • ParameterMetaData
      • Prepared Statement
    • ResultSetMetaData
      • Get Column Name, Data Type of “SELECT *” query
  • 44. Connection - Query - Retrieve First Connection, (Query) & Retrieval took significantly longer – 4,800 records
  • 45. Fetch Size
  • 46. Compiere JDBC Use
    • Async worker to load data
    • Select
      • Prepared Statement - Read-Only, Forward Only
      • Statement & ResultSet (cursor) closed after read
    • Insert
      • ResultSet from Prepared Statement (WHERE 1=2)
      • rs.moveToInsertRow(); rs.insertRow();
      • Re-read from Database using PK (trigger, ..)
    • Update
      • Re-read using ROWID or OID (“for update”)
      • Check, if data* not changed
        • rs.updateXX(); rs.updateRow();
        • or rs.cancelRowUpdates();
      • Re-read from Database using PK
    deterministic, optimistic locking
  • 47. Performance Hints
    • Qualify Columns in SELECT Clause
      • Gets only data needed
    • Use position of Column – not Name
      • rs.getString(1);
    • Give as much info as possible
      • WHERE clause
      • Scrollable/Updateable/…
    • Use Locks carefully
      • … and release them explicitly
    • Close Statements & ResultSets – closes DB cursors
      • rs.close(); stmt.close();
  • 48. Performance Hints (2)
    • Use Batches where possible
    • Connection.setAutoCommit (false)
    • Database Meta-Data Queries are slow
      • Quantify arguments, search patterns
    • Consider Use of
      • Statement.setFetchSize (int no)
      • Statement.setMaxRows (int no)
      • Statement.setQueryTimeout (int seconds)
      • Statement.setMaxFieldSize (int byte)
      • Hidden unique ID: RowID, OID
      • RowSet for complex mass update
    • Use Prepared Statements (*)
  • 49. Resources (Selection)
    • JDBC
      • http://java.sun.com/products/jdbc
        • Several courses and introductions
      • http://www.jguru.com/faq/JDBC
      • http://www.javaskyline.com/learnjdbc.html
    • JDO
      • http://access1.sun.com/jdo
      • http://www.jdocentral.com
  • 50. Thanks
    • You can download the presentation
      • http://www.compiere.org/download/
    • If you have questions
      • [email_address]
      • (203) 445-8182
    • Have a look
      • http://www.compiere.com/consulting.html

×