Java Developers, make the database work for you (NLJUG JFall 2010)


Published on

The general consensus among Java developers has evolved from a dogmatic strive for database independence to a much more pragmatic wish to leverage the power of the database. This session demonstrates some of the (hidden) powers of the database and how these can be utilized from Java applications using either straight JDBC or working through JPA. The Oracle database is used as example: SQL for Aggregation and Analysis, Flashback Queries for historical comparison and trends, Virtual Private Database, complex validation, PL/SQL and collections for bulk data manipulation, view and instead-of triggers for data model morphing, server push of relevant data changes, edition based redefinition for release management.

- overview of role of database in JEE architecture (and a little history on how the database is perceived through the years)
- discussion on the development of database functionality
- demonstration of some powerful database features
- description of how we leveraged these features in our JSF (RichFaces)/JPA (Hibernate) application
- demo of web application based on these features
- discussion on how to approach the database

Published in: Technology
1 Comment
  • Thanks Lucas, you are absolute right: Java developer, use database power where appropriate. This philosophy also counts for the 'classic PL/SQL developer': learn an object-oriented language like Java and use it's power. A real professional has more tools in his box.
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Most Java Applications work with persistent data – one way or another
    Various technologies are available for Java to connect to a RDBMS
    Almost all access to the database happens across JDBC
  • Database SystemImplementation LanguageMicrosoft SQL ServerTransact-SQL and various .NET Framework languagesOraclePL/SQL or JavaDB2SQL/PL or JavaInformixSPLPostgreSQLPL/pgSQL, can also use own function languages such as pl/perl or pl/phpFirebirdPSQL (Fyracle also supports portions of Oracle's PL/SQL)MySQLown stored procedures, closely adhering to SQL:2003 standard.
  • Screenshot:
    Frank sends email to Maggie – with a query on Employees
    After some time, a response is sent to this particular email – by the queue listener using the JSP to send
    (list of employee data, corresponding with “query”)
  • Java Developers, make the database work for you (NLJUG JFall 2010)

    1. 1. Java developers: make the database work for you Lucas Jellema AMIS
    2. 2. Plain JDBC Ibatis, Spring JPA (Hibernate) EJB (CMP) WS* JDBC RDBMSRDBMS “NO SQL” Cache Data Grid Java Applications & Database
    3. 3. Position of Database • It almost feels like “a necessary evil” • Database is abstracted away as much as possible • It’s the persistent data store – It does CRUD (Create, Retrieve, Update & Delete) • What else could there be to it?
    4. 4. Database (Vendor) Independence • Applications should not depend on a specific vendor’s database – Only use common functionality (available in ‘all’) – Do not leverage special features of any database • Abstract database away through frameworks – Use generic and/or generated SQL • Do as little as possible relating to the RDBMS – … even if the organization happens to have enterprise editions and specialized db veterans
    5. 5. “We could also do that in the database” • in the database? Huh? RDBMSRDBMS ≈
    6. 6. Stored Procedures • Stored Procedures executing procedural programming units – PL/SQL, Transact-SQL, SQL/PL, SPL, pl/perl, pl/php, … – Java Stored Procedures
    7. 7. After the polarization (peak 2002) pragmatism struck… • EJB 2.x => JPA and EJB 3.0 (JEE 5) • Consensus – Leverage enterprise database for what it is good at (you pay for it, may as well use it) – Most applications will only be used on one vendor’s database ever – Even portable applications should still leverage database strengths • Through generic APIs with database specific implementations
    8. 8. Project: VP - Rich UI, Complex Data Manipulation JSF (Rich Faces) SEAM JPA (Hibernate) Oracle RDBMS Oracle RDBMS
    9. 9. Team & Design that combines strengths of all technologies… - Ease and Elegance of Implementation - Functionality (in an affordable way) - Productivity - Performance JSF (Rich Faces) SEAM JPA (Hibernate) Oracle RDBMSOracle RDBMS
    10. 10. Database Strengths • Integrity • Fine grained (data) security and auditing • Data Retrieval – joining tables together, leveraging indexes – hierarchical, network-like traversals – advanced analytics, historical queries, mining • Aggregation and Sorting • Complex & Massive Data Manipulation
    11. 11. Zooming in – using Oracle • Have to pick one • Largest market-share • Fairly representative (e.g. ANSI SQL) • The one I personally know best Oracle RDBMSOracle RDBMS
    12. 12. Sample Data Model: Departments & Employees
    13. 13. Primary, Unique and Foreign Key Constraints • Definition in Database is Declarative • Implementation is optimized • Imagine the programming and performance cost of a middle tier based implementation
    14. 14. Database RDBMS not always exclusively accessed through one Java API SOA, ESB, WebServices Data Replication & Synchronization Batch Bulk Processes Standard Applications Legacy Applications
    15. 15. Other data constraints • Not Null • Data Type: – string, numeric, date (time), xml – maximum length, integer/floating point • Data Rules – COMM < 0.3 * SAL – COMM IS NULL or JOB = ‘SALESMAN’ – MGR != EMPNO • Implemented using Column Definitions and Check Constraints
    16. 16. Database Triggers – decorating Data Manipulation • Triggers execute before or after Insert, Update or Delete of database records Employees Before Insert trigger: sal=… insert,update ,delete
    17. 17. Purpose of triggers • Set default values on new records – if :new.job=‘SALESMAN’ then :new.comm = 1000 • Calculate & Derive values upon insert, update or delete • Notify third parties of data manipulation • Perform complex validation on the data changes applied by the transaction – Per Department: Max Salary < 1.8 * Average – Per Manager: #subordinates < 15
    18. 18. JPA refreshing entities after triggers have applied new values @Entity @Table(name = "EMP") public class Employee … @Column(name=“sal”) private Double salary persist Employees Before Insert trigger: sal=… @ReturnInsert //EclLnk @Generated (value= GenerationTime.INSERT) // Hibernate
    19. 19. The far reaches of SQL vs the limit(itation)s of JPQL • Many Java ORM-frameworks – including JPA via Hibernate or EclipseLink – generate SQL – Usually “database independent” SQL – By and large only leveraging the common functionality across databases • As a consequence: – Many Java applications do not exploit the wealth of (the SQL of) their databases – And use what they do leverage in a what is possibly a suboptimal way
    20. 20. Aggregation & Rollup • Data for reporting purposes can be prepared by database queries – Including aggregations (max/min/avg/count/sum) – and Sub Totals – and Grand Total – and String Aggregation
    21. 21. Sub and Grand totals with Rollup • Rollup instructs database to aggregate at every level starting from the right – deptno, job – deptno – (grand total) • Also see: – Cube – Grouping Sets
    22. 22. Analytical Functions – spreadsheet-style row processing • Analytical Functions allow SQL queries to perform inter-row comparison & aggregation • For example: in a single query, for each employee – show salary rank in department and job – show salary difference with colleague next higher in rank (on the list per department) – show average salary in the department – show csv list of colleagues in department
    23. 23. Analytical Functions - example
    24. 24. Flashback Query select emp.* , dept.dname from emp AS OF TIMESTAMP (SYSTIMESTAMP - INTERVAL '1' DAY) , dept where emp.deptno = dept.deptno
    25. 25. Flashback Versions • Retrieve all states each record has been in – Every transaction that touched a row left a version of it – Pseudocolumns: xid, operation, starttime, endtime
    26. 26. Employee Version-history with Analytical and Flashback
    27. 27. Trees
    28. 28. Trees
    29. 29. Retrieving Hierarchical data sets with single SQL statements • Database has optimized algorithms – Starting at any node in the tree or network – Drilling down to the specified number of levels – Order siblings within parent – Indicate leaf and parent nodes; detect cycles EMPID ENAME MGR DEPTNO LEVEL --------------- ---------- ---------- ---------- ---------- 7839 KING 10 1 7698 BLAKE 7839 30 2 7499 ALLEN 7698 30 3 7900 JAMES 7698 30 3 7654 MARTIN 7698 30 3 7844 TURNER 7698 30 3 7521 WARD 7698 30 3 7782 CLARK 7839 10 2 7934 MILLER 7782 10 3
    30. 30. Encapsulate Database specific SQL in a View API • Views – for encapsulation of data model, multi-table join, (advanced) SQL hiding, authorization rules – Note: a view looks like a table to the client View
    31. 31. The read-only cursor API • A Cursor is a reference to a query result set • Database can open a cursor for a SQL query • And return it to the application to fetch the rows from • Cursor == JDBC ResultSet • A cursor can be nested: contain details … Employees Stored Procedure Departments cursor JDBC ResultSet while { … }
    32. 32. Cursor for Master-Detail resultset Stored Procedure
    33. 33. Using Complex Views for Hiding Legacy Data Models
    34. 34. Providing a ‘business object’ API • DML API: a View – aided by an Instead Of trigger • Insert of one new row in USERS_VW (e.g. a JPA persist operation) can actually be four new records – USER, PERSON, EMAIL_TYPE EMAIL_ADDRESS USERS PERSONS EMAIL_ ADDRESSES EMAIL_TYPE USERS Instead Of DML trigger * * * *
    35. 35. The Hollywood Principle: Query ResultSet Change Notification Cache
    36. 36. Cache Refresh triggered by DB PL/SQL Cache SQL query Register DatabaseChange Notification Oracle RDBMS invokes Java Listener with event details
    37. 37. Reaching out from the database Database
    38. 38. Email conversations
    39. 39. Database receiving and sending emails – from people or applications
    40. 40. RESTful resource navigation
    41. 41. RESTful architecture RESTful PL/SQL API exposed through dbms_epg http http http
    42. 42. JEE Application Server Enterprise Service Bus ? Database informing and leveraging the middle tier HTTP calls using the UTL_HTTP package
    43. 43. Other Database Features worth investigating • Virtual Private Database & Fine Grained Authorization • XMLType, XMLDB & FTP/HTTP/WEBDAV server • Object Types and Collections • Data type Interval & Time Zone support • Fine Grained Auditing • System Triggers, for example “after logon” • (Global) Application Context • Autonomous Transaction • Advanced Queuing (& JMS interaction) • Creating advanced job execution schedules • Edition Based Redefinition (versioning of database objects) • Statistics and Data Mining • Virtual Columns
    44. 44. Summary & Conclusions • Databases can do much more than • Java applications can benefit! • Strike the right balance: – Leverage database for what it can do best • Make Java and Database work together in a smooth way
    45. 45. Use the right tool for the job • Render HTML • Enforce Application Logic • Handle User Interaction • Create graphics • Interact with Internet • (bulk) copy of data • Guard Uniqueness • (large) Sort or Aggregation • (complex) SQL • Enforce data rules
    46. 46. Summary & Conclusions • Databases can do much more than • Java applications can benefit! • Strike the right balance: • Make Java and Database work together • Cater for ‘multiple database consumers’ • Acquire expertise on your team • Arrive at architectural design choices and best development practices
    47. 47. Best Practices & Principles • Prevent data to travel to the middle tier unless it has to – Performance (network and object instantiation) & Resource Usage (memory) • When data is on the middle tier: ensure it has the required freshness • Encapsulate database (specific) functionality – NO SQL (in the middle tier) – Decoupling and database (vendor) & framework independence
    48. 48. Best Practices & Principles • Use Views and Stored Procedures to create APIs that encapsulate database functionality – Note: the database brings constraints and triggers to the party – weaved in like Aspects – Cursors mapping to ResultSets allow retrieval of nested data structures through simple calls • Leverage the database for what it’s worth • Include ‘database developer’ in your team • Never be dogmatic
    49. 49. Want to know more? • Have the sources for the demos • Have this presentation presented & discussed at your organization • Learn about Java and the Database (Oracle) • Inject (Oracle) Database expertise – in the context of Java development - into your team • Receive a paper with more details on ‘making the database work for you & for ’ • Send me an email: • Visit our blog:
    50. 50. Master Class ‘Java Developers make the database work for you’ • Friday 17 December 2010 (AMIS, Nieuwegein): – One day master class: ‘Java Developer make the database work for you’ • For information and registration: –