Hibernate Performance Tuning (JEEConf 2012)

Performance Tuning

@Sander_Mak


branchandbound.net
Hibernate sucks!
      ... because it’s slow
Hibernate sucks!
      ... because it’s slow
     ‘The problem is sort of cultural [..] developers
     use Hibernate because they are uncomfortable
     with SQL and with RDBMSes. You should be
     very comfortable with SQL and JDBC before you
     start using Hibernate - Hibernate builds on
     JDBC, it doesn’t replace it. That is the cost of
     extra abstraction [..] save yourself effort,
     pay attention to the database at all stages
     of development.’
                               - Gavin King (creator)
Hibernate Performance Tuning (JEEConf 2012)
‘Most of the performance problems we have come
up against have been solved not by code
optimizations, but by adding new functionality.’

                       - Gavin King (creator)
‘You can't communicate complexity,
only an awareness of it.’

     - Alan J. Perlis (1st Turing Award winner)
Outline
 Optimization
 Lazy loading
 Examples ‘from the trenches’
   Search queries
   Large collections
   Batching
 Odds & Ends
Optimization
Optimization is hard
 Performance blame
   Framework vs. You
 When to optimize?

 Preserve correctness at all times
   Unit tests ok, but not enough
   Automated integration tests
Optimization is hard
 Performance blame
   Framework vs. You
 When to optimize?

 Preserve correctness at all times
   Unit tests ok, but not enough
   Automated integration tests

  Premature optimization is the root of all evil
                                     - Donald Knuth
Optimization guidelines
Measurement
 Ensure stable, production-like environment
 Measure time and space
   Time: isolate timings in different layers
   Space: more heap -> longer GC -> slower
 Try to measure in RDBMS as well
   IO statistics (hot cache or disk thrashing?)
   Query plans
 Make many measurements -> automation
Optimization guidelines
Practical
 Profiler on DAO/Session.query() methods
 VisualVM etc. for heap usage
     many commercial tools also have
     built-in JDBC profiling
 Hibernate JMX
  <property name="hibernate.generate_statistics">true
  </property>




 RDBMS monitoring tools
Analyzing Hibernate
Log SQL:   <property name="show_sql">true</property>
           <property name="format_sql">true</property>


Log4J configuration:
  org.hibernate.SQL -> DEBUG
  org.hibernate.type -> TRACE (see bound params)

Or use P6Spy/Log4JDBC on JDBC connection
Analyzing Hibernate
    2011-07-28 09:57:12,061 DEBUG org.hibernate.SQL - insert into BASKET_LINE_ALLOC (LAST_UPDATED,      QUANTITY,
    CUSTOMER_REF, NOTES, BRANCH_ID, FUND_ID, TEMPLATE_ID,
    BASKET_LINE_ALLOC_ID) values (?, ?, ?, ?, ?, ?, ?, ?)


Log SQL:
    2011-07-28 09:57:12,081 DEBUG org.hibernate.type.TimestampType - binding '2006-07-28 09:57:12' to   parameter: 1
    2011-07-28 09:57:12,081 DEBUG org.hibernate.type.IntegerType - binding '3' to parameter: 2
                        <property name="show_sql">true</property>
    2011-07-28 09:57:12,082 DEBUG org.hibernate.type.StringType - binding '' to parameter: 3
    2011-07-28 09:57:12,082 DEBUG org.hibernate.type.StringType - binding '' to parameter: 4

                        <property name="format_sql">true</property>
    2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding '511' to parameter: 5
    2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding '512' to parameter: 6
    2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding null to parameter: 7
    2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding '180030' to parameter: 8
    Hibernate: INSERT INTO mkyong.stock_transaction (CHANGE, CLOSE, DATE, OPEN, STOCK_ID, VOLUME)

Log4J configuration:
    VALUES (?, ?, ?, ?, ?, ?)
    2011-07-28 13:33:07,253 DEBUG FloatType:133 - binding '10.0' to parameter: 1
    2011-07-28 13:33:07,253 DEBUG FloatType:133 - binding '1.1' to parameter: 2
    2011-07-28 13:33:07,253 DEBUG DateType:133 - binding '30 December 2009' to parameter: 3


  org.hibernate.SQL -> DEBUG
    2011-07-28 13:33:07,269 DEBUG FloatType:133 - binding '1.2' to parameter: 4
    2011-07-28 13:33:07,269 DEBUG IntegerType:133 - binding '11' to parameter: 5
    2011-07-28 13:33:07,269 DEBUG LongType:133 - binding '1000000' to parameter: 6
    2011-07-28 09:57:12,061 DEBUG org.hibernate.SQL - insert into BASKET_LINE_ALLOC (LAST_UPDATED,      QUANTITY,
    CUSTOMER_REF, NOTES, BRANCH_ID, FUND_ID, TEMPLATE_ID,

  org.hibernate.type -> TRACE (see bound params)
    BASKET_LINE_ALLOC_ID) values (?, ?, ?, ?, ?, ?, ?, ?)
    2011-07-28 09:57:12,081 DEBUG org.hibernate.type.TimestampType - binding '2006-07-28 09:57:12' to
    2011-07-28 09:57:12,081 DEBUG org.hibernate.type.IntegerType - binding '3' to parameter: 2
                                                                                                        parameter: 1

    2011-07-28 09:57:12,082 DEBUG org.hibernate.type.StringType - binding '' to parameter: 3
    2011-07-28 09:57:12,082 DEBUG org.hibernate.type.StringType - binding '' to parameter: 4
    2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding '511' to parameter: 5
    2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding '512' to parameter: 6
    2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding null to parameter: 7


Or use P6Spy/Log4JDBC on JDBC connection
    2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding '180030' to parameter: 8
    Hibernate: INSERT INTO mkyong.stock_transaction (CHANGE, CLOSE, DATE, OPEN, STOCK_ID, VOLUME)
    VALUES (?, ?, ?, ?, ?, ?)
    2011-07-28 13:33:07,253 DEBUG FloatType:133 - binding '10.0' to parameter: 1
    2011-07-28 13:33:07,253 DEBUG FloatType:133 - binding '1.1' to parameter: 2
    2011-07-28 13:33:07,253 DEBUG DateType:133 - binding '30 December 2009' to parameter: 3
    2011-07-28 13:33:07,269 DEBUG FloatType:133 - binding '1.2' to parameter: 4
    2011-07-28 13:33:07,269 DEBUG IntegerType:133 - binding '11' to parameter: 5
    2011-07-28 13:33:07,269 DEBUG LongType:133 - binding '1000000' to parameter: 6
Lazy loading
Lazy loading
One entity to rule them all                   Request

Mostly sane defaults:                                  1..1




@OneToOne,
@OneToMany,                                      User
@ManyToMany: LAZY                             1..*          *..1


@ManyToOne : EAGER
(Due to JPA spec.)            Authorization


Extra-lazy: Hibernate                            *..1



specific                         Global
                               Company
                                                     1..*          Company
Hibernate Performance Tuning (JEEConf 2012)
LazyInitializationException
Lazy loading          N+1 Selects problem

Select list of N users      HQL: SELECT u FROM User

Authorizations necessary:   SQL 1 query:
                            SELECT * FROM User
  N select queries on         LEFT JOIN Company c
  Authorization executed!     WHERE u.worksForCompany =
                              c.id
Solution:
  FETCH JOINS
  @Fetch(FetchMode.JOI
  N)
  @FetchProfile (enable
  per session
  don’t call .size()
Lazy loading          N+1 Selects problem

Select list of N users      HQL: SELECT u FROM User

Authorizations necessary:   SQL N queries:
                            SELECT * FROM Authorization
  N select queries on         WHERE userId = N
  Authorization executed!
Solution:
  FETCH JOINS
  @Fetch(FetchMode.JOI
  N)
  @FetchProfile (enable
  per session
  don’t call .size()
Lazy loading          N+1 Selects problem
                            HQL: SELECT u FROM User OUTER
Select list of N users      JOIN FETCH u.authorizations
Authorizations necessary:
                            SQL 1 query:
  N select queries on       SELECT * FROM User
  Authorization executed!     LEFT JOIN Company c LEFT
                              OUTER JOIN Authorization
Solution:                     ON .. WHERE
                              u.worksForCompany = c.id
  FETCH JOINS
  @Fetch(FetchMode.JOI
  N)
  @FetchProfile (enable
  per session
  don’t call .size()
Lazy loading
Some guidelines
 Laziness by default = mostly good
 However, architectural impact:
   Session lifetime (‘OpenSessionInView’ pattern)
   Extended Persistence Context
   Proxy usage (runtime code generation)
 Eagerness can be forced with HQL/JPAQL/Criteria
 But eagerness cannot be reverted
   exception: Session.load()/EntityManager.getReference()
Search queries
Search queries



                                     Search

                   User



                                                Result list
                1..*          *..1




Authorization

                   *..1


  Global               1..*           Company
 Company

                                                              Detail
Search queries
Obvious solution:

Too much information!
Use summary objects:



UserSummary = POJO
  not attached, only necessary fields (no relations)
Search queries
Obvious solution:

Too much information!
Use summary objects:



UserSummary = POJO
  not attached, only necessary fields (no relations)
  Or: drop down to JDBC to fetch id + fields
Search queries
Alternative:

Taking it further:




Pagination in queries, not in app. code
Extra count query may be necessary (totals)
         Ordering necessary for paging!
Search queries
          Subtle:
Alternative: effect of applying setMaxResults
          “The
           or setFirstResult to a query involving
         fetch joins over collections is undefined”
Taking it further:
         Cause: the emitted join possibly returns
             several rows per entity. So LIMIT,
            rownums, TOP cannot be used!
          Instead Hibernate must fetch all rows
         WARNING: firstResult/maxResults specified with
Pagination in queries, not in app. code
                  collection fetch; applying in memory!


Extra count query may be necessary (totals)
         Ordering necessary for paging!
Analytics & reporting
  ORM less relevant: no entities, but complex
  aggregations
  Simple sum/avg/counts possible over entities

  Specialized db calls for complex reports:
     Partitioning/windowing etc.
  Integrate using native query
  Or create a database view and map entity
Large collections
Large collections
      Frontend:
       Request
     CompanyGroup
          1..*


                             Backend:
        Company
                            CompanyGroup
                                  1..*
                                             Meta-data


Company read-only entity,   CompanyInGroup
backed by expensive view          *..1




                               Company
Large collections
  Frontend:
   Request
 CompanyGroup
      1..*


                    Backend:
   Company
                    CompanyGroup
                         1..*
                                   Meta-data

                CompanyInGroup
                         *..1




                      Company
Large collections
  Frontend:
   Request
 CompanyGroup
      1..*


                    Backend:
   Company
                    CompanyGroup
                         1..*
                                   Meta-data

                CompanyInGroup
                         *..1




                      Company
Large collections
 Opening large groups sluggish
 Improved performance:

 Fetches many uninitialized collections in 1 query
 Also possible on entity:

                                                 Request
                                               CompanyGroup
                                                     1..*




                                                 Company
Large collections
 Opening large groups sluggish
 Improved performance:

 Fetches many uninitialized collections in 1 query
 Also possible on entity:

                                                    Request
                                                  CompanyGroup
                                                       1..*




                                                    Company
       Better solution in hindsight: fetch join
Large collections
 Extra lazy collection fetching



 Efficient:
   companies.size() -> count query
   companies.contains() -> select 1 where ...
   companies.get(n) -> select * where index = n
Large collections
 Saving large group slow: >15 sec.
 Problem: Hibernate inserts row by row
   Query creation overhead, network latency
 Solution: <property name="hibernate.jdbc.batch_size">100
              </property>


 Enables JDBC batched statements
 Caution: global property                                 Request
                                                        CompanyGroup

 Also: <property name="hibernate.order_inserts">true
                                                              1..*




        </property>                                         Company
        <property name="hibernate.order_updates">true
        </property>
Large collections
  Frontend:
   Request
 CompanyGroup
      1..*


                    Backend:
   Company
                    CompanyGroup
                         1..*
                                   Meta-data

                CompanyInGroup
                         *..1




                      Company
Large collections


                    Backend:

                    CompanyGroup
                         1..*
                                   Meta-data

                CompanyInGroup
                         *..1




                      Company
Large collections
Process   CreateGroup (Soap)   Business
Service                         Service

 CreateGroup: ~10 min. for thousands of companies
 @BatchSize on Company improved demarshalling
 JDBC batch_size property marginal improvement
 INFO: INSERT INTO CompanyInGroup VALUES (?,...,?)
 INFO: SELECT @identity
 INFO: INSERT INTO CompanyInGroup VALUES (?,...,?)
 INFO: SELECT @identity
 .. 1000 times                                       CompanyGroup
                                                           1..*
                                                                      Meta-data


 Insert/select interleaved: due to gen. id           CompanyInGroup
                                                           *..1




                                                        Company
Large collections
Process    CreateGroup (Soap)   Business
Service                          Service

 Solution: generate id in app. (not always feasible)
 Running in ~3 minutes with batched inserts
 Next problem: heap usage spiking
 Use StatelessSession
  ✦ Bypass first-level cache
  ✦ No automatic dirty checking                     CompanyGroup

  ✦ Bypass Hibernate event model and interceptors         1..*
                                                                     Meta-data

  ✦ No cascading of operations                      CompanyInGroup
  ✦ Collections on entities are ignored                   *..1




                                                       Company
Large collections
Process   CreateGroup (Soap)   Business
Service                         Service

 Solution: generate id in app. (not always feasible)
 Running in ~3 minutes with batched inserts
 Next problem: heap usage spiking
 Use StatelessSession

                                              CompanyGroup
                                                    1..*
                                                               Meta-data

                                              CompanyInGroup
                                                    *..1




                                                 Company
Large collections
Process   CreateGroup (Soap)   Business
Service                         Service

 Now <1 min., everybody happy!




                                          CompanyGroup
                                                1..*
                                                           Meta-data

                                          CompanyInGroup
                                                *..1




                                             Company
Large collections
Process   CreateGroup (Soap)   Business
Service                         Service

 Now <1 min., everybody happy!


      Data loss detected!

                                          CompanyGroup
                                                1..*
                                                           Meta-data

                                          CompanyInGroup
                                                *..1




                                             Company
Large collections
Process   CreateGroup (Soap)   Business
Service                         Service   Data loss detected!

 StatelessSession and JDBC batch_size bug




 HHH-4042: Closed, won’t fix :
                                                    CompanyGroup
                                                         1..*
                                                                    Meta-data

                                                   CompanyInGroup
                                                         *..1




                                                      Company
Odds & Ends
Dirty little secret
                                            validated(item) performs
                                            read-only queries


                          select   currentItem from Catalog where ..
 Dirty collection after   select   spendingLimit from User where ..
 each iteration           insert   into Item values (?, ?, ?)
                          select   currentItem from Catalog where ..
                          select   spendingLimit from User where ..
 Batching fails           insert   into Item values (?, ?, ?)

 Flushmode.AUTO

 Loops always suspect: relational, set-based thinking
Dirty little secret
                                  validated(item) performs
                                  read-only queries



 Dirty collection after
 each iteration
 Batching fails
 Flushmode.AUTO

 Loops always suspect: relational, set-based thinking
Query hints
Speed up read-only service calls:



Hibernate Query.setHint():




Also: never use 2nd level cache just ‘because we can’
Query hints
Speed up read-only service calls:



Hibernate Query.setHint():




Also: never use 2nd level cache just ‘because we can’
     @org.hibernate.annotations.Immutable
Large updates
Naive approach:



Entities are not always necessary:



Changes are not reflected in persistence context
With optimistic concurrency: VERSIONED keyword
Large updates
Naive approach:



Entities are not always necessary:



Changes are not reflected in persistence context
With optimistic concurrency: VERSIONED keyword
       Consider use of stored procedures
Cherish your database
Data and schema outlive your application
Good indexes make a world of difference
Stored procedures etc. are not inherently evil
Do not let Hibernate dictate your schema
  Befriend a DBA instead!

There are other solutions (there I said it)
  MyBatis
  Squeryl (Scala)
Thanks for listening!


@Sander_Mak
                         Join me later today:
                     Elevate your webapps
                        with Scala & Lift!

                         17:00 Room C
branchandbound.net
1 of 54

Recommended

Hibernate performance tuning by
Hibernate performance tuningHibernate performance tuning
Hibernate performance tuningSander Mak (@Sander_Mak)
29.6K views53 slides
Hibernate Developer Reference by
Hibernate Developer ReferenceHibernate Developer Reference
Hibernate Developer ReferenceMuthuselvam RS
4.4K views105 slides
Spring Batch Workshop (advanced) by
Spring Batch Workshop (advanced)Spring Batch Workshop (advanced)
Spring Batch Workshop (advanced)lyonjug
4.8K views55 slides
Java EE 7 Batch processing in the Real World by
Java EE 7 Batch processing in the Real WorldJava EE 7 Batch processing in the Real World
Java EE 7 Batch processing in the Real WorldRoberto Cortez
23.5K views47 slides
Spring - Part 3 - AOP by
Spring - Part 3 - AOPSpring - Part 3 - AOP
Spring - Part 3 - AOPHitesh-Java
495 views25 slides
Data access by
Data accessData access
Data accessJoshua Yoon
264 views19 slides

More Related Content

What's hot

Java IO, Serialization by
Java IO, Serialization Java IO, Serialization
Java IO, Serialization Hitesh-Java
297 views45 slides
Testing database content with DBUnit. My experience. by
Testing database content with DBUnit. My experience.Testing database content with DBUnit. My experience.
Testing database content with DBUnit. My experience.Serhii Kartashov
1.6K views21 slides
Oracle High Availabiltity for application developers by
Oracle High Availabiltity for application developersOracle High Availabiltity for application developers
Oracle High Availabiltity for application developersAlexander Tokarev
753 views46 slides
Persistence hibernate by
Persistence hibernatePersistence hibernate
Persistence hibernateKrishnakanth Goud
629 views38 slides
P9 speed of-light faceted search via oracle in-memory option by alexander tok... by
P9 speed of-light faceted search via oracle in-memory option by alexander tok...P9 speed of-light faceted search via oracle in-memory option by alexander tok...
P9 speed of-light faceted search via oracle in-memory option by alexander tok...Alexander Tokarev
436 views68 slides
JDBC Part - 2 by
JDBC Part - 2JDBC Part - 2
JDBC Part - 2Hitesh-Java
414 views67 slides

What's hot(20)

Java IO, Serialization by Hitesh-Java
Java IO, Serialization Java IO, Serialization
Java IO, Serialization
Hitesh-Java297 views
Testing database content with DBUnit. My experience. by Serhii Kartashov
Testing database content with DBUnit. My experience.Testing database content with DBUnit. My experience.
Testing database content with DBUnit. My experience.
Serhii Kartashov1.6K views
Oracle High Availabiltity for application developers by Alexander Tokarev
Oracle High Availabiltity for application developersOracle High Availabiltity for application developers
Oracle High Availabiltity for application developers
Alexander Tokarev753 views
P9 speed of-light faceted search via oracle in-memory option by alexander tok... by Alexander Tokarev
P9 speed of-light faceted search via oracle in-memory option by alexander tok...P9 speed of-light faceted search via oracle in-memory option by alexander tok...
P9 speed of-light faceted search via oracle in-memory option by alexander tok...
Alexander Tokarev436 views
Silicon Valley JUG - How to generate customized java 8 code from your database by Speedment, Inc.
Silicon Valley JUG - How to generate customized java 8 code from your databaseSilicon Valley JUG - How to generate customized java 8 code from your database
Silicon Valley JUG - How to generate customized java 8 code from your database
Speedment, Inc.267 views
Java Hibernate Programming with Architecture Diagram and Example by kamal kotecha
Java Hibernate Programming with Architecture Diagram and ExampleJava Hibernate Programming with Architecture Diagram and Example
Java Hibernate Programming with Architecture Diagram and Example
kamal kotecha6.8K views
Hibernate - Part 1 by Hitesh-Java
Hibernate - Part 1Hibernate - Part 1
Hibernate - Part 1
Hitesh-Java131 views
Oracle SQL Tuning for Day-to-Day Data Warehouse Support by nkarag
Oracle SQL Tuning for Day-to-Day Data Warehouse SupportOracle SQL Tuning for Day-to-Day Data Warehouse Support
Oracle SQL Tuning for Day-to-Day Data Warehouse Support
nkarag1.1K views
Dao example by myrajendra
Dao exampleDao example
Dao example
myrajendra1.7K views
Dao pattern by ciriako
Dao patternDao pattern
Dao pattern
ciriako1.7K views
JSON, A Splash of SODA, and a SQL Chaser: Real-World Use Cases for Autonomous... by Jim Czuprynski
JSON, A Splash of SODA, and a SQL Chaser: Real-World Use Cases for Autonomous...JSON, A Splash of SODA, and a SQL Chaser: Real-World Use Cases for Autonomous...
JSON, A Splash of SODA, and a SQL Chaser: Real-World Use Cases for Autonomous...
Jim Czuprynski110 views
Oracle Text in APEX by Scott Wesley
Oracle Text in APEXOracle Text in APEX
Oracle Text in APEX
Scott Wesley2.8K views
Connecting Hadoop and Oracle by Tanel Poder
Connecting Hadoop and OracleConnecting Hadoop and Oracle
Connecting Hadoop and Oracle
Tanel Poder38.6K views

Viewers also liked

Java 7, 8 & 9 - Moving the language forward by
Java 7, 8 & 9 - Moving the language forwardJava 7, 8 & 9 - Moving the language forward
Java 7, 8 & 9 - Moving the language forwardMario Fusco
4.3K views13 slides
No more loops with lambdaj by
No more loops with lambdajNo more loops with lambdaj
No more loops with lambdajMario Fusco
34K views31 slides
Modularity in the Cloud by
Modularity in the CloudModularity in the Cloud
Modularity in the CloudSander Mak (@Sander_Mak)
2.6K views34 slides
Cross-Build Injection attacks: how safe is your Java build? by
Cross-Build Injection attacks: how safe is your Java build?Cross-Build Injection attacks: how safe is your Java build?
Cross-Build Injection attacks: how safe is your Java build?Sander Mak (@Sander_Mak)
3.7K views38 slides
Modular JavaScript by
Modular JavaScriptModular JavaScript
Modular JavaScriptSander Mak (@Sander_Mak)
4.1K views38 slides
Scala & Lift (JEEConf 2012) by
Scala & Lift (JEEConf 2012)Scala & Lift (JEEConf 2012)
Scala & Lift (JEEConf 2012)Sander Mak (@Sander_Mak)
2.5K views33 slides

Viewers also liked(20)

Java 7, 8 & 9 - Moving the language forward by Mario Fusco
Java 7, 8 & 9 - Moving the language forwardJava 7, 8 & 9 - Moving the language forward
Java 7, 8 & 9 - Moving the language forward
Mario Fusco4.3K views
No more loops with lambdaj by Mario Fusco
No more loops with lambdajNo more loops with lambdaj
No more loops with lambdaj
Mario Fusco34K views
Performance Tuning with JPA 2.1 and Hibernate (Geecon Prague 2015) by Thorben Janssen
Performance Tuning with JPA 2.1 and Hibernate (Geecon Prague 2015)Performance Tuning with JPA 2.1 and Hibernate (Geecon Prague 2015)
Performance Tuning with JPA 2.1 and Hibernate (Geecon Prague 2015)
Thorben Janssen3.4K views
Hibernate performance tuning by Igor Dmitriev
Hibernate performance tuningHibernate performance tuning
Hibernate performance tuning
Igor Dmitriev257 views
Delphi ORM SOA MVC SQL NoSQL JSON REST mORMot by Arnaud Bouchez
Delphi ORM SOA MVC SQL NoSQL JSON REST mORMotDelphi ORM SOA MVC SQL NoSQL JSON REST mORMot
Delphi ORM SOA MVC SQL NoSQL JSON REST mORMot
Arnaud Bouchez13.2K views
Hoons닷넷 좌충우돌 10년, 그리고 새로운 패러다임 by KH Park (박경훈)
Hoons닷넷 좌충우돌 10년, 그리고 새로운 패러다임Hoons닷넷 좌충우돌 10년, 그리고 새로운 패러다임
Hoons닷넷 좌충우돌 10년, 그리고 새로운 패러다임
KH Park (박경훈)4.1K views
2013 빅데이터 및 API 기술 현황과 전망- 윤석찬 by Channy Yun
2013 빅데이터 및 API 기술 현황과 전망- 윤석찬2013 빅데이터 및 API 기술 현황과 전망- 윤석찬
2013 빅데이터 및 API 기술 현황과 전망- 윤석찬
Channy Yun61.1K views
RPC에서 REST까지 간단한 개념소개 by Wonchang Song
RPC에서 REST까지 간단한 개념소개RPC에서 REST까지 간단한 개념소개
RPC에서 REST까지 간단한 개념소개
Wonchang Song43K views
Mysql Fulltext Search by johnymas
Mysql Fulltext SearchMysql Fulltext Search
Mysql Fulltext Search
johnymas3.9K views
Monetising your startup from the word go with advertising & affiliates by Digi Joe
Monetising your startup from the word go with advertising & affiliatesMonetising your startup from the word go with advertising & affiliates
Monetising your startup from the word go with advertising & affiliates
Digi Joe2.1K views
Lean startup Methodology by ali raza
Lean startup MethodologyLean startup Methodology
Lean startup Methodology
ali raza665 views

Similar to Hibernate Performance Tuning (JEEConf 2012)

Accelerated data access by
Accelerated data accessAccelerated data access
Accelerated data accessgordonyorke
772 views19 slides
Expanding your impact with programmability in the data center by
Expanding your impact with programmability in the data centerExpanding your impact with programmability in the data center
Expanding your impact with programmability in the data centerCisco Canada
348 views27 slides
Mastering Test Automation: How To Use Selenium Successfully by
Mastering Test Automation: How To Use Selenium SuccessfullyMastering Test Automation: How To Use Selenium Successfully
Mastering Test Automation: How To Use Selenium SuccessfullySpringPeople
1.5K views61 slides
Testing the frontend by
Testing the frontendTesting the frontend
Testing the frontendHeiko Hardt
284 views40 slides
JavaOne 2009 - TS-5276 - RESTful Protocol Buffers by
JavaOne 2009 - TS-5276 - RESTful  Protocol BuffersJavaOne 2009 - TS-5276 - RESTful  Protocol Buffers
JavaOne 2009 - TS-5276 - RESTful Protocol BuffersMatt O'Keefe
11.7K views38 slides
Apache Calcite Tutorial - BOSS 21 by
Apache Calcite Tutorial - BOSS 21Apache Calcite Tutorial - BOSS 21
Apache Calcite Tutorial - BOSS 21Stamatis Zampetakis
662 views83 slides

Similar to Hibernate Performance Tuning (JEEConf 2012)(20)

Accelerated data access by gordonyorke
Accelerated data accessAccelerated data access
Accelerated data access
gordonyorke772 views
Expanding your impact with programmability in the data center by Cisco Canada
Expanding your impact with programmability in the data centerExpanding your impact with programmability in the data center
Expanding your impact with programmability in the data center
Cisco Canada348 views
Mastering Test Automation: How To Use Selenium Successfully by SpringPeople
Mastering Test Automation: How To Use Selenium SuccessfullyMastering Test Automation: How To Use Selenium Successfully
Mastering Test Automation: How To Use Selenium Successfully
SpringPeople1.5K views
Testing the frontend by Heiko Hardt
Testing the frontendTesting the frontend
Testing the frontend
Heiko Hardt284 views
JavaOne 2009 - TS-5276 - RESTful Protocol Buffers by Matt O'Keefe
JavaOne 2009 - TS-5276 - RESTful  Protocol BuffersJavaOne 2009 - TS-5276 - RESTful  Protocol Buffers
JavaOne 2009 - TS-5276 - RESTful Protocol Buffers
Matt O'Keefe11.7K views
performancetestingjmeter-121109061704-phpapp02 by Gopi Raghavendra
performancetestingjmeter-121109061704-phpapp02performancetestingjmeter-121109061704-phpapp02
performancetestingjmeter-121109061704-phpapp02
Gopi Raghavendra372 views
performancetestingjmeter-121109061704-phpapp02 (1) by QA Programmer
performancetestingjmeter-121109061704-phpapp02 (1)performancetestingjmeter-121109061704-phpapp02 (1)
performancetestingjmeter-121109061704-phpapp02 (1)
QA Programmer251 views
Angular Optimization Web Performance Meetup by David Barreto
Angular Optimization Web Performance MeetupAngular Optimization Web Performance Meetup
Angular Optimization Web Performance Meetup
David Barreto386 views
Practical SQL query monitoring and optimization by Ivo Andreev
Practical SQL query monitoring and optimizationPractical SQL query monitoring and optimization
Practical SQL query monitoring and optimization
Ivo Andreev2.2K views
Adding a modern twist to legacy web applications by Jeff Durta
Adding a modern twist to legacy web applicationsAdding a modern twist to legacy web applications
Adding a modern twist to legacy web applications
Jeff Durta345 views
Productionalizing ML : Real Experience by Ihor Bobak
Productionalizing ML : Real ExperienceProductionalizing ML : Real Experience
Productionalizing ML : Real Experience
Ihor Bobak519 views
Breaking Dependencies To Allow Unit Testing - Steve Smith | FalafelCON 2014 by FalafelSoftware
Breaking Dependencies To Allow Unit Testing - Steve Smith | FalafelCON 2014Breaking Dependencies To Allow Unit Testing - Steve Smith | FalafelCON 2014
Breaking Dependencies To Allow Unit Testing - Steve Smith | FalafelCON 2014
FalafelSoftware671 views
Breaking Dependencies to Allow Unit Testing by Steven Smith
Breaking Dependencies to Allow Unit TestingBreaking Dependencies to Allow Unit Testing
Breaking Dependencies to Allow Unit Testing
Steven Smith1.6K views
6 tips for improving ruby performance by Engine Yard
6 tips for improving ruby performance6 tips for improving ruby performance
6 tips for improving ruby performance
Engine Yard6.6K views
Intro To Spring Python by gturnquist
Intro To Spring PythonIntro To Spring Python
Intro To Spring Python
gturnquist5.8K views
Performance testing and j meter by Purna Chandar
Performance testing and j meterPerformance testing and j meter
Performance testing and j meter
Purna Chandar460 views
Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery and DB2 by Vladimir Bacvanski, PhD
Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery and DB2Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery and DB2
Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery and DB2
High Performance NodeJS by Dicoding
High Performance NodeJSHigh Performance NodeJS
High Performance NodeJS
Dicoding1.7K views
Googleappengineintro 110410190620-phpapp01 by Tony Frame
Googleappengineintro 110410190620-phpapp01Googleappengineintro 110410190620-phpapp01
Googleappengineintro 110410190620-phpapp01
Tony Frame758 views

More from Sander Mak (@Sander_Mak)

Scalable Application Development @ Picnic by
Scalable Application Development @ PicnicScalable Application Development @ Picnic
Scalable Application Development @ PicnicSander Mak (@Sander_Mak)
239 views20 slides
Coding Your Way to Java 13 by
Coding Your Way to Java 13Coding Your Way to Java 13
Coding Your Way to Java 13Sander Mak (@Sander_Mak)
414 views80 slides
Coding Your Way to Java 12 by
Coding Your Way to Java 12Coding Your Way to Java 12
Coding Your Way to Java 12Sander Mak (@Sander_Mak)
2K views69 slides
Java Modularity: the Year After by
Java Modularity: the Year AfterJava Modularity: the Year After
Java Modularity: the Year AfterSander Mak (@Sander_Mak)
913 views109 slides
Desiging for Modularity with Java 9 by
Desiging for Modularity with Java 9Desiging for Modularity with Java 9
Desiging for Modularity with Java 9Sander Mak (@Sander_Mak)
1.9K views80 slides
Modules or microservices? by
Modules or microservices?Modules or microservices?
Modules or microservices?Sander Mak (@Sander_Mak)
5.9K views117 slides

More from Sander Mak (@Sander_Mak)(20)

Recently uploaded

The Research Portal of Catalonia: Growing more (information) & more (services) by
The Research Portal of Catalonia: Growing more (information) & more (services)The Research Portal of Catalonia: Growing more (information) & more (services)
The Research Portal of Catalonia: Growing more (information) & more (services)CSUC - Consorci de Serveis Universitaris de Catalunya
79 views25 slides
Top 10 Strategic Technologies in 2024: AI and Automation by
Top 10 Strategic Technologies in 2024: AI and AutomationTop 10 Strategic Technologies in 2024: AI and Automation
Top 10 Strategic Technologies in 2024: AI and AutomationAutomationEdge Technologies
18 views14 slides
Data-centric AI and the convergence of data and model engineering: opportunit... by
Data-centric AI and the convergence of data and model engineering:opportunit...Data-centric AI and the convergence of data and model engineering:opportunit...
Data-centric AI and the convergence of data and model engineering: opportunit...Paolo Missier
39 views40 slides
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas... by
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...Bernd Ruecker
33 views69 slides
Info Session November 2023.pdf by
Info Session November 2023.pdfInfo Session November 2023.pdf
Info Session November 2023.pdfAleksandraKoprivica4
11 views15 slides
Microsoft Power Platform.pptx by
Microsoft Power Platform.pptxMicrosoft Power Platform.pptx
Microsoft Power Platform.pptxUni Systems S.M.S.A.
52 views38 slides

Recently uploaded(20)

Data-centric AI and the convergence of data and model engineering: opportunit... by Paolo Missier
Data-centric AI and the convergence of data and model engineering:opportunit...Data-centric AI and the convergence of data and model engineering:opportunit...
Data-centric AI and the convergence of data and model engineering: opportunit...
Paolo Missier39 views
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas... by Bernd Ruecker
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
Bernd Ruecker33 views
PharoJS - Zürich Smalltalk Group Meetup November 2023 by Noury Bouraqadi
PharoJS - Zürich Smalltalk Group Meetup November 2023PharoJS - Zürich Smalltalk Group Meetup November 2023
PharoJS - Zürich Smalltalk Group Meetup November 2023
Noury Bouraqadi126 views
Black and White Modern Science Presentation.pptx by maryamkhalid2916
Black and White Modern Science Presentation.pptxBlack and White Modern Science Presentation.pptx
Black and White Modern Science Presentation.pptx
maryamkhalid291616 views
Piloting & Scaling Successfully With Microsoft Viva by Richard Harbridge
Piloting & Scaling Successfully With Microsoft VivaPiloting & Scaling Successfully With Microsoft Viva
Piloting & Scaling Successfully With Microsoft Viva
Unit 1_Lecture 2_Physical Design of IoT.pdf by StephenTec
Unit 1_Lecture 2_Physical Design of IoT.pdfUnit 1_Lecture 2_Physical Design of IoT.pdf
Unit 1_Lecture 2_Physical Design of IoT.pdf
StephenTec12 views
Perth MeetUp November 2023 by Michael Price
Perth MeetUp November 2023 Perth MeetUp November 2023
Perth MeetUp November 2023
Michael Price19 views
Special_edition_innovator_2023.pdf by WillDavies22
Special_edition_innovator_2023.pdfSpecial_edition_innovator_2023.pdf
Special_edition_innovator_2023.pdf
WillDavies2217 views
handbook for web 3 adoption.pdf by Liveplex
handbook for web 3 adoption.pdfhandbook for web 3 adoption.pdf
handbook for web 3 adoption.pdf
Liveplex22 views
STPI OctaNE CoE Brochure.pdf by madhurjyapb
STPI OctaNE CoE Brochure.pdfSTPI OctaNE CoE Brochure.pdf
STPI OctaNE CoE Brochure.pdf
madhurjyapb13 views
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive by Network Automation Forum
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLiveAutomating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Business Analyst Series 2023 - Week 3 Session 5 by DianaGray10
Business Analyst Series 2023 -  Week 3 Session 5Business Analyst Series 2023 -  Week 3 Session 5
Business Analyst Series 2023 - Week 3 Session 5
DianaGray10237 views
The details of description: Techniques, tips, and tangents on alternative tex... by BookNet Canada
The details of description: Techniques, tips, and tangents on alternative tex...The details of description: Techniques, tips, and tangents on alternative tex...
The details of description: Techniques, tips, and tangents on alternative tex...
BookNet Canada126 views
Case Study Copenhagen Energy and Business Central.pdf by Aitana
Case Study Copenhagen Energy and Business Central.pdfCase Study Copenhagen Energy and Business Central.pdf
Case Study Copenhagen Energy and Business Central.pdf
Aitana16 views

Hibernate Performance Tuning (JEEConf 2012)

  • 2. Hibernate sucks! ... because it’s slow
  • 3. Hibernate sucks! ... because it’s slow ‘The problem is sort of cultural [..] developers use Hibernate because they are uncomfortable with SQL and with RDBMSes. You should be very comfortable with SQL and JDBC before you start using Hibernate - Hibernate builds on JDBC, it doesn’t replace it. That is the cost of extra abstraction [..] save yourself effort, pay attention to the database at all stages of development.’ - Gavin King (creator)
  • 5. ‘Most of the performance problems we have come up against have been solved not by code optimizations, but by adding new functionality.’ - Gavin King (creator)
  • 6. ‘You can't communicate complexity, only an awareness of it.’ - Alan J. Perlis (1st Turing Award winner)
  • 7. Outline Optimization Lazy loading Examples ‘from the trenches’ Search queries Large collections Batching Odds & Ends
  • 9. Optimization is hard Performance blame Framework vs. You When to optimize? Preserve correctness at all times Unit tests ok, but not enough Automated integration tests
  • 10. Optimization is hard Performance blame Framework vs. You When to optimize? Preserve correctness at all times Unit tests ok, but not enough Automated integration tests Premature optimization is the root of all evil - Donald Knuth
  • 11. Optimization guidelines Measurement Ensure stable, production-like environment Measure time and space Time: isolate timings in different layers Space: more heap -> longer GC -> slower Try to measure in RDBMS as well IO statistics (hot cache or disk thrashing?) Query plans Make many measurements -> automation
  • 12. Optimization guidelines Practical Profiler on DAO/Session.query() methods VisualVM etc. for heap usage many commercial tools also have built-in JDBC profiling Hibernate JMX <property name="hibernate.generate_statistics">true </property> RDBMS monitoring tools
  • 13. Analyzing Hibernate Log SQL: <property name="show_sql">true</property> <property name="format_sql">true</property> Log4J configuration: org.hibernate.SQL -> DEBUG org.hibernate.type -> TRACE (see bound params) Or use P6Spy/Log4JDBC on JDBC connection
  • 14. Analyzing Hibernate 2011-07-28 09:57:12,061 DEBUG org.hibernate.SQL - insert into BASKET_LINE_ALLOC (LAST_UPDATED, QUANTITY, CUSTOMER_REF, NOTES, BRANCH_ID, FUND_ID, TEMPLATE_ID, BASKET_LINE_ALLOC_ID) values (?, ?, ?, ?, ?, ?, ?, ?) Log SQL: 2011-07-28 09:57:12,081 DEBUG org.hibernate.type.TimestampType - binding '2006-07-28 09:57:12' to parameter: 1 2011-07-28 09:57:12,081 DEBUG org.hibernate.type.IntegerType - binding '3' to parameter: 2 <property name="show_sql">true</property> 2011-07-28 09:57:12,082 DEBUG org.hibernate.type.StringType - binding '' to parameter: 3 2011-07-28 09:57:12,082 DEBUG org.hibernate.type.StringType - binding '' to parameter: 4 <property name="format_sql">true</property> 2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding '511' to parameter: 5 2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding '512' to parameter: 6 2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding null to parameter: 7 2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding '180030' to parameter: 8 Hibernate: INSERT INTO mkyong.stock_transaction (CHANGE, CLOSE, DATE, OPEN, STOCK_ID, VOLUME) Log4J configuration: VALUES (?, ?, ?, ?, ?, ?) 2011-07-28 13:33:07,253 DEBUG FloatType:133 - binding '10.0' to parameter: 1 2011-07-28 13:33:07,253 DEBUG FloatType:133 - binding '1.1' to parameter: 2 2011-07-28 13:33:07,253 DEBUG DateType:133 - binding '30 December 2009' to parameter: 3 org.hibernate.SQL -> DEBUG 2011-07-28 13:33:07,269 DEBUG FloatType:133 - binding '1.2' to parameter: 4 2011-07-28 13:33:07,269 DEBUG IntegerType:133 - binding '11' to parameter: 5 2011-07-28 13:33:07,269 DEBUG LongType:133 - binding '1000000' to parameter: 6 2011-07-28 09:57:12,061 DEBUG org.hibernate.SQL - insert into BASKET_LINE_ALLOC (LAST_UPDATED, QUANTITY, CUSTOMER_REF, NOTES, BRANCH_ID, FUND_ID, TEMPLATE_ID, org.hibernate.type -> TRACE (see bound params) BASKET_LINE_ALLOC_ID) values (?, ?, ?, ?, ?, ?, ?, ?) 2011-07-28 09:57:12,081 DEBUG org.hibernate.type.TimestampType - binding '2006-07-28 09:57:12' to 2011-07-28 09:57:12,081 DEBUG org.hibernate.type.IntegerType - binding '3' to parameter: 2 parameter: 1 2011-07-28 09:57:12,082 DEBUG org.hibernate.type.StringType - binding '' to parameter: 3 2011-07-28 09:57:12,082 DEBUG org.hibernate.type.StringType - binding '' to parameter: 4 2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding '511' to parameter: 5 2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding '512' to parameter: 6 2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding null to parameter: 7 Or use P6Spy/Log4JDBC on JDBC connection 2011-07-28 09:57:12,082 DEBUG org.hibernate.type.LongType - binding '180030' to parameter: 8 Hibernate: INSERT INTO mkyong.stock_transaction (CHANGE, CLOSE, DATE, OPEN, STOCK_ID, VOLUME) VALUES (?, ?, ?, ?, ?, ?) 2011-07-28 13:33:07,253 DEBUG FloatType:133 - binding '10.0' to parameter: 1 2011-07-28 13:33:07,253 DEBUG FloatType:133 - binding '1.1' to parameter: 2 2011-07-28 13:33:07,253 DEBUG DateType:133 - binding '30 December 2009' to parameter: 3 2011-07-28 13:33:07,269 DEBUG FloatType:133 - binding '1.2' to parameter: 4 2011-07-28 13:33:07,269 DEBUG IntegerType:133 - binding '11' to parameter: 5 2011-07-28 13:33:07,269 DEBUG LongType:133 - binding '1000000' to parameter: 6
  • 16. Lazy loading One entity to rule them all Request Mostly sane defaults: 1..1 @OneToOne, @OneToMany, User @ManyToMany: LAZY 1..* *..1 @ManyToOne : EAGER (Due to JPA spec.) Authorization Extra-lazy: Hibernate *..1 specific Global Company 1..* Company
  • 19. Lazy loading N+1 Selects problem Select list of N users HQL: SELECT u FROM User Authorizations necessary: SQL 1 query: SELECT * FROM User N select queries on LEFT JOIN Company c Authorization executed! WHERE u.worksForCompany = c.id Solution: FETCH JOINS @Fetch(FetchMode.JOI N) @FetchProfile (enable per session don’t call .size()
  • 20. Lazy loading N+1 Selects problem Select list of N users HQL: SELECT u FROM User Authorizations necessary: SQL N queries: SELECT * FROM Authorization N select queries on WHERE userId = N Authorization executed! Solution: FETCH JOINS @Fetch(FetchMode.JOI N) @FetchProfile (enable per session don’t call .size()
  • 21. Lazy loading N+1 Selects problem HQL: SELECT u FROM User OUTER Select list of N users JOIN FETCH u.authorizations Authorizations necessary: SQL 1 query: N select queries on SELECT * FROM User Authorization executed! LEFT JOIN Company c LEFT OUTER JOIN Authorization Solution: ON .. WHERE u.worksForCompany = c.id FETCH JOINS @Fetch(FetchMode.JOI N) @FetchProfile (enable per session don’t call .size()
  • 22. Lazy loading Some guidelines Laziness by default = mostly good However, architectural impact: Session lifetime (‘OpenSessionInView’ pattern) Extended Persistence Context Proxy usage (runtime code generation) Eagerness can be forced with HQL/JPAQL/Criteria But eagerness cannot be reverted exception: Session.load()/EntityManager.getReference()
  • 24. Search queries Search User Result list 1..* *..1 Authorization *..1 Global 1..* Company Company Detail
  • 25. Search queries Obvious solution: Too much information! Use summary objects: UserSummary = POJO not attached, only necessary fields (no relations)
  • 26. Search queries Obvious solution: Too much information! Use summary objects: UserSummary = POJO not attached, only necessary fields (no relations) Or: drop down to JDBC to fetch id + fields
  • 27. Search queries Alternative: Taking it further: Pagination in queries, not in app. code Extra count query may be necessary (totals) Ordering necessary for paging!
  • 28. Search queries Subtle: Alternative: effect of applying setMaxResults “The or setFirstResult to a query involving fetch joins over collections is undefined” Taking it further: Cause: the emitted join possibly returns several rows per entity. So LIMIT, rownums, TOP cannot be used! Instead Hibernate must fetch all rows WARNING: firstResult/maxResults specified with Pagination in queries, not in app. code collection fetch; applying in memory! Extra count query may be necessary (totals) Ordering necessary for paging!
  • 29. Analytics & reporting ORM less relevant: no entities, but complex aggregations Simple sum/avg/counts possible over entities Specialized db calls for complex reports: Partitioning/windowing etc. Integrate using native query Or create a database view and map entity
  • 31. Large collections Frontend: Request CompanyGroup 1..* Backend: Company CompanyGroup 1..* Meta-data Company read-only entity, CompanyInGroup backed by expensive view *..1 Company
  • 32. Large collections Frontend: Request CompanyGroup 1..* Backend: Company CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  • 33. Large collections Frontend: Request CompanyGroup 1..* Backend: Company CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  • 34. Large collections Opening large groups sluggish Improved performance: Fetches many uninitialized collections in 1 query Also possible on entity: Request CompanyGroup 1..* Company
  • 35. Large collections Opening large groups sluggish Improved performance: Fetches many uninitialized collections in 1 query Also possible on entity: Request CompanyGroup 1..* Company Better solution in hindsight: fetch join
  • 36. Large collections Extra lazy collection fetching Efficient: companies.size() -> count query companies.contains() -> select 1 where ... companies.get(n) -> select * where index = n
  • 37. Large collections Saving large group slow: >15 sec. Problem: Hibernate inserts row by row Query creation overhead, network latency Solution: <property name="hibernate.jdbc.batch_size">100 </property> Enables JDBC batched statements Caution: global property Request CompanyGroup Also: <property name="hibernate.order_inserts">true 1..* </property> Company <property name="hibernate.order_updates">true </property>
  • 38. Large collections Frontend: Request CompanyGroup 1..* Backend: Company CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  • 39. Large collections Backend: CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  • 40. Large collections Process CreateGroup (Soap) Business Service Service CreateGroup: ~10 min. for thousands of companies @BatchSize on Company improved demarshalling JDBC batch_size property marginal improvement INFO: INSERT INTO CompanyInGroup VALUES (?,...,?) INFO: SELECT @identity INFO: INSERT INTO CompanyInGroup VALUES (?,...,?) INFO: SELECT @identity .. 1000 times CompanyGroup 1..* Meta-data Insert/select interleaved: due to gen. id CompanyInGroup *..1 Company
  • 41. Large collections Process CreateGroup (Soap) Business Service Service Solution: generate id in app. (not always feasible) Running in ~3 minutes with batched inserts Next problem: heap usage spiking Use StatelessSession ✦ Bypass first-level cache ✦ No automatic dirty checking CompanyGroup ✦ Bypass Hibernate event model and interceptors 1..* Meta-data ✦ No cascading of operations CompanyInGroup ✦ Collections on entities are ignored *..1 Company
  • 42. Large collections Process CreateGroup (Soap) Business Service Service Solution: generate id in app. (not always feasible) Running in ~3 minutes with batched inserts Next problem: heap usage spiking Use StatelessSession CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  • 43. Large collections Process CreateGroup (Soap) Business Service Service Now <1 min., everybody happy! CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  • 44. Large collections Process CreateGroup (Soap) Business Service Service Now <1 min., everybody happy! Data loss detected! CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  • 45. Large collections Process CreateGroup (Soap) Business Service Service Data loss detected! StatelessSession and JDBC batch_size bug HHH-4042: Closed, won’t fix : CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  • 47. Dirty little secret validated(item) performs read-only queries select currentItem from Catalog where .. Dirty collection after select spendingLimit from User where .. each iteration insert into Item values (?, ?, ?) select currentItem from Catalog where .. select spendingLimit from User where .. Batching fails insert into Item values (?, ?, ?) Flushmode.AUTO Loops always suspect: relational, set-based thinking
  • 48. Dirty little secret validated(item) performs read-only queries Dirty collection after each iteration Batching fails Flushmode.AUTO Loops always suspect: relational, set-based thinking
  • 49. Query hints Speed up read-only service calls: Hibernate Query.setHint(): Also: never use 2nd level cache just ‘because we can’
  • 50. Query hints Speed up read-only service calls: Hibernate Query.setHint(): Also: never use 2nd level cache just ‘because we can’ @org.hibernate.annotations.Immutable
  • 51. Large updates Naive approach: Entities are not always necessary: Changes are not reflected in persistence context With optimistic concurrency: VERSIONED keyword
  • 52. Large updates Naive approach: Entities are not always necessary: Changes are not reflected in persistence context With optimistic concurrency: VERSIONED keyword Consider use of stored procedures
  • 53. Cherish your database Data and schema outlive your application Good indexes make a world of difference Stored procedures etc. are not inherently evil Do not let Hibernate dictate your schema Befriend a DBA instead! There are other solutions (there I said it) MyBatis Squeryl (Scala)
  • 54. Thanks for listening! @Sander_Mak Join me later today: Elevate your webapps with Scala & Lift! 17:00 Room C branchandbound.net

Editor's Notes

  1. Sander Mak - lead developer Java - Info Support\nDutch accent\nHibernate experience, not committer who knows everything\n
  2. Q: who has heard / said / thought this?\nLeaky abstraction -&gt; not going to defend ORM, many advantages\n1) mapping problems -&gt; impedance mismatch\n2) performance problems -&gt; stop treating Hibernate as blackbox!!\n
  3. Tangle of 37\nRed = bad -&gt; cyclic dependency\nHibernate implementation complex, but battle-tested \nJPA tutorials rosy picture: Using Hibernate can be quite hard!\n
  4. Tangle of 37\nRed = bad -&gt; cyclic dependency\nHibernate implementation complex, but battle-tested \nJPA tutorials rosy picture: Using Hibernate can be quite hard!\n
  5. Tangle of 37\nRed = bad -&gt; cyclic dependency\nHibernate implementation complex, but battle-tested \nJPA tutorials rosy picture: Using Hibernate can be quite hard!\n
  6. Examples use Hibernate/JPA API interchangeably: start with JPA, you will Hibernate specifics\n\n
  7. \n
  8. Tuning performance is a bit like refactoring: don&amp;#x2019;t change the semantics, just the how.\n\nPreserving correctness: unit tests! However, the more you reach the edges of the DBMS, the easier you will hit an obscure bug in query optimizer, caching strategy etc.\n\n
  9. Know your RDBMS! Database independence is nice when porting is necessary, but focus on particular DB for production situation (document!), that is what counts!\n (once you get into nitty-gritty opt. details, you will have to know the RDBMS intimately)\nIndex, covering indexes, locking strategies, &amp;#x2018;vacuuming&amp;#x2019;/&amp;#x2018;transaction logs&amp;#x2019;/reset statistics\n
  10. Hardware vs. virtualized, real data volumes, simulate real workloads\n
  11. SQL Server mgmt studio\n
  12. Question hear most often: how to see parameter values\n
  13. \n
  14. Beware: you might retrieve your whole database in one go...\n\nCode example: will load Company eager, Auths. lazy\nExtra-lazy: discuss later with large collections\n
  15. First encounter with lazy loading :)\n\nExtended persistence contexts, OpenSessionInView pattern and other band-aids\n\n
  16. First encounter with lazy loading :)\n\nExtended persistence contexts, OpenSessionInView pattern and other band-aids\n\n
  17. First encounter with lazy loading :)\n\nExtended persistence contexts, OpenSessionInView pattern and other band-aids\n\n
  18. Eager vs. lazy is contract specifying WHEN relations are retrieved, not HOW. For the HOW you can define fetching strategies.\n\nAlso possible to define fetch join on Criteria queries\n\n\n\n\n
  19. Eager vs. lazy is contract specifying WHEN relations are retrieved, not HOW. For the HOW you can define fetching strategies.\n\nAlso possible to define fetch join on Criteria queries\n\n\n\n\n
  20. Eager vs. lazy is contract specifying WHEN relations are retrieved, not HOW. For the HOW you can define fetching strategies.\n\nAlso possible to define fetch join on Criteria queries\n\n\n\n\n
  21. Eager vs. lazy is contract specifying WHEN relations are retrieved, not HOW. For the HOW you can define fetching strategies.\n\nAlso possible to define fetch join on Criteria queries\n\n\n\n\n
  22. Eager vs. lazy is contract specifying WHEN relations are retrieved, not HOW. For the HOW you can define fetching strategies.\n\nAlso possible to define fetch join on Criteria queries\n\n\n\n\n
  23. Eager vs. lazy is contract specifying WHEN relations are retrieved, not HOW. For the HOW you can define fetching strategies.\n\nAlso possible to define fetch join on Criteria queries\n\n\n\n\n
  24. Lazy as default, tune eager loading in queries specifically for your usecase (DAO pattern not that bad after all)\n
  25. \n
  26. \n
  27. Zelfde overwegingen gelden voor reporting queries, zoveel mogelijk in de query oplossen en geen entities teruggeven als niet nodig\n
  28. \n
  29. Zelfde overwegingen gelden voor reporting queries, zoveel mogelijk in de query oplossen en geen entities teruggeven als niet nodig\n
  30. Hibernate is specifically not for bulk manipulation: use stored procs for that. But when is something bulk? Collections with thousands of elements routinely in OLTP applications.\n
  31. \n
  32. \n
  33. \n
  34. \n
  35. Example: 20 groups with uninitialized collections, access first collection: all are initialized with 1 query.\nMeasured: opening first group was slightly slower, general user experience better\n
  36. \n
  37. \n
  38. \n
  39. \n
  40. \n
  41. \n
  42. \n
  43. stateless session ideaal for fire-and-forget service calls, minder in user-facing applicatie waar consistent houden persistence context van belang is.\n
  44. stateless session ideaal for fire-and-forget service calls, minder in user-facing applicatie waar consistent houden persistence context van belang is.\n
  45. \n
  46. \n
  47. \n
  48. Only &amp;#x2018;full batches&amp;#x2019; were performed\n
  49. \n
  50. Each item is flushed automatically before doing the lookup queries, because the collection becomes dirty after adding element\n
  51. Each item is flushed automatically before doing the lookup queries, because the collection becomes dirty after adding element\n
  52. Hibernate does some optimizing for read-only entities:\nIt saves execution time by not dirty-checking simple properties or single-ended associations. \nIt saves memory by deleting database snapshot\n\ncache increases load on memory, possibly more GC pauses for app. if co-located with application\n
  53. Interesting: all instances of entity are evicted from second level cache with such a query, even if WHERE clause limits affected entities\nAlso, no events fired as Hibernate normally would do.\n
  54. Interesting: all instances of entity are evicted from second level cache with such a query, even if WHERE clause limits affected entities\nAlso, no events fired as Hibernate normally would do.\n
  55. \n
  56. \n