Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Performance Tuning
Sander Mak
Outline
 Optimization
 Lazy loading
 Examples ‘from the trenches’
   Search queries
   Large collections
   Batching
 Odds...
Optimization
Hibernate Sucks!
      ... because it’s slow
Hibernate Sucks!
      ... because it’s slow
     ‘The problem is sort of cultural [..] developers
     use Hibernate beca...
Optimization is hard
 Performance blamed on?
   Framework <-> You
 How to optimize?
 When to optimize?
 Production vs. dev...
Optimization is hard
 Performance blamed on?
   Framework <-> You
 How to optimize?
 When to optimize?
 Production vs. dev...
Optimization guidelines
 Always measure
   Establish baseline and measure improvement
   Be prepared to detect performance...
Optimization guidelines
Measurement
 Ensure stable, production-like environment
 Measure time and space
   Time: isolate t...
Optimization guidelines
Measurement
 Ensure stable, production-like environment
 Measure time and space
   Time: isolate t...
Optimization guidelines
Practical
 Profiler on DAO/Session.query() methods
 VisualVM etc. for heap usage
   Also: JRockit M...
Optimization guidelines
Practical
 Example from BMG tuning session:
Analyzing Hibernate
Log SQL:   <property name="show_sql">true</property>
           <property name="format_sql">true</prop...
Analyzing Hibernate
Log SQL:   <property name="show_sql">true</property>
           <property name="format_sql">true</prop...
Analyzing Hibernate
Lazy loading
Lazy loading
One entity to rule them all                   Request

Mostly sane defaults:                                 ...
Lazy loading
                               Request
                                        1..1




                     ...
LazyInitializationException
Lazy loading          N+1 Selects problem

Select list of N users      HQL: SELECT u FROM User

Authorizations necessary: ...
Lazy loading          N+1 Selects problem

Select list of N users      HQL: SELECT u FROM User

Authorizations necessary: ...
Lazy loading          N+1 Selects problem

Select list of N users      HQL: SELECT u FROM User
                           ...
Lazy loading
Some guidelines
 Laziness by default = mostly good
 However, architectural impact:
   Session lifetime (‘Open...
Search queries
Search queries




                   User

                1..*          *..1




Authorization

                   *..1
...
Search queries
Obvious solution:

Too much information!
Use summary objects:



UserSummary = POJO
  not attached, only ne...
Search queries
Obvious solution:

Too much information!
Use summary objects:



UserSummary = POJO
  not attached, only ne...
Search queries
Alternative:

Taking it further:




Pagination in queries, not in app. code
Extra count query may be neces...
Search queries
          Subtle:
Alternative: effect of applying setMaxResults
          “The
           or setFirstResult...
Large collections
Large collections
      Frontend:
       Request
     CompanyGroup
          1..*


                             Backend:
...
Large collections
  Frontend:
   Request
 CompanyGroup
      1..*


                    Backend:
   Company
              ...
Large collections
  Frontend:
   Request
 CompanyGroup
      1..*


                    Backend:
   Company
              ...
Large collections
 Opening large groups sluggish
 Improved performance:

 Fetches many uninitialized collections in 1 quer...
Large collections
 Opening large groups sluggish
 Improved performance:

 Fetches many uninitialized collections in 1 quer...
Large collections
 Saving large group slow: >15 sec.
 Problem: Hibernate inserts row by row
   Query creation overhead, ne...
Large collections
  Frontend:
   Request
 CompanyGroup
      1..*


                    Backend:
   Company
              ...
Large collections


                    Backend:

                    CompanyGroup
                         1..*
         ...
Large collections
Process   CreateGroup (Soap)   Business
Service                         Service

 CreateGroup took ~10 m...
Large collections
Process    CreateGroup (Soap)   Business
Service                          Service

 Solution: generate i...
Large collections
Process   CreateGroup (Soap)   Business
Service                         Service

 Solution: generate id ...
Large collections
Process   CreateGroup (Soap)   Business
Service                         Service

 Now ~1 min., everybody...
Large collections
Process   CreateGroup (Soap)   Business
Service                         Service

 Now ~1 min., everybody...
Large collections
Process   CreateGroup (Soap)   Business
Service                         Service   Data loss detected!

 ...
Odds & Ends
Query hints
Speed up read-only service calls:



Hibernate Query.setHint():




Also: never use 2nd level cache just ‘beca...
Query hints
Speed up read-only service calls:



Hibernate Query.setHint():




Also: never use 2nd level cache just ‘beca...
Large updates
Naive approach:



Entities are not always necessary:



Changes are not reflected in persistence context
Wit...
Large updates
Naive approach:



Entities are not always necessary:



Changes are not reflected in persistence context
Wit...
Dynamic updates
@org.hibernate.annotations.Entity (dynamicInsert =
true, dynamicUpdate = true )

Only changed columns upda...
Cherish your database
Data and schema probably live longer than app.
Good indexes make a world of difference
Stored proced...
Questions
Hibernate performance tuning
Upcoming SlideShare
Loading in …5
×

Hibernate performance tuning

48,182 views

Published on

This presentation explores several common performance pitfalls when using Hibernate for persistence, both from a theoretic and practical perspective.

Published in: Technology
  • Very nice introduction. You can also check by High-Performance Hibernate tutorials: http://vladmihalcea.com/tutorials/hibernate/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @sujitdhaygude3 Thanks for the kind words! I suggest heading over to https://forum.hibernate.org/ for discussing specific Hibernate questions.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • good consolidated tips for hibernate, where should i post the questions i have would like to discuss
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @mipper it's an internal company guideline, sorry (this slidedeck was originally created for my co-workers, but I've since presented it at other occassions as well)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • What's the endeavour JPA guideline referenced on slide 52? A google search doesn't find anything relevant.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Hibernate performance tuning

  1. 1. Performance Tuning Sander Mak
  2. 2. Outline Optimization Lazy loading Examples ‘from the trenches’ Search queries Large collections Batching Odds & Ends
  3. 3. Optimization
  4. 4. Hibernate Sucks! ... because it’s slow
  5. 5. Hibernate Sucks! ... because it’s slow ‘The problem is sort of cultural [..] developers use Hibernate because they are uncomfortable with SQL and with RDBMSes. You should be very comfortable with SQL and JDBC before you start using Hibernate - Hibernate builds on JDBC, it doesn’t replace it. That is the cost of extra abstraction [..] save yourself effort, pay attention to the database at all stages of development. - Gavin King (creator)
  6. 6. Optimization is hard Performance blamed on? Framework <-> You How to optimize? When to optimize? Production vs. development Preserve correctness at all times Unit tests ok, but not enough
  7. 7. Optimization is hard Performance blamed on? Framework <-> You How to optimize? When to optimize? Production vs. development Preserve correctness at all times Unit tests ok, but not enough Premature optimization is the root of all evil - Donald Knuth
  8. 8. Optimization guidelines Always measure Establish baseline and measure improvement Be prepared to detect performance degradation! Favor obvious over obscure Even if latter performs better Know your RDBMS (hook up with a good DBA) Details do matter Database is not a dumb ‘bit-bucket’
  9. 9. Optimization guidelines Measurement Ensure stable, production-like environment Measure time and space Time: isolate timings in different layers Space: more heap -> longer GC -> slower Try to measure in RDBMS as well Memory usage (hot cache or disk thrashing?) Query plans Make many measurements
  10. 10. Optimization guidelines Measurement Ensure stable, production-like environment Measure time and space Time: isolate timings in different layers Space: more heap -> longer GC -> slower Try to measure in RDBMS as well Memory usage (hot cache or disk thrashing?) Query plans Make many measurements Remember: measurements are relative
  11. 11. Optimization guidelines Practical Profiler on DAO/Session.query() methods VisualVM etc. for heap usage Also: JRockit Mission Control Hibernate JMX <property name="hibernate.generate_statistics">true </property> RDBMS monitoring tools
  12. 12. Optimization guidelines Practical Example from BMG tuning session:
  13. 13. Analyzing Hibernate Log SQL: <property name="show_sql">true</property> <property name="format_sql">true</property> Log4J configuration: org.hibernate -> DEBUG org.hibernate.type -> FINE (see bound params) Or use P6Spy to snoop on JDBC connection
  14. 14. Analyzing Hibernate Log SQL: <property name="show_sql">true</property> <property name="format_sql">true</property> Log4J configuration: org.hibernate -> DEBUG org.hibernate.type -> FINE (see bound params) Or use P6Spy to snoop on JDBC connection
  15. 15. Analyzing Hibernate
  16. 16. Lazy loading
  17. 17. Lazy loading One entity to rule them all Request Mostly sane defaults: 1..1 @OneToOne, @OneToMany, User @ManyToMany: LAZY 1..* *..1 @ManyToOne : EAGER Due to JPA spec. Authorization *..1 Global 1..* Company Company
  18. 18. Lazy loading Request 1..1 User 1..* *..1 Authorization *..1 Global 1..* Company Company
  19. 19. LazyInitializationException
  20. 20. Lazy loading N+1 Selects problem Select list of N users HQL: SELECT u FROM User Authorizations necessary: SQL 1 query: SELECT * FROM User N select queries on LEFT JOIN Company c Authorization executed! WHERE u.worksForCompany = c.id Solution: FETCH JOINS
  21. 21. Lazy loading N+1 Selects problem Select list of N users HQL: SELECT u FROM User Authorizations necessary: SQL N queries: SELECT * FROM Authorization N select queries on WHERE userId = N Authorization executed! Solution: FETCH JOINS
  22. 22. Lazy loading N+1 Selects problem Select list of N users HQL: SELECT u FROM User JOIN FETCH u.authorizations Authorizations necessary: SQL 1 query: N select queries on SELECT * FROM User Authorization executed! LEFT JOIN Company c LEFT OUTER JOIN Authorization ON .. WHERE Solution: u.worksForCompany = c.id FETCH JOINS
  23. 23. Lazy loading Some guidelines Laziness by default = mostly good However, architectural impact: Session lifetime (‘OpenSessionInView’ pattern) Extended Persistence Context Proxy usage (runtime code generation) Eagerness can be forced with HQL/JPAQL/Criteria But eagerness cannot be revoked exception: Session.load()/EntityManager.getReference()
  24. 24. Search queries
  25. 25. Search queries User 1..* *..1 Authorization *..1 Global 1..* Company Company
  26. 26. Search queries Obvious solution: Too much information! Use summary objects: UserSummary = POJO not attached, only necessary fields (no relations)
  27. 27. Search queries Obvious solution: Too much information! Use summary objects: UserSummary = POJO not attached, only necessary fields (no relations) Or: drop down to JDBC to fetch id + fields
  28. 28. Search queries Alternative: Taking it further: Pagination in queries, not in app. code Extra count query may be necessary (totals) Ordering necessary for paging!
  29. 29. Search queries Subtle: Alternative: effect of applying setMaxResults “The or setFirstResult to a query involving fetch joins over collections is undefined” Taking it further: Cause: the emitted join possibly returns several rows per entity. So LIMIT, rownums, TOP cannot be used! Instead Hibernate must fetch all rows WARNING: firstResult/maxResults specified with Pagination in queries, not in app. code collection fetch; applying in memory! Extra count query may be necessary (totals) Ordering necessary for paging!
  30. 30. Large collections
  31. 31. Large collections Frontend: Request CompanyGroup 1..* Backend: Company CompanyGroup 1..* Meta-data Company read-only entity, CompanyInGroup backed by expensive view *..1 Company
  32. 32. Large collections Frontend: Request CompanyGroup 1..* Backend: Company CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  33. 33. Large collections Frontend: Request CompanyGroup 1..* Backend: Company CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  34. 34. Large collections Opening large groups sluggish Improved performance: Fetches many uninitialized collections in 1 query Also possible on entity: Request CompanyGroup 1..* Company
  35. 35. Large collections Opening large groups sluggish Improved performance: Fetches many uninitialized collections in 1 query Also possible on entity: Request CompanyGroup 1..* Company Better solution in hindsight: fetch join
  36. 36. Large collections Saving large group slow: >15 sec. Problem: Hibernate inserts row by row Query creation overhead, network latency Solution: <property name="hibernate.jdbc.batch_size">100 </property> Enables JDBC batched statements Caution: global property Request CompanyGroup Also: <property name="hibernate.order_inserts">true 1..* </property> Company <property name="hibernate.order_updates">true </property>
  37. 37. Large collections Frontend: Request CompanyGroup 1..* Backend: Company CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  38. 38. Large collections Backend: CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  39. 39. Large collections Process CreateGroup (Soap) Business Service Service CreateGroup took ~10 min. for 1000 companies @BatchSize on Company improved demarshalling JDBC batch_size property marginal improvement INFO: INSERT INTO CompanyInGroup VALUES (?,...,?) INFO: SELECT @identity INFO: INSERT INTO CompanyInGroup VALUES (?,...,?) INFO: SELECT @identity .. 1000 times CompanyGroup 1..* Meta-data Insert/select interleaved: due to gen. id CompanyInGroup *..1 Company
  40. 40. Large collections Process CreateGroup (Soap) Business Service Service Solution: generate id in app. (not always feasible) Running in ~3 minutes with batched inserts Next problem: heap usage spiking Use StatelessSession ✦ Bypass first-level cache ✦ No automatic dirty checking CompanyGroup ✦ Bypass Hibernate event model and interceptors 1..* Meta-data ✦ No cascading of operations CompanyInGroup ✦ Collections on entities are ignored *..1 Company
  41. 41. Large collections Process CreateGroup (Soap) Business Service Service Solution: generate id in app. (not always feasible) Running in ~3 minutes with batched inserts Next problem: heap usage spiking Use StatelessSession CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  42. 42. Large collections Process CreateGroup (Soap) Business Service Service Now ~1 min., everybody happy! CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  43. 43. Large collections Process CreateGroup (Soap) Business Service Service Now ~1 min., everybody happy! Data loss detected! CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  44. 44. Large collections Process CreateGroup (Soap) Business Service Service Data loss detected! StatelessSession and JDBC batch_size bug Only ‘full’ batches are performed! HHH-4042: Closed, won’t fix : CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
  45. 45. Odds & Ends
  46. 46. Query hints Speed up read-only service calls: Hibernate Query.setHint(): Also: never use 2nd level cache just ‘because we can’
  47. 47. Query hints Speed up read-only service calls: Hibernate Query.setHint(): Also: never use 2nd level cache just ‘because we can’ @org.hibernate.annotations.Entity(mutable = false)
  48. 48. Large updates Naive approach: Entities are not always necessary: Changes are not reflected in persistence context With optimistic concurrency: VERSIONED keyword
  49. 49. Large updates Naive approach: Entities are not always necessary: Changes are not reflected in persistence context With optimistic concurrency: VERSIONED keyword Consider use of stored procedures
  50. 50. Dynamic updates @org.hibernate.annotations.Entity (dynamicInsert = true, dynamicUpdate = true ) Only changed columns updated/inserted Beneficial for tables with many columns Downsides Runtime SQL generation overhead No PreparedStatement (caching) used anymore
  51. 51. Cherish your database Data and schema probably live longer than app. Good indexes make a world of difference Stored procedures etc. are not inherently evil Do not let Hibernate dictate your schema Though sometimes you are forced There are other solutions (there I said it) Read the Endeavour JPA Guideline!
  52. 52. Questions

×