Hibernate performance tuning

44,861 views

Published on

This presentation explores several common performance pitfalls when using Hibernate for persistence, both from a theoretic and practical perspective.

Published in: Technology
7 Comments
84 Likes
Statistics
Notes
  • Very nice introduction. You can also check by High-Performance Hibernate tutorials: http://vladmihalcea.com/tutorials/hibernate/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @sujitdhaygude3 Thanks for the kind words! I suggest heading over to https://forum.hibernate.org/ for discussing specific Hibernate questions.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • good consolidated tips for hibernate, where should i post the questions i have would like to discuss
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @mipper it's an internal company guideline, sorry (this slidedeck was originally created for my co-workers, but I've since presented it at other occassions as well)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • What's the endeavour JPA guideline referenced on slide 52? A google search doesn't find anything relevant.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
44,861
On SlideShare
0
From Embeds
0
Number of Embeds
443
Actions
Shares
0
Downloads
0
Comments
7
Likes
84
Embeds 0
No embeds

No notes for slide

  • Typisch meer ‘kunst dan wetenschap’. Mijn aanpak niet heilig, hoor graag tijdens de presentatie ideeen ‘wat als je dit of dat had gedaan’, goed om te discussieren.


  • Preserving correctness: unit tests! However, the more you reach the edges of the DBMS, the easier you will hit an obscure bug in query optimizer, caching strategy etc.
  • Know your RDBMS! Database independence is nice when porting is necessary, but focus on particular DB for production situation (document!), that is what counts!
    (once you get into nitty-gritty opt. details, you will have to know the RDBMS intimately)
  • Know your RDBMS! Database independence is nice when porting is necessary, but focus on particular DB for production situation (document!), that is what counts!
    (once you get into nitty-gritty opt. details, you will have to know the RDBMS intimately)
  • Know your RDBMS! Database independence is nice when porting is necessary, but focus on particular DB for production situation (document!), that is what counts!
    (once you get into nitty-gritty opt. details, you will have to know the RDBMS intimately)





  • Beware: you might retrieve your whole database in one go...

    Code example: will load Company eager, Auths. lazy
  • Beware: you might retrieve your whole database in one go...

    Code example: will load Company eager, Auths. lazy
  • Zonder Hibernate zou je niet eens over zo’n soort scenario nadenken, nu moet je wel.
  • Zonder Hibernate zou je niet eens over zo’n soort scenario nadenken, nu moet je wel.
  • Zonder Hibernate zou je niet eens over zo’n soort scenario nadenken, nu moet je wel.









  • Zelfde overwegingen gelden voor reporting queries, zoveel mogelijk in de query oplossen en geen entities teruggeven als niet nodig






  • Example: 20 groups with uninitialized collections, access first collection: all are initialized with 1 query.
    Measured: opening first group was slightly slower, general user experience better






  • stateless session ideaal for fire-and-forget service calls, minder in user-facing applicatie waar consistent houden persistence context van belang is.
  • stateless session ideaal for fire-and-forget service calls, minder in user-facing applicatie waar consistent houden persistence context van belang is.





  • Hibernate does some optimizing for read-only entities:
    It saves execution time by not dirty-checking simple properties or single-ended associations.
    It saves memory by deleting database snapshot

    cache increases load on memory, possibly more GC pauses for app. if co-located with application
  • Interesting: all instances of entity are evicted from second level cache with such a query, even if WHERE clause limits affected entities
    Also, no events fired as Hibernate normally would do.
  • Interesting: all instances of entity are evicted from second level cache with such a query, even if WHERE clause limits affected entities
    Also, no events fired as Hibernate normally would do.


  • Hibernate performance tuning

    1. 1. Performance Tuning Sander Mak
    2. 2. Outline Optimization Lazy loading Examples ‘from the trenches’ Search queries Large collections Batching Odds & Ends
    3. 3. Optimization
    4. 4. Hibernate Sucks! ... because it’s slow
    5. 5. Hibernate Sucks! ... because it’s slow ‘The problem is sort of cultural [..] developers use Hibernate because they are uncomfortable with SQL and with RDBMSes. You should be very comfortable with SQL and JDBC before you start using Hibernate - Hibernate builds on JDBC, it doesn’t replace it. That is the cost of extra abstraction [..] save yourself effort, pay attention to the database at all stages of development. - Gavin King (creator)
    6. 6. Optimization is hard Performance blamed on? Framework <-> You How to optimize? When to optimize? Production vs. development Preserve correctness at all times Unit tests ok, but not enough
    7. 7. Optimization is hard Performance blamed on? Framework <-> You How to optimize? When to optimize? Production vs. development Preserve correctness at all times Unit tests ok, but not enough Premature optimization is the root of all evil - Donald Knuth
    8. 8. Optimization guidelines Always measure Establish baseline and measure improvement Be prepared to detect performance degradation! Favor obvious over obscure Even if latter performs better Know your RDBMS (hook up with a good DBA) Details do matter Database is not a dumb ‘bit-bucket’
    9. 9. Optimization guidelines Measurement Ensure stable, production-like environment Measure time and space Time: isolate timings in different layers Space: more heap -> longer GC -> slower Try to measure in RDBMS as well Memory usage (hot cache or disk thrashing?) Query plans Make many measurements
    10. 10. Optimization guidelines Measurement Ensure stable, production-like environment Measure time and space Time: isolate timings in different layers Space: more heap -> longer GC -> slower Try to measure in RDBMS as well Memory usage (hot cache or disk thrashing?) Query plans Make many measurements Remember: measurements are relative
    11. 11. Optimization guidelines Practical Profiler on DAO/Session.query() methods VisualVM etc. for heap usage Also: JRockit Mission Control Hibernate JMX <property name="hibernate.generate_statistics">true </property> RDBMS monitoring tools
    12. 12. Optimization guidelines Practical Example from BMG tuning session:
    13. 13. Analyzing Hibernate Log SQL: <property name="show_sql">true</property> <property name="format_sql">true</property> Log4J configuration: org.hibernate -> DEBUG org.hibernate.type -> FINE (see bound params) Or use P6Spy to snoop on JDBC connection
    14. 14. Analyzing Hibernate Log SQL: <property name="show_sql">true</property> <property name="format_sql">true</property> Log4J configuration: org.hibernate -> DEBUG org.hibernate.type -> FINE (see bound params) Or use P6Spy to snoop on JDBC connection
    15. 15. Analyzing Hibernate
    16. 16. Lazy loading
    17. 17. Lazy loading One entity to rule them all Request Mostly sane defaults: 1..1 @OneToOne, @OneToMany, User @ManyToMany: LAZY 1..* *..1 @ManyToOne : EAGER Due to JPA spec. Authorization *..1 Global 1..* Company Company
    18. 18. Lazy loading Request 1..1 User 1..* *..1 Authorization *..1 Global 1..* Company Company
    19. 19. LazyInitializationException
    20. 20. Lazy loading N+1 Selects problem Select list of N users HQL: SELECT u FROM User Authorizations necessary: SQL 1 query: SELECT * FROM User N select queries on LEFT JOIN Company c Authorization executed! WHERE u.worksForCompany = c.id Solution: FETCH JOINS
    21. 21. Lazy loading N+1 Selects problem Select list of N users HQL: SELECT u FROM User Authorizations necessary: SQL N queries: SELECT * FROM Authorization N select queries on WHERE userId = N Authorization executed! Solution: FETCH JOINS
    22. 22. Lazy loading N+1 Selects problem Select list of N users HQL: SELECT u FROM User JOIN FETCH u.authorizations Authorizations necessary: SQL 1 query: N select queries on SELECT * FROM User Authorization executed! LEFT JOIN Company c LEFT OUTER JOIN Authorization ON .. WHERE Solution: u.worksForCompany = c.id FETCH JOINS
    23. 23. Lazy loading Some guidelines Laziness by default = mostly good However, architectural impact: Session lifetime (‘OpenSessionInView’ pattern) Extended Persistence Context Proxy usage (runtime code generation) Eagerness can be forced with HQL/JPAQL/Criteria But eagerness cannot be revoked exception: Session.load()/EntityManager.getReference()
    24. 24. Search queries
    25. 25. Search queries User 1..* *..1 Authorization *..1 Global 1..* Company Company
    26. 26. Search queries Obvious solution: Too much information! Use summary objects: UserSummary = POJO not attached, only necessary fields (no relations)
    27. 27. Search queries Obvious solution: Too much information! Use summary objects: UserSummary = POJO not attached, only necessary fields (no relations) Or: drop down to JDBC to fetch id + fields
    28. 28. Search queries Alternative: Taking it further: Pagination in queries, not in app. code Extra count query may be necessary (totals) Ordering necessary for paging!
    29. 29. Search queries Subtle: Alternative: effect of applying setMaxResults “The or setFirstResult to a query involving fetch joins over collections is undefined” Taking it further: Cause: the emitted join possibly returns several rows per entity. So LIMIT, rownums, TOP cannot be used! Instead Hibernate must fetch all rows WARNING: firstResult/maxResults specified with Pagination in queries, not in app. code collection fetch; applying in memory! Extra count query may be necessary (totals) Ordering necessary for paging!
    30. 30. Large collections
    31. 31. Large collections Frontend: Request CompanyGroup 1..* Backend: Company CompanyGroup 1..* Meta-data Company read-only entity, CompanyInGroup backed by expensive view *..1 Company
    32. 32. Large collections Frontend: Request CompanyGroup 1..* Backend: Company CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
    33. 33. Large collections Frontend: Request CompanyGroup 1..* Backend: Company CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
    34. 34. Large collections Opening large groups sluggish Improved performance: Fetches many uninitialized collections in 1 query Also possible on entity: Request CompanyGroup 1..* Company
    35. 35. Large collections Opening large groups sluggish Improved performance: Fetches many uninitialized collections in 1 query Also possible on entity: Request CompanyGroup 1..* Company Better solution in hindsight: fetch join
    36. 36. Large collections Saving large group slow: >15 sec. Problem: Hibernate inserts row by row Query creation overhead, network latency Solution: <property name="hibernate.jdbc.batch_size">100 </property> Enables JDBC batched statements Caution: global property Request CompanyGroup Also: <property name="hibernate.order_inserts">true 1..* </property> Company <property name="hibernate.order_updates">true </property>
    37. 37. Large collections Frontend: Request CompanyGroup 1..* Backend: Company CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
    38. 38. Large collections Backend: CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
    39. 39. Large collections Process CreateGroup (Soap) Business Service Service CreateGroup took ~10 min. for 1000 companies @BatchSize on Company improved demarshalling JDBC batch_size property marginal improvement INFO: INSERT INTO CompanyInGroup VALUES (?,...,?) INFO: SELECT @identity INFO: INSERT INTO CompanyInGroup VALUES (?,...,?) INFO: SELECT @identity .. 1000 times CompanyGroup 1..* Meta-data Insert/select interleaved: due to gen. id CompanyInGroup *..1 Company
    40. 40. Large collections Process CreateGroup (Soap) Business Service Service Solution: generate id in app. (not always feasible) Running in ~3 minutes with batched inserts Next problem: heap usage spiking Use StatelessSession ✦ Bypass first-level cache ✦ No automatic dirty checking CompanyGroup ✦ Bypass Hibernate event model and interceptors 1..* Meta-data ✦ No cascading of operations CompanyInGroup ✦ Collections on entities are ignored *..1 Company
    41. 41. Large collections Process CreateGroup (Soap) Business Service Service Solution: generate id in app. (not always feasible) Running in ~3 minutes with batched inserts Next problem: heap usage spiking Use StatelessSession CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
    42. 42. Large collections Process CreateGroup (Soap) Business Service Service Now ~1 min., everybody happy! CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
    43. 43. Large collections Process CreateGroup (Soap) Business Service Service Now ~1 min., everybody happy! Data loss detected! CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
    44. 44. Large collections Process CreateGroup (Soap) Business Service Service Data loss detected! StatelessSession and JDBC batch_size bug Only ‘full’ batches are performed! HHH-4042: Closed, won’t fix : CompanyGroup 1..* Meta-data CompanyInGroup *..1 Company
    45. 45. Odds & Ends
    46. 46. Query hints Speed up read-only service calls: Hibernate Query.setHint(): Also: never use 2nd level cache just ‘because we can’
    47. 47. Query hints Speed up read-only service calls: Hibernate Query.setHint(): Also: never use 2nd level cache just ‘because we can’ @org.hibernate.annotations.Entity(mutable = false)
    48. 48. Large updates Naive approach: Entities are not always necessary: Changes are not reflected in persistence context With optimistic concurrency: VERSIONED keyword
    49. 49. Large updates Naive approach: Entities are not always necessary: Changes are not reflected in persistence context With optimistic concurrency: VERSIONED keyword Consider use of stored procedures
    50. 50. Dynamic updates @org.hibernate.annotations.Entity (dynamicInsert = true, dynamicUpdate = true ) Only changed columns updated/inserted Beneficial for tables with many columns Downsides Runtime SQL generation overhead No PreparedStatement (caching) used anymore
    51. 51. Cherish your database Data and schema probably live longer than app. Good indexes make a world of difference Stored procedures etc. are not inherently evil Do not let Hibernate dictate your schema Though sometimes you are forced There are other solutions (there I said it) Read the Endeavour JPA Guideline!
    52. 52. Questions

    ×