Dispatcher Oom


Published on

Published in: Technology
1 Comment
  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Dispatcher Oom

  1. 1. www.day.com Top Support Issues Dispatcher & OutOfMemory Situations Speaker: Honwai Wong Duration: 45 min Feedback: techsummit@day.com Day Technical Summit 2008 1
  2. 2. www.day.com Agenda  Dispatcher  Basics  Request Handling  Best-Practice  Feature: Permission Sensitive Caching  OutOfMemory  Symptoms  Causes  Measures  Analysis  DayCare Day Technical Summit 2008 2
  3. 3. www.day.com Dispatcher Basics  Webserver module, e.g. Apache, IIS, iPlanet  Reverse proxy  Loadbalancing  Caching  Simple caching and invalidation rules  No cache dependencies resolution  Only caches GET requests  with an extension  without query parameters Day Technical Summit 2008 3
  4. 4. www.day.com Dispatcher Architecture www.day.com partnernet.day.com Render farm www.day.com Communiqué Render farm partnernet.day.com Dispatcher Module Webserver Day Technical Summit 2008 4
  5. 5. www.day.com Dispatcher Request Handling - Chain deliver page Dispatcher Module Is NO Communiqué request Webserver cacheable? Incoming GET YES request NO Already in cache? YES YES NO Is document up-to-date? cache and deliver page Day Technical Summit 2008 5
  6. 6. www.day.com Dispatcher Flush request process Dispatcher Module Communiqué Webserver touch .stat-files up to configured level FLUSH request to invalidate files in respective folders flush cache: remove all files linked to handle of replicated page Day Technical Summit 2008 6
  7. 7. www.day.com Dispatcher Best-Practice  Always use the dispatcher module, even for author  Apply tree-based cache-invalidation if possible  configuration of /statfileslevel in dispatcher.any  Distribute load among multiple redundant publish instances  dispatcher module or native loadbalancer (hardware)  Initiate flush requests from publish instance  avoid potential timing conflicts (ensures that cache flush is initiated AFTER page has actually reached publish instance)  performance gain of author by shifting cache management to publish Day Technical Summit 2008 7
  8. 8. www.day.com Dispatcher Feature: Permission Sensitive Caching  Problem: public site contains content with restricted access  Solution (so far): exclude such pages from being cached to force authentication and authorization on a CQ level  potential performance impact  New feature: Permission Sensitive Caching  Check whether user has access to cached content prior delivery  Benefit: lightweight query sent to CQ to determine access, static delivery of actual content from webserver cache Day Technical Summit 2008 8
  9. 9. www.day.com Dispatcher Feature: Permission Sensitive Caching  Requires CQ to handle authentication requests from Dispatcher by returning appropriate response status codes  Additional configuration section in dispatcher.any  Fully integrated in upcoming CQ version 4.3.0  Will work with Dispatcher version 4.0.1 and above Day Technical Summit 2008 9
  10. 10. www.day.com Dispatcher Feature: Permission Sensitive Caching Dispatcher Module Is NO request Webserver CQ Instance cacheable? Incoming GET YES request NO Already in cache? YES NO Is document up-to-date? YES Send HEAD request CQ returns YES 200 ? NO, return 40x HTTP status Day Technical Summit 2008 10
  11. 11. www.day.com Dispatcher Feature: Permission Sensitive Caching DEMO Day Technical Summit 2008 11
  12. 12. www.day.com OutOfMemory Symptoms  CQ instance crashes  Performance degradation  Seemingly endless loop of repeated garbage collections  Instance restart restores normal operation Day Technical Summit 2008 12
  13. 13. www.day.com OutOfMemory Causes  Maximum heap size is insufficient for the application  Application code makes heavy use of in-memory operations  e.g. caching objects/data in memory for fast access  Bug in product which causes memory leaks  e.g. maintaining references to unused objects, thus preventing proper garbage collection by the JVM Day Technical Summit 2008 13
  14. 14. www.day.com OutOfMemory Measures  Inspect OutOfMemory error message  Java heap space -> increase maximum heap-size  PermGen space -> increase maximum PermGen size  Add GC specific JVM options to monitor garbage collection  –verbose:gc –xloggc:gc.log –XX:+PrintGCDetails –XX: +PrintGCTimeStamps  Add JVM options to record heap-dump once OutOfMemory occurs  -XX:HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=<path> Day Technical Summit 2008 14
  15. 15. www.day.com OutOfMemory Analysis  Check application code for memory intensive operations  Inspect heapdump generated by JVM to identify potential memory leaks  Available tools:  jhat, the Java Heap Analysis Tool  YourKit, Java Profiler with powerful GUI (needs license)  SAP Memory Analyzer, Eclipse Open Source Project  Open DayCare ticket if potential memory leak discovered Day Technical Summit 2008 15
  16. 16. www.day.com OutOfMemory DayCare  Provide Day support with detailled information:  Recent changes introduced to the system that might be related  JVM options  Any problem related log-files (server-log files, error.log)  Heapdump, ideally compressed and uploaded to ftp.day.com Day Technical Summit 2008 16
  17. 17. www.day.com Questions? Day Technical Summit 2008 17