Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Winning With Java at Market Open

400 views

Published on

In this presentation, Azul CTO Gil Tene describes why many Java warm-up strategies don't work, and how to slash warm-up time without rearchitecting. Learn why your warm-up strategies don't always work, how Java trading systems can start fast and stay fast, how to save and re-use optimizations across runs and how new JVM innovations can help you win.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Winning With Java at Market Open

  1. 1. ©2013 Azul Systems, Inc. Winning with Java At Market Open Getting Fast & Staying Fast Gil Tene, CTO & co-Founder, Azul Systems
  2. 2. ©2013 Azul Systems, Inc. Are you fast at Market Open?
  3. 3. ©2013 Azul Systems, Inc. Java at Market Open . . . Market Open
  4. 4. ©2013 Azul Systems, Inc. Java’s “Just In Time” Reality
  5. 5. ©2013 Azul Systems, Inc. Java’s “Just In Time” Reality Starts slow, learns fast
  6. 6. ©2013 Azul Systems, Inc. Java’s “Just In Time” Reality Starts slow, learns fast Lazy loading & initialization
  7. 7. ©2013 Azul Systems, Inc. Java’s “Just In Time” Reality Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case
  8. 8. ©2013 Azul Systems, Inc. Java’s “Just In Time” Reality Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case (temporarily) Reverts to slower execution to adapt
  9. 9. ©2013 Azul Systems, Inc. Java’s “Just In Time” Reality Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case (temporarily) Reverts to slower execution to adapt
  10. 10. ©2013 Azul Systems, Inc. Java’s “Just In Time” Reality Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case (temporarily) Reverts to slower execution to adapt . . .
  11. 11. ©2013 Azul Systems, Inc. Java’s “Just In Time” Reality Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case (temporarily) Reverts to slower execution to adapt Warmup . . .
  12. 12. ©2013 Azul Systems, Inc. Java’s “Just In Time” Reality Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case (temporarily) Reverts to slower execution to adapt Warmup Deoptimization . . .
  13. 13. Speculative compiler tricks JIT compilers can do things that static compilers can have a hard time with…
  14. 14. ©2012 Azul Systems, Inc. Untaken path example
  15. 15. ©2012 Azul Systems, Inc. Untaken path example “Never taken” paths can be optimized away with benefits: void computeMagnitude(int val) { if (val > 10) { bias = computeBias(val); else { bias = 1; } return Math.log10(bias + 99); }
  16. 16. ©2012 Azul Systems, Inc. Untaken path example “Never taken” paths can be optimized away with benefits: void computeMagnitude(int val) { if (val > 10) { bias = computeBias(val); else { bias = 1; } return Math.log10(bias + 99); } When all values so far were <= 10 , can be optimized to: void computeMagnitude(int val) { if (val > 10) uncommonTrap(); return 2; }
  17. 17. ©2012 Azul Systems, Inc. Implicit Null Check example
  18. 18. ©2012 Azul Systems, Inc. Implicit Null Check example All field and array access in Java is null checked x = foo.x; is (in equivalent required machine code): if (foo == null) throw new NullPointerException(); x = foo.x;
  19. 19. ©2012 Azul Systems, Inc. Implicit Null Check example All field and array access in Java is null checked x = foo.x; is (in equivalent required machine code): if (foo == null) throw new NullPointerException(); x = foo.x; But compiler can “hope” for non-nulls, and handle SEGV <Point where later SEGV will appear to throw> x = foo.x;
  20. 20. ©2012 Azul Systems, Inc. Implicit Null Check example All field and array access in Java is null checked x = foo.x; is (in equivalent required machine code): if (foo == null) throw new NullPointerException(); x = foo.x; But compiler can “hope” for non-nulls, and handle SEGV <Point where later SEGV will appear to throw> x = foo.x; This is faster *IF* no nulls are encountered…
  21. 21. Deoptimization
  22. 22. ©2012 Azul Systems, Inc. Deoptimization: Adaptive compilation is… adaptive
  23. 23. ©2012 Azul Systems, Inc. Deoptimization: Adaptive compilation is… adaptive JIT Compilers are adaptive and speculative
  24. 24. ©2012 Azul Systems, Inc. Deoptimization: Adaptive compilation is… adaptive JIT Compilers are adaptive and speculative Leverage observations of behavior
  25. 25. ©2012 Azul Systems, Inc. Deoptimization: Adaptive compilation is… adaptive JIT Compilers are adaptive and speculative Leverage observations of behavior Can “hope” about things and can recover if wrong
  26. 26. ©2012 Azul Systems, Inc. Deoptimization: Adaptive compilation is… adaptive JIT Compilers are adaptive and speculative Leverage observations of behavior Can “hope” about things and can recover if wrong Warming up code is a black art
  27. 27. ©2012 Azul Systems, Inc. Deoptimization: Adaptive compilation is… adaptive JIT Compilers are adaptive and speculative Leverage observations of behavior Can “hope” about things and can recover if wrong Warming up code is a black art Running code long enough to compile is just the start…
  28. 28. ©2012 Azul Systems, Inc. Deoptimization: Adaptive compilation is… adaptive JIT Compilers are adaptive and speculative Leverage observations of behavior Can “hope” about things and can recover if wrong Warming up code is a black art Running code long enough to compile is just the start… Deoptimization can occur at any time
  29. 29. ©2012 Azul Systems, Inc. Deoptimization: Adaptive compilation is… adaptive JIT Compilers are adaptive and speculative Leverage observations of behavior Can “hope” about things and can recover if wrong Warming up code is a black art Running code long enough to compile is just the start… Deoptimization can occur at any time often occur after you *think* the code is warmed up.
  30. 30. ©2012 Azul Systems, Inc. Deoptimization: Adaptive compilation is… adaptive JIT Compilers are adaptive and speculative Leverage observations of behavior Can “hope” about things and can recover if wrong Warming up code is a black art Running code long enough to compile is just the start… Deoptimization can occur at any time often occur after you *think* the code is warmed up. Deoptimization has many potential causes
  31. 31. ©2012 Azul Systems, Inc. Warmup often doesn’t cut it…
  32. 32. ©2012 Azul Systems, Inc. Warmup often doesn’t cut it… Common Example: Trading system wants to have the first trade be fast So run 20,000 “fake” messages through the system to warm up let JIT compilers optimize, learn, and deopt before actual trades
  33. 33. ©2012 Azul Systems, Inc. Warmup often doesn’t cut it… Common Example: Trading system wants to have the first trade be fast So run 20,000 “fake” messages through the system to warm up let JIT compilers optimize, learn, and deopt before actual trades What really happens Code is written to do different things “if this is a fake message” e.g. “Don’t send to the exchange if this is a fake message” JITs optimize for fake path, including speculatively assuming “fake” First real message through deopts...
  34. 34. ©2012 Azul Systems, Inc. A few example causes for depotimization
  35. 35. ©2012 Azul Systems, Inc. A few example causes for depotimization Newly encountered code breaking prior assumptions
  36. 36. ©2012 Azul Systems, Inc. A few example causes for depotimization Newly encountered code breaking prior assumptions First calls to method in an uninitialized class
  37. 37. ©2012 Azul Systems, Inc. A few example causes for depotimization Newly encountered code breaking prior assumptions First calls to method in an uninitialized class First call to a method in a not-yet-loaded class
  38. 38. ©2012 Azul Systems, Inc. A few example causes for depotimization Newly encountered code breaking prior assumptions First calls to method in an uninitialized class First call to a method in a not-yet-loaded class Dynamic branch elimination hitting an unexpected path
  39. 39. ©2012 Azul Systems, Inc. A few example causes for depotimization Newly encountered code breaking prior assumptions First calls to method in an uninitialized class First call to a method in a not-yet-loaded class Dynamic branch elimination hitting an unexpected path Implicit Null Check hitting a null
  40. 40. ©2013 Azul Systems, Inc. Java’s “Just In Time” Reality Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case (temporarily) Reverts to slower execution to adapt Warmup Deoptimization . . .
  41. 41. ©2013 Azul Systems, Inc. Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case (temporarily) Reverts to slower execution to adapt What we have Java’s “Just In Time” Reality
  42. 42. ©2013 Azul Systems, Inc. Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case (temporarily) Reverts to slower execution to adapt What we have What we want Java’s “Just In Time” Reality
  43. 43. ©2013 Azul Systems, Inc. Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case (temporarily) Reverts to slower execution to adapt What we have What we want No Slow Ops or Trades Java’s “Just In Time” Reality
  44. 44. ©2013 Azul Systems, Inc. Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case (temporarily) Reverts to slower execution to adapt What we have What we want No Slow Ops or Trades ReadyNow! to the rescue Java’s “Just In Time” Reality
  45. 45. ©2012 Azul Systems, Inc. What can we do about it?
  46. 46. ©2012 Azul Systems, Inc. What can we do about it? Azul’s Zing JVM has a feature set called “ReadyNow!” Avoids deoptimization while keeping code fast Allows code to reach optimized levels without requiring “exercise”
  47. 47. ©2012 Azul Systems, Inc. What can we do about it? Azul’s Zing JVM has a feature set called “ReadyNow!” Avoids deoptimization while keeping code fast Allows code to reach optimized levels without requiring “exercise” Learning and retaining optimizations With Zing, your next run can learn from the previous one
  48. 48. ©2012 Azul Systems, Inc. What can we do about it? Azul’s Zing JVM has a feature set called “ReadyNow!” Avoids deoptimization while keeping code fast Allows code to reach optimized levels without requiring “exercise” Learning and retaining optimizations With Zing, your next run can learn from the previous one Fine grained control options Per method compiler directives …
  49. 49. ©2012 Azul Systems, Inc. Logging & “replaying”optimizations
  50. 50. ©2012 Azul Systems, Inc. Logging & “replaying”optimizations Zing’s ReadyNow includes optimization logging Records ongoing optimization decisions and stats records optimization dependencies Establishes “stable optimization state” at end of previous run
  51. 51. ©2012 Azul Systems, Inc. Logging & “replaying”optimizations Zing’s ReadyNow includes optimization logging Records ongoing optimization decisions and stats records optimization dependencies Establishes “stable optimization state” at end of previous run Zing can then read prior logs at startup Prime JVM with knowledge of prior stable optimizations Optimizations are applied as their dependencies get resolved
  52. 52. ©2012 Azul Systems, Inc. Logging & “replaying”optimizations Zing’s ReadyNow includes optimization logging Records ongoing optimization decisions and stats records optimization dependencies Establishes “stable optimization state” at end of previous run Zing can then read prior logs at startup Prime JVM with knowledge of prior stable optimizations Optimizations are applied as their dependencies get resolved ReadyNow workflow promotes confidence You’ll know if/when all optimizations have been applied If some optimization haven’t been applied, you’ll know why…
  53. 53. ©2013 Azul Systems, Inc. Java at “Load Start” . . . Load Start
  54. 54. ©2013 Azul Systems, Inc. Java at “Load Start” . . . Load Start Deoptimization
  55. 55. ©2013 Azul Systems, Inc. Java at “Load Start” . . . Load Start Deoptimization ReadyNow! avoids deoptimization
  56. 56. ©2013 Azul Systems, Inc. Java at “Load Start” . . . Load Start With Zing & ReadyNow!
  57. 57. ©2013 Azul Systems, Inc. Java at “Load Start” . . . Load Start With ReadyNow! Warmup?
  58. 58. ©2013 Azul Systems, Inc. Java at “Load Start” . . . Load Start With ReadyNow! Warmup? Fast From The Start
  59. 59. ©2013 Azul Systems, Inc. . . . Load Start With ReadyNow! and Start Fast & Stay Fast Java at “Load Start”
  60. 60. ©2013 Azul Systems, Inc. . . . Load Start With Zing & ReadyNow! Start Fast & Stay Fast Java at “Load Start”
  61. 61. ©2013 Azul Systems, Inc. . . . Load Start With Zing & ReadyNow! Start Fast & Stay Fast Java at “Load Start”
  62. 62. ©2013 Azul Systems, Inc. . . . Load Start With Zing & ReadyNow! Start Fast & Stay Fast Java at “Load Start” With Confidence
  63. 63. ©2013 Azul Systems, Inc. One liner takeaway . . . Load Start
  64. 64. ©2013 Azul Systems, Inc. One liner takeaway . . . Zing: A cure for the Java hiccups Load Start

×