This document discusses issues with updating Java applications in production and proposes solutions. It finds that 27% of organizations forbid downtime during updates, 19% lose on average $3,230 per minute on each update. Only 27% are satisfied with their current update process, which commonly faces issues with tools, downtime, reliability and off-hours updates. The document proposes techniques like in-app updates, multi-process JVMs and automated rolling restarts to help solve the majority of issues by preventing downtime during updates.
43. Conclusions Container redeploys are unreliable Most popular update method is restart in off-hours Biggest issues are lack of automation and downtime during updates Another issue is lack of standard language for lightweight processes In-App Updates and automated rolling restarts will solve majority of issues
Hi guys! My name is Jevgeni Kabanov and I’m here to present the “Do you really get changes?” talk.
Here’s my life in a tag cloud. I’m the original creator of JRebel and now the CTO of ZeroTurnaround, I used to write Haskell and still am a bit snooty about other languages, but I’ve been digging deep in Java for the past 7 years. I’m one article away from my PhD and have a wife, two cats and a really fast bike :)Before we go any further I’d like to ask you if you heard about JRebel?
Bringing developers and administrators together. Automation, lightweight process, fast reaction to errors v/s preventing errors at all costs
ClassLoader abuse
The majority of process restrictions stem from the technical issues.
Lower than expected, but is it because they don’t care or because it’s so hard to do without downtime?
This was calculated from those who allow downtime v/s those who actually are subject to it. The price is the average one, average length of an update is 1.6 hours. Could be more than $300,000 per update for some unlucky folks.
Only 16% fully automated! 54% majority has a manual step, 30% manual to a (varyingly) large extent. 7% have an “It’s the last copy of a list of arcane steps printed out three years ago. (May or may not involve sacrificing a goat)”
Lack of automation revealed nicely by the most popular tool, Maven & Ant mentioned often too. No real tooling available for automation!
Under 12% do fully transparent updates with no downtimes, 46% take the app offline (off-hours are a common complain), 42% risk side-effects with hacks causing complaints about reliability and downtime. Only 12% use container redeployment, why is that?
Very low.
We’ll examine all of them down the line.
33% report onlyOutOfMemoryErrors, 46% multiple additional issues including PermGen or OutOfMemoryErrors, lack of support for database updates, thread races and deadlocks, thread and resource leaks, security concerns, native library issues, a multitude of application server caching issues, performance overhead, rollback difficulties and lack of reliability.
33% report onlyOutOfMemory issues, 46% multiple additional issues including lack of support for database updates, thread races and deadlocks, thread and resource leaks, security concerns, native library issues, a multitude of application server caching issues, performance overhead, rollback difficulties and lack of reliability.
33% report onlyOutOfMemory issues, 46% multiple additional issues including lack of support for database updates, thread races and deadlocks, thread and resource leaks, security concerns, native library issues, a multitude of application server caching issues, performance overhead, rollback difficulties and lack of reliability.
Tomcat 7, Weblogic 9.x+
The absolute majority of current ready-made tools support only two update methods: container redeployment and restarts with downtime
Session clustering is subject to state structure changes and heisenbugs
If the app is actively used it may take forever. Hard to automate due to lack of APIs and ready-made tooling. Needs manual control.
33% report onlyOutOfMemory issues, 46% multiple additional issues including lack of support for database updates, thread races and deadlocks, thread and resource leaks, security concerns, native library issues, a multitude of application server caching issues, performance overhead, rollback difficulties and lack of reliability.
Rolling
33% report onlyOutOfMemory issues, 46% multiple additional issues including lack of support for database updates, thread races and deadlocks, thread and resource leaks, security concerns, native library issues, a multitude of application server caching issues, performance overhead, rollback difficulties and lack of reliability.
LiveRebel does in-app hot updates, with no downtime, no lost sessions, no OOME, no side effect. It’s fully automated and instant.
33% report onlyOutOfMemory issues, 46% multiple additional issues including lack of support for database updates, thread races and deadlocks, thread and resource leaks, security concerns, native library issues, a multitude of application server caching issues, performance overhead, rollback difficulties and lack of reliability.
33% report onlyOutOfMemory issues, 46% multiple additional issues including lack of support for database updates, thread races and deadlocks, thread and resource leaks, security concerns, native library issues, a multitude of application server caching issues, performance overhead, rollback difficulties and lack of reliability.
Rolling restarts are hard because changes are hard in general!