Your SlideShare is downloading. ×
Concurrency in Eclipse:Best Practices and Gotchas<br />Andrew McCulloch<br />andrew.mcculloch@oracle.com<br />Carlin Roger...
About Us<br />Andrew McCulloch Andrew.McCulloch@oracle.com<br />Principle Member of Technical Staff at Oracle working on t...
Reactions to Concurrency<br />"...we always get deadlock messages, and have no idea what they mean (in context) and we jus...
Agenda<br />Overview concurrency<br />Describe our project and experience<br />Problems and gotchas we encountered<br />Pr...
Java Concurrency Overview<br /><ul><li>Basic Building Blocks
java.lang.Thread, java.lang.Runnable
synchronized keyword
java.util.concurrent package (Java SE 5)
Eclipse Building Blocks
org.eclipse.core.runtime.jobs.Job
org.eclipse.core.runtime.jobs.ISchedulingRule
org.eclipse.core.runtime.jobs.ILock
org.eclipse.swt.widgets.Display#asyncExec</li></li></ul><li>Common Concurrency Issues<br /><ul><li>Race Condition
Interleaving of threads results in undefined or incorrect state
Deadlock / Livelock
Progress can not be made due to competing locks (deadlock), broken lock avoidance (livelock)
Starvation / Fairness
Starvation usually occurs when the scheduling algorithm gives priority to one thread over another, repeatedly</li></li></u...
Problems We Encountered<br />Race Conditions<br />Lazy initialization of, and access to, models that were not thread safe ...
Race Conditions Exceptions from EMF models<br />Exceptions during access/initialization of EMF Models [Bugzilla 228748]<br...
Debugging Tip<br />Intermittent bugs can be very difficult to reproduce in a system test.<br />Difficult to reproduce race...
Best Practices<br />“Synchronize access to shared mutable data” – Joshua Bloch<br />
Problems We Encountered<br />Race Conditions<br />Lazy initialization of, and access to, models that were not thread safe ...
Lock Order and Third Party Code<br />Deadlocks from calls in other libraries that require a scheduling rule. For example…<...
Debugging Tip<br /><ul><li>Carefully analyze call stacks of threads to find root cause. To review thread stacks and locks,...
Eclipse Debugger - turn on "Show Monitors" to see locks for synchronized blocks
Mission Control with Oracle JRockit JVM
jstack with a process ID for a thread dump
Create a simple test case that explicitly launches two jobs. Put break points in strategic places to control timing of tak...
Problems We Encountered<br />Race Conditions<br />Lazy initialization of, and access to, models that were not thread safe ...
Upcoming SlideShare
Loading in...5
×

Concurrency in Eclipse: Best Practices and Gotchas

5,145

Published on

Slide from presentation titles Concurrency in Eclipse: Best Practices and Gotchas by Andrew McCulloch and Carlin Rogers at EclipseCon 2011 in Santa Clara, CA.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,145
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Transcript of "Concurrency in Eclipse: Best Practices and Gotchas"

  1. 1. Concurrency in Eclipse:Best Practices and Gotchas<br />Andrew McCulloch<br />andrew.mcculloch@oracle.com<br />Carlin Rogers<br />carlin.rogers@oracle.com<br />
  2. 2. About Us<br />Andrew McCulloch Andrew.McCulloch@oracle.com<br />Principle Member of Technical Staff at Oracle working on the Oracle Enterprise Pack for Eclipse (OEPE)<br />Lead on AppXray feature which manages dependencies in a user's workspace<br />Area of interest include concurrency and performance tuning<br />Carlin Rogers Carlin.Rogers@oracle.com<br />Principle Member of Technical Staff at Oracle working on the Oracle Enterprise Pack for Eclipse (OEPE)<br />Committer on the JavaServer Faces Tools Project at Eclipse<br />Areas of interest include web frameworks, concurrency, and performance tuning<br />
  3. 3.
  4. 4. Reactions to Concurrency<br />"...we always get deadlock messages, and have no idea what they mean (in context) and we just ignore them.“<br /> - Anonymous SQL Server Developer<br />
  5. 5. Agenda<br />Overview concurrency<br />Describe our project and experience<br />Problems and gotchas we encountered<br />Practical debugging tips<br />Best practices<br />Questions<br />
  6. 6. Java Concurrency Overview<br /><ul><li>Basic Building Blocks
  7. 7. java.lang.Thread, java.lang.Runnable
  8. 8. synchronized keyword
  9. 9. java.util.concurrent package (Java SE 5)
  10. 10. Eclipse Building Blocks
  11. 11. org.eclipse.core.runtime.jobs.Job
  12. 12. org.eclipse.core.runtime.jobs.ISchedulingRule
  13. 13. org.eclipse.core.runtime.jobs.ILock
  14. 14. org.eclipse.swt.widgets.Display#asyncExec</li></li></ul><li>Common Concurrency Issues<br /><ul><li>Race Condition
  15. 15. Interleaving of threads results in undefined or incorrect state
  16. 16. Deadlock / Livelock
  17. 17. Progress can not be made due to competing locks (deadlock), broken lock avoidance (livelock)
  18. 18. Starvation / Fairness
  19. 19. Starvation usually occurs when the scheduling algorithm gives priority to one thread over another, repeatedly</li></li></ul><li>Our Story Begins…<br />AppXray<br />Respond to every IResource change<br />Respond to every IDocument change<br />Multiple concurrent Jobs using ISchedulingRules<br />Must leverage legacy code using Java synchronization<br />WTP<br />Use WTP DOM model for JSP, faces-config.xml, web.xml parsing<br />Use WTP provided EMF Models when convenient<br />WTP EMF Models do not always protected against concurrent access<br />Performance is important!<br />
  20. 20. Problems We Encountered<br />Race Conditions<br />Lazy initialization of, and access to, models that were not thread safe - uncontrolled concurrent access to mutable data.<br />
  21. 21. Race Conditions Exceptions from EMF models<br />Exceptions during access/initialization of EMF Models [Bugzilla 228748]<br />java.lang.NullPointerException <br />at org.eclipse.emf.ecore.util.EContentAdapter.addAdapter(EContentAdapter.java:352) <br />at org.eclipse.emf.ecore.util.EContentAdapter.setTarget(EContentAdapter.java:225) <br />at org.eclipse.emf.ecore.util.EContentAdapter.setTarget(EContentAdapter.java:188) <br />at org.eclipse.emf.common.notify.impl.BasicNotifierImpl$EAdapterList.didAdd(BasicNotifierImpl.java:77)<br />at org.eclipse.emf.common.util.BasicEList.addUnique(BasicEList.java:646) <br />at org.eclipse.emf.common.util.BasicEList.add(BasicEList.java:626) <br />at org.eclipse.emf.common.notify.impl.BasicNotifierImpl$EAdapterList.add(BasicNotifierImpl.java:129) <br />at org.eclipse.jst.jsf.core.jsfappconfig.JSFAppConfigManager.addFacesConfigChangeAdapter(<br /> JSFAppConfigManager.java:769) <br />…<br />at org.eclipse.jst.jsf.core.jsfappconfig.JSFAppConfigManager.getFacesConfigModels( JSFAppConfigManager.java:409)<br />
  22. 22. Debugging Tip<br />Intermittent bugs can be very difficult to reproduce in a system test.<br />Difficult to reproduce race condition in EMF code using debugger.<br />Narrow the scope - create simple controlled test of the specific code path in the stack trace from the exception.<br />Example: launch multiple threads, each calling the JSFAppConfigManager.getFacesConfigModels() method<br />
  23. 23. Best Practices<br />“Synchronize access to shared mutable data” – Joshua Bloch<br />
  24. 24. Problems We Encountered<br />Race Conditions<br />Lazy initialization of, and access to, models that were not thread safe - uncontrolled concurrent access to mutable data.<br />Deadlocks<br />Know your third party code - it may take a scheduling rule.<br />
  25. 25. Lock Order and Third Party Code<br />Deadlocks from calls in other libraries that require a scheduling rule. For example…<br />Thread A - AppXRay Job<br />Thread B – Main<br />- Calls beginRule with the IProject as the scheduling rule.<br />- UI component calls to access information from shared model, enters synchronized block.<br />- Tries to access information from shared model, waits to enter synchronized block locked by Thread B.<br />- Third party code requires JDT to resolve classpaths for the projects, and tries to take the IProjectscheduling rule.<br />
  26. 26. Debugging Tip<br /><ul><li>Carefully analyze call stacks of threads to find root cause. To review thread stacks and locks, use…
  27. 27. Eclipse Debugger - turn on "Show Monitors" to see locks for synchronized blocks
  28. 28. Mission Control with Oracle JRockit JVM
  29. 29. jstack with a process ID for a thread dump
  30. 30. Create a simple test case that explicitly launches two jobs. Put break points in strategic places to control timing of taking the locks to force the deadlock condition. Use this technique to confirm a fix.</li></li></ul><li>Best Practices<br />“Synchronize access to shared mutable data”<br />Enforce a consistent locking order (Resource scheduling rule, then lock) and “avoid excessive synchronization” - Joshua Bloch<br />
  31. 31. Problems We Encountered<br />Race Conditions<br />Lazy initialization of, and access to, models that were not thread safe - uncontrolled concurrent access to mutable data.<br />Deadlocks<br />Know your third party code - it may take a scheduling rule.<br />Thread Starvation<br />JobManager.waitForRun prevented jobs with complex relationships from running in a timely manner.<br />
  32. 32. Scheduler issues [Bugzilla 320329]<br />Not a typical Thread Starvation case, but illustrates a point.<br />Job dependencies can be complex when multiple jobs launch in response to a single event.<br />In the UI progress dialog the running job appeared to be deadlocked when it was not at fault.<br />Dynamic Analysis tools can pinpoint the true problem<br />Our profiling tool pointed out that a loop in the scheduler was executed millions of times.<br />The issue was in the Job scheduler in open source code<br />After several iterations at a patch the issue was resolved<br />
  33. 33. Debugging Tip<br />Be familiar with dynamic analysis tools. We use several including Oracle JRockit Mission Control and third-party tools.<br />Don’t ignore org.eclipse code in your analysis. Eclipse is a quality product but bugs do get through and the community can help resolve them.<br />
  34. 34. Best Practices<br />“Synchronize access to shared mutable data”<br />Enforce a consistent locking order. “Avoid excessive synchronization”<br />Don’t be hesitant to debug into open source code, submit questions, bugs, and patches. This make your product and Eclipse better<br />
  35. 35. Gotchas<br />Job.yieldRule()<br />Third party code may move your job to waiting state and yield the scheduling rule to other jobs to avoid potential deadlocks. (bug 283449) <br />
  36. 36. Job.yieldRule()<br />Thread dump from deadlock<br />java.lang.Object.wait(Native Method)<br /> at org.eclipse.core.internal.jobs.ThreadJob.waitForRun(ThreadJob.java:270)<br /> at org.eclipse.core.internal.jobs.ThreadJob.joinRun(ThreadJob.java:199)<br /> at org.eclipse.core.internal.jobs.JobManager.yieldRule(JobManager.java:1398)<br /> at org.eclipse.core.internal.jobs.InternalJob.yieldRule(InternalJob.java:600)<br /> at org.eclipse.core.runtime.jobs.Job.yieldRule(Job.java:709)<br /> at org.eclipse.wst.sse.core.internal.model.ModelManagerImpl$<br /> SharedObject.waitForLoadAttempt(ModelManagerImpl.java:139)<br /> at org.eclipse.wst.sse.core.internal.model.ModelManagerImpl.getExistingModel(<br /> ModelManagerImpl.java:1137)<br />
  37. 37. Debugging Tip<br /><ul><li>Carefully analyze call stacks of threads to find root cause.
  38. 38. Look for yieldRule() in the stack.
  39. 39. Determine if you can move the call to third party code outside of the synchronization.
  40. 40. Review the state of the DeadlockDetector in the JobManager’s LockManager. It has a graph of threads and locks to identify which jobs hold which locks.</li></li></ul><li>Best Practices<br />“Synchronize access to shared mutable data”<br />Enforce a consistent locking order. “Avoid excessive synchronization”<br />Don’t be hesitant to work with open source code.<br />Use ILock for synchronization – supports deadlock detection and recovery with other locks and scheduling rules.<br />
  41. 41. Gotchas<br />Job.yieldRule()<br />Third party code may move your job to waiting state and yield the scheduling rule to other jobs to avoid potential deadlocks. (bug 283449) <br />EventLoopProgressMonitor<br />Starting a rule on the UI thread will have unintended consequences regardless of how long the rule is held.<br />
  42. 42. Event loop will be interleaved<br />Acquiring a scheduling rule from the UI thread may appear as a UI freeze.<br />Alternatively in some instances acquiring a scheduling rule on the UI thread may cause calls to the progress monitor to execute other UI events which will eliminate the appearance of a freeze but can cause errors with non-reentrant code.<br />RunnableWithProgress also replaces null progress monitors.<br />
  43. 43. Debugging Tip<br />Analyze stack trace for EventLoopProgressMonitor<br />Look for entries on the stack that you would not expect to be executing from within your code.<br />Look for tell-tale exception <br /> “Attempted reentrant call …”<br />Determine where the scheduling rule is acquired<br />Can that code be moved off of the UI thread?<br />Can the UI widget be redesigned to update asynchronously?<br />
  44. 44. Best Practices<br />“Synchronize access to shared mutable data”<br />Enforce a consistent locking order. “Avoid excessive synchronization”<br />Don’t be hesitant to work with open source code.<br />Use ILock for synchronization <br />Run as little as possible on the UI thread. Move code requiring ISchedulingRules to a Job.<br />
  45. 45. In Conclusion…<br />“Synchronize access to shared mutable data”<br />Enforce a consistent locking order. “Avoid excessive synchronization”<br />Don’t be hesitant to work with open source code.<br />Use ILock for synchronization <br />Run as little as possible on the UI thread. Move code requiring ISchedulingRules to a Job.<br />
  46. 46. Suggested References<br />Java Concurrency in Practice<br />Brian Goetz (2006)<br />Effective Java (2nd Edition)<br />Joshua Bloch (2008)<br />Concurrent Programming in Java<br />Doug Lea (1999)<br />
  47. 47. Any Questions?<br />

×