Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deadlock Victim

2,532 views

Published on

Support de la présentation "Deadlock Victim" à Devoxx-fr 2012, par Heinz Kabutz (javaspecialists.eu) et Olivier Croisier (zenika.com, thecodersbreakfast.net).

Published in: Technology, News & Politics
  • Be the first to comment

Deadlock Victim

  1. 1. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Deadlock Victim by Dr Heinz Kabutz && Olivier CroisierThe Java Specialists Newsletter && The Coders Breakfast heinz@javaspecialists.eu && olivier.croisier@zenika.com twitter: @heinzkabutz && twitter: @oliviercroisier 1
  2. 2. The Java Specialists Newsletter - javaspecialists.eu / zenika.frYou discover a race condition 2
  3. 3. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Data is being corrupted 3
  4. 4. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr What ya gonna do? 4
  5. 5. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Who ya gonna call? 5
  6. 6. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Ghostbusters? 6
  7. 7. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr you do what we all do 7
  8. 8. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr you use synchronized 8
  9. 9. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr synchronized 9
  10. 10. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr synchronized 9
  11. 11. The Java Specialists Newsletter - javaspecialists.eu / zenika.frsynchronized can be bad medicine 10
  12. 12. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr deadlock awaits you! 11
  13. 13. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr 12
  14. 14. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Deadlock Victim 13
  15. 15. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Correctness fix can lead to liveness fault 14
  16. 16. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr lock ordering deadlock 15
  17. 17. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr at least two locks 16
  18. 18. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr and at least two threads 17
  19. 19. The Java Specialists Newsletter - javaspecialists.eu / zenika.frlocked in different orders 18
  20. 20. The Java Specialists Newsletter - javaspecialists.eu / zenika.frcan cause a deadly embrace 19
  21. 21. The Java Specialists Newsletter - javaspecialists.eu / zenika.frrequires global analysis to fix 20
  22. 22. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr resource deadlocks 21
  23. 23. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr bounded queues 22
  24. 24. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr bounded thread pools 23
  25. 25. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr semaphores 24
  26. 26. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr class loaders 25
  27. 27. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr lost signals 26
  28. 28. The Java Specialists Newsletter - javaspecialists.eu / zenika.frThe Drinking Philosophers 27
  29. 29. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr@ the Java Specialist Symposium in Crete 28
  30. 30. The Java Specialists Newsletter - javaspecialists.eu / zenika.frSymposium == sym + poto Literally - to drink together 29
  31. 31. The Java Specialists Newsletter - javaspecialists.eu / zenika.frour philosophers need two cups 30
  32. 32. The Java Specialists Newsletter - javaspecialists.eu / zenika.frthey either want to drink or think 31
  33. 33. The Java Specialists Newsletter - javaspecialists.eu / zenika.frTable is ready, all philosophers are thinking 1 5 2 4 3 32
  34. 34. The Java Specialists Newsletter - javaspecialists.eu / zenika.frphilosophers 5 wants to drink, takes right cup 1 5 2 4 3 33
  35. 35. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 5 is now drinking with both cups 1 5 2 4 3 34
  36. 36. The Java Specialists Newsletter - javaspecialists.eu / zenika.frphilosophers 3 wants to drink, takes right cup 1 5 2 4 3 35
  37. 37. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 is now drinking with both cups 1 5 2 4 3 36
  38. 38. The Java Specialists Newsletter - javaspecialists.eu / zenika.frphilosophers 2 wants to drink, takes right cup 1 5 2 4 3 37
  39. 39. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 finished drinking, returns right cup 1 5 2 4 3 38
  40. 40. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 2 is now drinking with both cups 1 5 2 4 3 39
  41. 41. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 returns Left cup 1 5 2 4 3 40
  42. 42. The Java Specialists Newsletter - javaspecialists.eu / zenika.frphilosophers can get stuck 41
  43. 43. The Java Specialists Newsletter - javaspecialists.eu / zenika.frif they all pick up the right cup 42
  44. 44. The Java Specialists Newsletter - javaspecialists.eu / zenika.frA deadlock can easily happen with this design 1 5 2 4 3 43
  45. 45. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 5 wants to drink, takes right cup 1 5 2 4 3 44
  46. 46. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 1 wants to drink, takes right cup 1 5 2 4 3 45
  47. 47. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 2 wants to drink, takes right cup 1 5 2 4 3 46
  48. 48. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 wants to drink, takes right cup 1 5 2 4 3 47
  49. 49. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 4 wants to drink, takes right cup 1 5 2 4 3 48
  50. 50. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Deadlock! 1 5 2 4 3 49
  51. 51. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr they will all stay thirsty 1 5 2 4 3 50
  52. 52. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr this is a deadly embrace 1 5 2 4 3 51
  53. 53. The Java Specialists Newsletter - javaspecialists.eu / zenika.frthis can be detected automatically 52
  54. 54. The Java Specialists Newsletter - javaspecialists.eu / zenika.frsearch the graph of call stacks 53
  55. 55. The Java Specialists Newsletter - javaspecialists.eu / zenika.frand look for circular dependencies 54
  56. 56. The Java Specialists Newsletter - javaspecialists.eu / zenika.frThreadMXBean does that for us 55
  57. 57. The Java Specialists Newsletter - javaspecialists.eu / zenika.frsimple deadlocks are quickly found 56
  58. 58. The Java Specialists Newsletter - javaspecialists.eu / zenika.frbut what do you DO with them? 57
  59. 59. The Java Specialists Newsletter - javaspecialists.eu / zenika.frwhat will the dog do with the car if he ever catches it? 58
  60. 60. The Java Specialists Newsletter - javaspecialists.eu / zenika.frdatabases choose deadlock victim 59
  61. 61. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr but what does Java do? 60
  62. 62. The Java Specialists Newsletter - javaspecialists.eu / zenika.frit should probably throw an Error 61
  63. 63. The Java Specialists Newsletter - javaspecialists.eu / zenika.frbut the threads simply hang up 62
  64. 64. The Java Specialists Newsletter - javaspecialists.eu / zenika.frit is best to avoid causing them! 63
  65. 65. The Java Specialists Newsletter - javaspecialists.eu / zenika.frglobal order of locks in program 64
  66. 66. The Java Specialists Newsletter - javaspecialists.eu / zenika.frpublic class LeftRightDeadlock { private final Object left = new Object(); private final Object right = new Object(); public void leftRight() { synchronized (left) { synchronized (right) { doSomething(); } } } public void rightLeft() { synchronized (right) { synchronized (left) { doSomethingElse(); } } }} 65
  67. 67. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Interleaving causes deadlock lock left trying to lock rightA leftRight() BLOCKED lock right trying to lock leftB rightLeft() BLOCKED 66
  68. 68. The Java Specialists Newsletter - javaspecialists.eu / zenika.frwe define a fixed global order 67
  69. 69. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr e.g. left before right 68
  70. 70. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr deadlock is solved! public void rightLeft() { synchronized (left) { synchronized (right) { doSomethingElse(); } } } 69
  71. 71. The Java Specialists Newsletter - javaspecialists.eu / zenika.frour drinking philosophers 70
  72. 72. The Java Specialists Newsletter - javaspecialists.eu / zenika.freach cup gets a unique number 71
  73. 73. The Java Specialists Newsletter - javaspecialists.eu / zenika.frlock the highest number cup first 72
  74. 74. The Java Specialists Newsletter - javaspecialists.eu / zenika.frthen lock the lower number cup 73
  75. 75. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr drink 74
  76. 76. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr unlock lower order cup 75
  77. 77. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr unlock higher order cup 76
  78. 78. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr think 77
  79. 79. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Global Lock ordering• We start with all the 1 philosophers thinking 1 2 5 2 5 3 4 4 3 78
  80. 80. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Philosopher 5 takes cup 5• Cup 5 has higher number 1 • Remember our rule! 1 2 5 2 5 3 4 4 3 79
  81. 81. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Philosopher 1 takes cup 2• Must take the cup with 1 the higher number 2 first 1 • In this case 5 2 cup 2 5 3 4 4 3 80
  82. 82. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 2 takes cup 3 1 2 1 5 2 5 3 4 4 3 81
  83. 83. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 takes cup 4 1 2 1 5 2 5 3 4 4 3 82
  84. 84. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 1 takes cup 1 - Drinking 1 1 2 5 2 5 3 4 4 3 83
  85. 85. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Philosopher 1 returns Cup 1• Cups are returned in the 1 opposite order to what 2 they are acquired 1 5 2 5 3 4 4 3 84
  86. 86. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 5 takes cup 1 - Drinking 1 2 1 5 2 5 3 4 4 3 85
  87. 87. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 5 returns cup 1 1 2 1 5 2 5 3 4 4 3 86
  88. 88. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 1 returns cup 2 1 1 2 5 2 5 3 4 4 3 87
  89. 89. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 2 takes cup 2 - Drinking 1 2 1 5 2 5 3 4 4 3 88
  90. 90. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 5 returns cup 5 1 2 1 5 2 3 5 4 4 3 89
  91. 91. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 4 takes cup 5 1 2 1 5 2 3 5 4 4 3 90
  92. 92. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 2 returns cup 2 1 1 2 5 2 3 5 4 4 3 91
  93. 93. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 2 returns cup 3 1 1 2 5 2 3 5 4 4 3 92
  94. 94. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 takes cup 3 - Drinking 1 1 2 5 2 5 3 4 4 3 93
  95. 95. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 Returns cup 3 1 1 2 5 2 3 5 4 4 3 94
  96. 96. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 Returns cup 4 1 1 2 5 2 3 5 4 4 3 95
  97. 97. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 4 takes cup 4 - Drinking 1 1 2 5 2 3 5 4 4 3 96
  98. 98. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 4 Returns cup 4 1 1 2 5 2 3 5 4 4 3 97
  99. 99. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 4 Returns cup 5 1 1 2 5 2 5 3 4 4 3 98
  100. 100. The Java Specialists Newsletter - javaspecialists.eu / zenika.frsorting locks by property is easy 99
  101. 101. The Java Specialists Newsletter - javaspecialists.eu / zenika.fre.g. account IBAN number 100
  102. 102. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr or employee number 101
  103. 103. The Java Specialists Newsletter - javaspecialists.eu / zenika.fror some other characteristic 102
  104. 104. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr maybe evenSystem.identityHashCode() 103
  105. 105. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr use "open calls" 104
  106. 106. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Vector synchronizes writeObject()private synchronized void writeObject(ObjectOutputStream s) throws IOException { s.defaultWriteObject();} 105
  107. 107. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr synchronized writeObject() is invoking s.defaultWriteObject()private synchronized void writeObject(ObjectOutputStream s) throws IOException { s.defaultWriteObject();} 106
  108. 108. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr this is not an "open call"private synchronized void writeObject(ObjectOutputStream s) throws IOException { s.defaultWriteObject();} 107
  109. 109. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr but can it cause problems?private synchronized void writeObject(ObjectOutputStream s) throws IOException { s.defaultWriteObject();} 108
  110. 110. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr yes it can!private synchronized void writeObject(ObjectOutputStream s) throws IOException { s.defaultWriteObject();} 109
  111. 111. The Java Specialists Newsletter - javaspecialists.eu / zenika.frserialize both vectors at same timeVector v1 = new Vector();Vector v2 = new Vector();v1.add(v2);v2.add(v1); 110
  112. 112. The Java Specialists Newsletter - javaspecialists.eu / zenika.frthis could happen with nested data structures 111
  113. 113. The Java Specialists Newsletter - javaspecialists.eu / zenika.frit happened to an IBM client 112
  114. 114. The Java Specialists Newsletter - javaspecialists.eu / zenika.frprivate void writeObject(ObjectOutputStream stream) throws IOException { Vector<E> cloned = null; // this specially fix is for a special dead-lock in customer // program: two vectors refer each other may meet dead-lock in // synchronized serialization. Refer CMVC-103316.1 synchronized (this) { try { cloned = (Vector<E>) super.clone(); cloned.elementData = elementData.clone(); } catch (CloneNotSupportedException e) { // no deep clone, ignore the exception } } cloned.writeObjectImpl(stream);}private void writeObjectImpl(ObjectOutputStream stream) throws IOException { stream.defaultWriteObject();} 113
  115. 115. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Java 7 uses an "open call"private void writeObject(ObjectOutputStream s) throws IOException { final ObjectOutputStream.PutField fields = s.putFields(); final Object[] data; synchronized (this) { fields.put("capacityIncrement", capacityIncrement); fields.put("elementCount", elementCount); data = elementData.clone(); } fields.put("elementData", data); s.writeFields();} 114
  116. 116. The Java Specialists Newsletter - javaspecialists.eu / zenika.frMore synchronized writeObject()s• File• StringBuffer• Throwable• URL• Hashtable• and others 115
  117. 117. The Java Specialists Newsletter - javaspecialists.eu / zenika.frwe have seen the deadly embrace 116
  118. 118. The Java Specialists Newsletter - javaspecialists.eu / zenika.frare there other deadlocks? 117
  119. 119. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr semaphore deadlocks 118
  120. 120. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Bounded DB Connection Pools• For example, say you have two DB connection pools • Some tasks might require connections to both databases • Thus thread A might hold semaphore for D1 and wait for D2, whereas thread B might hold semaphore for D2 and be waiting for D1 119
  121. 121. The Java Specialists Newsletter - javaspecialists.eu / zenika.frpublic class DatabasePool { private final Semaphore connections; public DatabasePool(int connections) { this.connections = new Semaphore(connections); } public void connect() { connections.acquireUninterruptibly(); System.out.println("DatabasePool.connect"); } public void disconnect() { System.out.println("DatabasePool.disconnect"); connections.release(); }} 120
  122. 122. The Java Specialists Newsletter - javaspecialists.eu / zenika.frThreadMXBean does not show this as a deadlock! 121
  123. 123. The Java Specialists Newsletter - javaspecialists.eu / zenika.frJVM does not detect this deadlock 122
  124. 124. The Java Specialists Newsletter - javaspecialists.eu / zenika.frJVM does not detect this deadlock 122
  125. 125. The Java Specialists Newsletter - javaspecialists.eu / zenika.frAvoiding and diagnosing deadlocks Avoiding Liveness Hazards 123
  126. 126. The Java Specialists Newsletter - javaspecialists.eu / zenika.frAvoiding and diagnosing deadlocks• You cannot get a lock-ordering deadlock with a single lock • Easiest way to avoid deadlocks • But not always practical 124
  127. 127. The Java Specialists Newsletter - javaspecialists.eu / zenika.frAvoiding and diagnosing deadlocks• For multiple locks we need to define a lock order • specify and document possible lock sequences • Identify where multiple locks could be acquired • Do a global analysis to ensure that lock ordering is consistent • This can be extremely difficult in large programs 125
  128. 128. The Java Specialists Newsletter - javaspecialists.eu / zenika.frUse open calls whenever possible 126
  129. 129. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Unit Testing for lock ordering deadlocks• Code typically has to be called many times before a deadlock happens• How many times do you need to call it to prove that there is no deadlock? • Nondeterministic unit tests are bad - they should either always pass or always fail 127
  130. 130. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Testing the Bank • In the transferMoney() method, a deadlock occurs if after the first lock is granted, the first thread is swapped out and another thread requests the second lockpublic class Bank { public boolean transferMoney(Account from, Account to, Amount amount) { synchronized (from) { // if thread is swapped out here, we can end up with a deadlock synchronized (to) { return doActualTransfer(from, to, amount); } } }} 128
  131. 131. The Java Specialists Newsletter - javaspecialists.eu / zenika.frWe can force the context switch 129
  132. 132. The Java Specialists Newsletter - javaspecialists.eu / zenika.frsimply add a sleep after first lock 130
  133. 133. The Java Specialists Newsletter - javaspecialists.eu / zenika.frAdding a sleep to cause deadlockspublic class Bank { public boolean transferMoney(Account from, Account to, Amount amount) { synchronized (from) { sleepAWhileForTesting(); synchronized (to) { return doActualTransfer(from, to, amount); } } } protected void sleepAWhileForTesting() {}} 131
  134. 134. The Java Specialists Newsletter - javaspecialists.eu / zenika.frIn our unit test we use a SlowBank public class SlowBank extends Bank { private final long timeout; private final TimeUnit unit; public SlowBank(long timeout, TimeUnit unit) { this.timeout = timeout; this.unit = unit; } protected void sleepAWhileForTesting() { try { unit.sleep(timeout); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } } 132
  135. 135. The Java Specialists Newsletter - javaspecialists.eu / zenika.frWont this cause problems in production? 133
  136. 136. The Java Specialists Newsletter - javaspecialists.eu / zenika.frIn production we only use Bank 134
  137. 137. The Java Specialists Newsletter - javaspecialists.eu / zenika.frHotSpot will optimize away empty sleepAWhileForTesting() 135
  138. 138. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr it is thus a zero cost 136
  139. 139. The Java Specialists Newsletter - javaspecialists.eu / zenika.fras long as SlowBank is never used 137
  140. 140. The Java Specialists Newsletter - javaspecialists.eu / zenika.frHow can you verify the lock- ordering deadlock? 138
  141. 141. The Java Specialists Newsletter - javaspecialists.eu / zenika.frCTRL+Break, jstack, kill -3, jconsole, etc. 139
  142. 142. The Java Specialists Newsletter - javaspecialists.eu / zenika.frbut that wont be a good unit test 140
  143. 143. The Java Specialists Newsletter - javaspecialists.eu / zenika.frBetter via Javas ThreadMXBean 141
  144. 144. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr ThreadMXBean• Get instance with ManagementFactory.getThreadMXBean()• findMonitorDeadlockedThreads() • Includes only "monitor" locks, i.e. synchronized • Only way to find deadlocks in Java 5 142
  145. 145. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr ThreadMXBean• Better is findDeadlockedThreads() • Includes "monitor" and "owned" (Java 5) locks • Preferred method to test for deadlocks • But, does not find deadlocks between semaphores• See http://www.javaspecialists.eu/archive/Issue130.html 143
  146. 146. The Java Specialists Newsletter - javaspecialists.eu / zenika.frpublic class BankDeadlockTest { private final static ThreadMXBean tmb = ManagementFactory.getThreadMXBean(); private boolean isThreadDeadlocked(long tid) { long[] ids = tmb.findDeadlockedThreads(); if (ids == null) return false; for (long id : ids) { if (id == tid) return true; } return false; } 144
  147. 147. The Java Specialists Newsletter - javaspecialists.eu / zenika.frprivate void checkThatThreadTerminates(Thread thread) throws InterruptedException { for (int i = 0; i < 2000; i++) { thread.join(50); if (!thread.isAlive()) return; if (isThreadDeadlocked(thread.getId())) { fail("Deadlock detected!"); } } fail(thread + " did not terminate in time");} 145
  148. 148. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr @Test public void testForTransferDeadlock() throws InterruptedException { final Account alpha = new Account(new Amount(1000)); final Account ubs = new Account(new Amount(1000000)); final Bank bank = new SlowBank(100, TimeUnit.MILLISECONDS); Thread alphaToUbs = new Thread("alphaToUbs") { public void run() { bank.transferMoney(alpha, ubs, new Amount(100)); } }; Thread ubsToAlpha = new Thread("ubsToAlpha") { public void run() { bank.transferMoney(ubs, alpha, new Amount(100)); } }; alphaToUbs.start(); ubsToAlpha.start(); checkThatThreadTerminates(alphaToUbs); }} 146
  149. 149. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Output with broken transferMoney() method • We see the deadlock within about 100 millisecondsjunit.framework.AssertionFailedError: Deadlock detected! at BankDeadlockTest.checkThatThreadTerminates(BankDeadlockTest.java:20) at BankDeadlockTest.testForTransferDeadlock(BankDeadlockTest.java:55) 147
  150. 150. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Output with broken transferMoney() method• If we fix the transferMoney() method, it also completes within about 100 milliseconds • This is the time that we are sleeping for testing purposes• Remember that the empty sleepAWhileForTesting() method will be optimized away by HotSpot 148
  151. 151. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Timed lock attempts• Another technique for solving deadlocks is to use the timed tryLock() method of explicit Java locks 149
  152. 152. The Java Specialists Newsletter - javaspecialists.eu / zenika.frHowever, if tryLock() fails it may be 150
  153. 153. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr a deadlock 151
  154. 154. The Java Specialists Newsletter - javaspecialists.eu / zenika.fror some thread taking a long time 152
  155. 155. The Java Specialists Newsletter - javaspecialists.eu / zenika.fror a thread in an infinite loop 153
  156. 156. The Java Specialists Newsletter - javaspecialists.eu / zenika.frBut ThreadMXBean shows it as deadlocked 154
  157. 157. The Java Specialists Newsletter - javaspecialists.eu / zenika.freven if it is a temporary condition 155
  158. 158. The Java Specialists Newsletter - javaspecialists.eu / zenika.frThe JVM has support for tryLock with synchronized 156
  159. 159. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr but it is hidden away 157
  160. 160. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr So here it is 158
  161. 161. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr 159
  162. 162. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Told you it was hidden! 160
  163. 163. The Java Specialists Newsletter - javaspecialists.eu / zenika.frIt is hidden for a good reason 161
  164. 164. The Java Specialists Newsletter - javaspecialists.eu / zenika.frif you need tryLock functionality 162
  165. 165. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr rather use explicit locks (ReentrantLock) 163
  166. 166. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr tryLock with synchronized• Technically possible in Sun JVM using • Unsafe.getUnsafe().tryMonitorEnter(monitor) • Unsafe.getUnsafe().monitorExit(monitor)• http://www.javaspecialists.eu/archive/Issue194.html 164
  167. 167. The Java Specialists Newsletter - javaspecialists.eu / zenika.frDeadlock analysis with thread dumps 165
  168. 168. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Deadlock analysis with thread dumps• We can also cause a thread dump in many ways: • Ctrl+Break on Windows or Ctrl- on Unix • Invoking "kill -3" on the process id • Calling jstack on the process id • Only shows deadlocks since Java 6• Intrinsic locks show more information of where they were acquired 166
  169. 169. The Java Specialists Newsletter - javaspecialists.eu / zenika.frCan a deadlock be resolved? 167
  170. 170. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr NO with synchronized 168
  171. 171. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr YES with ReentrantLock 169
  172. 172. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr synchronized() goes into BLOCKED state 170
  173. 173. The Java Specialists Newsletter - javaspecialists.eu / zenika.frReentrantLock.lock() goes into WAITING state 171
  174. 174. The Java Specialists Newsletter - javaspecialists.eu / zenika.frWe can write a DealockArbitrator 172
  175. 175. The Java Specialists Newsletter - javaspecialists.eu / zenika.frThis stops one of the threads with a DeadlockVictimError 173
  176. 176. The Java Specialists Newsletter - javaspecialists.eu / zenika.frOur special Error for deadlocks public class DeadlockVictimError extends Error { private final Thread victim; public DeadlockVictimError(Thread victim) { super("Deadlock victim: " + victim); this.victim = victim; } public Thread getVictim() { return victim; } } 174
  177. 177. The Java Specialists Newsletter - javaspecialists.eu / zenika.frpublic class DeadlockArbitrator { private static final ThreadMXBean tmb = ManagementFactory.getThreadMXBean(); public boolean tryResolveDeadlock( int attempts, long timeout, TimeUnit unit) throws InterruptedException { for (int i = 0; i < attempts; i++) { long[] ids = tmb.findDeadlockedThreads(); if (ids == null) return true; Thread t = findThread(ids[i % ids.length]); if (t == null) throw new IllegalStateException("Could not find thread"); t.stop(new DeadlockVictimError(t)); unit.sleep(timeout); } return false; } 175
  178. 178. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr finding threads by id is tricky public boolean tryResolveDeadlock() throws InterruptedException { return tryResolveDeadlock(3, 1, TimeUnit.SECONDS); } private Thread findThread(long id) { for (Thread thread : Thread.getAllStackTraces().keySet()) { if (thread.getId() == id) return thread; } return null; }} 176
  179. 179. The Java Specialists Newsletter - javaspecialists.eu / zenika.frApplicability of DeadlockArbitrator• Only use in extreme circumstances • Code that is outside your control and that deadlocks • Where you cannot prevent the deadlock• Remember, it only works with ReentrantLock 177
  180. 180. The Java Specialists Newsletter - javaspecialists.eu / zenika.frDeadlock Victim Questions? by Dr Heinz Kabutz && Olivier CroisierThe Java Specialists Newsletter && The Coders Breakfast heinz@javaspecialists.eu && olivier.croisier@zenika.com twitter: @heinzkabutz && twitter: @oliviercroisier 178

×