Deadlock Victim

2,310 views
2,238 views

Published on

Support de la présentation "Deadlock Victim" à Devoxx-fr 2012, par Heinz Kabutz (javaspecialists.eu) et Olivier Croisier (zenika.com, thecodersbreakfast.net).

Published in: Technology, News & Politics
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,310
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
15
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Deadlock Victim

  1. 1. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Deadlock Victim by Dr Heinz Kabutz && Olivier CroisierThe Java Specialists Newsletter && The Coders Breakfast heinz@javaspecialists.eu && olivier.croisier@zenika.com twitter: @heinzkabutz && twitter: @oliviercroisier 1
  2. 2. The Java Specialists Newsletter - javaspecialists.eu / zenika.frYou discover a race condition 2
  3. 3. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Data is being corrupted 3
  4. 4. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr What ya gonna do? 4
  5. 5. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Who ya gonna call? 5
  6. 6. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Ghostbusters? 6
  7. 7. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr you do what we all do 7
  8. 8. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr you use synchronized 8
  9. 9. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr synchronized 9
  10. 10. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr synchronized 9
  11. 11. The Java Specialists Newsletter - javaspecialists.eu / zenika.frsynchronized can be bad medicine 10
  12. 12. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr deadlock awaits you! 11
  13. 13. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr 12
  14. 14. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Deadlock Victim 13
  15. 15. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Correctness fix can lead to liveness fault 14
  16. 16. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr lock ordering deadlock 15
  17. 17. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr at least two locks 16
  18. 18. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr and at least two threads 17
  19. 19. The Java Specialists Newsletter - javaspecialists.eu / zenika.frlocked in different orders 18
  20. 20. The Java Specialists Newsletter - javaspecialists.eu / zenika.frcan cause a deadly embrace 19
  21. 21. The Java Specialists Newsletter - javaspecialists.eu / zenika.frrequires global analysis to fix 20
  22. 22. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr resource deadlocks 21
  23. 23. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr bounded queues 22
  24. 24. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr bounded thread pools 23
  25. 25. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr semaphores 24
  26. 26. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr class loaders 25
  27. 27. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr lost signals 26
  28. 28. The Java Specialists Newsletter - javaspecialists.eu / zenika.frThe Drinking Philosophers 27
  29. 29. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr@ the Java Specialist Symposium in Crete 28
  30. 30. The Java Specialists Newsletter - javaspecialists.eu / zenika.frSymposium == sym + poto Literally - to drink together 29
  31. 31. The Java Specialists Newsletter - javaspecialists.eu / zenika.frour philosophers need two cups 30
  32. 32. The Java Specialists Newsletter - javaspecialists.eu / zenika.frthey either want to drink or think 31
  33. 33. The Java Specialists Newsletter - javaspecialists.eu / zenika.frTable is ready, all philosophers are thinking 1 5 2 4 3 32
  34. 34. The Java Specialists Newsletter - javaspecialists.eu / zenika.frphilosophers 5 wants to drink, takes right cup 1 5 2 4 3 33
  35. 35. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 5 is now drinking with both cups 1 5 2 4 3 34
  36. 36. The Java Specialists Newsletter - javaspecialists.eu / zenika.frphilosophers 3 wants to drink, takes right cup 1 5 2 4 3 35
  37. 37. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 is now drinking with both cups 1 5 2 4 3 36
  38. 38. The Java Specialists Newsletter - javaspecialists.eu / zenika.frphilosophers 2 wants to drink, takes right cup 1 5 2 4 3 37
  39. 39. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 finished drinking, returns right cup 1 5 2 4 3 38
  40. 40. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 2 is now drinking with both cups 1 5 2 4 3 39
  41. 41. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 returns Left cup 1 5 2 4 3 40
  42. 42. The Java Specialists Newsletter - javaspecialists.eu / zenika.frphilosophers can get stuck 41
  43. 43. The Java Specialists Newsletter - javaspecialists.eu / zenika.frif they all pick up the right cup 42
  44. 44. The Java Specialists Newsletter - javaspecialists.eu / zenika.frA deadlock can easily happen with this design 1 5 2 4 3 43
  45. 45. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 5 wants to drink, takes right cup 1 5 2 4 3 44
  46. 46. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 1 wants to drink, takes right cup 1 5 2 4 3 45
  47. 47. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 2 wants to drink, takes right cup 1 5 2 4 3 46
  48. 48. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 wants to drink, takes right cup 1 5 2 4 3 47
  49. 49. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 4 wants to drink, takes right cup 1 5 2 4 3 48
  50. 50. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Deadlock! 1 5 2 4 3 49
  51. 51. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr they will all stay thirsty 1 5 2 4 3 50
  52. 52. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr this is a deadly embrace 1 5 2 4 3 51
  53. 53. The Java Specialists Newsletter - javaspecialists.eu / zenika.frthis can be detected automatically 52
  54. 54. The Java Specialists Newsletter - javaspecialists.eu / zenika.frsearch the graph of call stacks 53
  55. 55. The Java Specialists Newsletter - javaspecialists.eu / zenika.frand look for circular dependencies 54
  56. 56. The Java Specialists Newsletter - javaspecialists.eu / zenika.frThreadMXBean does that for us 55
  57. 57. The Java Specialists Newsletter - javaspecialists.eu / zenika.frsimple deadlocks are quickly found 56
  58. 58. The Java Specialists Newsletter - javaspecialists.eu / zenika.frbut what do you DO with them? 57
  59. 59. The Java Specialists Newsletter - javaspecialists.eu / zenika.frwhat will the dog do with the car if he ever catches it? 58
  60. 60. The Java Specialists Newsletter - javaspecialists.eu / zenika.frdatabases choose deadlock victim 59
  61. 61. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr but what does Java do? 60
  62. 62. The Java Specialists Newsletter - javaspecialists.eu / zenika.frit should probably throw an Error 61
  63. 63. The Java Specialists Newsletter - javaspecialists.eu / zenika.frbut the threads simply hang up 62
  64. 64. The Java Specialists Newsletter - javaspecialists.eu / zenika.frit is best to avoid causing them! 63
  65. 65. The Java Specialists Newsletter - javaspecialists.eu / zenika.frglobal order of locks in program 64
  66. 66. The Java Specialists Newsletter - javaspecialists.eu / zenika.frpublic class LeftRightDeadlock { private final Object left = new Object(); private final Object right = new Object(); public void leftRight() { synchronized (left) { synchronized (right) { doSomething(); } } } public void rightLeft() { synchronized (right) { synchronized (left) { doSomethingElse(); } } }} 65
  67. 67. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Interleaving causes deadlock lock left trying to lock rightA leftRight() BLOCKED lock right trying to lock leftB rightLeft() BLOCKED 66
  68. 68. The Java Specialists Newsletter - javaspecialists.eu / zenika.frwe define a fixed global order 67
  69. 69. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr e.g. left before right 68
  70. 70. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr deadlock is solved! public void rightLeft() { synchronized (left) { synchronized (right) { doSomethingElse(); } } } 69
  71. 71. The Java Specialists Newsletter - javaspecialists.eu / zenika.frour drinking philosophers 70
  72. 72. The Java Specialists Newsletter - javaspecialists.eu / zenika.freach cup gets a unique number 71
  73. 73. The Java Specialists Newsletter - javaspecialists.eu / zenika.frlock the highest number cup first 72
  74. 74. The Java Specialists Newsletter - javaspecialists.eu / zenika.frthen lock the lower number cup 73
  75. 75. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr drink 74
  76. 76. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr unlock lower order cup 75
  77. 77. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr unlock higher order cup 76
  78. 78. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr think 77
  79. 79. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Global Lock ordering• We start with all the 1 philosophers thinking 1 2 5 2 5 3 4 4 3 78
  80. 80. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Philosopher 5 takes cup 5• Cup 5 has higher number 1 • Remember our rule! 1 2 5 2 5 3 4 4 3 79
  81. 81. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Philosopher 1 takes cup 2• Must take the cup with 1 the higher number 2 first 1 • In this case 5 2 cup 2 5 3 4 4 3 80
  82. 82. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 2 takes cup 3 1 2 1 5 2 5 3 4 4 3 81
  83. 83. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 takes cup 4 1 2 1 5 2 5 3 4 4 3 82
  84. 84. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 1 takes cup 1 - Drinking 1 1 2 5 2 5 3 4 4 3 83
  85. 85. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Philosopher 1 returns Cup 1• Cups are returned in the 1 opposite order to what 2 they are acquired 1 5 2 5 3 4 4 3 84
  86. 86. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 5 takes cup 1 - Drinking 1 2 1 5 2 5 3 4 4 3 85
  87. 87. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 5 returns cup 1 1 2 1 5 2 5 3 4 4 3 86
  88. 88. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 1 returns cup 2 1 1 2 5 2 5 3 4 4 3 87
  89. 89. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 2 takes cup 2 - Drinking 1 2 1 5 2 5 3 4 4 3 88
  90. 90. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 5 returns cup 5 1 2 1 5 2 3 5 4 4 3 89
  91. 91. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 4 takes cup 5 1 2 1 5 2 3 5 4 4 3 90
  92. 92. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 2 returns cup 2 1 1 2 5 2 3 5 4 4 3 91
  93. 93. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 2 returns cup 3 1 1 2 5 2 3 5 4 4 3 92
  94. 94. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 takes cup 3 - Drinking 1 1 2 5 2 5 3 4 4 3 93
  95. 95. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 Returns cup 3 1 1 2 5 2 3 5 4 4 3 94
  96. 96. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 3 Returns cup 4 1 1 2 5 2 3 5 4 4 3 95
  97. 97. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 4 takes cup 4 - Drinking 1 1 2 5 2 3 5 4 4 3 96
  98. 98. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 4 Returns cup 4 1 1 2 5 2 3 5 4 4 3 97
  99. 99. The Java Specialists Newsletter - javaspecialists.eu / zenika.frPhilosopher 4 Returns cup 5 1 1 2 5 2 5 3 4 4 3 98
  100. 100. The Java Specialists Newsletter - javaspecialists.eu / zenika.frsorting locks by property is easy 99
  101. 101. The Java Specialists Newsletter - javaspecialists.eu / zenika.fre.g. account IBAN number 100
  102. 102. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr or employee number 101
  103. 103. The Java Specialists Newsletter - javaspecialists.eu / zenika.fror some other characteristic 102
  104. 104. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr maybe evenSystem.identityHashCode() 103
  105. 105. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr use "open calls" 104
  106. 106. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Vector synchronizes writeObject()private synchronized void writeObject(ObjectOutputStream s) throws IOException { s.defaultWriteObject();} 105
  107. 107. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr synchronized writeObject() is invoking s.defaultWriteObject()private synchronized void writeObject(ObjectOutputStream s) throws IOException { s.defaultWriteObject();} 106
  108. 108. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr this is not an "open call"private synchronized void writeObject(ObjectOutputStream s) throws IOException { s.defaultWriteObject();} 107
  109. 109. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr but can it cause problems?private synchronized void writeObject(ObjectOutputStream s) throws IOException { s.defaultWriteObject();} 108
  110. 110. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr yes it can!private synchronized void writeObject(ObjectOutputStream s) throws IOException { s.defaultWriteObject();} 109
  111. 111. The Java Specialists Newsletter - javaspecialists.eu / zenika.frserialize both vectors at same timeVector v1 = new Vector();Vector v2 = new Vector();v1.add(v2);v2.add(v1); 110
  112. 112. The Java Specialists Newsletter - javaspecialists.eu / zenika.frthis could happen with nested data structures 111
  113. 113. The Java Specialists Newsletter - javaspecialists.eu / zenika.frit happened to an IBM client 112
  114. 114. The Java Specialists Newsletter - javaspecialists.eu / zenika.frprivate void writeObject(ObjectOutputStream stream) throws IOException { Vector<E> cloned = null; // this specially fix is for a special dead-lock in customer // program: two vectors refer each other may meet dead-lock in // synchronized serialization. Refer CMVC-103316.1 synchronized (this) { try { cloned = (Vector<E>) super.clone(); cloned.elementData = elementData.clone(); } catch (CloneNotSupportedException e) { // no deep clone, ignore the exception } } cloned.writeObjectImpl(stream);}private void writeObjectImpl(ObjectOutputStream stream) throws IOException { stream.defaultWriteObject();} 113
  115. 115. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Java 7 uses an "open call"private void writeObject(ObjectOutputStream s) throws IOException { final ObjectOutputStream.PutField fields = s.putFields(); final Object[] data; synchronized (this) { fields.put("capacityIncrement", capacityIncrement); fields.put("elementCount", elementCount); data = elementData.clone(); } fields.put("elementData", data); s.writeFields();} 114
  116. 116. The Java Specialists Newsletter - javaspecialists.eu / zenika.frMore synchronized writeObject()s• File• StringBuffer• Throwable• URL• Hashtable• and others 115
  117. 117. The Java Specialists Newsletter - javaspecialists.eu / zenika.frwe have seen the deadly embrace 116
  118. 118. The Java Specialists Newsletter - javaspecialists.eu / zenika.frare there other deadlocks? 117
  119. 119. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr semaphore deadlocks 118
  120. 120. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Bounded DB Connection Pools• For example, say you have two DB connection pools • Some tasks might require connections to both databases • Thus thread A might hold semaphore for D1 and wait for D2, whereas thread B might hold semaphore for D2 and be waiting for D1 119
  121. 121. The Java Specialists Newsletter - javaspecialists.eu / zenika.frpublic class DatabasePool { private final Semaphore connections; public DatabasePool(int connections) { this.connections = new Semaphore(connections); } public void connect() { connections.acquireUninterruptibly(); System.out.println("DatabasePool.connect"); } public void disconnect() { System.out.println("DatabasePool.disconnect"); connections.release(); }} 120
  122. 122. The Java Specialists Newsletter - javaspecialists.eu / zenika.frThreadMXBean does not show this as a deadlock! 121
  123. 123. The Java Specialists Newsletter - javaspecialists.eu / zenika.frJVM does not detect this deadlock 122
  124. 124. The Java Specialists Newsletter - javaspecialists.eu / zenika.frJVM does not detect this deadlock 122
  125. 125. The Java Specialists Newsletter - javaspecialists.eu / zenika.frAvoiding and diagnosing deadlocks Avoiding Liveness Hazards 123
  126. 126. The Java Specialists Newsletter - javaspecialists.eu / zenika.frAvoiding and diagnosing deadlocks• You cannot get a lock-ordering deadlock with a single lock • Easiest way to avoid deadlocks • But not always practical 124
  127. 127. The Java Specialists Newsletter - javaspecialists.eu / zenika.frAvoiding and diagnosing deadlocks• For multiple locks we need to define a lock order • specify and document possible lock sequences • Identify where multiple locks could be acquired • Do a global analysis to ensure that lock ordering is consistent • This can be extremely difficult in large programs 125
  128. 128. The Java Specialists Newsletter - javaspecialists.eu / zenika.frUse open calls whenever possible 126
  129. 129. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Unit Testing for lock ordering deadlocks• Code typically has to be called many times before a deadlock happens• How many times do you need to call it to prove that there is no deadlock? • Nondeterministic unit tests are bad - they should either always pass or always fail 127
  130. 130. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Testing the Bank • In the transferMoney() method, a deadlock occurs if after the first lock is granted, the first thread is swapped out and another thread requests the second lockpublic class Bank { public boolean transferMoney(Account from, Account to, Amount amount) { synchronized (from) { // if thread is swapped out here, we can end up with a deadlock synchronized (to) { return doActualTransfer(from, to, amount); } } }} 128
  131. 131. The Java Specialists Newsletter - javaspecialists.eu / zenika.frWe can force the context switch 129
  132. 132. The Java Specialists Newsletter - javaspecialists.eu / zenika.frsimply add a sleep after first lock 130
  133. 133. The Java Specialists Newsletter - javaspecialists.eu / zenika.frAdding a sleep to cause deadlockspublic class Bank { public boolean transferMoney(Account from, Account to, Amount amount) { synchronized (from) { sleepAWhileForTesting(); synchronized (to) { return doActualTransfer(from, to, amount); } } } protected void sleepAWhileForTesting() {}} 131
  134. 134. The Java Specialists Newsletter - javaspecialists.eu / zenika.frIn our unit test we use a SlowBank public class SlowBank extends Bank { private final long timeout; private final TimeUnit unit; public SlowBank(long timeout, TimeUnit unit) { this.timeout = timeout; this.unit = unit; } protected void sleepAWhileForTesting() { try { unit.sleep(timeout); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } } 132
  135. 135. The Java Specialists Newsletter - javaspecialists.eu / zenika.frWont this cause problems in production? 133
  136. 136. The Java Specialists Newsletter - javaspecialists.eu / zenika.frIn production we only use Bank 134
  137. 137. The Java Specialists Newsletter - javaspecialists.eu / zenika.frHotSpot will optimize away empty sleepAWhileForTesting() 135
  138. 138. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr it is thus a zero cost 136
  139. 139. The Java Specialists Newsletter - javaspecialists.eu / zenika.fras long as SlowBank is never used 137
  140. 140. The Java Specialists Newsletter - javaspecialists.eu / zenika.frHow can you verify the lock- ordering deadlock? 138
  141. 141. The Java Specialists Newsletter - javaspecialists.eu / zenika.frCTRL+Break, jstack, kill -3, jconsole, etc. 139
  142. 142. The Java Specialists Newsletter - javaspecialists.eu / zenika.frbut that wont be a good unit test 140
  143. 143. The Java Specialists Newsletter - javaspecialists.eu / zenika.frBetter via Javas ThreadMXBean 141
  144. 144. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr ThreadMXBean• Get instance with ManagementFactory.getThreadMXBean()• findMonitorDeadlockedThreads() • Includes only "monitor" locks, i.e. synchronized • Only way to find deadlocks in Java 5 142
  145. 145. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr ThreadMXBean• Better is findDeadlockedThreads() • Includes "monitor" and "owned" (Java 5) locks • Preferred method to test for deadlocks • But, does not find deadlocks between semaphores• See http://www.javaspecialists.eu/archive/Issue130.html 143
  146. 146. The Java Specialists Newsletter - javaspecialists.eu / zenika.frpublic class BankDeadlockTest { private final static ThreadMXBean tmb = ManagementFactory.getThreadMXBean(); private boolean isThreadDeadlocked(long tid) { long[] ids = tmb.findDeadlockedThreads(); if (ids == null) return false; for (long id : ids) { if (id == tid) return true; } return false; } 144
  147. 147. The Java Specialists Newsletter - javaspecialists.eu / zenika.frprivate void checkThatThreadTerminates(Thread thread) throws InterruptedException { for (int i = 0; i < 2000; i++) { thread.join(50); if (!thread.isAlive()) return; if (isThreadDeadlocked(thread.getId())) { fail("Deadlock detected!"); } } fail(thread + " did not terminate in time");} 145
  148. 148. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr @Test public void testForTransferDeadlock() throws InterruptedException { final Account alpha = new Account(new Amount(1000)); final Account ubs = new Account(new Amount(1000000)); final Bank bank = new SlowBank(100, TimeUnit.MILLISECONDS); Thread alphaToUbs = new Thread("alphaToUbs") { public void run() { bank.transferMoney(alpha, ubs, new Amount(100)); } }; Thread ubsToAlpha = new Thread("ubsToAlpha") { public void run() { bank.transferMoney(ubs, alpha, new Amount(100)); } }; alphaToUbs.start(); ubsToAlpha.start(); checkThatThreadTerminates(alphaToUbs); }} 146
  149. 149. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Output with broken transferMoney() method • We see the deadlock within about 100 millisecondsjunit.framework.AssertionFailedError: Deadlock detected! at BankDeadlockTest.checkThatThreadTerminates(BankDeadlockTest.java:20) at BankDeadlockTest.testForTransferDeadlock(BankDeadlockTest.java:55) 147
  150. 150. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Output with broken transferMoney() method• If we fix the transferMoney() method, it also completes within about 100 milliseconds • This is the time that we are sleeping for testing purposes• Remember that the empty sleepAWhileForTesting() method will be optimized away by HotSpot 148
  151. 151. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Timed lock attempts• Another technique for solving deadlocks is to use the timed tryLock() method of explicit Java locks 149
  152. 152. The Java Specialists Newsletter - javaspecialists.eu / zenika.frHowever, if tryLock() fails it may be 150
  153. 153. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr a deadlock 151
  154. 154. The Java Specialists Newsletter - javaspecialists.eu / zenika.fror some thread taking a long time 152
  155. 155. The Java Specialists Newsletter - javaspecialists.eu / zenika.fror a thread in an infinite loop 153
  156. 156. The Java Specialists Newsletter - javaspecialists.eu / zenika.frBut ThreadMXBean shows it as deadlocked 154
  157. 157. The Java Specialists Newsletter - javaspecialists.eu / zenika.freven if it is a temporary condition 155
  158. 158. The Java Specialists Newsletter - javaspecialists.eu / zenika.frThe JVM has support for tryLock with synchronized 156
  159. 159. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr but it is hidden away 157
  160. 160. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr So here it is 158
  161. 161. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr 159
  162. 162. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Told you it was hidden! 160
  163. 163. The Java Specialists Newsletter - javaspecialists.eu / zenika.frIt is hidden for a good reason 161
  164. 164. The Java Specialists Newsletter - javaspecialists.eu / zenika.frif you need tryLock functionality 162
  165. 165. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr rather use explicit locks (ReentrantLock) 163
  166. 166. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr tryLock with synchronized• Technically possible in Sun JVM using • Unsafe.getUnsafe().tryMonitorEnter(monitor) • Unsafe.getUnsafe().monitorExit(monitor)• http://www.javaspecialists.eu/archive/Issue194.html 164
  167. 167. The Java Specialists Newsletter - javaspecialists.eu / zenika.frDeadlock analysis with thread dumps 165
  168. 168. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr Deadlock analysis with thread dumps• We can also cause a thread dump in many ways: • Ctrl+Break on Windows or Ctrl- on Unix • Invoking "kill -3" on the process id • Calling jstack on the process id • Only shows deadlocks since Java 6• Intrinsic locks show more information of where they were acquired 166
  169. 169. The Java Specialists Newsletter - javaspecialists.eu / zenika.frCan a deadlock be resolved? 167
  170. 170. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr NO with synchronized 168
  171. 171. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr YES with ReentrantLock 169
  172. 172. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr synchronized() goes into BLOCKED state 170
  173. 173. The Java Specialists Newsletter - javaspecialists.eu / zenika.frReentrantLock.lock() goes into WAITING state 171
  174. 174. The Java Specialists Newsletter - javaspecialists.eu / zenika.frWe can write a DealockArbitrator 172
  175. 175. The Java Specialists Newsletter - javaspecialists.eu / zenika.frThis stops one of the threads with a DeadlockVictimError 173
  176. 176. The Java Specialists Newsletter - javaspecialists.eu / zenika.frOur special Error for deadlocks public class DeadlockVictimError extends Error { private final Thread victim; public DeadlockVictimError(Thread victim) { super("Deadlock victim: " + victim); this.victim = victim; } public Thread getVictim() { return victim; } } 174
  177. 177. The Java Specialists Newsletter - javaspecialists.eu / zenika.frpublic class DeadlockArbitrator { private static final ThreadMXBean tmb = ManagementFactory.getThreadMXBean(); public boolean tryResolveDeadlock( int attempts, long timeout, TimeUnit unit) throws InterruptedException { for (int i = 0; i < attempts; i++) { long[] ids = tmb.findDeadlockedThreads(); if (ids == null) return true; Thread t = findThread(ids[i % ids.length]); if (t == null) throw new IllegalStateException("Could not find thread"); t.stop(new DeadlockVictimError(t)); unit.sleep(timeout); } return false; } 175
  178. 178. The Java Specialists Newsletter - javaspecialists.eu / zenika.fr finding threads by id is tricky public boolean tryResolveDeadlock() throws InterruptedException { return tryResolveDeadlock(3, 1, TimeUnit.SECONDS); } private Thread findThread(long id) { for (Thread thread : Thread.getAllStackTraces().keySet()) { if (thread.getId() == id) return thread; } return null; }} 176
  179. 179. The Java Specialists Newsletter - javaspecialists.eu / zenika.frApplicability of DeadlockArbitrator• Only use in extreme circumstances • Code that is outside your control and that deadlocks • Where you cannot prevent the deadlock• Remember, it only works with ReentrantLock 177
  180. 180. The Java Specialists Newsletter - javaspecialists.eu / zenika.frDeadlock Victim Questions? by Dr Heinz Kabutz && Olivier CroisierThe Java Specialists Newsletter && The Coders Breakfast heinz@javaspecialists.eu && olivier.croisier@zenika.com twitter: @heinzkabutz && twitter: @oliviercroisier 178

×