スローダウン、ハングを一発解決 スレッドダンプはトラブルシューティングの味方 #wlstudy
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

スローダウン、ハングを一発解決 スレッドダンプはトラブルシューティングの味方 #wlstudy

  • 4,291 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
4,291
On Slideshare
4,262
From Embeds
29
Number of Embeds
1

Actions

Shares
Downloads
53
Comments
0
Likes
13

Embeds 29

https://twitter.com 29

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. スローダウン、ハングを一発解決スレッドダンプはトラブルシューティングの味方 山本 裕介 @yusuke Yusuke Yamamoto @yusuke #wlstudy
  • 2. スローダウン、ハングYusuke Yamamoto @yusuke #wlstudy
  • 3. スローダウン• アプリケーションは動いている• レスポンスがなかなか帰ってこないYusuke Yamamoto @yusuke #wlstudy
  • 4. ハング• アプリケーションが固まる• 応答がない• 503が帰ってくるYusuke Yamamoto @yusuke #wlstudy
  • 5. スローダウン、ハングの原因• マシンスペック不足 • キャパシティプランニングに間違い ‣ キャパシティプランニングをしていない ‣ 台数不足• 割り当てCPU/メモリ不足 • 仮想化時のリソース配分Yusuke Yamamoto @yusuke #wlstudy
  • 6. スローダウン、ハングの原因• チューニング不足 • JVMオプション • カーネルオプション• アプリケーションサーバ設定Yusuke Yamamoto @yusuke #wlstudy
  • 7. スローダウン、ハングの原因• アプリケーション • 非効率なロジック • リソースの奪い合い • I/O待ちYusuke Yamamoto @yusuke #wlstudy
  • 8. 切り分け• GCログの取得• GC時の挙動を確認• スレッドダンプの取得、解析Yusuke Yamamoto @yusuke #wlstudy
  • 9. スレッドダンプとは• JVMのスレッド状態のスナップショット• わかること • ある時点で • それぞれのスレッドが • 何をしているかYusuke Yamamoto @yusuke #wlstudy
  • 10. スレッドダンプでわからないこと• クラッシュ / ストールの原因• プロセスが落ちたらスレッドダンプは 取れないYusuke Yamamoto @yusuke #wlstudy
  • 11. スレッドダンプの利点• 追加ツール不要 • JProfiler • JProbe • YourKit Java Profiler• 取得時に負荷がかからない • 運用環境でも手軽に使える• オフラインで解析できるYusuke Yamamoto @yusuke #wlstudy
  • 12. スレッドダンプの見た目Yusuke Yamamoto @yusuke #wlstudy
  • 13. 2013-01-23 16:32:13Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.6-b04 mixed mode): at java.lang.Object.wait(Native Method)"Attach Listener" daemon prio=5 tid=0x00007f8dbd9dd000 nid=0x9407 waiting on condition [0x0000000000000000] at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab16fa8> (a weblogic.work.ExecuteThread) java.lang.Thread.State: RUNNABLE at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab16fa8> (a weblogic.work.ExecuteThread)"[STANDBY] ExecuteThread: 40 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcd3c000 nid=0x9303 in Object.wait() [0x0000000144526000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 18 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba9a3000 nid=0x7d03 in Object.wait() [0x0000000142d1b000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 18 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba9a3000 nid=0x7d03 in Object.wait() [0x0000000142d1b000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011c40fc88> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab18448> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab18448> (a weblogic.work.ExecuteThread)"[STANDBY] ExecuteThread: 39 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbdab9800 nid=0x9203 in Object.wait() [0x000000014437b000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[STANDBY] ExecuteThread: 17 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbae29800 nid=0x7c03 in Object.wait() [0x0000000142c18000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[STANDBY] ExecuteThread: 17 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbae29800 nid=0x7c03 in Object.wait() [0x0000000142c18000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab2f568> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab15510> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab15510> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 38 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd1bf000 nid=0x9103 in Object.wait() [0x0000000144278000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 16 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd0b7000 nid=0x7b03 in Object.wait() [0x0000000142b15000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 16 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd0b7000 nid=0x7b03 in Object.wait() [0x0000000142b15000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab2e4a0> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab17fc8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab17fc8> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 37 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba80b000 nid=0x9003 in Object.wait() [0x0000000144175000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 15 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcaa6800 nid=0x7a03 in Object.wait() [0x0000000142a12000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 15 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcaa6800 nid=0x7a03 in Object.wait() [0x0000000142a12000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab2d1b0> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab1aeb8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab1aeb8> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 36 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba479800 nid=0x8f03 in Object.wait() [0x0000000144072000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 14 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc5c800 nid=0x7903 in Object.wait() [0x0000000142856000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 14 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc5c800 nid=0x7903 in Object.wait() [0x0000000142856000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab27da0> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab1bed8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab1bed8> (a weblogic.work.ExecuteThread)"[STANDBY] ExecuteThread: 35 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db959f000 nid=0x8e03 in Object.wait() [0x0000000143f6f000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 13 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbb83c000 nid=0x7803 in Object.wait() [0x0000000142650000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 13 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbb83c000 nid=0x7803 in Object.wait() [0x0000000142650000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab26c90> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab144c8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab144c8> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 34 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba9a4000 nid=0x8d03 in Object.wait() [0x0000000143e6c000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 12 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd9c0000 nid=0x7703 in Object.wait() [0x0000000140a29000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 12 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd9c0000 nid=0x7703 in Object.wait() [0x0000000140a29000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab29978> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab19548> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab19548> (a weblogic.work.ExecuteThread)"[STANDBY] ExecuteThread: 33 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc5b800 nid=0x8c03 in Object.wait() [0x0000000143d69000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 11 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba94d800 nid=0x7603 in Object.wait() [0x0000000142347000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 11 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba94d800 nid=0x7603 in Object.wait() [0x0000000142347000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab29a28> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab0d3f8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab0d3f8> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 32 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc5b000 nid=0x8b03 in Object.wait() [0x0000000143c66000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[STANDBY] ExecuteThread: 10 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbe196000 nid=0x7503 in Object.wait() [0x0000000140c23000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[STANDBY] ExecuteThread: 10 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbe196000 nid=0x7503 in Object.wait() [0x0000000140c23000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab25c70> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab0e418> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab0e418> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 31 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbd54800 nid=0x8a03 in Object.wait() [0x00000001435d3000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 9 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbe195800 nid=0x7403 in Object.wait() [0x00000001408be000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 9 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbe195800 nid=0x7403 in Object.wait() [0x00000001408be000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab28da8> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab10b38> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab10b38> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 30 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbd8f800 nid=0x8903 in Object.wait() [0x00000001434d0000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 8 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbba6d800 nid=0x7303 in Object.wait() [0x000000013f0a9000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 8 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbba6d800 nid=0x7303 in Object.wait() [0x000000013f0a9000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab2a9a0> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab0c2d0> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab0c2d0> (a weblogic.work.ExecuteThread)"[STANDBY] ExecuteThread: 29 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbd8f000 nid=0x8803 in Object.wait() [0x0000000143b63000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 7 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbc812800 nid=0x7203 in Object.wait() [0x00000001407bb000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 7 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbc812800 nid=0x7203 in Object.wait() [0x00000001407bb000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab229f0> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab01498> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab01498> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 28 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcb4e000 nid=0x8703 in Object.wait() [0x0000000143a60000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 6 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db96b5800 nid=0x7103 in Object.wait() [0x00000001406b8000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 6 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db96b5800 nid=0x7103 in Object.wait() [0x00000001406b8000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab23a10> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab024b8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab024b8> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 27 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc91800 nid=0x8603 in Object.wait() [0x000000014395d000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[STANDBY] ExecuteThread: 5 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db971b000 nid=0x7003 in Object.wait() [0x00000001403de000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[STANDBY] ExecuteThread: 5 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db971b000 nid=0x7003 in Object.wait() [0x00000001403de000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab24a30> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab06208> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab06208> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 26 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbb015800 nid=0x8503 in Object.wait() [0x000000014385a000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 4 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbd5d000 nid=0x6f03 in Object.wait() [0x00000001401d1000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 4 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbd5d000 nid=0x6f03 in Object.wait() [0x00000001401d1000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab1f7b0> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab07228> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab07228> (a weblogic.work.ExecuteThread)"[STANDBY] ExecuteThread: 25 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba709800 nid=0x8403 in Object.wait() [0x0000000143757000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "DynamicListenThread[Default]" daemon prio=5 tid=0x00007f8dbd82b000 nid=0x6e03 runnable [0x0000000142753000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: RUNNABLE at java.lang.Object.wait(Native Method) "DynamicListenThread[Default]" daemon prio=5 tid=0x00007f8dbd82b000 nid=0x6e03 runnable [0x0000000142753000] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: RUNNABLE at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.ServerSocket.implAccept(ServerSocket.java:522) - locked <0x000000011ab207d0> (a weblogic.work.ExecuteThread) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.accept(ServerSocket.java:490) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at java.net.ServerSocket.implAccept(ServerSocket.java:522) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at java.net.ServerSocket.accept(ServerSocket.java:490) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535)"[ACTIVE] ExecuteThread: 24 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcb54800 nid=0x8303 in Object.wait() [0x000000013f1cd000] at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) java.lang.Thread.State: WAITING (on object monitor) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173) at java.lang.Object.wait(Native Method) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) at java.lang.Thread.run(Thread.java:722) at java.lang.Object.wait(Object.java:503) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Thread.run(Thread.java:722) "DynamicListenThread[Default[1]]" daemon prio=5 tid=0x00007f8db97d7800 nid=0x6c03 runnable [0x000000014254d000] - locked <0x000000011ab21860> (a weblogic.work.ExecuteThread) java.lang.Thread.State: RUNNABLE at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "DynamicListenThread[Default[1]]" daemon prio=5 tid=0x00007f8db97d7800 nid=0x6c03 runnable [0x000000014254d000] at java.net.PlainSocketImpl.socketAccept(Native Method) java.lang.Thread.State: RUNNABLE at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)"[ACTIVE] ExecuteThread: 23 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbc945800 nid=0x8203 in Object.wait() [0x000000014322a000] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.ServerSocket.implAccept(ServerSocket.java:522) java.lang.Thread.State: WAITING (on object monitor) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.accept(ServerSocket.java:490) at java.lang.Object.wait(Native Method) at java.net.ServerSocket.implAccept(ServerSocket.java:522) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at java.lang.Object.wait(Object.java:503) at java.net.ServerSocket.accept(ServerSocket.java:490) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) - locked <0x000000011ab1d8c8> (a weblogic.work.ExecuteThread) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) at java.lang.Thread.run(Thread.java:722) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173)"[STANDBY] ExecuteThread: 22 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db959d800 nid=0x8103 in Object.wait() [0x0000000143127000] at java.lang.Thread.run(Thread.java:722) "DynamicListenThread[Default[7]]" daemon prio=5 tid=0x00007f8dbe02c800 nid=0x6b03 runnable [0x000000014244a000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: RUNNABLE at java.lang.Object.wait(Native Method) "DynamicListenThread[Default[7]]" daemon prio=5 tid=0x00007f8dbe02c800 nid=0x6b03 runnable [0x000000014244a000] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: RUNNABLE at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.ServerSocket.implAccept(ServerSocket.java:522) - locked <0x000000011ab12600> (a weblogic.work.ExecuteThread) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.accept(ServerSocket.java:490) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at java.net.ServerSocket.implAccept(ServerSocket.java:522) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at java.net.ServerSocket.accept(ServerSocket.java:490) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535)"[ACTIVE] ExecuteThread: 21 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcaa7000 nid=0x8003 in Object.wait() [0x0000000143024000] at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) java.lang.Thread.State: WAITING (on object monitor) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173) at java.lang.Object.wait(Native Method) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) at java.lang.Thread.run(Thread.java:722) at java.lang.Object.wait(Object.java:503) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Thread.run(Thread.java:722) "DynamicListenThread[Default[ - locked <0x000000011ab13620> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "DynamicListenThread[Default[4]]" daemon prio=5 tid=0x00007f8dbc8c7800 nid=0x6903 runnable [0x0000000142244000] java.lang.Thread.State: RUNNABLE"[STANDBY] ExecuteThread: 20 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbb832800 nid=0x7f03 in Object.wait() [0x0000000142f21000] at java.net.PlainSocketImpl.socketAccept(Native Method) java.lang.Thread.State: WAITING (on object monitor) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.lang.Object.wait(Native Method) at java.net.ServerSocket.implAccept(ServerSocket.java:522) at java.lang.Object.wait(Object.java:503) at java.net.ServerSocket.accept(ServerSocket.java:490) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) - locked <0x000000011ab164e0> (a weblogic.work.ExecuteThread) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173)"[ACTIVE] ExecuteThread: 19 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc5d800 nid=0x7e03 in Object.wait() [0x0000000142e1e000] at java.lang.Thread.run(Thread.java:722) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "DynamicListenThread[Default[8]]" daemon prio=5 tid=0x00007f8dba13f800 nid=0x6803 runnable [0x0000000142141000] at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: RUNNABLE at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.net.PlainSocketImpl.socketAccept(Native Method) - locked <0x000000011ab16fa8> (a weblogic.work.ExecuteThread) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at java.net.ServerSocket.implAccept(ServerSocket.java:522) at java.net.ServerSocket.accept(ServerSocket.java:490)"[ACTIVE] ExecuteThread: 18 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba9a3000 nid=0x7d03 in Object.wait() [0x0000000142d1b000] at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) java.lang.Thread.State: WAITING (on object monitor) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at java.lang.Object.wait(Native Method) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) at java.lang.Object.wait(Object.java:503) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Thread.run(Thread.java:722) - locked <0x000000011ab18448> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "DynamicListenThread[Default[2]]" daemon prio=5 tid=0x00007f8dbbd06000 nid=0x6703 runnable [0x000000014203e000] java.lang.Thread.State: RUNNABLE"[STANDBY] ExecuteThread: 17 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbae29800 nid=0x7c03 in Object.wait() [0x0000000142c18000] at java.net.PlainSocketImpl.socketAccept(Native Method) java.lang.Thread.State: WAITING (on object monitor) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.lang.Object.wait(Native Method) at java.net.ServerSocket.implAccept(ServerSocket.java:522) at java.lang.Object.wait(Object.java:503) at java.net.ServerSocket.accept(ServerSocket.java:490) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) - locked <0x000000011ab15510> (a weblogic.work.ExecuteThread) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173)"[ACTIVE] ExecuteThread: 16 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd0b7000 nid=0x7b03 in Object.wait() [0x0000000142b15000] at java.lang.Thread.run(Thread.java:722) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "weblogic.GCMonitor" daemon prio=5 tid=0x00007f8dbe0c0800 nid=0x4507 in Object.wait() [0x0000000140eed000] at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: TIMED_WAITING (on object monitor) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) - locked <0x000000011ab17fc8> (a weblogic.work.ExecuteThread) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011a6d83f0> (a java.lang.ref.ReferenceQueue$Lock) at weblogic.platform.GCMonitorThread.waitForNotification(GCMonitorThread.java:88)"[ACTIVE] ExecuteThread: 15 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcaa6800 nid=0x7a03 in Object.wait() [0x0000000142a12000] at weblogic.platform.GCMonitorThread.run(GCMonitorThread.java:64) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "Thread-11" prio=5 tid=0x00007f8dbc884000 nid=0x4007 waiting on condition [0x0000000140dea000] at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (parking) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at sun.misc.Unsafe.park(Native Method) - locked <0x000000011ab1aeb8> (a weblogic.work.ExecuteThread) - parking to wait for <0x000000011a50cfa8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)"[ACTIVE] ExecuteThread: 14 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc5c800 nid=0x7903 in Object.wait() [0x0000000142856000] at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.lang.Thread.State: WAITING (on object monitor) at weblogic.utils.concurrent.JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89) at java.lang.Object.wait(Native Method) at weblogic.store.internal.PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:676) at java.lang.Object.wait(Object.java:503) at weblogic.store.internal.PersistentStoreImpl.run(PersistentStoreImpl.java:751) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Thread.run(Thread.java:722) - locked <0x000000011ab1bed8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 3 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba33e800 nid=0x430f in Object.wait() [0x000000013f4ca000] java.lang.Thread.State: WAITING (on object monitor)"[ACTIVE] ExecuteThread: 13 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbb83c000 nid=0x7803 in Object.wait() [0x0000000142650000] at java.lang.Object.wait(Native Method) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Object.java:503) - locked <0x000000011a4b00e8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab144c8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 2 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbad9f800 nid=0x4207 in Object.wait() [0x00000001404e7000] java.lang.Thread.State: WAITING (on object monitor)"[ACTIVE] ExecuteThread: 12 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd9c0000 nid=0x7703 in Object.wait() [0x0000000140a29000] at java.lang.Object.wait(Native Method) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Object.java:503) - locked <0x000000011a4b0d30> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab19548> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "Thread-10" prio=5 tid=0x00007f8dbad89000 nid=0x6603 waiting on condition [0x00000001402db000] java.lang.Thread.State: WAITING (parking)"[ACTIVE] ExecuteThread: 11 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba94d800 nid=0x7603 in Object.wait() [0x0000000142347000] at sun.misc.Unsafe.park(Native Method) java.lang.Thread.State: WAITING (on object monitor) - parking to wait for <0x000000011a208ea0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.lang.Object.wait(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.lang.Object.wait(Object.java:503) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) - locked <0x000000011ab0d3f8> (a weblogic.work.ExecuteThread) at weblogic.utils.concurrent.JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.store.internal.PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:676) at weblogic.store.internal.PersistentStoreImpl.run(PersistentStoreImpl.java:751)"[STANDBY] ExecuteThread: 10 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbe196000 nid=0x7503 in Object.wait() [0x0000000140c23000] at java.lang.Thread.run(Thread.java:722) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "DoSManager" daemon prio=5 tid=0x00007f8dbd84c800 nid=0x6503 waiting on condition [0x00000001400ce000] at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: TIMED_WAITING (sleeping) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Thread.sleep(Native Method) - locked <0x000000011ab0e418> (a weblogic.work.ExecuteThread) at com.octetstring.vde.DoSManager.run(DoSManager.java:433) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "VDE Transaction Processor Thread" daemon prio=5 tid=0x00007f8dbc82b000 nid=0x6303 in Object.wait() [0x000000013ffcb000]"[ACTIVE] ExecuteThread: 9 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbe195800 nid=0x7403 in Object.wait() [0x00000001408be000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at java.lang.Object.wait(Object.java:503) at com.octetstring.vde.backend.standard.TransactionProcessor.waitChange(TransactionProcessor.java:367) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x0000000119edae88> (a com.octetstring.vde.backend.standard.TransactionProcessor) - locked <0x000000011ab10b38> (a weblogic.work.ExecuteThread) at com.octetstring.vde.backend.standard.TransactionProcessor.run(TransactionProcessor.java:212) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "ExecuteThread: 3 for queue: weblogic.socket.Muxer" daemon prio=5 tid=0x00007f8db9712800 nid=0x6203 waiting for monitor entry [0x000000013fec8000]"[ACTIVE] ExecuteThread: 8 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbba6d800 nid=0x7303 in Object.wait() [0x000000013f0a9000] java.lang.Thread.State: BLOCKED (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93) at java.lang.Object.wait(Native Method) - waiting to lock <0x0000000119f22540> (a weblogic.socket.PosixSocketMuxer$1) at java.lang.Object.wait(Object.java:503) at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42) - locked <0x000000011ab0c2d0> (a weblogic.work.ExecuteThread) at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119)"[ACTIVE] ExecuteThread: 7 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbc812800 nid=0x7203 in Object.wait() [0x00000001407bb000] "ExecuteThread: 2 for queue: weblogic.socket.Muxer" daemon prio=5 tid=0x00007f8dbac3a000 nid=0x6103 runnable [0x000000013fdc5000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: RUNNABLE at java.lang.Object.wait(Native Method) at weblogic.socket.PosixSocketMuxer.poll(Native Method) at java.lang.Object.wait(Object.java:503) at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:102) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x0000000119f22540> (a weblogic.socket.PosixSocketMuxer$1) - locked <0x000000011ab01498> (a weblogic.work.ExecuteThread) at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42) at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147)"[ACTIVE] ExecuteThread: 6 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db96b5800 nid=0x7103 in Object.wait() [0x00000001406b8000] at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "ExecuteThread: 1 for queue: weblogic.socket.Muxer" daemon prio=5 tid=0x00007f8db973b000 nid=0x6003 waiting for monitor entry [0x000000013fcc2000] at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: BLOCKED (on object monitor) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93) - locked <0x000000011ab024b8> (a weblogic.work.ExecuteThread) - waiting to lock <0x0000000119f22540> (a weblogic.socket.PosixSocketMuxer$1) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29) at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)"[STANDBY] ExecuteThread: 5 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db971b000 nid=0x7003 in Object.wait() [0x00000001403de000] at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147) java.lang.Thread.State: WAITING (on object monitor) at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) "ExecuteThread: 0 for queue: weblogic.socket.Muxer" daemon prio=5 tid=0x00007f8dbaa1b800 nid=0x5f03 waiting for monitor entry [0x000000013fbbf000] at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) java.lang.Thread.State: BLOCKED (on object monitor) - locked <0x000000011ab06208> (a weblogic.work.ExecuteThread) at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - waiting to lock <0x0000000119f22540> (a weblogic.socket.PosixSocketMuxer$1) at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)"[ACTIVE] ExecuteThread: 4 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbd5d000 nid=0x6f03 in Object.wait() [0x00000001401d1000] at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42) java.lang.Thread.State: WAITING (on object monitor) at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147) at java.lang.Object.wait(Native Method) at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) "[ACTIVE] ExecuteThread: 1 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba4cc800 nid=0x5e03 in Object.wait() [0x000000013fabc000] - locked <0x000000011ab07228> (a weblogic.work.ExecuteThread) java.lang.Thread.State: WAITING (on object monitor) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503)"DynamicListenThread[Default]" daemon prio=5 tid=0x00007f8dbd82b000 nid=0x6e03 runnable [0x0000000142753000] at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) java.lang.Thread.State: RUNNABLE - locked <0x0000000119fa2d20> (a weblogic.work.ExecuteThread) at java.net.PlainSocketImpl.socketAccept(Native Method) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:522) "weblogic.timers.TimerThread" daemon prio=5 tid=0x00007f8db9738000 nid=0x5d03 in Object.wait() [0x000000013f9b9000] at java.net.ServerSocket.accept(ServerSocket.java:490) java.lang.Thread.State: TIMED_WAITING (on object monitor) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at java.lang.Object.wait(Native Method) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.timers.internal.TimerThread$Thread.run(TimerThread.java:262) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) - locked <0x0000000119e7f158> (a weblogic.timers.internal.TimerThread) "weblogic.time.TimeEventGenerator" daemon prio=5 tid=0x00007f8db9724000 nid=0x5c03 in Object.wait() [0x000000013f8b6000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:286) - locked <0x0000000119dfc2e0> (a weblogic.time.common.internal.TimeTable) at weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:117) at java.lang.Thread.run(Thread.java:722) "[ACTIVE] ExecuteThread: 0 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbb232000 nid=0x5b03 in Object.wait() [0x000000013f7b3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x0000000119fa37e0> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) Yusuke Yamamoto @yusuke #wlstudy
  • 14. 2013-01-23 16:32:13Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.6-b04 mixed mode): at java.lang.Object.wait(Native Method)"Attach Listener" daemon prio=5 tid=0x00007f8dbd9dd000 nid=0x9407 waiting on condition [0x0000000000000000] at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab16fa8> (a weblogic.work.ExecuteThread) java.lang.Thread.State: RUNNABLE at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab16fa8> (a weblogic.work.ExecuteThread)"[STANDBY] ExecuteThread: 40 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcd3c000 nid=0x9303 in Object.wait() [0x0000000144526000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 18 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba9a3000 nid=0x7d03 in Object.wait() [0x0000000142d1b000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 18 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba9a3000 nid=0x7d03 in Object.wait() [0x0000000142d1b000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011c40fc88> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab18448> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab18448> (a weblogic.work.ExecuteThread)"[STANDBY] ExecuteThread: 39 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbdab9800 nid=0x9203 in Object.wait() [0x000000014437b000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[STANDBY] ExecuteThread: 17 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbae29800 nid=0x7c03 in Object.wait() [0x0000000142c18000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[STANDBY] ExecuteThread: 17 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbae29800 nid=0x7c03 in Object.wait() [0x0000000142c18000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab2f568> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab15510> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab15510> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 38 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd1bf000 nid=0x9103 in Object.wait() [0x0000000144278000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 16 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd0b7000 nid=0x7b03 in Object.wait() [0x0000000142b15000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 16 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd0b7000 nid=0x7b03 in Object.wait() [0x0000000142b15000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab2e4a0> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab17fc8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab17fc8> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 37 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba80b000 nid=0x9003 in Object.wait() [0x0000000144175000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 15 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcaa6800 nid=0x7a03 in Object.wait() [0x0000000142a12000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 15 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcaa6800 nid=0x7a03 in Object.wait() [0x0000000142a12000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab2d1b0> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab1aeb8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab1aeb8> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 36 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba479800 nid=0x8f03 in Object.wait() [0x0000000144072000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 14 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc5c800 nid=0x7903 in Object.wait() [0x0000000142856000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 14 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc5c800 nid=0x7903 in Object.wait() [0x0000000142856000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab27da0> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab1bed8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab1bed8> (a weblogic.work.ExecuteThread)"[STANDBY] ExecuteThread: 35 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db959f000 nid=0x8e03 in Object.wait() [0x0000000143f6f000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 13 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbb83c000 nid=0x7803 in Object.wait() [0x0000000142650000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 13 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbb83c000 nid=0x7803 in Object.wait() [0x0000000142650000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab26c90> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab144c8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab144c8> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 34 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba9a4000 nid=0x8d03 in Object.wait() [0x0000000143e6c000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 12 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd9c0000 nid=0x7703 in Object.wait() [0x0000000140a29000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 12 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd9c0000 nid=0x7703 in Object.wait() [0x0000000140a29000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab29978> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab19548> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab19548> (a weblogic.work.ExecuteThread)"[STANDBY] ExecuteThread: 33 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc5b800 nid=0x8c03 in Object.wait() [0x0000000143d69000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 11 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba94d800 nid=0x7603 in Object.wait() [0x0000000142347000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 11 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba94d800 nid=0x7603 in Object.wait() [0x0000000142347000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab29a28> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab0d3f8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab0d3f8> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 32 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc5b000 nid=0x8b03 in Object.wait() [0x0000000143c66000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[STANDBY] ExecuteThread: 10 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbe196000 nid=0x7503 in Object.wait() [0x0000000140c23000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[STANDBY] ExecuteThread: 10 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbe196000 nid=0x7503 in Object.wait() [0x0000000140c23000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab25c70> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab0e418> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab0e418> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 31 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbd54800 nid=0x8a03 in Object.wait() [0x00000001435d3000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 9 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbe195800 nid=0x7403 in Object.wait() [0x00000001408be000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 9 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbe195800 nid=0x7403 in Object.wait() [0x00000001408be000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab28da8> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab10b38> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab10b38> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 30 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbd8f800 nid=0x8903 in Object.wait() [0x00000001434d0000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 8 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbba6d800 nid=0x7303 in Object.wait() [0x000000013f0a9000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 8 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbba6d800 nid=0x7303 in Object.wait() [0x000000013f0a9000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab2a9a0> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab0c2d0> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab0c2d0> (a weblogic.work.ExecuteThread)"[STANDBY] ExecuteThread: 29 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbd8f000 nid=0x8803 in Object.wait() [0x0000000143b63000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 7 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbc812800 nid=0x7203 in Object.wait() [0x00000001407bb000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 7 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbc812800 nid=0x7203 in Object.wait() [0x00000001407bb000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab229f0> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab01498> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab01498> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 28 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcb4e000 nid=0x8703 in Object.wait() [0x0000000143a60000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 6 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db96b5800 nid=0x7103 in Object.wait() [0x00000001406b8000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 6 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db96b5800 nid=0x7103 in Object.wait() [0x00000001406b8000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab23a10> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab024b8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab024b8> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 27 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc91800 nid=0x8603 in Object.wait() [0x000000014395d000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[STANDBY] ExecuteThread: 5 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db971b000 nid=0x7003 in Object.wait() [0x00000001403de000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[STANDBY] ExecuteThread: 5 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db971b000 nid=0x7003 in Object.wait() [0x00000001403de000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab24a30> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab06208> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab06208> (a weblogic.work.ExecuteThread)"[ACTIVE] ExecuteThread: 26 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbb015800 nid=0x8503 in Object.wait() [0x000000014385a000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 4 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbd5d000 nid=0x6f03 in Object.wait() [0x00000001401d1000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "[ACTIVE] ExecuteThread: 4 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbd5d000 nid=0x6f03 in Object.wait() [0x00000001401d1000] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011ab1f7b0> (a weblogic.work.ExecuteThread) at java.lang.Object.wait(Object.java:503) - locked <0x000000011ab07228> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab07228> (a weblogic.work.ExecuteThread)"[STANDBY] ExecuteThread: 25 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba709800 nid=0x8403 in Object.wait() [0x0000000143757000] at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "DynamicListenThread[Default]" daemon prio=5 tid=0x00007f8dbd82b000 nid=0x6e03 runnable [0x0000000142753000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: RUNNABLE at java.lang.Object.wait(Native Method) "DynamicListenThread[Default]" daemon prio=5 tid=0x00007f8dbd82b000 nid=0x6e03 runnable [0x0000000142753000] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: RUNNABLE at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.ServerSocket.implAccept(ServerSocket.java:522) - locked <0x000000011ab207d0> (a weblogic.work.ExecuteThread) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.accept(ServerSocket.java:490) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at java.net.ServerSocket.implAccept(ServerSocket.java:522) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at java.net.ServerSocket.accept(ServerSocket.java:490) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535)"[ACTIVE] ExecuteThread: 24 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcb54800 nid=0x8303 in Object.wait() [0x000000013f1cd000] at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) java.lang.Thread.State: WAITING (on object monitor) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173) at java.lang.Object.wait(Native Method) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) at java.lang.Thread.run(Thread.java:722) at java.lang.Object.wait(Object.java:503) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Thread.run(Thread.java:722) "DynamicListenThread[Default[1]]" daemon prio=5 tid=0x00007f8db97d7800 nid=0x6c03 runnable [0x000000014254d000] - locked <0x000000011ab21860> (a weblogic.work.ExecuteThread) java.lang.Thread.State: RUNNABLE at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "DynamicListenThread[Default[1]]" daemon prio=5 tid=0x00007f8db97d7800 nid=0x6c03 runnable [0x000000014254d000] at java.net.PlainSocketImpl.socketAccept(Native Method) java.lang.Thread.State: RUNNABLE at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)"[ACTIVE] ExecuteThread: 23 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbc945800 nid=0x8203 in Object.wait() [0x000000014322a000] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.ServerSocket.implAccept(ServerSocket.java:522) java.lang.Thread.State: WAITING (on object monitor) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.accept(ServerSocket.java:490) at java.lang.Object.wait(Native Method) at java.net.ServerSocket.implAccept(ServerSocket.java:522) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at java.lang.Object.wait(Object.java:503) at java.net.ServerSocket.accept(ServerSocket.java:490) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) 2013-01-23 16:32:13 - locked <0x000000011ab1d8c8> (a weblogic.work.ExecuteThread) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) at java.lang.Thread.run(Thread.java:722) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173)"[STANDBY] ExecuteThread: 22 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db959d800 nid=0x8103 in Object.wait() [0x0000000143127000] at java.lang.Thread.run(Thread.java:722) "DynamicListenThread[Default[7]]" daemon prio=5 tid=0x00007f8dbe02c800 nid=0x6b03 runnable [0x000000014244a000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: RUNNABLE at java.lang.Object.wait(Native Method) "DynamicListenThread[Default[7]]" daemon prio=5 tid=0x00007f8dbe02c800 nid=0x6b03 runnable [0x000000014244a000] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: RUNNABLE at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.net.PlainSocketImpl.socketAccept(Native Method) Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.6-b04 mixed mode): at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:522) - locked <0x000000011ab12600> (a weblogic.work.ExecuteThread) at java.net.ServerSocket.accept(ServerSocket.java:490) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at java.net.ServerSocket.implAccept(ServerSocket.java:522) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at java.net.ServerSocket.accept(ServerSocket.java:490) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535)"[ACTIVE] ExecuteThread: 21 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcaa7000 nid=0x8003 in Object.wait() [0x0000000143024000] at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) java.lang.Thread.State: WAITING (on object monitor) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173) at java.lang.Object.wait(Native Method) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) at java.lang.Thread.run(Thread.java:722) at java.lang.Object.wait(Object.java:503) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Thread.run(Thread.java:722) "DynamicListenThread[Default[ - locked <0x000000011ab13620> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "DynamicListenThread[Default[4]]" daemon prio=5 tid=0x00007f8dbc8c7800 nid=0x6903 runnable [0x0000000142244000] java.lang.Thread.State: RUNNABLE"[STANDBY] ExecuteThread: 20 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbb832800 nid=0x7f03 in Object.wait() [0x0000000142f21000] at java.net.PlainSocketImpl.socketAccept(Native Method) java.lang.Thread.State: WAITING (on object monitor) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.lang.Object.wait(Native Method) at java.net.ServerSocket.implAccept(ServerSocket.java:522) at java.lang.Object.wait(Object.java:503) at java.net.ServerSocket.accept(ServerSocket.java:490) "Attach Listener" daemon prio=5 tid=0x00007f8dbd9dd000 nid=0x9407 waiting on condition at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) - locked <0x000000011ab164e0> (a weblogic.work.ExecuteThread) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173)"[ACTIVE] ExecuteThread: 19 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc5d800 nid=0x7e03 in Object.wait() [0x0000000142e1e000] at java.lang.Thread.run(Thread.java:722) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "DynamicListenThread[Default[8]]" daemon prio=5 tid=0x00007f8dba13f800 nid=0x6803 runnable [0x0000000142141000] [0x0000000000000000] at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: RUNNABLE at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.net.PlainSocketImpl.socketAccept(Native Method) - locked <0x000000011ab16fa8> (a weblogic.work.ExecuteThread) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at java.net.ServerSocket.implAccept(ServerSocket.java:522) at java.net.ServerSocket.accept(ServerSocket.java:490)"[ACTIVE] ExecuteThread: 18 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba9a3000 nid=0x7d03 in Object.wait() [0x0000000142d1b000] at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) java.lang.Thread.State: WAITING (on object monitor) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at java.lang.Object.wait(Native Method) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) java.lang.Thread.State: RUNNABLE at java.lang.Object.wait(Object.java:503) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Thread.run(Thread.java:722) - locked <0x000000011ab18448> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "DynamicListenThread[Default[2]]" daemon prio=5 tid=0x00007f8dbbd06000 nid=0x6703 runnable [0x000000014203e000] java.lang.Thread.State: RUNNABLE"[STANDBY] ExecuteThread: 17 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbae29800 nid=0x7c03 in Object.wait() [0x0000000142c18000] at java.net.PlainSocketImpl.socketAccept(Native Method) java.lang.Thread.State: WAITING (on object monitor) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.lang.Object.wait(Native Method) at java.net.ServerSocket.implAccept(ServerSocket.java:522) at java.lang.Object.wait(Object.java:503) at java.net.ServerSocket.accept(ServerSocket.java:490) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) - locked <0x000000011ab15510> (a weblogic.work.ExecuteThread) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) at weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173)"[ACTIVE] ExecuteThread: 16 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd0b7000 nid=0x7b03 in Object.wait() [0x0000000142b15000] at java.lang.Thread.run(Thread.java:722) java.lang.Thread.State: WAITING (on object monitor) "[STANDBY] ExecuteThread: 40 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 at java.lang.Object.wait(Native Method) "weblogic.GCMonitor" daemon prio=5 tid=0x00007f8dbe0c0800 nid=0x4507 in Object.wait() [0x0000000140eed000] at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: TIMED_WAITING (on object monitor) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Native Method) - locked <0x000000011ab17fc8> (a weblogic.work.ExecuteThread) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011a6d83f0> (a java.lang.ref.ReferenceQueue$Lock) at weblogic.platform.GCMonitorThread.waitForNotification(GCMonitorThread.java:88)"[ACTIVE] ExecuteThread: 15 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbcaa6800 nid=0x7a03 in Object.wait() [0x0000000142a12000] at weblogic.platform.GCMonitorThread.run(GCMonitorThread.java:64) tid=0x00007f8dbcd3c000 nid=0x9303 in Object.wait() [0x0000000144526000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "Thread-11" prio=5 tid=0x00007f8dbc884000 nid=0x4007 waiting on condition [0x0000000140dea000] at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: WAITING (parking) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at sun.misc.Unsafe.park(Native Method) - locked <0x000000011ab1aeb8> (a weblogic.work.ExecuteThread) - parking to wait for <0x000000011a50cfa8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)"[ACTIVE] ExecuteThread: 14 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbc5c800 nid=0x7903 in Object.wait() [0x0000000142856000] at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at weblogic.utils.concurrent.JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89) at java.lang.Object.wait(Native Method) at weblogic.store.internal.PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:676) at java.lang.Object.wait(Object.java:503) at weblogic.store.internal.PersistentStoreImpl.run(PersistentStoreImpl.java:751) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Thread.run(Thread.java:722) - locked <0x000000011ab1bed8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 3 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba33e800 nid=0x430f in Object.wait() [0x000000013f4ca000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method)"[ACTIVE] ExecuteThread: 13 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbb83c000 nid=0x7803 in Object.wait() [0x0000000142650000] at java.lang.Object.wait(Native Method) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Object.java:503) - locked <0x000000011a4b00e8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab144c8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 2 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbad9f800 nid=0x4207 in Object.wait() [0x00000001404e7000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503)"[ACTIVE] ExecuteThread: 12 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbd9c0000 nid=0x7703 in Object.wait() [0x0000000140a29000] at java.lang.Object.wait(Native Method) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Object.java:503) at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Object.wait(Object.java:503) - locked <0x000000011a4b0d30> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - locked <0x000000011ab19548> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "Thread-10" prio=5 tid=0x00007f8dbad89000 nid=0x6603 waiting on condition [0x00000001402db000] at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) java.lang.Thread.State: WAITING (parking)"[ACTIVE] ExecuteThread: 11 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba94d800 nid=0x7603 in Object.wait() [0x0000000142347000] at sun.misc.Unsafe.park(Native Method) java.lang.Thread.State: WAITING (on object monitor) - parking to wait for <0x000000011a208ea0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.lang.Object.wait(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.lang.Object.wait(Object.java:503) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) - locked <0x000000011ab0d3f8> (a weblogic.work.ExecuteThread) at weblogic.utils.concurrent.JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.store.internal.PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:676) - locked <0x000000011c40fc88> (a weblogic.work.ExecuteThread) at weblogic.store.internal.PersistentStoreImpl.run(PersistentStoreImpl.java:751)"[STANDBY] ExecuteThread: 10 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbe196000 nid=0x7503 in Object.wait() [0x0000000140c23000] at java.lang.Thread.run(Thread.java:722) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "DoSManager" daemon prio=5 tid=0x00007f8dbd84c800 nid=0x6503 waiting on condition [0x00000001400ce000] at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: TIMED_WAITING (sleeping) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.lang.Thread.sleep(Native Method) - locked <0x000000011ab0e418> (a weblogic.work.ExecuteThread) at com.octetstring.vde.DoSManager.run(DoSManager.java:433) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "VDE Transaction Processor Thread" daemon prio=5 tid=0x00007f8dbc82b000 nid=0x6303 in Object.wait() [0x000000013ffcb000]"[ACTIVE] ExecuteThread: 9 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbe195800 nid=0x7403 in Object.wait() [0x00000001408be000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at java.lang.Object.wait(Object.java:503) at com.octetstring.vde.backend.standard.TransactionProcessor.waitChange(TransactionProcessor.java:367) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x0000000119edae88> (a com.octetstring.vde.backend.standard.TransactionProcessor) - locked <0x000000011ab10b38> (a weblogic.work.ExecuteThread) at com.octetstring.vde.backend.standard.TransactionProcessor.run(TransactionProcessor.java:212) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "ExecuteThread: 3 for queue: weblogic.socket.Muxer" daemon prio=5 tid=0x00007f8db9712800 nid=0x6203 waiting for monitor entry [0x000000013fec8000]"[ACTIVE] ExecuteThread: 8 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbba6d800 nid=0x7303 in Object.wait() [0x000000013f0a9000] java.lang.Thread.State: BLOCKED (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93) at java.lang.Object.wait(Native Method) - waiting to lock <0x0000000119f22540> (a weblogic.socket.PosixSocketMuxer$1) at java.lang.Object.wait(Object.java:503) at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42) "[STANDBY] ExecuteThread: 39 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 - locked <0x000000011ab0c2d0> (a weblogic.work.ExecuteThread) at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119)"[ACTIVE] ExecuteThread: 7 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbc812800 nid=0x7203 in Object.wait() [0x00000001407bb000] "ExecuteThread: 2 for queue: weblogic.socket.Muxer" daemon prio=5 tid=0x00007f8dbac3a000 nid=0x6103 runnable [0x000000013fdc5000] java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: RUNNABLE at java.lang.Object.wait(Native Method) at weblogic.socket.PosixSocketMuxer.poll(Native Method) at java.lang.Object.wait(Object.java:503) at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:102) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x0000000119f22540> (a weblogic.socket.PosixSocketMuxer$1) tid=0x00007f8dbdab9800 nid=0x9203 in Object.wait() [0x000000014437b000] - locked <0x000000011ab01498> (a weblogic.work.ExecuteThread) at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42) at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147)"[ACTIVE] ExecuteThread: 6 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db96b5800 nid=0x7103 in Object.wait() [0x00000001406b8000] at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119) java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) "ExecuteThread: 1 for queue: weblogic.socket.Muxer" daemon prio=5 tid=0x00007f8db973b000 nid=0x6003 waiting for monitor entry [0x000000013fcc2000] at java.lang.Object.wait(Object.java:503) java.lang.Thread.State: BLOCKED (on object monitor) java.lang.Thread.State: WAITING (on object monitor) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93) - locked <0x000000011ab024b8> (a weblogic.work.ExecuteThread) - waiting to lock <0x0000000119f22540> (a weblogic.socket.PosixSocketMuxer$1) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29) at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)"[STANDBY] ExecuteThread: 5 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8db971b000 nid=0x7003 in Object.wait() [0x00000001403de000] at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147) java.lang.Thread.State: WAITING (on object monitor) at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) "ExecuteThread: 0 for queue: weblogic.socket.Muxer" daemon prio=5 tid=0x00007f8dbaa1b800 nid=0x5f03 waiting for monitor entry [0x000000013fbbf000] at java.lang.Object.wait(Native Method) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) java.lang.Thread.State: BLOCKED (on object monitor) - locked <0x000000011ab06208> (a weblogic.work.ExecuteThread) at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) - waiting to lock <0x0000000119f22540> (a weblogic.socket.PosixSocketMuxer$1) at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)"[ACTIVE] ExecuteThread: 4 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbbd5d000 nid=0x6f03 in Object.wait() [0x00000001401d1000] at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42) java.lang.Thread.State: WAITING (on object monitor) at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147) at java.lang.Object.wait(Native Method) at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119) at java.lang.Object.wait(Object.java:503) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) "[ACTIVE] ExecuteThread: 1 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dba4cc800 nid=0x5e03 in Object.wait() [0x000000013fabc000] - locked <0x000000011ab07228> (a weblogic.work.ExecuteThread) java.lang.Thread.State: WAITING (on object monitor) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503)"DynamicListenThread[Default]" daemon prio=5 tid=0x00007f8dbd82b000 nid=0x6e03 runnable [0x0000000142753000] at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) java.lang.Thread.State: RUNNABLE - locked <0x0000000119fa2d20> (a weblogic.work.ExecuteThread) at java.net.PlainSocketImpl.socketAccept(Native Method) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:522) "weblogic.timers.TimerThread" daemon prio=5 tid=0x00007f8db9738000 nid=0x5d03 in Object.wait() [0x000000013f9b9000] at java.net.ServerSocket.accept(ServerSocket.java:490) java.lang.Thread.State: TIMED_WAITING (on object monitor) at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38) at java.lang.Object.wait(Native Method) at weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535) at weblogic.timers.internal.TimerThread$Thread.run(TimerThread.java:262) at weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417) - locked <0x0000000119e7f158> (a weblogic.timers.internal.TimerThread) - locked <0x000000011ab2f568> (a weblogic.work.ExecuteThread) "weblogic.time.TimeEventGenerator" daemon prio=5 tid=0x00007f8db9724000 nid=0x5c03 in Object.wait() [0x000000013f8b6000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:286) - locked <0x0000000119dfc2e0> (a weblogic.time.common.internal.TimeTable) at weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:117) at java.lang.Thread.run(Thread.java:722) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) "[ACTIVE] ExecuteThread: 0 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5 tid=0x00007f8dbb232000 nid=0x5b03 in Object.wait() [0x000000013f7b3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x0000000119fa37e0> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) Yusuke Yamamoto @yusuke #wlstudy
  • 15. スレッドダンプ取得方法• jps / jstack• kill -3 $PID (Unix / Linux)• ctrl-Break (Windows)• VisualVMYusuke Yamamoto @yusuke #wlstudy
  • 16. スレッドダンプ取得方法• jps / jstack• kill -3 $PID (Unix / Linux)• ctrl-Break (Windows)• VisualVMYusuke Yamamoto @yusuke #wlstudy
  • 17. スレッドダンプ取得方法• jps JVMプロセスID,mainクラス名一覧Yusuke Yamamoto @yusuke #wlstudy
  • 18. スレッドダンプ取得方法• jps JVMプロセスID,mainクラス名一覧 “Server”がWebLogicのプロセス mainクラスはweblogic.ServerなのでYusuke Yamamoto @yusuke #wlstudy
  • 19. スレッドダンプ取得方法• jstack [pid]Yusuke Yamamoto @yusuke #wlstudy
  • 20. スレッドダンプ取得のコツ• 時間を空けて複数回取得• スレッドの動きを見るため• 取得した複数のスレッドダンプを比較Yusuke Yamamoto @yusuke #wlstudy
  • 21. スレッドダンプ取得方法• jstack [pid] >> dump.txt 同じファイルに追記するのが現実的Yusuke Yamamoto @yusuke #wlstudy
  • 22. WebLogic独自のスレッドダンプ取得方法 • WLSTconnect(weblogic,weblogic ,t3://localhost:7001)cd (Servers)cd (AdminServer)threadDump() Yusuke Yamamoto @yusuke #wlstudy
  • 23. WebLogic独自のスレッドダンプ取得方法• 管理コンソールYusuke Yamamoto @yusuke #wlstudy
  • 24. スレッドダンプ解析は・・・• 結構難しい• トラブルが起きる前に練習をYusuke Yamamoto @yusuke #wlstudy
  • 25. スレッドヘッダとスタックトレース • スタックトレースの見た目"ExecuteThread: 3 for queue: weblogic.socket.Muxer" daemon prio=5tid=0x00007f8db9712800 nid=0x6203 waiting for monitor entry [0x000000013fec8000]      java.lang.Thread.State: BLOCKED (on object monitor)      at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93)      - waiting to lock <0x000000011c797450> (a weblogic.socket.PosixSocketMuxer$1)      at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)      at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)      at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147)      at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119) Yusuke Yamamoto @yusuke #wlstudy
  • 26. スレッドヘッダ • スレッドの先頭行 スレッドヘッダ"ExecuteThread: 3 for queue: weblogic.socket.Muxer" daemon prio=5tid=0x00007f8db9712800 nid=0x6203 waiting for monitor entry [0x000000013fec8000]      java.lang.Thread.State: BLOCKED (on object monitor)      at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93)      - waiting to lock <0x000000011c797450> (a weblogic.socket.PosixSocketMuxer$1)      at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)      at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)      at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147)      at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119) Yusuke Yamamoto @yusuke #wlstudy
  • 27. スレッド名 • スレッドに付けられた区別用の名前 • weblogic.socket.Muxer[n] : ソケットマクサー • weblogic.kernel.Default[n] : 実行スレッド ‣ 通常weblogic.kernel.Default[n]を解析"ExecuteThread: 3 for queue: weblogic.socket.Muxer" daemon prio=5tid=0x00007f8db9712800 nid=0x6203 waiting for monitor entry [0x000000013fec8000]      java.lang.Thread.State: BLOCKED (on object monitor) スレッド名      at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93)      - waiting to lock <0x000000011c797450> (a weblogic.socket.PosixSocketMuxer$1)      at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)      at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)      at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147)      at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119) Yusuke Yamamoto @yusuke #wlstudy
  • 28. スレッドの状態 • スレッドがしていることの概要 • runnable • waiting for monitor entry • in Object.wait()"ExecuteThread: 3 for queue: weblogic.socket.Muxer" daemon prio=5tid=0x00007f8db9712800 nid=0x6203 waiting for monitor entry [0x000000013fec8000]      java.lang.Thread.State: BLOCKED (on object monitor)      at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93) スレッドの状態      - waiting to lock <0x000000011c797450> (a weblogic.socket.PosixSocketMuxer$1)      at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)      at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)      at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147)      at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119) Yusuke Yamamoto @yusuke #wlstudy
  • 29. スレッドの状態 • runnable • Javaコードを実行中 • ネイティブコードを実行中 • 基本的に正常 • データース待ちも runnable"DynamicListenThread[Default]" daemon prio=5 tid=0x00007f8dbd82b000 nid=0x6e03 runnable java.lang.Thread.State: RUNNABLE at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:522) at java.net.ServerSocket.accept(ServerSocket.java:490) Yusuke Yamamoto @yusuke #wlstudy
  • 30. スレッドの状態 • 待ち状態 • in Object.wait() • 他のスレッドから叩かれるのを待っ ている"[ACTIVE] ExecuteThread: 3 for queue: weblogic.kernel.Default (self-tuning)" daemon prio=5tid=0x00007f8dba33e800 nid=0x430f in Object.wait() [0x000000013f4ca000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205) - locked <0x000000011a4b00e8> (a weblogic.work.ExecuteThread) at weblogic.work.ExecuteThread.run(ExecuteThread.java:226) Yusuke Yamamoto @yusuke #wlstudy
  • 31. スレッドの状態 • 待ち状態 • waiting for monitor entry"ExecuteThread: 1 for queue: weblogic.socket.Muxer" daemon prio=5 tid=0x00007f8db973b000nid=0x6003 waiting for monitor entry [0x000000013fcc2000] java.lang.Thread.State: BLOCKED (on object monitor) at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93) - waiting to lock <0x0000000119f22540> (a weblogic.socket.PosixSocketMuxer$1) at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119)"ExecuteThread: 1 for queue: weblogic.socket.Muxer" daemon prio=5 tid=0x00007f8db973b000nid=0x6003 runnable [0x000000013fcc2000] java.lang.Thread.State: RUNNABLE at weblogic.socket.PosixSocketMuxer.poll(Native Method) at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:102) - locked <0x0000000119f22540> (a weblogic.socket.PosixSocketMuxer$1) at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29) at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119) Yusuke Yamamoto @yusuke #wlstudy
  • 32. スレッドダンプ解析の肝• スレッドの状態を解析• waiting for monitor entryのスレッドが多 い場合は注目• ロックしているスレッドで握っている リソースは? • 外部リソース • 大きなファイルの読み込み Yusuke Yamamoto @yusuke #wlstudy
  • 33. スレッドダンプ解析の肝• waiting for monitor entryがほとんどない• runnableなスレッドは何をしている?• 何度とっても同じ、似たスタックト レースのスレッドは注意 Yusuke Yamamoto @yusuke #wlstudy
  • 34. スレッドダンプ解析は面倒 • ツールで解析 •侍http://samuraism.jp/samurai/ja/index.html • Java Thread Dump Analyserhttp://mchr3k.github.com/javathreaddumpanalyser/ Yusuke Yamamoto @yusuke #wlstudy
  • 35. 侍 http://samuraism.jp/samurai/Yusuke Yamamoto @yusuke #wlstudy
  • 36. 侍の起動• Java Web Startで起動 クリックYusuke Yamamoto @yusuke #wlstudy
  • 37. 侍の起動• アプリケーションをダウンロード(Mac) http://samuraism.jp/samurai/ja/samurai.zip ダブルクリックYusuke Yamamoto @yusuke #wlstudy
  • 38. スレッドダンプを読み込ませるファイルをドラッグ&ドロップするだけYusuke Yamamoto @yusuke #wlstudy
  • 39. スレッドダンプをビジュアルに表現Yusuke Yamamoto @yusuke #wlstudy
  • 40. スレッドダンプをビジュアルに表現 列: スレッドダンプ1回分Yusuke Yamamoto @yusuke #wlstudy
  • 41. スレッドダンプをビジュアルに表現 行: 同じスレッドの状態遷移Yusuke Yamamoto @yusuke #wlstudy
  • 42. スレッドの状態で色分け表示 色: スレッドの状態Yusuke Yamamoto @yusuke #wlstudy
  • 43. 1つのスレッドを時系列で確認Yusuke Yamamoto @yusuke #wlstudy
  • 44. 状態に変化のないスレッド検出Yusuke Yamamoto @yusuke #wlstudy
  • 45. 状態に変化のないスレッド検出• テーブルビューの場合Yusuke Yamamoto @yusuke #wlstudy
  • 46. 状態に変化のないスレッド検出• 時系列ビューの場合Yusuke Yamamoto @yusuke #wlstudy
  • 47. スレッドダンプパターンYusuke Yamamoto @yusuke #wlstudy
  • 48. アイドル状態• ほとんどのスレッドが灰色Yusuke Yamamoto @yusuke #wlstudy
  • 49. がんばっている状態• ほとんどのスレッドが緑色Yusuke Yamamoto @yusuke #wlstudy
  • 50. 詰まっている状態• ほとんどのスレッドが赤、一部オレンジYusuke Yamamoto @yusuke #wlstudy
  • 51. デッドロックYusuke Yamamoto @yusuke #wlstudy
  • 52. WebLogicのデッドロック自動検出• BEA-000394Yusuke Yamamoto @yusuke #wlstudy
  • 53. WebLogicのデッドロック自動検出Yusuke Yamamoto @yusuke #wlstudy