Android Crash analysis and The Dalvik Garbage collector – Tools and Tips

9,693 views

Published on

You released an Android Application to the Play store. You even have some Advertising to make some money. The next step is to improve the code by looking at the crashes the Users are experiencing, and fixing them. The talk will cover current services to collect crash data, explain how to analyze the information. We will present an example crash analysis, and go in depth into the Android Garbage Collector, and the way it works for different versions of Android. We will also provide with a GC benchmarking tool, and some tips and tricks.

Published in: Technology
1 Comment
14 Likes
Statistics
Notes
No Downloads
Views
Total views
9,693
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
137
Comments
1
Likes
14
Embeds 0
No embeds

No notes for slide

Android Crash analysis and The Dalvik Garbage collector – Tools and Tips

  1. 1. Android Crash Analysis and The Dalvik Garbage Collector - Tips Oren BarAd Mobile Security Team Leader AVG
  2. 2. 2 100,000,000
  3. 3. 3 Intro • App on the market • Crashes will happen • Need to Fix the Bugs! • Identify the root issues
  4. 4. 4 Who? • Developing for Android since 2008 • The first employee of DroidSecurity – First company to release An antivirus to the Android market (2009) – Later acquired by AVG (2011) • Mobile Security Research Team Leader @AVG • B.Sc Information System Engineering from BGU
  5. 5. 5 Agenda • Android app crash Collection & Analysis • The Android Garbage Collector • Memory analysis Tools • Tips and tricks
  6. 6. 6 How to collect crash information?
  7. 7. 7 What to fix first? • Easy to fix • Affects most users • The ones users complain most about • Affects the most popular device
  8. 8. 8 Root cause analysis • Stare at data: –Crash type –Exception Type –Device –Manufacturer –OS version –Locale • And stare some more…
  9. 9. 9 Ultimate debugging method! • What is the answer to this bug? – 42 • We don’t know the answer – Brainstorming – Developing tools • Learning something new…
  10. 10. 10 Root cause analysis – Real world example • Concurrent.Timeout exception – ~10% of total crashes for AVG antivirus. – Unhelpful stack traces – Second place in top 10 crashes (and number 5, and number 9) java.util.concurrent.TimeoutException: android.os.BinderProxy.finalize() timed out after 10 seconds at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:401) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:102) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:73) at java.lang.Daemons$FinalizerDaemon.run(Daemons.java:170) at java.lang.Thread.run(Thread.java:841) java.util.concurrent.TimeoutException: com.android.internal.os.BinderInternal$GcWatcher.finalize() timed out after 10 seconds java.util.concurrent.TimeoutException: android.os.Parcel.finalize() timed out after 10 seconds java.util.concurrent.TimeoutException: java.io.FileInputStream.finalize() timed out after 10 seconds
  11. 11. 11 Device and OS analysis 33% 66% Device Galaxy S3 49% 38% 13% 19% 10% 71% OS Crash Dashboard 4.2.2 4.3 Other OS Versions
  12. 12. 12 What next? • After going deep into the Code… • Get S3 and use
  13. 13. 13 Android GC – little bit of history Gingerbread
  14. 14. 14 Complex Object Tricolor collector at work roots rootsroots object object object object object object object object object object
  15. 15. 15 object Tricolor collector at work roots rootsroots object object object object object object object object object object
  16. 16. 16 object Tricolor collector at work roots rootsroots object object object object object object object object object object
  17. 17. 17 object Tricolor collector at work roots rootsroots object object object object object object object object object object
  18. 18. 18 Complex Object Tricolor collector at work roots rootsroots object object object object object object object object object object
  19. 19. 19 Complex Object Tricolor collector at work roots rootsroots object object object object object object object object object object
  20. 20. 20 Garbage collector observed behavior Cause Effect GC_FOR_MALLOC Stop The world GC Use complex Objects GC takes longer A lot of small objects GC take Longer Use more memory GC takes longer USE_LARGE_HEAP All of the above
  21. 21. 21 Trying to replicate the problem • The GC_Test project – Test the GC performance under load – Android app to test the behavior of the GC on android – https://github.com/oba2cat3/GCTest • Useful logcat python script – https://github.com/oba2cat3/logcat2memorygraph • Available from GitHub
  22. 22. 22 Logcat output 02-04 12:40:33.185: D/dalvikvm(2496): GC_EXPLICIT freed <1K, 19% free 14383K/17720K, paused 1ms+1ms, total 17ms 02-04 12:40:33.265: D/dalvikvm(3279): GC_CONCURRENT freed 661K, 27% free 14934K/20336K, paused 5ms+3ms, total 21ms 02-04 12:40:33.460: D/dalvikvm(5637): GC_CONCURRENT freed 834K, 22% free 15516K/19644K, paused 6ms+3ms, total 33ms 02-04 12:40:33.480: D/dalvikvm(3279): GC_CONCURRENT freed 313K, 27% free 15022K/20336K, paused 3ms+9ms, total 30ms 02-04 12:40:33.665: D/dalvikvm(5637): GC_CONCURRENT freed 369K, 21% free 15658K/19644K, paused 2ms+2ms, total 18ms 02-04 12:40:33.725: D/dalvikvm(5730): GC_CONCURRENT freed 1166K, 23% free 15188K/19644K, paused 4ms+2ms, total 25ms 02-04 12:40:33.765: D/dalvikvm(5637): GC_CONCURRENT freed 430K, 20% free 15740K/19644K, paused 7ms+2ms, total 22ms 02-04 12:40:33.865: D/dalvikvm(5730): GC_CONCURRENT freed 325K, 23% free 15256K/19644K, paused 2ms+3ms, total 22ms 02-04 12:40:34.065: D/dalvikvm(5637): GC_CONCURRENT freed 1811K, 25% free 15934K/21040K, paused 3ms+2ms, total 29ms 02-04 12:40:34.160: D/dalvikvm(5637): GC_CONCURRENT freed 99K, 23% free 16354K/21040K, paused 2ms+4ms, total 31ms 02-04 12:40:34.250: D/dalvikvm(3403): GC_EXPLICIT freed 172K, 24% free 14956K/19628K, paused 3ms+4ms, total 28ms 02-04 12:40:34.315: D/dalvikvm(5766): GC_CONCURRENT freed 1185K, 23% free 15148K/19628K, paused 3ms+3ms, total 28ms 02-04 12:40:34.350: D/dalvikvm(5637): GC_CONCURRENT freed 1140K, 26% free 15766K/21040K, paused 3ms+3ms, total 30ms 02-04 12:40:34.495: D/dalvikvm(4155): GC_CONCURRENT freed 1274K, 24% free 15093K/19660K, paused 7ms+5ms, total 56ms 02-04 12:40:34.635: D/dalvikvm(3333): GC_CONCURRENT freed 574K, 24% free 14982K/19620K, paused 3ms+2ms, total 19ms 02-04 12:40:34.780: D/dalvikvm(5694): GC_CONCURRENT freed 1268K, 24% free 15100K/19656K, paused 3ms+1ms, total 25ms 02-04 12:40:34.840: D/dalvikvm(2960): GC_EXPLICIT freed 1236K, 10% free 53524K/58860K, paused 4ms+7ms, total 130ms 02-04 12:40:34.915: D/dalvikvm(5637): GC_FOR_ALLOC freed 143K, 25% free 15978K/21040K, paused 26ms, total 26ms 02-04 12:40:33.480: D/dalvikvm(3279): GC_CONCURRENT freed 313K, 27% free 15022K/20336K, paused 3ms+9ms, total 30ms 02-04 12:40:33.665: D/dalvikvm(5637): GC_CONCURRENT freed 369K, 21% free 15658K/19644K, paused 2ms+2ms, total 18ms 02-04 12:40:33.725: D/dalvikvm(5730): GC_CONCURRENT freed 1166K, 23% free 15188K/19644K, paused 4ms+2ms, total 25ms 02-04 12:40:33.765: D/dalvikvm(5637): GC_CONCURRENT freed 430K, 20% free 15740K/19644K, paused 7ms+2ms, total 22ms 02-04 12:40:33.865: D/dalvikvm(5730): GC_CONCURRENT freed 325K, 23% free 15256K/19644K, paused 2ms+3ms, total 22ms 02-04 12:40:34.065: D/dalvikvm(5637): GC_CONCURRENT freed 1811K, 25% free 15934K/21040K, paused 3ms+2ms, total 29ms
  23. 23. 23 Script result - Heap size and GC delays 0 0.2 0.4 0.6 0.8 1 1.2 0 20000 40000 60000 80000 100000 120000 140000 160000 Time in MSheap size in KB dalvikvm event timestamps current heap max heap
  24. 24. 24 Script result - Heap size and GC delays 0 50 100 150 200 250 300 350 400 0 20000 40000 60000 80000 100000 120000 140000 160000 Time in MSheap size in KB dalvikvm event timestamps current heap max heap ui pause time total pause
  25. 25. 25 Script result - Heap size and GC delays 0 50 100 150 200 250 300 350 400 0 20000 40000 60000 80000 100000 120000 140000 160000 Time in MSheap size in KB dalvikvm event timestamps current heap max heap ui pause time total pause webview start
  26. 26. 26 Memory Analyzer Tool tricks
  27. 27. 27 Memory Analyzer Tool tricks
  28. 28. 28 Memory Analyzer Tool tricks
  29. 29. 29 Memory Analyzer Tool tricks
  30. 30. 30 Memory Analyzer Tool tricks
  31. 31. 31 Memory Analyzer Tool tricks
  32. 32. 32 Memory Analyzer Tool tricks
  33. 33. 33 Memory Analyzer Tool tricks
  34. 34. 34 Tips for managing GC Do How Avoid GC_FOR_ALLOC Load stuff in Splash Use Object Pools For large/heavy objects Avoid Complex Objects Use simpler Objects Perform Heap analysis Use MAT and DDMS Reduce memory use …
  35. 35. 35 Summary • Application will crash –Collect and Analyze –Understand and Fix • Fixing the bug is half the fun
  36. 36. 36 Back to the problem • JNI objects are a special case • Dalvik will finalize and Destroy • Timeouts will happen... And they Do!
  37. 37. 37 Q&A • Questions? • Contact: Oren.barad@avg.com • We are hiring! • Come talk to us in the AVG booth
  38. 38. 38 Links and resources • http://android-developers.blogspot.co.il/2011/03/memory- analysis-for-android.html • http://www.brpreiss.com/books/opus5/html/page424.html • http://java-is-the-new-c.blogspot.co.il/2013/07/tuning-and- benchmarking-java-7s-garbage.html • http://www.slideshare.net/muhammedshakirmisarwala/java- performance-monitoring-tuning • http://www.slideshare.net/VikasBalikai/gc-in-android • http://www.javacodegeeks.com/2012/01/practical-garbage- collection-part-1.html
  39. 39. 39 Links and resources • https://android.googlesource.com/platform/dalvik/+/android- 4.3_r2/vm/alloc/MarkSweep.cpp • https://android.googlesource.com/platform/dalvik/+log/android- 4.4.2_r2/vm/alloc/MarkSweep.cpp • https://android.googlesource.com/platform/dalvik/+/gingerbread/vm/alloc/MarkSwe ep.c • https://android.googlesource.com/platform/dalvik/+/froyo- release/vm/alloc/MarkSweep.c

×