Android crash debugging


Published on

Android logging and crash debugging

Published in: Technology

Android crash debugging

  1. 1. Android logging and debugging ● By: Ashish Agrawal 1
  2. 2. Androids boot up process Stage Steps CommentsBoot-loader - Location: bootablebootloaderlegacyusbloader init.S Initializes stacks, zeros the BSS segment, call _main() in main.c main.c Initializes hardware (clocks, board, keypad, console), creates Linux tags Displays "USB FastBoot". Boot from flash, or loops while usb_poll() awaits host PC connectionLinux kernel - Sets up the system, loads drivers, and starts running the first process initThe init process Setup file system Create and mount directories like /dev, /proc, /sys Execute init.rc This is the boot-up script, commands are using Android-specific syntax Setup console Display "A N D R This is just a text msg written to /dev/tty0 O I D" Zygote Zygot process in init.rc brings up Dalvik Java VM and starts the system server bootanimation Shows the animation during boot-up 2Framework …. ….
  3. 3. Overview of Android Logging system 3
  4. 4. * Main - the main application logThis log contains output from android.util.Log class on Java side andLOGE(LOGV, LOGI, LOGW.) macro on the native side.* Events - for system event informationEvents log reflects system diagnostic events which outputs usingandroid.util.EventLog class e.g:System diagnostic events are used to record certain system-level events(such as garbage collection, activity manager state, system watchdogs, andother low level activity)* Radio - for radio and phone-related informationThe logging system automatically routes messages with specific tags (“RIL”,“GSM” etc.) into the radio buffer.* System - a log for low-level system messagesand debugging.Many classes in the Android framework utilize the system log to keep theirmessages separate from (possibly noisy) application log messages. 4Slog.i(“Tag I want to see in the system log”, “Hello system log");
  5. 5. dumpsys/dumpstateDumps huge amounts of information about the system, includingstatus, counts and statistics• Dumpstate reproduces lots of stuff from /proc– Does a dumpsys as well• Dumpsys show status information from Android services– e.g. dumpsys alarm 5
  6. 6. Dumpsys Eg:adb shell dumpsys batteryYou will get output:Current Battery Service state:AC powered: falseAC capacity: 500000USB powered: truestatus: 5health: 2present: truelevel: 100scale: 100voltage:4201temperature: 271 <---------- Battery temperature! %)technology: Li-poly <---------- Battery technology! %) 6
  7. 7. How to get kernel messages from Android?● To invoke the "dmesg" from the control PC, one can simply run● # adb shell dmesg● However, because "syslogd" is possibly not included in Android, there is no such logs, and one may find that the directory "/var" is not even created.One can just run the following command to continuously dump the kernel messages.● adb shell cat /proc/kmsg 7
  8. 8. Redirecting kernel messages● If the phone doesnt even boot up to a stable state where the ADB commands can be used, one might be interested in redirecting the kernel messages to some places where they can be seen. This is to leverage the "console=" command for the kernel.● For different devices, there may be different ports where the messages can be redirected to. USB SERIAL PORT, SERIAL PORT, UART port● Eg: console=ttyMSM2,115200n8, console=ttyUSB0,9600n8 etc● In order to be used as a console, a device driver has to register itself as a console provider by calling register_console in kernel/printk.c, and it has to provide some callbacks for printk to write kernel messages.
  9. 9. Java Application CrashWhen a Java application crashes, the Dalvik VMwill receive a SIGQUIT, it dumps stack traces for all threads and parse to plain text typo. Developer could use this dump together with AndroidRuntime error level log to locate error. 9
  10. 10. Example----- pid 182 at 2009-03-06 06:15:22 -----Cmd line: THREADS:"main" prio=5 tid=3 NATIVE | group="main" sCount=1 dsCount=0 s=0 obj=0x40018dd8 | sysTid=182 nice=0 sched=0/0 handle=-1096356708 at android.os.BinderProxy.transact(Native Method) at$UncaughtHandler.uncaughtException( at java.lang.ThreadGroup.uncaughtException( at java.lang.ThreadGroup.uncaughtException( at dalvik.system.NativeStart.main(Native Method) 10
  11. 11. ANR In Android, the system guards againstapplications that are insufficiently responsive for a period of time by displaying a dialog to the user, called the “Application Not Responding” (ANR) dialog Note that system cannot show this dialogsometimes due to internal problems e.g. if ANR occurred in the Activity or Window Manager. 11
  12. 12. What triggers ANRIn Android, application responsiveness ismonitored by the Activity Manager and WindowManager system services. Android will displaythe ANR dialog for a particular application whenit detects one of the following conditions: * No response to an input event (e.g. keypress, screen touch) within 5 seconds * A BroadcastReceiver hasnt finishedexecuting within 10 seconds 12
  13. 13. Click to11-28 12:30:56.258 489 505 W ActivityManager: Timeout executingservice: ServiceRecord{}11-28 12:30:59.937 489 505 E ActivityManager: ANR 12:30:59.937 489 505 E ActivityManager: Reason: Executing 12:30:59.937 489 505 E ActivityManager: Load: 96.68 / 22.15 / 7.4911-28 12:30:59.937 489 505 E ActivityManager: CPU usage from 6970msto 960ms ago:11-28 12:30:59.937 489 505 E ActivityManager: 90% 9297/coredump:82% user + 7.6% kernelR11-28 12:31:00.875 489 505 W ActivityManager: KillingProcessRecord{}:background ANR11-28 12:31:01.203 489 794 I ActivityManager: (pid 5361) has died. 13
  14. 14. ANR hints * Look the “SIG: 9” line in the main thread and analyze nearbymessages. Sometimes system output information why it decidedthat the thread is in ANR state. * In ANR log start your analysis from the “main” thread of theprocess since this is a thread where UI works. The ANR conceptwas specially invented with struggling “not responding” UI threads. * In ANR log check in which state is your main thread. If it is inMONITOR state it can be in “dead lock” state. TIMED_WAIT statecan also point to the problems with locking of your thread by ‘sleep’or ‘wait’ functions. * If the system itself (Activity or Window Managers) is in ANRcondition and cannot raise ANR so you cannot differentiate whichthread in which process is responsible for this analyze “Just Now”14log.
  15. 15. Thread Status* ZOMBIE – terminated thread* RUNNABLE – runnable or running now* TIMED_WAIT – timed waiting in Object.wait()* MONITOR – blocked on a monitor* WAIT – waiting in Object.wait()* INITIALIZING - allocated, not yet running* STARTING - started, not yet on thread list* NATIVE - off in a JNI native method* VMWAIT - waiting on a VM resource* SUSPENDED - suspended, usually by GC or debugger* UNKNOWN – thread is in the undefined state 15
  16. 16. How to read Android native crash log and stack trace● An Android crash in C/C++ code often generates some crash log which looks like the following.● The first is the build information in the system property "" I/DEBUG ( 730): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** I/DEBUG ( 730): Build fingerprint: generic/generic/generic/:1.5/... 16●
  17. 17. How to read Android crash log and stack trace● Then, it shows the process ID number (pid) and the thread id (tid). In this example, the PID and TID are the same. However, if the crash happens in a child thread, the thread ID tid will be different from pid. I/DEBUG ( 730): pid: 876, tid: 876 >>> /system/bin/mediaserver <<< 17●
  18. 18. How to read Android crash log and stack trace● The following shows the signal which caused the process to abort, in this case, its a segmentation fault. This is followed by the register values. I/DEBUG ( 730): signal 11 (SIGSEGV), fault addr 00000010 I/DEBUG ( 730): r0 00000000 r1 00016618 r2 80248f78 r3 00000000 I/DEBUG ( 730): r4 80248f78 r5 0000d330 r6 80248f78 r7 beaf9974 I/DEBUG ( 730): r8 00000000 r9 00000000 10 00000000 fp 00000000 I/DEBUG ( 730): ip afd020c8 sp beaf98d8 lr 8021edcd pc 8021c630 cpsr a0000030● 18●
  19. 19. How to read Android crash log and stack trace● This is the call stack trace. #00 is the depth of the stack pointer. The "pc <addr>" is the PC address in the stack. Sometimes, the "lr" link register containing the return address is shown instead of PC. It is followed by the file containing the code. I/DEBUG ( 730): #00 pc 0001c630 /system/lib/ I/DEBUG ( 730): #01 pc 0001edca /system/lib/ I/DEBUG ( 730): #02 pc 0001ff0a /system/lib/ I/DEBUG ( 730): #03 pc 000214e0 /system/lib/ I/DEBUG ( 730): #04 pc 0000e322 /system/lib/ ... I/DEBUG ( 730): #15 pc b0001516 /system/bin/linker● 19●
  20. 20. How to read Android crash log and stack trace● The following is actually the current stack with the stack pointer address and code dump. Each line contains 4 bytes (one machine word), and the address is in ascending order. The words in the stack are mapped onto the memory region it belongs to. I/DEBUG ( 730): stack: I/DEBUG ( 730): beaf9898 00016618 [heap] I/DEBUG ( 730): beaf989c beaf98d0 [stack] I/DEBUG ( 730): beaf98a0 0000db28 [heap] I/DEBUG ( 730): beaf98a4 beaf98f8 [stack] I/DEBUG ( 730): beaf98b8 8021cf4d /system/lib/ I/DEBUG ( 730): beaf98bc 80248f78 I/DEBUG ( 730): #00 beaf98d8 0000d330 [heap] I/DEBUG ( 730): beaf98dc 00000000 I/DEBUG ( 730): beaf98e0 0000d330 [heap] I/DEBUG ( 730): beaf98e4 8021edcd /system/lib/ I/DEBUG ( 730): #01 beaf98e8 80248f78● 20●
  21. 21. Unix SignalsSIGHUP 1 Exit HangupSIGINT 2 Exit InterruptSIGQUIT 3 Core QuitSIGILL 4 Core Illegal InstructionSIGTRAP 5 Core Trace/Breakpoint TrapSIGABRT 6 Core AbortSIGEMT 7 Core Emulation TrapSIGFPE 8 Core Arithmetic ExceptionSIGKILL 9 Exit KilledSIGBUS 10 Core Bus ErrorSIGSEGV 11 Core Segmentation FaultSIGSYS 12 Core Bad System CallSIGPIPE 13 Exit Broken Pipe 21
  22. 22. Android WatchdogAndroid frameworks watchdog is meant to deal withcases when any of the following locks is held formore than a minute or when ServerThread is busy.ActivityManagerService.thisPowerManagerService.mLocksWindowManagerService.mWindowMapWindowManagerService.mKeyguardTokenWatcherWindowManagerService.mKeyWaiter 22
  23. 23. Watchdog thread posts a message MONITOR looper thread would read allpending messages including watchdogs MONITOR message andwould invoke an appropriate handler.The handler of MONITOR message would simply check foravailability of above mentioned locks.If all, locks are available variable mCompleted ( be set to true and watchdog would continue to postMONITOR messages once every minute.mCompleted stays false only when any of the above locks is heldby any thread of system_server for more than a minute or if the23MONITOR message isnt handled by ServerThread.
  24. 24. In this case, MONITOR is handled but cant be serviced dueto unavailability of lock (ActivityManagerService)"android.server.ServerThread" prio=5 tid=8 MONITORgroup="main" sCount=1 dsCount=0 s=N obj=0x4690be50self=0x54e440sysTid=206 nice=-2 sched=0/0 cgrp=unknown waiting to lock (0x4691a9b8) ( held by threadid=41(Binder Thread #7)$HeartbeatHandler.handleMessage( android.os.Handler.dispatchMessage( android.os.Looper.loop( 24
  25. 25. When you are no longer wanted..Android open source project supports amaximum of 15 hidden applications running atany point and any attempt to launch new appskills the least recently used ones. This is done toreduce the load on RAM and has been the casesince early version of Android. 02-19 13:43:55.194 I/ActivityManager( 1096): No longer want com.lge.omadmclient:remote (pid 23052): hidden #16 25
  26. 26. References: 26