Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

ProbeDroid - Crafting Your Own Dynamic Instrument Tool on Android for App Behavior Exploration


Published on

The design memo and hack note of ProbeDroid
A dynamic binary instrumentation kit targeting Android(Lollipop) 5.0 and above

This is the first complete draft.
Improved version will be updated in a few days.

Published in: Technology

ProbeDroid - Crafting Your Own Dynamic Instrument Tool on Android for App Behavior Exploration

  1. 1. ProbeDroid: CraftingYour Own Dynamic Instrument Tool on Android for App Behavior Exploration ZongXian Shen
  2. 2. About Me • Passionate Security Researcher and Developer • Earned Master in CS from NCTU,Taiwan • Involved in DSNS Lab ZSShen @AndyZSShen
  3. 3. Disclaimer • A research project run in my spare time • Still a lot of features to enhance • Slide is the design memo and hack notes
  4. 4. Outline • Intro to dynamic binary instrumentation • Intro to ProbeDroid instrument kit • Design and implementation of ProbeDroid • App analysis with custom instrument tools
  5. 5. Intro to Dynamic Binary Instrumentation
  6. 6. Dynamic Binary Instrumentation • Technique that inserts code into a process to get runtime information or change process behavior without modifying original program binary Definition Source: Intel PIN manual
  7. 7. DBI Application • Performance tuning and bug hunting • Cache performance, memory access footprint • Behavior tracing and data logging • API call sequence, code block control flow relation • Changing process behavior on the fly • Execution path exploration, product hacking
  8. 8. DBI for Android App • Current Android app is mainly coded with Java and run on custom Java runtime named ART • Due to semantic gap, prefer Java level DBI rather than instrumentation directly on native ISA • Class field and object content inspection • Method call sequence and parameter profiling • Changing Java artifacts on the fly
  9. 9. Inspiring Work • ADBI and DDI introduced by Collin Mulliner • • • DBI framework based on Dalvik runtime • Demonstrate how to hook interested Java method and manipulate class field And now ProbeDroid, targeting on ART runtime with enhanced user interface
  10. 10. Intro to ProbeDroid Instrument Kit
  11. 11. ProbeDroid Instrument Kit • Programmable instrumentation • Code your own instrument tools with Java practice • Flexible APIs for you to • Hook interested library or app defined methods • Customize instrument gadgets for different analysis purposes • Modify method in/output to hack app at runtime • Succinct deployment • Only ProbeDroid engine and instrument tools are required • No need to customize Android framework
  12. 12. ProbeDroid Usage • Source building • Compile launcher, engine, and exported jar • Import jar to Android Studio project for tool creation • Play and hack • Push launcher, engine and tool to experiment device • Run launcher to inject engine to target app ProbeDroid ProjectWiki
  13. 13. Sample Tool Signature of to be instrumented method Create custom instrument gadget Register the gadget to ProbeDroid engine
  14. 14. Sample Tool • Manipulate the trapped input parameters • Do hacks before entering the hooked method • Manipulate the trapped return value • Do hacks after leaving the hooked method
  15. 15. • Be injected into the target app process • Work as a mini runtime • Load and execute the instrument tool • Hook the specified methods and install gadgets • Marshal control flow for hooked methods and gadgets ProbeDroid Tasks Currently support X86 & ARM ISAs and Lollipop 5.0
  16. 16. ProbeDroid Design and Implementation
  17. 17. ProbeDroid Overview Inject Engine Library Launcher Run Library Inited Load Instrument Tool Get Runtime Utilities Hook Specified Methods Method Entered Call Pre-Method Gadget Call Original Method Call Post-Method Gadget 3. Play Music 2. Compose Gadget 1. Deploy Engine 1 & 2 Before App Execution 3 During App Execution
  18. 18. Stage: Deploy Engine libART Android & Java Base Classes libProbeDroid Launcher App Process libART Android & Java Base Classes Zygote Process Attach and Wait for child Control the process Fork process • Library injection • Catch the newly forked app process • Force the app process to load engine library Engine binary Load library
  19. 19. Library Injection • No convenient APIs like WriteProcessMemory() and CreateRemoteThread() dedicated forWindows • Manually crafting ptrace() operation sequence to simulate the procedure on Android
  20. 20. Injection Procedure 1. Attach to Zygote and wait for target app to be forked 2. Attach to app process and release Zygote 3. Resolve the address of mmap() and dlopen() in app process 4. Force app to execute mmap() for stashing library pathname 5. Force app to execute dlopen() for loading engine library Known technique with proper modification
  21. 21. Resolve Function Address /system/lib/ … … … … mmap … … /system/bin/linker … … … … dlopen … … • Resolve /proc/pid/maps for library base address in both processes • Use dlopen() and dlsym() for symbol address in launcher process • Use relative offset to resolve symbol address in app process base_linker Launcher Process addr_dlopen base_libc addr_mmap /system/lib/ … … … … mmap … … /system/bin/linker … … … … dlopen … … App Process (addr_dlopen - base_linker) + base_linker’ base_linker’’ base_libc’’ (addr_mmap - base_libc) + base_libc’’
  22. 22. Remote Function Call 0 0 4096 PROT_READ|WRITE MAP_ANONYMOUS 0 0 base_stack Invalid return address esp  base_stack eip  mmap() eax  addr_libpath 0 4096 PROT_READ|WRITE MAP_ANONYMOUS 0 0 base_stack ARM_sp  base_stack ARM_pc  mmap() ARM_r0  addr_libpath ARM_lr  0 Invalid return address 0 addr_libpath RTLD_NOW Invalid return address base_stack esp  base_stack eip  dlopen() eax  handle_lib ARM_lr  0 addr_libpath RTLD_NOW Invalid return address base_stack ARM_sp  base_stack ARM_pc  dlopen() ARM_r0  handle_lib • Invoke mmap() to allocate space for library path • Invoke dlopen() to load the specified library Force SIGSEGV for wait()
  23. 23. Stage: Compose Gadget libART Android & Java Base Classes App Classes Instrument Classes libProbeDroid Instrument Tool APK App Oat App Process • Bootstrapping • Acquire ART context to start instrument world • Hooks installation • Load instrument tool • Modify ART artifacts to hook interested methods
  24. 24. Bootstrapping • JNI exports helpful utilities for native code to access Java runtime features • ProbeDroid must acquire JNI interface pointer for all the instrument related tasks
  25. 25. JNI Interface Pointer jint JNI_GetCreatedJavaVMs (JavaVM** vms, jsize size, jsize* vm_count); Return Java vm instance pointer of current runtime • First retrieve JavaVM pointer and stash it for subsequent tasks • Then apply JavaVM pointer to acquire per-thread JNIEnv pointer jint AttachCurrentThread (JavaVM* vm, JNIEnv** p_env, void* thr_args); Return JNI interface pointer bound to current thread
  26. 26. Install Hooks • Load instrument gadgets defined in tool APK • Modify ART artifacts to divert the control flow to our space when the specified methods are called
  27. 27. ART Metadata ClassA ClassB ClassZ …… DexCache ArtMethodA ArtMethodB ArtMethodZ …… ArtFieldA ArtFieldB ArtFieldZ …… ClassLoader dex_class_def_idx dex_type_idx Class entry_point_from_ interpreter entry_point_from_ portable_compiled_code entry_point_from_ quick_compiled_code dex_code_item_offset dex_method_index ArtMethod push {r5,r6,r7,lr} sub sp, sp, #16 mov r7, r0 str r0, [sp, #0] …… blx lr code_item ins_size outs_size insns_size insns invoke-super {v1}, AB iget_object v0, v1 if-eqz v0, +3 const/4 v0, #+1 …… native code DEX bytecode
  28. 28. App Compiled Code …… 0x0a: invoke-virtual {v6}, java.lang.ClassLoader android.content.Context.getClassLoader() 0x0d: move-result-object v3 0x0e: invoke-virtual {v3, v7}, java.lang.Class java.lang.ClassLoader.loadClass(java.lang.String) 0x11: move-result-object v0 …… DEX bytecode dex PC: 0x000a …… mov r7, r0 mov r1, r7 mov r2, r11 ldr r0, [r1, #0] dex PC: 0x000e ldr.w r0, [r0, #484] ldr.w lr, [r0, #40] blx lr …… native code Get ClassLoader object Get loadClass() ArtMethod pointer Get compiled code function pointer Branch and link to the callee
  29. 29. Compiled Code Pointer Normal method call flow in most cases Get ArtMethod pointer Get quick compiled code pointer Indirect jump to method code Modify quick compiled code pointer to let it point to hook trampoline Get ArtMethod pointer Get quick compiled code pointer Indirect jump to trampoline Call pre-method gedget Call original method Call post-method gedget
  30. 30. ArtMethod Pointer jmethodID (*GetMethodID) (JNIEnv*, jclass, const char*, const char*); jmethodID (*GetStaticMethodID)(JNIEnv*, jclass, const char*, const char*); jclass (*FindClass)(JNIEnv*, const char*); Get loaded class id by specifying class name Get method id by specifying class id, method name and signature • jmethodID is actually the pointer to ArtMethod class • Refer to art/runtime/art_method.h for ArtMethod class definition and resolve member offset for entry_point_from_quick_compiled_code
  31. 31. About Install Timing • Must be done before the execution of app code • How early is that moment ?
  32. 32. App Init Flow ZygoteInit. runSelectLoop() Wait for forking task from ActivityManager ActivityManagerService. attachApplicationLocked() Handle app initialization and register the relevant metadata ZygoteConnection. handleChildProc() Prepare to specialize the process runtime Call Zygote. forkAndSpecialize() ActivityThread. main() Create an event handler to serve the requests from ActivityManager ActivityThread$H. handleMessage()call ActivityThread.handleBindApplication() Serve requests in event handler: • Load the app Application class • Call its Application.onCreate() Zygote Process App Process ActivityManager Process The base runtime context is ready and the lifecycle management for ContentProvider, Activity, and Service is now started
  33. 33. Install Timing • Before ActivityThread executes Application.onCreate() • The declaring classes of the to be instrumented methods may not be loaded at that early moment • Must intercept the ClassLoader used by ActivityThread to load non-system classes containing app defined methods
  34. 34. Intercept ClassLoader Deploy a special hook for ClassLoader.loadClass() ActivityThread calls ClassLoader.loadClass() Load app Application class ActivityThread calls ClassLoader.loadClass() Intercept app ClassLoader Load declaring classes of target methods Install hooks for target methods Load app Application class• Restore work of ActivityThread • Resolve ArtMethod pointer • Substitute native code pointer • Patch instrument gadget
  35. 35. Stage: Play Music libART Android & Java Base Classes App Classes Instrument Classes libProbeDroid App Process • Method in/output • Extract different types of parameters and return value • Gadget marshalling • Deliver modifiable parameters • Original method invocation • Handle generic method call for different signature types Call method Trap to ProbeDroid Call gadget Call original method
  36. 36. Extract In/Output • Control flow is trapped into trampoline • Extract input parameters for gadget and pass them to the original method • Return from original method • Extract return value for gadget and pass it to caller • How to extract correct data on different ISAs?
  37. 37. Calling Convention Method Register  EAX Entry Spill 1st Chunk  ECX 2nd Chunk  EDX 3rd Chunk  EBX Others stored on stack Return Register Float and Double  XMM0 Long  EAX_EDX Other types  EAX Method Register  R0 Entry Spill 1st Chunk  R1 2nd Chunk  R2 3rd Chunk  R3 Others stored on stack Return Register Double and Long  R0_R1 Float  R0 Other types  R0 X86 ISA ARM ISA • Defined in art/compiler/jni/quick/.* source files • Trampoline must follow the conventions defined on different ISAs
  38. 38. Entry Spill R1 ObjPtr R2 Char R3 Long (Hi) Stk Long (Lo) Int Virtual Method Signature  Func1(CJI)V 1st param 4th param 3rd param 2nd param R1 Byte R2 Double(Hi) R3 Double(Lo) Stk ObjPtr Float Static Method Signature  Func2(BD[SF)V 1st param 4th param 3rd param 2nd param • Parameter index is determined by data type and is not always equal to chunk index • Trampoline must be smart enough to handle this
  39. 39. Marshal Gadgets • Deliver modifiable method in/output for gadgets • Manage object pointer and reference in trampoline
  40. 40. Box In/Output • Trampoline applies JNIs to invoke gadgets • To make primitive in/output modifiable for gadgets, we can box them in wrapper classes byte  Byte short  Short int  Integer long  Long float  Float double  Double boolean  Boolean char  Character • Gadgets manipulate wrapper objects and trampoline must unbox objects for record update when gadgets return
  41. 41. Object Pointer and Reference • Non-primitive data is compiled and managed as object pointer in native code • Object pointer in JNI is boxed and managed as indirect reference that maps to the real pointer jstring str = env->NewStringUTF(“jni0”); env->CallVoidMethod(receiver, method_id, str); Shuffle Function str ObjPtr receiver ObjPtr Indirect Reference Table Ref 1 Idx 1 Ref 2 Idx 2
  42. 42. • Receiver object and non-primitive in/output trapped by trampoline are object pointers • Since JNI only accepts reference, it is necessary to • Wrap object pointers as references to call gadgets • Unwrap reference to object pointer when gadgets return • No exported interface, so manually resolve ART functions relevant to reference management Object Pointer and Reference
  43. 43. Indirect Reference Table struct JNIEnvExt : public JNIEnv { …… void* thread_; void* jvm_; uint32_t local_ref_cookie_; void* local_refs_table_; …… }; Actual definition of JNI interface struct JNIEnvExt : public JNIEnv { …… Thread* const self; JavaVMExt* vm; // Cookie used when using the local // indirect reference table. uint32_t local_ref_cookie; // JNI local references. IndirectReferenceTable locals; …… } Our mirrored structure • Extract the indirect reference table defined in JNIEnvExt structure • Craft a mirrored structure for space layout resolution • Cast JNIEnv pointer to this type for member field access
  44. 44. IndirectRef IndirectReferenceTable:: Add(uint32_t cookie, mirror::Object* obj) bool IndirectReferenceTable:: Remove(uint32_t cookie, IndirectRef iref) Indirect Reference Table mirror::Object* Thread::DecodeJObject(jobject obj) Insert an object pointer and the reference is returned Remove a table entry with the specified reference Get the object pointer with the specified reference • Use dlsym() to resolve ART functions relevant to table manipulation • To fulfill C++ calling convention, pass the table pointer extracted from the casted JNIEnvExt structure as this pointer
  45. 45. Call Original Method • Trampoline never knows the signature type of to be called method until runtime • Impossible to enumerate all the combinations of parameter list and return type with C/C++ • Solution is to make generic function call with assembly programming that fits native ISA
  46. 46. LibFFI • Portable Foreign Function Interface library • Generic C interface to various ISAs that allows users to call functions with signatures specified at runtime • Users apply it to bridge the interpreted and natively compiled code including CPython, OpenJDK, DalvikVM • ProbeDroid uses it to reduce native programming effort LibFFI Project Page
  47. 47. LibFFI Usage ffi_status ffi_prep_cif (ffi_cif *cif, ffi_abi abi, unsigned int nargs, ffi_type *rtype, ffi_type **argtypes) void ffi_call (ffi_cif *cif, void *fn, void *rvalue, void **avalues) FFI context ABI to use Number of parameters Return type specification Parameter types specification FFI context Function pointer To be filled with return value Parameter values to be passed in • Aim to specify function signature • Aim to make generic function call LibFFI API Reference
  48. 48. Generate Function Call at Runtime boolean String.startsWith(String, int) JniEnv* ObjPtr* jmethodID* ObjPtr* Int* ffi_type_ pointer ffi_type_ pointer ffi_type_ pointer ffi_type_ pointer ffi_type_ sint32 Mini Interpreter FFI parameter value array Craft FFI call payload bool* ffi_type_ uint8 FFI parameter type array return value return type
  49. 49. App Analysis with Custom Tools (To be fine tuned with more demo and samples)
  50. 50. Google Maps Demo Forensics for the strings converted from StringBuffer and StringBuilder buffer with tool Click the Picture for demo link StringInspector
  51. 51. Obfuscated App • Detect dynamic code loading and intercept the payload DexClassLoader <init>(String dexPath, String optimizedDirectory, String librarySearchPath, ClassLoader parent) More relevant methods … … Field Class.getDeclaredField(String name) Method Class.getDeclaredMethod(String name) Object Field.get(Object object) Object Method.invoke(Object receiver, Object[] args) More relevant methods … … • Resolve the method and field mutated via Java reflection
  52. 52. Instant Messaging App javax.crypto.Cipher.* relevant methods • Useful to analyze crypto algorithm • Resolve app credential (evil ?)* relevant methods • Useful to analyze authentication protocol • Resolve message content (evil ?)
  53. 53. And more app analysis result …