Advanced Dynamic Analysis for Leak Detection (Apple Internship 2008)

471 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Advanced Dynamic Analysis for Leak Detection (Apple Internship 2008)

  1. 1. Advanced dynamic analysisfor leak detectionJim ClauseChris Friesen - ManagerAnalysis Tools Group
  2. 2. Current analysis toolsShark Instruments
  3. 3. ≈X-rayCurrent analysis toolsShark Instruments
  4. 4. ≈X-rayMRICurrent analysis toolsShark Instruments
  5. 5. ≈X-rayMRICurrent analysis toolsShark Instruments≈?
  6. 6. ≈X-rayMRICurrent analysis toolsShark InstrumentsCAB312Z3Dynamic taint analysis≈
  7. 7. Dynamic taint analysisCAB Z
  8. 8. Dynamic taint analysis1 Assigntaint marksCAB Z
  9. 9. Dynamic taint analysis1 Assigntaint marksCAB312Z
  10. 10. Dynamic taint analysis1 Assigntaint marks2 Propagatetaint marksCAB312Z
  11. 11. Dynamic taint analysis1 Assigntaint marks2 Propagatetaint marksCAB312Z
  12. 12. Dynamic taint analysis1 Assigntaint marks3 Checktaint marks2 Propagatetaint marksCAB312Z
  13. 13. Dynamic taint analysis1 Assigntaint marks3 Checktaint marks2 Propagatetaint marksCAB312ZCAB312Z3
  14. 14. Attack detection / preventionInformation policy enforcementTestingData lifetimeApplications of dynamic taintingMemory errors
  15. 15. Attack detection / preventionPrevent stack smashing, SQL injection, buffer overruns, etc.Attack detection / preventionInformation policy enforcementTestingData lifetimeApplications of dynamic taintingMemory errors
  16. 16. Information policy enforcementensure classified information does not leave the systemAttack detection / preventionInformation policy enforcementTestingData lifetimeApplications of dynamic taintingMemory errors
  17. 17. TestingCoverage metrics, test data generation heuristic, etc.✔/✘Attack detection / preventionInformation policy enforcementTestingData lifetimeApplications of dynamic taintingMemory errors
  18. 18. Data lifetimetrack how long sensitive data remain in the applicationAttack detection / preventionInformation policy enforcementTestingData lifetimeApplications of dynamic taintingMemory errors
  19. 19. Attack detection / preventionInformation policy enforcementTestingData lifetimeApplications of dynamic taintingMemory errorsMemory errorsDetect illegal memory access, leak detection, etc.
  20. 20. Attack detection / preventionInformation policy enforcementTestingData lifetimeApplications of dynamic taintingMemory errorsMemory errorsDetect illegal memory access, leak detection, etc.leak detection
  21. 21. Detecting leaks is easy, fixing them is hard
  22. 22. Detecting leaks is easy, fixing them is hard@interface Container:NSObject { id _object;}@end@implementation Container- (void) dealloc { //[_object release]; [super dealloc];}- (void) setObject:(id)obj { [_object release]; _object = [obj retain];}@end
  23. 23. Detecting leaks is easy, fixing them is hard@interface Container:NSObject { id _object;}@end@implementation Container- (void) dealloc { //[_object release]; [super dealloc];}- (void) setObject:(id)obj { [_object release]; _object = [obj retain];}@endContainer *create() { Container *c = [[Container alloc] init]; NSObject *o = [[NSObject alloc] init]; [c setObject:o]; [o release]; return c;}int main(...) { Container *c = create(); … [c release];}
  24. 24. Detecting leaks is easy, fixing them is hard@interface Container:NSObject { id _object;}@end@implementation Container- (void) dealloc { //[_object release]; [super dealloc];}- (void) setObject:(id)obj { [_object release]; _object = [obj retain];}@endContainer *create() { Container *c = [[Container alloc] init]; NSObject *o = [[NSObject alloc] init]; [c setObject:o]; [o release]; return c;}int main(...) { Container *c = create(); … [c release];}leaks:This object is leaked
  25. 25. Leakpoint overviewDiscover where the last pointer to un-freed memory is lost
  26. 26. Leakpoint overviewAssigntaint marksPropagatetaint marksChecktaint marksptr1 = malloc(...) ➔ ptr1ptr2 = calloc(...) ➔ ptr2ptr3 = ptr1 ➔ ptr3 , ptr1ptr1 = NULL ➔ ptr1 , ptr3ptr4 = ptr2 + 1 ➔ ptr4 , ptr2Report error if taint mark’s count is zero andmemory has not been freed.1 11Discover where the last pointer to un-freed memory is lost
  27. 27. Leakpoint overviewAssigntaint marksPropagatetaint marksChecktaint marksptr1 = malloc(...) ➔ ptr1ptr2 = calloc(...) ➔ ptr2ptr3 = ptr1 ➔ ptr3 , ptr1ptr1 = NULL ➔ ptr1 , ptr3ptr4 = ptr2 + 1 ➔ ptr4 , ptr2Report error if taint mark’s count is zero andmemory has not been freed.21 111 22211 2 2Discover where the last pointer to un-freed memory is lost
  28. 28. Leakpoint overviewAssigntaint marksPropagatetaint marksChecktaint marksptr1 = malloc(...) ➔ ptr1ptr2 = calloc(...) ➔ ptr2ptr3 = ptr1 ➔ ptr3 , ptr1ptr1 = NULL ➔ ptr1 , ptr3ptr4 = ptr2 + 1 ➔ ptr4 , ptr2Report error if taint mark’s count is zero andmemory has not been freed.21 111 22211 2 2In general propagation follows standard pointer arithmetic rulesDiscover where the last pointer to un-freed memory is lost
  29. 29. Leakpoint overviewAssigntaint marksPropagatetaint marksChecktaint marksptr1 = malloc(...) ➔ ptr1ptr2 = calloc(...) ➔ ptr2ptr3 = ptr1 ➔ ptr3 , ptr1ptr1 = NULL ➔ ptr1 , ptr3ptr4 = ptr2 + 1 ➔ ptr4 , ptr2Report error if taint mark’s count is zero andmemory has not been freed.231 111 22211 2 2In general propagation follows standard pointer arithmetic rulesDiscover where the last pointer to un-freed memory is lost
  30. 30. @interface Container:NSObject { id _object;}@end@implementation Container- (void) dealloc { [super dealloc];}- (void) setObject:(id)obj { [_object release]; _object = [obj retain];}@endContainer *create() { Container *c = [[Container alloc] init]; NSObject *o = [[NSObject alloc] init]; [c setObject:o]; [o release]; return c;}int main(...) { Container *c = create(); … [c release];}Detecting leaks is easy, fixing them is easier
  31. 31. @interface Container:NSObject { id _object;}@end@implementation Container- (void) dealloc { [super dealloc];}- (void) setObject:(id)obj { [_object release]; _object = [obj retain];}@endContainer *create() { Container *c = [[Container alloc] init]; NSObject *o = [[NSObject alloc] init]; [c setObject:o]; [o release]; return c;}int main(...) { Container *c = create(); … [c release];}leakpoint:This object is leakedDetecting leaks is easy, fixing them is easier
  32. 32. @interface Container:NSObject { id _object;}@end@implementation Container- (void) dealloc { [super dealloc];}- (void) setObject:(id)obj { [_object release]; _object = [obj retain];}@endContainer *create() { Container *c = [[Container alloc] init]; NSObject *o = [[NSObject alloc] init]; [c setObject:o]; [o release]; return c;}int main(...) { Container *c = create(); … [c release];}leakpoint:Last reference was lost hereleakpoint:This object is leakedDetecting leaks is easy, fixing them is easier
  33. 33. @interface Container:NSObject { id _object;}@end@implementation Container- (void) dealloc { [super dealloc];}- (void) setObject:(id)obj { [_object release]; _object = [obj retain];}@endContainer *create() { Container *c = [[Container alloc] init]; NSObject *o = [[NSObject alloc] init]; [c setObject:o]; [o release]; return c;}int main(...) { Container *c = create(); … [c release];} [_object release];leakpoint:Last reference was lost hereleakpoint:This object is leakedDetecting leaks is easy, fixing them is easier
  34. 34. Leakpoint implementation• Implemented as aValgrind tool (www.valgrind.org)■ intercept libc memory management functions■ instrument binary instructions to perform propagation
  35. 35. Lost pointer to 0x1C93AC0 (16 bytes) allocated at:  at calloc+105  by _internal_class_createInstanceFromZone+149  by _internal_class_createInstance+31  by +[NSObject allocWithZone:]+155 (NSObject.m:445)  by +[NSObject alloc]+41 (NSObject.m:432)  by create+97 (main.m:29)  by main+17 (main.m:38) leaked at:  at free+103  by _internal_object_dispose+81  by NSDeallocateObject+223 (NSObject.m:207)  by -[Container dealloc]+53 (container.m:13)  by main+43 (main.m:40)Leakpoint implementation• Implemented as aValgrind tool (www.valgrind.org)■ intercept libc memory management functions■ instrument binary instructions to perform propagation
  36. 36. leaksLost pointer to 0x1C93AC0 (16 bytes) allocated at:  at calloc+105  by _internal_class_createInstanceFromZone+149  by _internal_class_createInstance+31  by +[NSObject allocWithZone:]+155 (NSObject.m:445)  by +[NSObject alloc]+41 (NSObject.m:432)  by create+97 (main.m:29)  by main+17 (main.m:38) leaked at:  at free+103  by _internal_object_dispose+81  by NSDeallocateObject+223 (NSObject.m:207)  by -[Container dealloc]+53 (container.m:13)  by main+43 (main.m:40)Leakpoint implementation• Implemented as aValgrind tool (www.valgrind.org)■ intercept libc memory management functions■ instrument binary instructions to perform propagation
  37. 37. leakpointleaksLost pointer to 0x1C93AC0 (16 bytes) allocated at:  at calloc+105  by _internal_class_createInstanceFromZone+149  by _internal_class_createInstance+31  by +[NSObject allocWithZone:]+155 (NSObject.m:445)  by +[NSObject alloc]+41 (NSObject.m:432)  by create+97 (main.m:29)  by main+17 (main.m:38) leaked at:  at free+103  by _internal_object_dispose+81  by NSDeallocateObject+223 (NSObject.m:207)  by -[Container dealloc]+53 (container.m:13)  by main+43 (main.m:40)Leakpoint implementation• Implemented as aValgrind tool (www.valgrind.org)■ intercept libc memory management functions■ instrument binary instructions to perform propagation
  38. 38. Leakpoint: current status
  39. 39. Leakpoint: current statusHandle basic C / C++ / Objective C
  40. 40. Leakpoint: current statusHandle basic C / C++ / Objective C✔
  41. 41. Leakpoint: current statusHandle basic C / C++ / Objective C✔Handle CoreFoundation
  42. 42. Leakpoint: current statusHandle basic C / C++ / Objective C✔Handle CoreFoundation✔
  43. 43. Leakpoint: current statusHandle basic C / C++ / Objective CHandle Cocoa✔Handle CoreFoundation✔
  44. 44. Need to investigate approximately 40false positive (probably) leak reports• Interface Builder unarchiving• CoreDataLeakpoint: current statusHandle basic C / C++ / Objective CHandle Cocoa✔Handle CoreFoundation✔
  45. 45. Need to investigate approximately 40false positive (probably) leak reports• Interface Builder unarchiving• CoreDataLeakpoint: current statusHandle basic C / C++ / Objective CHandle Cocoa✔Handle CoreFoundation✔64bit compatible
  46. 46. Need to investigate approximately 40false positive (probably) leak reports• Interface Builder unarchiving• CoreDataLeakpoint: current statusHandle basic C / C++ / Objective CHandle Cocoa✔Handle CoreFoundation✔64bit compatible✔
  47. 47. A real leak?: _NSImageMallocvoid *_NSImageMalloc(NSZone* zone, size_t size) {// allocate storage aligned to 32 bytes. we do this by// allocating an extra 32 bytes, finding the address in the proper// location and storing the delta in one of the previous 32 bytes.void *unaligned = NSZoneMalloc(zone, size + BITMAP_DATA_ALIGNMENT);if(unaligned != NULL) {uintptr_t aligned = ((uintptr_t)unaligned + BITMAP_DATA_ALIGNMENT)& ~(BITMAP_DATA_ALIGNMENT - 1);(unsigned char*)aligned[-1] = aligned - (uintptr_t) unaligned;return (void*)aligned;}else {return NULL;}}
  48. 48. OverheadPowerful but expensive50 -100x overheads are common
  49. 49. OverheadPowerful but expensive50 -100x overheads are commonRecommended usage:run cheap tools to check for errorsrun expensive tools to diagnose errors
  50. 50. Future work+ Leakpoint( )
  51. 51. Future workImpact+ Leakpoint( )
  52. 52. Future work• Apple■ new leak detection tool■ experience with dynamic taint analysisImpact+ Leakpoint( )
  53. 53. Future work• Apple■ new leak detection tool■ experience with dynamic taint analysis• Me■ experience withValgrind■ experience analyzing large commercial code baseImpact+ Leakpoint( )
  54. 54. Questions?

×