Getting Intimate with Images on Android with James Halpern

1,483 views

Published on

Save 10% off ANY FITC event with discount code 'slideshare'
See our upcoming events at www.fitc.ca

OVERVIEW
As most Android developers know, dealing with the extreme degree of fragmentation in the Android ecosystem is often challenging. Among the more difficult challenges is managing memory usage, as devices that are in the market today can have as little as 13MB of memory. Now imagine the pains that developers go through when faced with the headache of having massive bitmaps eat up memory in a millisecond.
In this presentation, James Halpern will talk about the complexities of image and memory management in Android and walk you through the creation of a successful, powerful and open source image management utility. Come to this presentation to learn about techniques that will help you optimize the performance of your apps. Learn about Android’s memory limitations and the role the garbage collector plays in your app’s performance and complexity. Learn how to communicate android graphics issues to developers, and how good design can create fewer bugs. James will conclude this presentation by briefly walking you through his open sourced image management solution that gracefully handles most of these issues in a simple to use package.

Published in: Internet, Art & Photos
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,483
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
17
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • ----- Meeting Notes (2013-05-02 14:18) -----NameLead Engineer
  • ----- Meeting Notes (2013-05-02 14:18) -----I'm hear to talk about images!They're complex.Need to learn how to deal with them.
  • ----- Meeting Notes (2013-05-02 14:18) -----How many devs? Android devs?Designers? Android designers?Product/management? Other?Dev focus. Good content for designers.
  • ----- Meeting Notes (2013-05-02 14:18) -----At Xtreme Labswe strive to build the best mobile appsfor the biggest companies
  • ----- Meeting Notes (2013-05-02 14:18) -----We hold a core set of values that help usToday, execution
  • ----- Meeting Notes (2013-05-02 14:18) -----Passion for librariesReusable codeMaking lives simple
  • ----- Meeting Notes (2013-05-02 14:18) -----HTC Status -- Challenging16MB of memorySlowWeird screen resolution
  • ----- Meeting Notes (2013-05-02 14:18) -----Four key areas with problemsBugs -- Flickering, wrong view, OOMDesign -- Scaling, aspect ratio
  • ----- Meeting Notes (2013-05-02 14:18) -----Available on github
  • ----- Meeting Notes (2013-05-02 14:18) -----Designers -- How design impacts devDevelopers -- How to solveProduct -- Why it takes devs a whileWalk you through problem to solution
  • ----- Meeting Notes (2013-05-02 14:18) -----Less of a problem on iOSStill a problem on iOSThis solution -- Relevant to iOS
  • ----- Meeting Notes (2013-05-02 14:18) -----Less RAM than Disk!Yet bigger in RAM!
  • ----- Meeting Notes (2013-05-02 14:18) -----Will crash the HTC Status & replenish
  • ----- Meeting Notes (2013-05-02 14:18) -----Same image fits in different views
  • ----- Meeting Notes (2013-05-02 14:18) -----Same view can require different sizes
  • Getting Intimate with Images on Android with James Halpern

    1. 1. | CAPABILITIES Getting Intimate with Images on Android @james_halpern | james.halpern@xtremelabs.com
    2. 2. Pulse
    3. 3. Pulse
    4. 4. THE SITUATION…
    5. 5. Why Kittens are Awesome
    6. 6. What happened?
    7. 7. | CAPABILITIES Introduction @james_halpern | james.halpern@xtremelabs.com
    8. 8. MISSION To become the leading provider of mobile solutions to the world’s most important companies as we help drive a revolution in computing.
    9. 9. VALUES Transparency Opportunity Meritocracy Execution
    10. 10. Introduction • 2+ years of Android • 6 product launches • 2 Google Play apps • 3+ Android libraries • 20+ Android projects impacted by tools development, and counting
    11. 11. Why I Started
    12. 12. Why I Started Performance Bugs Design Time
    13. 13. My Library Image Utilities Open Source https://github.com/xtremelabs/xl-image_utils_lib-android Available under Apache 2.0
    14. 14. Objectives Simple Fast Powerful
    15. 15. Goal Problems Tools API Debugging Solve the images problem, Build great apps
    16. 16. | CAPABILITIES Production Quality Software Problems Tools API Debugging @james_halpern | james.halpern@xtremelabs.com
    17. 17. Non-Production Quality try { // 5 BILLION lines of code! } catch (Exception e) { // Just in case there is a bug, // let’s not crash. }
    18. 18. Non-Production Quality new AsyncTask<Void, Void, Bitmap>() { public Bitmap doInBackground() { Bitmap bitmap = fetchBitmap(url); return bitmap; } public void onPostExecute(Bitmap result) { imageView.setImageBitmap(result); } }
    19. 19. What Users Want The app should just work.
    20. 20. Lists that just work Resources that adapt to screen size and resolution Lazy loading that does not interfere with list items Optimizations for ImageView size Images that are available before you scroll to them Smooth, flawless scrolling, across all devices Anticipate the user’s next action
    21. 21. Practices 1. Clean Code 2. Design Patterns 3. Unit Testing 4. Test Driven Development
    22. 22. | CAPABILITIES Platform Fragmentation Problems Tools API Debugging @james_halpern | james.halpern@xtremelabs.com
    23. 23. Fragmentation
    24. 24. Android vs. iOS Android • 16MB – 96MB iOS • 128MB – 512MB
    25. 25. Why Images are Tough • Available memory differs by phone 16MB? 32MB? 64MB?
    26. 26. Why Images are Tough • Screen Density/Resolution Lower Density = Fewer Pixels = Less Memory Used Higher Density = More Pixels = More Memory Used
    27. 27. | CAPABILITIES Image Decoding Problems Tools API Debugging @james_halpern | james.halpern@xtremelabs.com
    28. 28. Bitmap Complexities
    29. 29. Bitmap Complexities Encoded Decoded Small File Size High Memory Usage
    30. 30. Effect of Decoding This cute kitten is: • ~20kB on disk • ~500kB in memory
    31. 31. Effect of Decoding • ~33kB on disk • ~1.5MB in memory
    32. 32. Effect of Decoding 5 megapixel image • ~1.8MB on disk • ~20MB in memory!!!
    33. 33. Approximating Bitmap Sizes Bit Depth * Number of Pixels = Approximate Size Examples of bit depth: ARGB-8888  4 bytes/pixel RGB-565  2 bytes/pixel
    34. 34. Big Image, Big Problem What happens if an image is too big?
    35. 35. Example: The 15x15dp Ad
    36. 36. Summary • Images are big in memory • Scaling can save memory • The API may provide bad images • Different image formats perform differently
    37. 37. | CAPABILITIES Threads Problems Tools API Debugging @james_halpern | james.halpern@xtremelabs.com
    38. 38. Context Switching Main Thread 1 Thread 2 UI will not update here!
    39. 39. Context Switching More Threads = Fewer UI Updates
    40. 40. Context Switching Main Network 1 Network 2 Network 3 Disk 1
    41. 41. Sources of Background Threads • Network • Disk access • Heavy CPU tasks – Renders – Encoding – Decoding – etc.
    42. 42. Typical App – Threads 1. Analytics system? (1-X threads) 2. Ad system? (1-X threads) 3. Network library? (1-10 threads) 4. Content providers? (1-255 threads) 5. Disk/File access threads (1-X threads)
    43. 43. Network Threads Network 1 Network 2 Network 3
    44. 44. Context Switching Disk 1
    45. 45. Strict Mode StrictMode.setThreadPolicy( new StrictMode.ThreadPolicy.Builder() .detectDiskReads() .detectDiskWrites() .detectNetwork() .penaltyDeath() .build() );
    46. 46. | CAPABILITIES Communication Problems Tools API Debugging @james_halpern | james.halpern@xtremelabs.com
    47. 47. Communication Product Design Engineering
    48. 48. The Density Independent Pixel px = dp * (dpi / 160) 1dp ~= 1/160th of an inch (but not really) 1dp may be smaller than a pixel on low density phones!
    49. 49. Forgetting Pixels This is not iPhone! Don’t think in pixels! Pixels are different sizes on different phones
    50. 50. With that said… Designers still need to consider pixels…
    51. 51. Talking to Designers Break it down! 1. Understand how it works 2. Know what to build
    52. 52. Android for Designers http://petrnohejl.github.io/Android-Cheatsheet-For-Graphic-Designers/
    53. 53. How it works Heading R Heading R
    54. 54. How it works Heading R Heading R H WH W
    55. 55. How it works Heading R Heading R H WH W
    56. 56. What to Build Back to pixels…
    57. 57. Targeting Resolutions Providing some Resolutions: 1. What devices? 2. Choose 2-4 resolutions. 3. Test ON DEVICE. 4. Alignment: Start? End? Center?
    58. 58. Targeting Resolutions http://en.wikipedia.org/wiki/Comparison_of_Android_devices (1080/1200) x 1920 (720/768/800) x 1280 480 x 800 320 x 480
    59. 59. Flexible Designs Only scale down. Don’t scale up!
    60. 60. Planning for the Small Screen Heading R Heading R
    61. 61. Communicate Performance Larger Image = Slower Performance Design With Small Images = Fast App 9-Patches > Images
    62. 62. Summary • Memory limits • Scrolling performance • Multiple densities • Multiple resolutions • ImageView references • Image Variety • Pixel Formats • Image Scaling • Performance • And on and on… The problem is not limited to engineering!
    63. 63. | CAPABILITIES Decoding Images Problems Tools API Debugging @james_halpern | james.halpern@xtremelabs.com
    64. 64. The BitmapFactory Bitmap File InputStream Resource Byte[]
    65. 65. The BitmapFactory Bitmap bitmap; bitmap = BitmapFactory.decodeFile(filePath); bitmap = BitmapFactory.decodeStream(inputStream); etc.
    66. 66. Options BitmapFactory.Options options; options = new BitmapFactory.Options(); // Full sized image options.inSampleSize = 1; // 75% smaller options.inSampleSize = 2;
    67. 67. Applying Scaling What is the optimal sample size?
    68. 68. Applying Scaling
    69. 69. Applying Scaling
    70. 70. Calculating Scaling Scaled Dimension = Image Dimension / Sample Size Optimal Sample Size: Sample Size = (Floor) Image Width / View Width OR Sample Size = (Floor) Image Height / View Height
    71. 71. Calculating Scaling Which one? Height or Width?
    72. 72. Calculating Scaling • Fit for largest size – Crop • Fit for smallest size – Fit • Always smaller than view – Saves memory, drops quality • Round to closest scaling – May hurt quality
    73. 73. Applying Scaling Use largest size -- Crop Use smallest size -- Fit
    74. 74. Summary Build for the kitten you want, Not for the kitten you have. Scale.
    75. 75. | CAPABILITIES API Design Problems Tools API Debugging @james_halpern | james.halpern@xtremelabs.com
    76. 76. The Learning Curve 0 10 20 30 40 50 60 70 80 90 Start Setup First Request Callbacks Placeholders ImageLoader Other
    77. 77. Objectives Simple.
    78. 78. Setup Time Get started in <5min: onCreate: mImageLoader = ImageLoader.buildImageLoaderForActivity(this); onDestroy: mImageLoader.destroy();
    79. 79. The First Call imageLoader.loadImage(imageView, url);
    80. 80. Adding Complexity • Pixel format • Placeholders • Scaling • Memory safety
    81. 81. Adding Complexity Options options = new Options(); options.autoDetectBounds = false;
    82. 82. Adding Complexity imageLoader.loadImage(imageView, url, options);
    83. 83. Calling Back ImageLoaderListener listener = new ImageloaderListener() { public void onSuccess(ImageResponse response) {} public void onFailure(ImageError error) {} } imageLoader.loadImage( imageView, url, options, listener);
    84. 84. Calling Back public void onSuccess( ImageResponse response) { ImageView view; Bitmap bitmap; bitmap = response.getBitmap(); view = response.getImageView(); // Could have animated here! view.setImageBitmap(bitmap); }
    85. 85. Calling Back public void onSuccess( ImageResponse response) { … ReturnedFrom from; from = response.getReturnedFrom(); switch (from) { case MEMORY: // Don’t animate! default: // Animate here! } … }
    86. 86. Too Many Parameters… imageLoader.loadImage( imageView, url, options, listener);
    87. 87. Box the Request ImageRequest imageRequest = new ImageRequest(url); imageRequest.setImageView(imageView); imageRequest.setOptions(options); imageRequest.setListener(listener); imageLoader.loadImage(imageRequest);
    88. 88. Requests from File System ImageRequest imageRequest = new ImageRequest(URL); ImageRequest imageRequest = new ImageRequest(URI);
    89. 89. Requests from File System String file = new File(path).toUri().toString(); ImageRequest request = new ImageRequest(file);
    90. 90. Why use the same system? 1. Image scaling 2. Memory cache 3. Pre-built adapter features 4. Will not retain Activities/Fragments
    91. 91. Precaching ImageLoader.precacheToDisk(context, uri);
    92. 92. Additional APIs • Custom Memory Caches • Cache Settings – Size, structure • Manual Evictions • Cancelling Requests
    93. 93. API Summary Save time. Keep it simple.
    94. 94. | CAPABILITIES Stack Traces Tell White Lies Problems Tools API Debugging @james_halpern | james.halpern@xtremelabs.com
    95. 95. Let’s start with a Stack Trace! 01-25 05:05:49.877: ERROR/dalvikvm-heap(3896): 6291456-byte external allocation too large for this process. 01-25 05:05:49.877: ERROR/(3896): VM wont let us allocate 6291456 bytes 01-25 05:05:49.877: ERROR/AndroidRuntime(3896): Uncaught handler: thread main exiting due to uncaught exception 01-25 05:05:49.917: ERROR/AndroidRuntime(3896): java.lang.OutOfMemoryError: bitmap size exceeds VM budget 01-25 05:05:49.917: ERROR/AndroidRuntime(3896): at android.graphics.BitmapFactory.nativeDecodeStream(Native Method) 01-25 05:05:49.917: ERROR/AndroidRuntime(3896): at android.graphics.BitmapFactory.decodeStream(BitmapFactory.java:304) 01-25 05:05:49.917: ERROR/AndroidRuntime(3896): at android.graphics.BitmapFactory.decodeFile(BitmapFactory.java:149) 01-25 05:05:49.917: ERROR/AndroidRuntime(3896): at android.graphics.BitmapFactory.decodeFile(BitmapFactory.java:174)
    96. 96. Let’s start with a Stack Trace! • java.lang.OutOfMemoryError: bitmap size exceeds VM budget • at: android.graphics.BitmapFactory.decodeStream
    97. 97. Errors are Symptoms
    98. 98. Example onCreate(…) { foo = new Foo(); } onStart() { foo.bar(); } onStop() { foo = null; }
    99. 99. Example – NullPointerException! onCreate(…) { foo = new Foo(); } onStart() { foo.bar(); // Crash! } onStop() { foo = null; //  Probably not intended }
    100. 100. Bitmap Factory - OOM Maximum Heap Size Maximum image size in caching system Memory leak increasing heap size
    101. 101. Bitmap Factory - OOM Maximum Heap Size
    102. 102. Example – Memory Leak App • Timer leaking memory every 4ms • Expected error: 09-13 10:51:14.401: E/AndroidRuntime(7170): java.lang.OutOfMemoryError 09-13 10:51:14.401: E/AndroidRuntime(7170): at com.example.outofmemoryomg.OutOfMemoryActivity$1.run(OutOfMemoryActivity.java:40) 09-13 10:51:14.401: E/AndroidRuntime(7170): at java.util.Timer$TimerImpl.run(Timer.java:284) • OutOfMemoryActivity.java:40
    103. 103. Example – Memory Leak App • Actual Error: 09-13 10:52:38.159: E/dalvikvm-heap(7471): Out of memory on a 576016-byte allocation. 09-13 10:52:38.166: E/AndroidRuntime(7471): FATAL EXCEPTION: pool-1-thread-1 09-13 10:52:38.166: E/AndroidRuntime(7471): java.lang.OutOfMemoryError 09-13 10:52:38.166: E/AndroidRuntime(7471): at android.graphics.BitmapFactory.nativeDecodeStream(Native Method) 09-13 10:52:38.166: E/AndroidRuntime(7471): at android.graphics.BitmapFactory.decodeStream(BitmapFactory.java:527) 09-13 10:52:38.166: E/AndroidRuntime(7471): at com.xtremelabs.imageutils.DiskLRUCacher.getBitmapSynchronouslyFromDisk(DiskLRUCacher.java:172) 09-13 10:52:38.166: E/AndroidRuntime(7471): at com.xtremelabs.imageutils.DiskLRUCacher.access$1(DiskLRUCacher.java:166) 09-13 10:52:38.166: E/AndroidRuntime(7471): at com.xtremelabs.imageutils.DiskLRUCacher$2.run(DiskLRUCacher.java:90) 09-13 10:52:38.166: E/AndroidRuntime(7471): at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076) 09-13 10:52:38.166: E/AndroidRuntime(7471): at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569) 09-13 10:52:38.166: E/AndroidRuntime(7471): at java.lang.Thread.run(Thread.java:856) • BitmapFactory.decodeStream(BitmapFact ory.java:527)
    104. 104. | CAPABILITIES Debugging: MAT & DDMS Problems Tools API Debugging @james_halpern | james.halpern@xtremelabs.com
    105. 105. Proving the Leak
    106. 106. Introducing MAT • Memory Analyzer Tool – http://www.eclipse.org/mat/ – http://download.eclipse.org/mat/1.2/update-site/ Link to Tutorial: http://www.google.com/events/io/2011/sessions/memory-management-for-android- apps.html
    107. 107. Leak proven!
    108. 108. What is DDMS? Dalvik Debug Monitor Server Provides massive amounts of debug information
    109. 109. What is DDMS? • Helps with: – Performance – Profiling – Memory – File system access – Emulator options
    110. 110. DDMS and Memory
    111. 111. DDMS and Memory
    112. 112. Tips for Hunting Leaks • Reduce image quality significantly! – Sample Size = 128 • Remove tested, big objects • Repeat the offending action • Use the “Leak Suspects Report”
    113. 113. DDMS – Threads and Performance • Debug number of threads running – Are you creating too many threads? – Threads lingering beyond their lifetime? – Threads eating up performance/hanging? UI Thread -- Are we blocking it?
    114. 114. DDMS – Threads and Performance
    115. 115. DDMS – Threads and Performance
    116. 116. DDMS – Threads and Performance Method-by-method breakdown of performance
    117. 117. | CAPABILITIES Conclusion https://github.com/xtremelabs/xl-image_utils_lib-android @james_halpern | james.halpern@xtremelabs.com Problems Tools API Debugging
    118. 118. | CAPABILITIES Q&A https://github.com/xtremelabs/xl-image_utils_lib-android @james_halpern | james.halpern@xtremelabs.com Problems Tools API Debugging
    119. 119. | CAPABILITIES Thank you! @james_halpern | james.halpern@xtremelabs.com

    ×