Getting Intimate with Images on Android with James Halpern

| CAPABILITIES
Getting Intimate
with Images on
Android
@james_halpern | james.halpern@xtremelabs.com

| CAPABILITIES
Introduction

MISSION
To become the leading provider of mobile
solutions to the world’s most important
companies as we help drive a revolution in
computing.

VALUES
Transparency
Opportunity
Meritocracy
Execution

Introduction
• 2+ years of Android
• 6 product launches
• 2 Google Play apps
• 3+ Android libraries
• 20+ Android projects impacted by tools
development, and counting

Why I Started
Performance Bugs
Design Time

My Library
Image Utilities
Open Source
https://github.com/xtremelabs/xl-image_utils_lib-android
Available under Apache 2.0

Objectives
Simple
Fast
Powerful

Goal
Problems
Tools
API
Debugging
Solve the images problem,
Build great apps

| CAPABILITIES
Production
Quality Software
Problems Tools API Debugging

Non-Production Quality
try {
// 5 BILLION lines of code!
} catch (Exception e) {
// Just in case there is a bug,
// let’s not crash.
}

Non-Production Quality
new AsyncTask<Void, Void, Bitmap>() {
public Bitmap doInBackground() {
Bitmap bitmap = fetchBitmap(url);
return bitmap;
}
public void onPostExecute(Bitmap result) {
imageView.setImageBitmap(result);
}
}

What Users Want
The app should just work.

Lists that just work
Resources that adapt to
screen size and resolution
Lazy loading that does not
interfere with list items
Optimizations for
ImageView size
Images that are available
before you scroll to them
Smooth, flawless
scrolling, across all devices
Anticipate the user’s next
action

Practices
1. Clean Code
2. Design Patterns
3. Unit Testing
4. Test Driven Development

| CAPABILITIES
Platform
Fragmentation

Android vs. iOS
Android
• 16MB – 96MB
iOS
• 128MB – 512MB

Why Images are Tough
• Available memory differs by phone
16MB?
32MB?
64MB?

Why Images are Tough
• Screen Density/Resolution
Lower Density = Fewer Pixels = Less Memory Used
Higher Density = More Pixels = More Memory Used

| CAPABILITIES
Image Decoding

Bitmap Complexities
Encoded Decoded
Small File Size High Memory Usage

Effect of Decoding
This cute kitten is:
• ~20kB on disk
• ~500kB in memory

Effect of Decoding
• ~33kB on disk
• ~1.5MB in memory

Effect of Decoding
5 megapixel image
• ~1.8MB on disk
• ~20MB in memory!!!

Approximating Bitmap Sizes
Bit Depth * Number of Pixels = Approximate Size
Examples of bit depth:
ARGB-8888  4 bytes/pixel
RGB-565  2 bytes/pixel

Big Image, Big Problem
What happens if an image is too big?

Summary
• Images are big in memory
• Scaling can save memory
• The API may provide bad images
• Different image formats perform differently

| CAPABILITIES
Threads

Context Switching
Main
Thread 1
Thread 2
UI will not update here!

Context Switching
More Threads = Fewer UI Updates

Context Switching
Main
Network 1
Network 2
Network 3
Disk 1

Sources of Background Threads
• Network
• Disk access
• Heavy CPU tasks
– Renders
– Encoding
– Decoding
– etc.

Typical App – Threads
1. Analytics system? (1-X threads)
2. Ad system? (1-X threads)
3. Network library? (1-10 threads)
4. Content providers? (1-255 threads)
5. Disk/File access threads (1-X threads)

Network Threads
Network 1
Network 2
Network 3

Strict Mode
StrictMode.setThreadPolicy(
new StrictMode.ThreadPolicy.Builder()
.detectDiskReads()
.detectDiskWrites()
.detectNetwork()
.penaltyDeath()
.build()
);

| CAPABILITIES
Communication

Communication
Product
Design
Engineering

The Density Independent Pixel
px = dp * (dpi / 160)
1dp ~= 1/160th of an inch (but not really)
1dp may be smaller than a pixel on low density
phones!

Forgetting Pixels
This is not iPhone!
Don’t think in pixels!
Pixels are different sizes on different phones

With that said…
Designers still need to consider pixels…

Talking to Designers
Break it down!
1. Understand how it works
2. Know what to build

Android for Designers
http://petrnohejl.github.io/Android-Cheatsheet-For-Graphic-Designers/

How it works
Heading R
Heading R

How it works
Heading R
Heading R
H WH W

What to Build
Back to pixels…

Targeting Resolutions
Providing some Resolutions:
1. What devices?
2. Choose 2-4 resolutions.
3. Test ON DEVICE.
4. Alignment: Start? End? Center?

Targeting Resolutions
http://en.wikipedia.org/wiki/Comparison_of_Android_devices
(1080/1200) x 1920
(720/768/800) x 1280
480 x 800
320 x 480

Flexible Designs
Only scale down.
Don’t scale up!

Planning for the Small Screen
Heading R
Heading R

Communicate Performance
Larger Image = Slower Performance
Design With Small Images = Fast App
9-Patches > Images

Summary
• Memory limits
• Scrolling performance
• Multiple densities
• Multiple resolutions
• ImageView references
• Image Variety
• Pixel Formats
• Image Scaling
• Performance
• And on and on…
The problem is not limited
to engineering!

| CAPABILITIES
Decoding Images

The BitmapFactory
Bitmap
File
InputStream Resource
Byte[]

The BitmapFactory
Bitmap bitmap;
bitmap = BitmapFactory.decodeFile(filePath);
bitmap = BitmapFactory.decodeStream(inputStream);
etc.

Options
BitmapFactory.Options options;
options = new BitmapFactory.Options();
// Full sized image
options.inSampleSize = 1;
// 75% smaller
options.inSampleSize = 2;

Applying Scaling
What is the optimal sample size?

Calculating Scaling
Scaled Dimension = Image Dimension / Sample Size
Optimal Sample Size:
Sample Size = (Floor) Image Width / View Width
OR
Sample Size = (Floor) Image Height / View Height

Calculating Scaling
Which one? Height or Width?

Calculating Scaling
• Fit for largest size
– Crop
• Fit for smallest size
– Fit
• Always smaller than view
– Saves memory, drops quality
• Round to closest scaling
– May hurt quality

Applying Scaling
Use largest size -- Crop Use smallest size -- Fit

Summary
Build for the
kitten you want,
Not for the
kitten you have.
Scale.

| CAPABILITIES
API Design

The Learning Curve
0
10
20
30
40
50
60
70
80
90
Start Setup First Request Callbacks Placeholders
ImageLoader
Other

Setup Time
Get started in <5min:
onCreate:
mImageLoader = ImageLoader.buildImageLoaderForActivity(this);
onDestroy:
mImageLoader.destroy();

The First Call
imageLoader.loadImage(imageView, url);

Adding Complexity
• Pixel format
• Placeholders
• Scaling
• Memory safety

Adding Complexity
Options options = new Options();
options.autoDetectBounds = false;

Adding Complexity
imageLoader.loadImage(imageView, url, options);

Calling Back
ImageLoaderListener listener =
new ImageloaderListener() {
public void onSuccess(ImageResponse response) {}
public void onFailure(ImageError error) {}
}
imageLoader.loadImage(
imageView, url, options, listener);

Calling Back
public void onSuccess(
ImageResponse response) {
ImageView view;
Bitmap bitmap;
bitmap = response.getBitmap();
view = response.getImageView();
// Could have animated here!
view.setImageBitmap(bitmap);
}

Calling Back
public void onSuccess(
ImageResponse response) {
…
ReturnedFrom from;
from = response.getReturnedFrom();
switch (from) {
case MEMORY:
// Don’t animate!
default:
// Animate here!
}
…
}

Too Many Parameters…
imageLoader.loadImage(
imageView, url, options, listener);

Box the Request
ImageRequest imageRequest =
new ImageRequest(url);
imageRequest.setImageView(imageView);
imageRequest.setOptions(options);
imageRequest.setListener(listener);
imageLoader.loadImage(imageRequest);

Requests from File System
new ImageRequest(URL);
new ImageRequest(URI);

Requests from File System
String file =
new File(path).toUri().toString();
ImageRequest request =
new ImageRequest(file);

Why use the same system?
1. Image scaling
2. Memory cache
3. Pre-built adapter features
4. Will not retain Activities/Fragments

Precaching
ImageLoader.precacheToDisk(context, uri);

Additional APIs
• Custom Memory Caches
• Cache Settings
– Size, structure
• Manual Evictions
• Cancelling Requests

API Summary
Save time.
Keep it simple.

| CAPABILITIES
Stack Traces Tell
White Lies

Let’s start with a Stack Trace!
01-25 05:05:49.877: ERROR/dalvikvm-heap(3896): 6291456-byte external allocation too large for this
process.
01-25 05:05:49.877: ERROR/(3896): VM wont let us allocate 6291456 bytes
01-25 05:05:49.877: ERROR/AndroidRuntime(3896): Uncaught handler: thread main exiting due to
uncaught exception
01-25 05:05:49.917: ERROR/AndroidRuntime(3896): java.lang.OutOfMemoryError: bitmap size
exceeds VM budget
01-25 05:05:49.917: ERROR/AndroidRuntime(3896): at
android.graphics.BitmapFactory.nativeDecodeStream(Native Method)
android.graphics.BitmapFactory.decodeStream(BitmapFactory.java:304)
android.graphics.BitmapFactory.decodeFile(BitmapFactory.java:149)
android.graphics.BitmapFactory.decodeFile(BitmapFactory.java:174)

Let’s start with a Stack Trace!
• java.lang.OutOfMemoryError: bitmap size
exceeds VM budget
• at:
android.graphics.BitmapFactory.decodeStream

Example
onCreate(…) {
foo = new Foo();
}
onStart() {
foo.bar();
}
onStop() {
foo = null;
}

Example – NullPointerException!
onCreate(…) {
foo = new Foo();
}
onStart() {
foo.bar(); // Crash!
}
onStop() {
foo = null; //  Probably not intended
}

Bitmap Factory - OOM
Maximum Heap Size
Maximum image
size in caching
system
Memory leak increasing heap size

Bitmap Factory - OOM
Maximum Heap Size

Example – Memory Leak App
• Timer leaking memory every 4ms
• Expected error:
09-13 10:51:14.401: E/AndroidRuntime(7170): java.lang.OutOfMemoryError
09-13 10:51:14.401: E/AndroidRuntime(7170): at
com.example.outofmemoryomg.OutOfMemoryActivity$1.run(OutOfMemoryActivity.java:40)
09-13 10:51:14.401: E/AndroidRuntime(7170): at java.util.Timer$TimerImpl.run(Timer.java:284)
• OutOfMemoryActivity.java:40

Example – Memory Leak App
• Actual Error:
09-13 10:52:38.159: E/dalvikvm-heap(7471): Out of memory on a 576016-byte allocation.
09-13 10:52:38.166: E/AndroidRuntime(7471): FATAL EXCEPTION: pool-1-thread-1
09-13 10:52:38.166: E/AndroidRuntime(7471): java.lang.OutOfMemoryError
09-13 10:52:38.166: E/AndroidRuntime(7471): at android.graphics.BitmapFactory.nativeDecodeStream(Native Method)
09-13 10:52:38.166: E/AndroidRuntime(7471): at android.graphics.BitmapFactory.decodeStream(BitmapFactory.java:527)
09-13 10:52:38.166: E/AndroidRuntime(7471): at
com.xtremelabs.imageutils.DiskLRUCacher.getBitmapSynchronouslyFromDisk(DiskLRUCacher.java:172)
09-13 10:52:38.166: E/AndroidRuntime(7471): at com.xtremelabs.imageutils.DiskLRUCacher.access$1(DiskLRUCacher.java:166)
09-13 10:52:38.166: E/AndroidRuntime(7471): at com.xtremelabs.imageutils.DiskLRUCacher$2.run(DiskLRUCacher.java:90)
09-13 10:52:38.166: E/AndroidRuntime(7471): at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
09-13 10:52:38.166: E/AndroidRuntime(7471): at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
09-13 10:52:38.166: E/AndroidRuntime(7471): at java.lang.Thread.run(Thread.java:856)
• BitmapFactory.decodeStream(BitmapFact
ory.java:527)

| CAPABILITIES
Debugging:
MAT & DDMS

Introducing MAT
• Memory Analyzer Tool
– http://www.eclipse.org/mat/
– http://download.eclipse.org/mat/1.2/update-site/
Link to Tutorial:
http://www.google.com/events/io/2011/sessions/memory-management-for-android-
apps.html

What is DDMS?
Dalvik Debug Monitor Server
Provides massive amounts of debug information

What is DDMS?
• Helps with:
– Performance
– Profiling
– Memory
– File system access
– Emulator options

Tips for Hunting Leaks
• Reduce image quality significantly!
– Sample Size = 128
• Remove tested, big objects
• Repeat the offending action
• Use the “Leak Suspects Report”

DDMS – Threads and Performance
• Debug number of threads running
– Are you creating too many threads?
– Threads lingering beyond their lifetime?
– Threads eating up performance/hanging?
UI Thread -- Are we blocking it?

Method-by-method breakdown of performance

| CAPABILITIES
Conclusion

| CAPABILITIES
Q&A

| CAPABILITIES
Thank you!

Getting Intimate with Images on Android with James Halpern

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (19)

Similar to Getting Intimate with Images on Android with James Halpern

Similar to Getting Intimate with Images on Android with James Halpern (20)

More from FITC

More from FITC (20)

Recently uploaded

Recently uploaded (20)

Getting Intimate with Images on Android with James Halpern

Editor's Notes