Explore tips, tools and best practices for building high-performance apps that use data more efficiently. Learn how to leverage KnowMyApp.org (by CTIA-The Wireless Association®) and AT&T's Applications Resource Optimizer (ARO) to measure and make the most of data usage
TK: Hi Everyone and Thank you for joining us. Today we’re going to be talking about App Performance, specifically we’ll be looking at real examples of mistakes that some of the top apps make, so that you can avoid them. We’ll also give you a simple testing plan to improve your own app, before the app store reviews get ahold of them.
TK When we talk about performance, we focus on how the network connections are impacting customers. And it may have a bigger impact than you think. Data connections account for 40% of the battery life when the screen is on, and 70% of the battery life with the screen off. When we first started testing apps, we focused our testing on actively using the app, but shortly afterwards, we found the biggest impacts were when the apps were in the background or even just idle.
As we look at examples today, I want you to have a little background information on the RRC state machine. The RRC state machine is essentially the antenna on the device, and many developers aren’t aware, that every time the antenna is opened, it says open for an extra 15 seconds after the last data is pulled down…. In the example here, you can see that RRC state machine powers up (here in Red) and stays active after the first group of data comes down. When it stays open the antenna is at full power. Then as another group of packets comes down at 50 seconds, the antenna stays open again until it finally turns off at 64 seconds.
While 24 seconds of antenna power isn’t extreme in this screen shot, We often see this behavior repeated for the entire time a user is in an app. 10, 15, 25 minutes or more of the app not giving the battery second to rest. I
So Just keep that in mind as we look at the rest of the examples, We’ll be showing how the RRC state machine is affected with each example.
TK
Some times we get push back …by focusing on the network, but honestly it’s the black box that can make or break your app. Just a 500 millisecond delay can increase user frustration by 26%, Decrease user engagement by 8%... And even a minor 100 millisecond delay can reduce Amazon’s revenue by 1% …. And our best stat of all is that High latency can lead to 4% of mobile users throwing their phones…. Admit it.. We’ve all been there.
Just think of your raise, when you walk into your bosses office and show them that you can reduce customer frustration, and increase engagement and revenue. It’s our job to help developers get there everyday…. Well, we can’t help with the raise, but the rest we can.
OK OK
So your boss, doesn’t think “Performance” is something you need to focus on……
Well chances they do want you to focus on User Retention Rate, App store rating, User Acquisition Cost and things like Average Revenue per user. The more we research performance, the more we discover how performance influences all of these important application stats.
DS
DS
DS Repeatedly we hear, “But not my developers… They‘re the best…. That issue is so minor it won’t affect my customers….”
98% of all the apps we test have some sort of optimization
What appear to be minor issues are responsible for killing user batteries and slowing down applications to a point where revenue is being lost….
“I can’t even test my own app without it being plugged into an outlet because it drains so much battery”
DS: Repeatedly we hear, “But not my developers… They‘re the best…. That issue is so minor it won’t affect my customers….”
98% of all the apps we test have some sort of optimization
What appear to be minor issues are responsible for killing user batteries and slowing down applications to a point where revenue is being lost….
“There was a 4kb file that came down each time the app was launched and I didn’t think it was a big issue, until at launch we had so many downloads and new users that it took down our servers. Then we realized how much revenue we were losing every minute our servers were down.”
TK
Those of us that come from a testing background, drive home “TEST TEST TEST. Test early test often” Sometimes I feel like I get a little preachy. But developers are getting hit hard by application reviews, and most users will only give you that one shot. We’ve got to put our best foot forward.
Users are getting smarter too. While crash screens are obvious, users expect their apps to load in less than 4 seconds… Actually that’s the stat from a few years ago, it’s moving down toward 2 seconds. When I heard that, I immediately pulled out my phone and started launching apps to see how long they take to load. 4 seconds goes by quicker than you think and 2 seconds well… it’s tough. Try it tonight when you’re in your hotel, pull out your phone and launch an app that you would expect to be at the top of it’s game. Preferably one that you haven’t used in a while. And see how long it takes….
Put your phones away, I didn’t say do it now.
TK:
We all know Doug loves his farm animals. He actually does have a few goats at home, and we like to incorporate goats into all our presentations. We’ve had a little fun with the examples we’re showing today, and put goats in as often as possible.
Find my Goat is an application that brings together Goats and riders. Riders request a goat and the goats mobile device lets the goat know where he needs to go.
Find my Goat has seen tremendous success, but one day there was trouble in Goatville, when there was a sudden drop in sales reported.
Image: https://www.flickr.com/photos/68919440@N05/6269603592/in/photolist-nuiq6T-bTKEXz-beHLNH-cWRPWA-7Ztgj6-bGT5rF-joN57z-irVKZ4-9y3tkW-8FDAqc-oKAvgN-buRoNN-6XLEt-ddRZJ-aEAwr-d2gnrh-gUbVCV-hqWXPa-b3FmqZ-hr81FU-92UxSM-aX9rHF-2VSy9v-negLWP-apVG8d-dNs5gT-dAog6U-dqmHYT-dH9Xfd-ei6orG-ay2mHE-bc6Ryg-fGnRGf-aRHExn-7JUXEc-9iy5ko-8X6JYv-bPzDS4-mS1Rbv-b3Fxuv-eXkN8A-b3FrRk-b3FsHT-aUusft-9CHUJ3-f47gFM-fZt3Sk-d9B6ps-f47fDD-6diGxW/
Usability starts with the design and architecture
OS
ie: Cross platform inter-relations (Drive app vs Customer app)
Small issues become magnified
No sandbox
Tiny issues become exponential when multiplied by an entire platform
Growth of product matrix needed early
TK:
During a network test of Find my Goat, we found that the app was connecting to the network every 5 seconds…….
So we know this is terrible for the battery, we can see that the RRC state machine is never able to rest, but on top of being hard on battery life. We discovered the app wasn’t reusing the same connection but creating a new connection each time…This turned out to be 18 times more connections than the architecture team expected and IP/port collisions were causing the loss in sales.
Again, this is something that normal UAT testing wouldn’t show you. If you have 15 testers in a room, they may not have any issues, but when you are able to take an image like this to your architects and say “Are our servers ready to handle this for our 100,000 or more users?” it paints a different picture. You start to be able to see how these small connections start scaling.
The other thing that I want to point out is the Throughput…. Notice how the throughput are a series of small blips. We often see this with streaming apps. We highly recommend grouping data, send out a larger batch of data, let the connection close and then reopen it. Often times we can cut loading time and reduce battery drain by just grouping data.
Usability starts with the design and architecture
OS
ie: Cross platform inter-relations (Drive app vs Customer app)
Small issues become magnified
No sandbox
Tiny issues become exponential when multiplied by an entire platform
Growth of product matrix needed early
TKWell… how many users are excited to see a whole screen of spinning circles.
Each of these Dumbnails, I mean Thumbnails are 250 KB * 9 = 1.75 MB
Progress indicators actually hurt the perception of how long it’s taking to load the images. At the very least remove the spinning circles, but there is so much more we can do here. Reducing the image size, and caching those images and again, just reuse them for the detail page. One level of loading screens is bad but two is worse.
TK
App is IDLE.
Ads downloaded every 30s.
Files are not compressed (extra data, more time)
Connections are not closed (blue bars) battery drain
TK
TK
DS
DS
Text Compression
Applications use text files in ways that the user wouldn’t expect and/or can’t see. Opportunities to gzip text files will be identified during a normal Active trace.
Duplicate Content
Testers will need to navigate back to some screens that they navigated through earlier in the trace to test caching and duplicate content.
Closing Connections
Opportunities to Close connections will be identified during a normal Active trace.
Bluetooth
If the application uses Bluetooth, it’s good to use the feature that launches Bluetooth early in the trace, as it will give an opportunity to see how long it stays on during the rest of the trace.
GPS
If the application uses GPS, it’s good to use the feature that launches GPS early in the trace, as it will give an opportunity to see how long it stays on during the rest of the trace.
DS
DS
DS:Drive mode - GPS
Battery impact
Utilize existing API’s to identify user movements
DS:
Netflix (Caching and Duplicates)
(Can use example from Doug’s deck)
Speed of app at startup
Labs study where 17% of all content is dups
How much of does that affect your servers if you remove 20% of all content served?
Types of Caching
Etags vs cache control
Etags up to 3 seconds latency for connection
DS:
Netflix (Caching and Duplicates)
(Can use example from Doug’s deck)
Speed of app at startup
Labs study where 17% of all content is dups
How much of does that affect your servers if you remove 20% of all content served?
Types of Caching
Etags vs cache control
Etags up to 3 seconds latency for connection