2. Background & Objectives
By taking actual measured behavior, the objective for this analysis
is to better understand how college students use mobile for
learning and how to improve on the existing and future mobile
apps.
This analysis is based on the compiled data that tracks the existing
mobile apps for Learning Studio.
2
4. Feedback Loop
1. App Annie
Downloads
Online Reviews
2. Email in apps
Subjective responses from users
3. Google Analytics
User demographics
User behavior and flows
4
5. App Annie
Downloads
5
Device Total April ‘15- Dec ‘15
iPad 14,202 4,875
Android Tablet 713 301
iPhone 7,018 7,018
Android Phone 1,708 1,708
6. Ipad (20)
• 5 liked the app
• 2 Pearson Haters
• 9 couldn’t find school (MLP users)
• 1 DropBox Request
• 3 Misc.,
Android Tablet (3)
• Needs more Features (DropBox)
• 2 unknown issues with potential EP configuration issues.
Iphone (4)
• 3 Needs more functionality
• 2 Trying to use on unsupported devices
Andriod Phone (4)
• All Configuration issues
6
App Annie Online
Reviews
7. Email Feedback
Ipad (231)
• 193 empty emails
• 38 Email Feedback with content
• 6 Emails to people at their school
• 4 DropBox Request
• 2 Journal Request
• 2 Exams Request
• Misc Functionality: UX Controls, Notifications, Instructor
Functionality, profile
• 1 Reset password
Android Tablet (18)
• 6 requests to reset Password
• 1 DropBox Request
• 5 EP Configuration Issues (3 from a Park - identified bug)
• 4 Users trying to things that were not supported
7
8. Email Feedback Phones
iPhone
• 180 (Est. 90% of 193) reported no school found - all schools
were identified as MLP users
• 4 requesting more features (dropbox..)
• 1 bug reported - EP configuration issue
Android Phone
• 13 (Est. 10% of 193) emails with no school found - all schools
were identified as MLP users
• EP Configuration Issue
8
10. iPad
In Market since April 2013
Total: 27,819 ( 25,676 in US) Total Downloaded
Features:
• Activity Stream
• Calendar
• Announcements
Course Content
• Discussions
• Gradebook
View Dropbox
• View Doc Sharing
View People & Profiles
10
11. iPad Analysis
• Email feedback design flawed, email sent when users intend to close
• Users want more functionality with top requested feature: Dropbox
• App is impacted by back end configuration of EPs - which Pearson
cannot control, however the app is judged by those factors
• Pearson is a target due to overall perception in market.
• Users do not adjust expectations based on Store literature and MLP
users do not differentiate between MLP and LearningStudio.
11
12. Android Tablet App
In market for 18 months
Total: 713 downloaded (All Time as of 1/11/15) 578 in US
Features:
• Course Content
• Activity Stream
• Upcoming
• Announcements
• Discussions
• Grade book
12
13. Android Tablet App Analysis
• Users regularly need help with forgotten passwords
• App is impacted by back end configuration of EPs - which
Pearson cannot control, however the app is judged by those
factors
• Users do not adjust expectations based on Store literature
• Users do not adjust expectations based on device.
• Additional observation from verbal feedback: “Coming soon”
messages just frustrate users, particularly if the features do not
come soon.
13
19. Downloads & Behavior
19
iPhone Android
Downloads from store 7,018 1,708
Users 4,666 1,943
Sessions 50,212 13,585
Screen Views 111,976 663,604
Screens per Session: 8.24 13.22
Avg. Session Duration: 00:03:39 00:04:09
Avg. Time on Screen 00:00:30 00:00:20
20. Behavior/Events
20
iphone Android
Event Action Total Unique Event Action Total Unique
Replied to a Thread 2,158 1,298 Replied to a Thread 1,362 581
Posted to a Thread 876 597 Posted to a Thread 332 228
Viewed Privacy Policy 107 106 Viewed Privacy Policy 91 82
Started filling a generic feedback 116 111 Started filling a generic feedback 88 66
Toggled between month/week views 959 524 Toggled between month to week views 278 207
Toggled between week to month views 290 190
Changed Read/Unread status manually 23 20
Refreshed activities manually 6,781 4,693 Refreshed activities manually 494 272
Selected a Course as STUD 74,382 35,162 Calendar 10,019 454
Selected a Course as PROF 731 446
21. Rating the apps
21
Iphone Android
Event Action Total Unique Average Event Action Total Unique Average
Rated the app 90 90 4.28 Rated the app 72 66 3.39
Skipped rating the app 121 121 Skipped rating the app 14 14
Started filling a rating
feedback 11 11
Started filling a rating
feedback 9 9
Visited App Store to
rate the app 11 11
Visited App Store to
rate the app 12 12
233 233 107 101
25. Login Errors
25
iPhone Android
Action Total Unique Total Unique
Logged in automatically
Non-SSO EP 34,605 26,907
Logged in manually Non-
SSO EP 3,930 3,798
Logged in manually
Non-SSO EP 1,435 1,290
Logged in manually
SSO EP 12,431 12,173
Logged in manually
SSO EP 1,632 1,447
Logged out manually 824 704 Logged out manually 857 637
Unsuccessful login attempt
Non-SSO EP 1,895 887
Login failed due to
Aggregation Layer / WSOD
issue (Non-SSO EP) 494 384
28. Schools iPhone top 10
28
Iphone Android
Event Label Total Events Unique Events Total Events Unique Events
Texas A&M University-
Commerce 63,533 17,151 1,375 945
SFMC College of
Nursing 29,117 16,500 247 127
Everest University
Online 15,992 8,221 13,508 2,224
Clarkson College 14,728 8,603 285 117
Indiana Wesleyan
University 13,906 2,019 393 230
Upper Iowa University 9,492 5,886 212 79
OSF Healthcare 8,915 5,037 298 170
Rutgers University -
Net ID Login 8,256 1,205 336 161
Santa Monica College 7,834 2,250 338 121
29. 29
Schools Android top 10
Android iPhone
Event Label Total Events Unique Events Total Events Unique Events
Everest University
Online 13,508 2,224 15,992 8,221
Texas A&M University-
Commerce 1,375 945 63,533 17,151
American College of
Healthcare Sciences 564 45 5,387 3,115
University of
Sunderland 441 281 5,909 1,487
Indiana Wesleyan
University 393 230 13,906 2,019
Santa Monica College 338 121 7,834 2,250
Rutgers University -
Net ID Login 336 161 8,256 1,205
OSF Healthcare 298 170 8,915 5,037
Clarkson College 285 117 14,728 8,603
SFMC College of
Nursing 247 127 29,117 16,500
30. Phone Analysis
Analysis
1. iPhone users outpaced Android by 7-1.
2. Iphone and Adroid user shares were not reflective of the general market, nor were
they consistent across schools.
3. Iphone Users were more satisfied with their mobile experiences
4. iPhone Users spent less time on each page
5. App is most judged by performance, regardless if those are factors Pearson cannot
control – i.e. EP configuration
6. Online Phone rates were driven by instructors, not satisfied students
7. Users were much higher satisfied with their experience than the store reviews
reflected.
8. MLP users do not differentiate between MLP and LearningStudio.
9. Primary Pattern of flow behavior was Course name – announcements – grades and
not threads as expected.
10. Responses to threads out numbered initial posts to threads.
11. Audio recording in threads was minimally used
12. Most requested feature was drop box, and not content.
30
31. Summary
iPhone is King
We found that despite market researching showing Android phones and tablets dominate
sales and usage in the general population, students use Apple phones and tablets by a 7-
1 ratio, far outpacing Android Devices. How should development resources be allocated
based on these numbers?
Performance is King
The foremost driver of ratings in the marketplace is performance. It is the number 1 driver
for most behavior on mobile. All feedback, and ratings mention performance in someway
or another. All the issues mentioned were based on configuration external to Pearson
although users place all responsibility on the publisher. Users want more functionality, but
if it impacts performance, they don’t really want it.
Look beyond the store ratings.
Ratings in the store don’t always reflect the actual value of an app. While the ratings
online were not positive, the overall ratings gathered in app were very high. 4+ for the
iPhone app and 3+ for the Android app.
31
32. Summary Con’t
Everyone wants mobile
Most students want a phone app to access information. MLP users
request to use the mobile apps in droves.
Edsel vs. Mustang
Real User Behavior is valuable
As with most customer interviews, what I say I want and what I
really want are two different things. The number one request for a
mobile app was discussions. The most common user behavior was
checking their grades.
32
The Edesel was a car in search of a market, The Mustang was a market in search of a car.
Editor's Notes
No revenues
Ratings and reviews online are usually done by users with extreme views.
Overall satisfaction with the app is different than reflected in the store…more later – in app ratings show different story.
Note: All data is April 2015 through Dec 2015
I recorded posting to a thread with audio recording.
High notes are on Mondays
Next Wednesdays
User flow with drop off
Standard Logon: Login Screen – Launch screen – Course List page – Course Home page
2. Predominant traffic between: Home – announcements – grade book
3. Fall off traffic to threads and some to activity screen
Initial thoughts prioritized discussion above gradebook and announcements.
Same behavior in the iPhone as the Android, although more fall out towards discussions and threads
Importance of tracking Error messages – login errors
1. Finding unreported issues which can impact your market ratings.
Main complaints:
Performance and outages as main complaints
46% of users will not return to a site that performs poorly
1 second faster experience up to a 27% increase in conversion rate. 1 second delay hurts mobile metrics.
The lesson learned was that people responding to the surveys tended to say what they thought other people would want in a car, not what they themselves wanted.
