The Definitive Guide to Qualitative Analytics

Quantitative Analytics
Qualitative Analytics
OPTIMIZE YOUR APP WITH
APPSEE UX APP ANALYTICS
The Definitive
Guide to Qualitative
Analytics

Introduction
Why Did We Write This Guide?
There’s a unique transformation occurring in the realm of mobile app
analytics. Analytics are not only becoming more specified but also more
robust in terms of what mobile teams are able to track and analyze.
Specifically, now app professionals have the ability to actually see and
qualify specific user behavior instead of just broadly defining it by numbers.
This is thanks to the advent of qualitative app analytics. We wanted to help
people understand exactly why this change is happening, what it means,
and how they can harness its full value.
Who is This Guide for?
This guide is for mobile product managers, CTO’s, mobile developers, UX
researchers, UI designers, mobile marketers, mobile startup leaders, or any
other type of “appreneur.” You do not need to have a Masters in Statistics
or Data Science in order to get the most from this guide. All you need is the
ambition to build the best app possible and the willingness to learn and
grow. Now, let’s get to the good stuff…























The Rise of App Analytics | Page 4
The Rise of App Analytics (cont.) | Page 5
Quantitative Analytics: A Break Down | Page 7
The Shortcomings of Quantitative Analytics | Page 10
The Need for Qualitative Analytics | Page 12
Qualitative Analytics in Action (Use Cases) | Page 15
Comparison Graph: Quantitative vs. Qualitative | Page 21
Table of Contents



It’s hard not to gape at the sheer mushrooming of the mobile app
ecosystem. As Eric Schmidt, Executive Chairman of Google, so perfectly
put it, “The trend has been mobile was winning; it’s now won.” Just think,
Schmidt said this back towards the end of 2013- almost three years ago.
Look how much growth has even occurred since then! The stats are
extensive and astonishing.
To start, US daily smartphone screen time has actually exceeded television
time. On top of that, now 80% of internet users own a smartphone and of
those users, 89% of their media time is spent within apps. By 2020, App
Annie projects that gross revenue across all app stores will exceed $101
billion globally. Plus, in terms of simple annual growth, from 2014 to 2015
global total time spent in android apps grew by a staggering 63%. The
amazing statistics go on and on. We can confidently look back over the last
3 years and reaffirm that mobile has definitely won but we can also say that
apps have been the most valuable player!
The Rise of App Analytics
What is qualitative analytics?
Page 12Page 4



Page 5
Just how have apps risen as the “most valuable player” in the mobile
industry? An upsurge of unique, innovative products and increased
smartphone adoption certainly play a role, but are just the tip of the
iceberg. Under the surface practically every app uses some sort of app
analytics.
Why? Well to start off the concept of “If you build it, they will come” lasted,
if anything, for a hot minute. In this exploding app ecosystem, mobile
publishers quickly learned that if they want to stand out from the millions
of other apps out there, they can’t just rely on building a “great app”. They
must engage, monetize, and retain your users (who are some of the most
selective and impatient users the digital world has ever seen). The only
way to successfully do this is to know what’s going on inside your app- and
you can only obtain this knowledge via app analytics. Even interviewing
your potential users and holding focus groups won’t give you the insights
you are seeking. As Jakob Nielsen so perfectly worded it, you must “pay
attention to what users do, not what they say.” By gathering valuable data
on your app’s real users right from the get-go, you will optimize your app
more efficiently and with more certainty.
Thanks to the explosion of apps and speedy specification of apps’ needs,
there are a multitude (hundreds) of app analytics options out there. While
these systems might differ greatly in specific attributes, capabilities, plans
& pricing, and platform support, they typically can be categorized as
“Quantitative” (traditional) app analytics.
The Rise of App Analytics (cont.)
Page 12

/22The Definitive Guide to Qualitative Analytics
























But staying true to the spirit of the rapidly evolving mobile app world,
there’s already a “new” category of app analytics on the scene-
“Qualitative” app analytics. Now the big question is which one is going to
deliver the type of exceptional value that will allow you to fully understand
your users and what they’re doing, effectively optimize and troubleshoot
your app, and ultimately keep your app not just up to par but prevailing
in the ultra-competitive app ecosystem? In this guide we will illuminate
the characteristics and capabilities of both types of analytics, explore
the reasons behind the advent of Qualitative app analytics, and help you
successfully conclude which type of analytics best suits your needs. Let’s
get started.
Page 12
Page 6



So let’s start with what exactly is “Quantitative Analytics”? To obtain the best
understanding, let’s first look at the definition of the word “quantitative.”
Merriam Webster notes the definition as follows:
1. of, relating to, or expressible in terms of quantity
2. of, relating to, or involving the measurement of quantity or amount
3. based on quantity; specifically of classical verse: based on temporal
quantity or duration of sounds
Numbers, numbers, numbers- that is the crux of the definition. So when
it comes to quantitative app analytics, it means you are analyzing and
gathering data in terms of quantities. All of the data and information
obtained through this type of analytics can be measured and written down
with numbers. Typically, one would utilize quantitative analytics to get hard
aggregate insights on certain user actions and trends. Below are some
examples of when you would use quantitative analytics:
Quantitative Analytics: A Break Down
Page 7
Page 12



1. Basic usage reporting
a. How many total users do you have?
b. How many times is your app launched on a daily basis?
c. How many new users do you have? What is the ratio of new to
returning users?
d. How long is the average user session?
e. How many users never return to your app after their first session?
f. How many of your users come from a particular country, a particular
campaign, a particular site?
2. Basic conversion reporting
a. How many users made it from Step A to Step B within a specific
conversion funnel?
b. How many users dropped off between Step A and Step B within a
specific conversion funnel?
c. How many users completed a certain conversion funnel? What is your
CR for a particular funnel?
3. Basic user trend reporting
a. How many times do users use a particular feature?
b. How many users quit your app on a particular screen?
c. How many users accept or deny a particular in-app permission?
Page 8
Page 12

























Notice a trend? The term “how many” is often used when asking
quantitative oriented questions. Now although the fundamentals of
quantitative analytics are to gather large amounts of numerical data,
let’s not forget how far quantitative analytics has come in terms of how
those numbers are obtained and displayed. Today, the technique of data
visualization, the presentation of data in a pictorial or graphical format,
plays a big role in quantitative analytics. Basically it enables mobile
professionals to better grasp difficult concepts and identify new patterns
within all the data on their app. There is no doubt that data visualization can
certainly help you pinpoint trends and correlations within your data, but at
the end of the day it cannot tell you the “whys” behind all those numbers.
This leads us into our next section: The Shortcomings of Quantitative
Analytics
Page 12
Page 9

Numerical data is of course important, but there is so much more that
you can (and need) to learn about your app. Essentially, this is where
quantitative analytics falls short, as it only satisfies your number based
inquires. As noted earlier, quantitative data can help you pinpoint
numerical-based concerns such as quit rate of certain screens, but cannot
really go beyond that in terms of analysis.
For example, let’s say you have a quit rate of over 50% for your login screen.
Clearly the majority of your users are not wanting to login to your app-
that’s certainly important to know. Unfortunately, with quantitative analytics
you have absolutely no idea as to why they are quitting. No matter whether
the data is displayed in a table or as a beautiful pie chart, at the end of
the day it is still a quantitative measurement and nothing more. In order to
hopefully understand the reason(s) behind this particular metric, you will
probably have to invest many hours, implement multiple tests and design
iterations, and potentially conduct additional primary user research. That’s
a lot- all just to get one answer for one bit of data.
The Shortcomings of Quantitative
Analytics
Page 12
Page 10

























Ultimately, a table of data can only take us so far, and often times the most
actionable insights cannot be deduced from numeric data. Due to the fact
that quantitative analytics was practically the only solution on the market
since the onset of mobile analytics, many mobile professionals initially
presumed that they were tracking and measuring their app in the best
way possible. Logically as the mobile app market proliferated, technology
evolved, and user experience standards skyrocketed, mobile professionals
quickly began to notice that “something” was missing in terms of their
analysis of their app. Their numeric data is only supplying them with a
glimpse of what is going on in their app. And of course, as anyone would,
they want the full picture.
Enter- the need for Qualitative App Analytics.
Page 12
Page 11



To properly grasp the essence of qualitative analytics, let’s again start out
with a simple definition:
1. Pertaining to or concerned with quality or qualities
2. Of or relating to how good something is: of or relating to the quality of something
Can qualities such as the color of one’s eyes, feelings on one’s birthday,
or softness of one’s cat be delineated by numbers? No- these are traits,
sentiments, and conditions.
So when it comes to the analysis of your app, what is the one essential
element that cannot be represented by numbers?
User experience.
User experience (UX) encompasses all aspects of the end-user’s
interaction with your product. Because the UX of your app is a qualitative
measurement, you need to essentially see what your users are
experiencing and how they behave in order to actually understand it.
Only qualitative app analytics, or as many now call it, “UX app analytics”
can measure this extremely critical and nuanced KPI.
The Need for Qualitative Analytics
Page 12

Capturing Visual Data
Drilling down to the individual
user level
Displaying Data in a
Comprehensive Manner
USER RECORDING
00:23
1
3
2
SETTINGS
Send feedback
Sign out
About
CHECKOUT
Phone
Name
Address
Crashing Screens
Touch Heatmap
*From this point on you will see us utilize
“Qualitative Analytics” and “UX App Analytics”
interchangeably. Fundamentally, UX App Analytics
is a type of qualitative analytics. In the case
of mobile app software, qualitative analytics
specifically focus on assessing user experience.
Let’s take a look at the key pillars that
define the essence of UX app analytics*.
Page 12
Page 13

The Need for Qualitative Analytics
(cont.)
0%
10%
20%
30%
0-3 sec 3-10 sec 10-30 sec 30-60 sec 1-3 min 3-10 min
Newusers
Time spent within App
Create account
Login
NEW USER SESSION
PLAYBACKUSER #0014
00:23 iPhone 6
iOS 8.1.2 wiﬁMay 2 4:24pm
TIMELINE
Action Time
Login 00:00
00:10Shopping List
(Appsee’s User Session
Recording Feature)
Quantitative VS Qualitative
So now you have a basic comprehension of:
1. Why there’s a considerable need for qualitative analytics.
2. What are the foundations of qualitative analytics.
To further highlight the power of qualitative analytics for your app, take
a look at this common app assessment scenario and the kind of data
obtained from each type of analytics. Below, a mobile app professional
is trying to understand new user behavior within their app.
Page 12
Page 14



Qualitative Analytics in Action (Use
Cases)
Not only can qualitative analytics greatly help you with analyzing and
optimizing your user onboarding experience, it can also address a plethora
of app optimization concerns.
Let’s dive in to some specific case studies:
Crashes
You can save your dev team over “20% additional
hours” like eBay did by harnessing the power
of qualitative analytics for crash analysis and
repairs. With qualitative data on crashes, such as
crash video recordings paired with symbolicated
crash reports, mobile development teams no
longer have to pour over crash logs for endless
hours. Instead, they can see the exact sequence of actions that led to
the crash with their own eyes and then easily pinpoint the precise line
of code that needs to be fixed. No more guesswork when it comes to
understanding and resolving crashes- which means a better overall user
experience for your users.
Page 12 Page 15



Checkout Process
Any m-commerce app understands the utter
importance of their checkout process and
screen. The slightest flaw or confusing element
can cost them a sale. In order to effectively and
thoroughly enhance this critical facet of your app,
you must go beyond quantitative data. Exactly
why did your user abandon purchase? Did they
experience a payment issue, did a form field freeze on them, did they have
trouble adding a promo code? All of these questions can only be answered
by UX app analytics. With UX app analytics you can filter users that did not
end up making a purchase and watch their session recordings. In seconds
you'll be able to detect the issue(s) that hindered them from converting.
BUY
User Support
Maybe you have a unique IoT app that allows
users to control their home heating system or
monitor their baby’s vitals? Your user support
team is even doubly crucial in this case seeing
as your app regulates a real-world, tangible
device in a realm that is relatively new in the
digital world. Basically, your mobile application is
your user’s home base and communicates the value of your IoT device. To
maintain a high level of performance, connectivity, and usability, your user
support team needs to know exactly how your users are behaving within
your app and what they are experiencing. Again, user experience is not
something that can be represented by numbers. By watching user session
recordings of real-world interactions with your app, your support team can
see the exact experiences of users, which users were affected by particular
issues, and reach out and resolve those matters accordingly. This can allow
your support team to be drastically more precise, personal, and prompt-
meeting your users’ high standards and even exceeding them.
Page 12 Page 16



Search Accuracy
For any app with an extensive search function,
such as the standard travel app, you need to
remain on top of your game in terms of search
accuracy, auto-correction/predictions, and
search display results. Whether your search
function is fully optimal can make the difference
between a conversion and a drop-off. What are
the most often searched terms, how are the results appearing (are you
providing too many results per page?), are users confused by results or
straining to find the search they want? Surprise- another instance where
only qualitative analytics can provide you with the actionable insights you
need regarding your search function.
Secure Sign-Up
The majority of financial and banking companies
are going mobile nowadays and it makes total
sense. Users want to be able to access banking
and financial functionalities on the go, and not
have to wait to physically be at a desktop. Yet
that doesn’t mean that there isn’t a sense of
apprehensiveness from the user when it comes
to logging in to these types of apps, especially due to mobile security
issues. Basically, a secure sign-up/account login screen can make or break
the adoption of your app and retention of your users. Maybe your users are
struggling to complete a certain “CAPTCHA” or password confirmation? Or
maybe your users are quickly abandoning your app due to a “suspicious”
login glitch? Once more, you can utilize qualitative analytics to understand
how your users are experiencing your app and affim that your signup/login
screen remains a safe, gateway and not a barrier.
Page 12 Page 17



In-app Ads
Let’s say you have a sports app, with your main
source of revenue being in-app ads. To put it
simply, no matter how much your monetization
tool and programmatic ad network “promise” that
they provide the best control and transparency in
the business, it is oftentimes not enough. When it
comes to monitoring your in-app ads, there are a
lot of moving pieces to keep track of, especially
if your app has a large amount of impressions to fill. Once an unsuitable ad
sneaks its way into your app, it can damage your brand name, ruin the UX,
trigger app abandonment, and spark negative reviews. A qualitative UX
tool, like user session recordings, can allow you to see through your users’
eyes how the ads (from banners to videos) are actually appearing on your
app’s screens.
These insights can reveal whether an ad potentially interrupted your users’
experience/journey, had a technical or quality issue, and/or led the person
to instantly quit your app among many other results. Many top apps are
beginning to harness the potential of qualitative analytics for monitoring
of in-app ads. For example, when the renowned sports app 365 Scores
utilized UX app analytics to monitor there in-app ads, they were able to
decrease user complaints by 44% and discontinue work with two unsuitable
ad networks.
Page 12 Page 18



Gesturization
Let’s say you have a gaming app with a unique
gesture pattern. You want to analyze how your
users grasp this gesture, their natural movement
on your screens, and whether the gesture is
ultimately the best choice for your user base.
Maybe users from a specific country interpret
the gesture differently than another geo? Maybe some users are lacking
the fine motor skills to successfully execute this gesture? Or maybe some
users are just frustrated with the gesture? Numerical data is not going to
provide you with actionable answers on your gestures. UX app analytics
systems, like Appsee, will provide you with the qualitative answers you
need to optimize your gestures and app usability. Via Appsee’s user session
recordings and automatic tagging of specific gestures, you will be able to
see for yourself exactly how specific users complete or don’t complete your
gesture. On an aggregate level of analysis, Appsee’s visual touch heatmaps
can highlight all the gestures users are completing on each screen and
where they focus the most. It can also reveal every instance and location
of an unresponsive gesture on each screen. Innately, gestures are a very
physical, nuanced user action that in order to be properly optimized, must
be analyzed on the qualitative level.
Page 12 Page 19



Fake Installs
Many mobile teams will run paid campaigns
to acquire downloads/new users. This is a
great tactic to boost visibility and increase
your user base, but it also has its inherent risks.
Unfortunately, there is always the potential that
some acquired users from paid campaigns can
actually be fake users/bots. The only way you
can truly separate the “wheat from the chaff” is by actually seeing how
users behave in your app. Surprise- you can only accurately execute this
by employing qualitative analytics. Qualitative analytics will allow you to
effectively pinpoint behaviors (or lack of behavior) that is characteristic
of fake users and then identify which source they originated from. This
will enable you to fine-tune your work with particular ad networks and
significantly boost your ROI.
Page 12 Page 20

In order to fully grasp the value of qualitative analytics, you must
understand that it is a powerful complement to quantitative analytics. As
discussed, quantitative allows you to identify important trends, issues,
and actions within your app on a numerical basis. Then, qualitative
analytics augments this data by supplying the crucial “whys” behind the
numbers. Basically, in order to paint the clearest picture of what’s going on
inside your app, all successful qualitative analytics systems reference at
least some quantitative data.
Below we have another nifty visual that illuminates how qualitative
analytics effectively expands on quantitative.
The Marriage of Quantitative and
Qualitative
QUALITATIVE
QUALITATIVE
Funn
el Metrics Crash
M
etrics
Usabilit
y
M
etrics User Navigati
on
Metrics
W
hy
did
users
drop
oﬀ
your funnel at a
certain
stage
W
hy
did
a
crash
occurand
w
hatsequance
ofevents
led
up
to
the
crash
H
ow
are
your users
actually
using
your app
and
w
here
do
they
encounter usability
issues
H
ow
is
each
usernavigating
through
yourapp
and
w
hatdoes
theirjourney
look
like?
QUANTITATIVE
Page 12
Page 21
























So are you ready to get the ball rolling and gain the best insights possible on your
app? Choose the definitive leader in qualitative app analytics- Appsee.
With Appsee's unique user recordings and touch heatmaps, you can make the
most informed decisions regarding your app's user experience and spend less
time sifting through mountains of data. Appsee also automatically detects all
screens, buttons and user actions within your app without requiring you to select
what to measure in advance. No more asking questions of your analytics system-
just pure, unambiguous answers regarding your app and its users.
Integrated with Fabric? You can easily install Appsee’s free trial with just one click.
You can also simply begin our Free Trial via our site. All it takes is one line of code
and less than 5 minutes. Now that’s user experience at its finest.
Get Started Now
Appsee is trusted by these amazing mobile teams:
The Industry Leader in Qualitative App Analytics
info@appsee.com
www.appsee.com

The Definitive Guide to Qualitative Analytics

More Related Content

What's hot

Similar to The Definitive Guide to Qualitative Analytics

The Definitive Guide to Qualitative Analytics