A lecture on consideration when launching digital products for technology entrepreneurship students in the New Venture Analytics course

1. Analytics
why, when, what, where, how & gotchas
@niallrochen.roche@ucl.ac.ukGet in touch
Niall Roche

3. why use analytics?
find out who users are
find out how they are using your product (or not)
find out what they are/are not interested in
saves wasted effort or can try differently
either way you have learned something!
can help with innovation accounting
can help improve product with A/B testing

4. provable learnings
compare the data to measure effectiveness new features
cohort analysis
identify new ideas from what people want to do, but can't
pretty dashboards and spreadsheets

5. know your users
where geographically (may not be accurate)
how did they get there (referrals & campaigns)
gender & age (may be implied)
what devices
if from mobile, how does the website look on those devices
what languages are they using
when are they using it (any interesting trends?)

6. creates lots of data
creates data points for further analysis
a data point to make predictions against
can dangerous!
can apply Machine Learning
can be a source of value and revenue
could be used for targeted advertising

7. without analytics
no idea how users are using your site/app/device
hard to prove to investors and prove metrics
how to measure against KPIs?
how to know if something new or different is working

8. when?
start tracking as early as possible (no really!)
missed data can be lost value
especially when it shows something is wrong
learnings as important as the technical aspect
maybe more!
figures don't lie (ish)
figures are not everything (converse with users!)

9. what?
decide what to track
and how to interpret
what are the user journeys to focus on
what goal performance to measure
look crash reports exceptions and bad things
don't forget about some feedback!

10. Industry Baselines
compare with other apps in same category
user baselines sourced from the industry
• app annie
• visionmobile
• Flurry
• apptopia

11. Examine Sessions
look at session length
compare length with journeys
is page wait time an issue?
where do users get bored move on?
user behaviour may be in and out vs linger
supermarket vs convenience store
use at bus stop vs bus
compare behaviour of different types of user groups

12. Examine Funnels

13. where?
identify milestones along a journey
different decision points for user
all major events
should be mapped to KPIs and metrics
if in doubt - track as much as possible!

14. Metrics
• Acquisition
• Activation
• Retention
• Referral
• Revenue
• many many more…..

15. cohorts
• cohort analysis
• match campaigns to cohorts
• ignore vanity metrics!
• no of page impressions + signups not as important as engaged
users
• can always buy users to visit your site
• what campaigns convert into paying customers for lowest
acquisition cost
• useful when calculating lifetime value of customers

18. how?
weblogs
limited but useful
event trackers
javascript event tracking
cookies to track repeat visitors
session capture & replay with heatmaps

19. tools
Google Analytics
MixPanel
Liquid
Flurry
Crashlytics
AppSee
UXCam
AppAnalytics
UsabilityTools
Localytics
Segment
Kissmetrics
customer.io
mparticle
many more

20. A/B Testing
implementation options
AB Tasty
Convert
Optimizely
Unbounce
many analytics tools have simple implementations built
in too with reports on variance

21. potential pitfalls

22. App pitfalls

23. app launch events
app icon
spotlight
push notification
from other app / device
url scheme
intent (Android)
Document Handler
proximity trigger
geofence
beacon/iBeacon/eddistone/NFC
system event (low battery)
background mode may bypass
initial start logic

24. multiple paths
A -> B -> C is not always straightforward
may be multiple routes to same point in app/site
how did the user get there?
Tab / Swipe / Press / Gesture
from Hamburger menu, context and other menus
tracking context of how an event was reached

25. • tracking signed in web users mush easier than using cookies as
they may not span different devices
• identify hits from bots / screen scrapers / web search tools /
developers & testers / uptime checkers
• page load time measurement (especially in areas where there are
slow connections such as emerging markets, and poor
connections)
• look for different behaviours and trends
• compare session length vs onboarding
• trying to determine reasons for dropoff

26. ga('send', {
hitType: 'event',
eventCategory: 'Video',
eventAction: 'play',
eventLabel: 'cats.mp4'
});
//for Single Page Apps
ga('send', 'pageview', '/
new-page');
Category -> Action -> Label + extra if needed

27. technical problems
• some technical issues for websites
• leaving page on form submit
• careful with placement of code
• missing images, load events may not fire
• add timeouts for error handling

28. actioning with data
platforms like Liquid (onliquid.com) can adapt an app
experience based on data
identify variables to change inside the app
trigger an actions on a funnel of users (e.g push
notification)
set rules to automatically intervene based on trends
personalised experiences

29. Analytics
why, when, what, where, how & gotchas
@niallrochen.roche@ucl.ac.ukGet in touch
Niall Roche