Session Replay or Screen Recording tools are now part of an arsenal of discovery toolkits that can drive optimisation, bug fixes, funnel and journey analysis - using qual and quant techniques. Without these tools, the analytics data misses emotion, frustration, friction and more - I've collated the best tips, tricks, tools and approaches to yield the most valuable insights for CRO / Growth Hacking.
3. @OptimiseOrDie
1. What are these tools weโre talking about?
2. What stuff do they do?
3. What do optimisers find most useful?
4. Summary
Sep 2015, Ruuhijarvi
4. What are these tools
weโre talking about?
@OptimiseOrDie
5. @OptimiseOrDie
1. What are these tools?
Naming these tools is quite hard!
What are the characteristics?
โข Session recordings (all of them)
โข Attention / Click / Scroll maps (all)
โข Polls, Surveys, Feedback (VOC) (most)
โข Funnel or segmentation tools (some)
โข Form Analytics (some)
6. @OptimiseOrDie
1. Who should you consider?
โข Helping you find out WHY!
Decibel Insight www.decibelinsight.com
Hotjar www.hotjar.com
Inspectlet www.inspectlet.com
SessionCam www.sessioncam.com
Clicktale www.clicktale.com
Mouseflow www.mouseflow.com
Ghostrec www.ghostrec.com
Yandex WebVisor (free) metrica.yandex.com/promo/webvisor
7. @OptimiseOrDie
4. If you have 4 hours
PLUS
โข Snap interviews (Sales, Customer Services, Tech Support)
โข Run a quick poll or survey (See my tools slides)
Less
Bullshit!
9. @OptimiseOrDie
1. Change Growth Trajectory1. Session Recording โ what?
PROS
โข Video recordings of people using the site
โข Vital to โseeโ a qualitative layer with data
โข Spot form problems, annoyance, friction,
validation issues, errors, repetitive or
confused behaviour.
โข Push code live, watch people interacting in
near realtime. Fix stuff โ very lean!
CONS
โข NOT a substitute for form analytics
โข Hard to โviewโ thousands of recordings
โข Will hit site performance!
โข Itโs always an abstraction of reality*
10. @OptimiseOrDie
1. Change Growth Trajectory1. Session Recording โ tips?
โข Record only when you need to
โข Use it tactically โ turn on for the landing page, collect 300
recordings, run them at high speed, crunch in an afternoon
โข Recording all users is fine, as long as you accept the drag on
experiences (especially on mobile)
โข Use the GA DOM timings report to check the hit (ask me)
โข Ensure that the tool records not just mouse but touch, swipe,
zoom layer and the right device groups.
โข Filtering is vital โ if you have shite filtering of session
recordings, itโs now a shite tool vomiting data.
โข JavaScript errors/browser & device problems โ use these tools
to mine for large bugs in your device experiences.
13. @OptimiseOrDie
1. Examples
โข US Underwear Brand x.xM per month, 2 bugs
โข European Travel site x.xM per month, 13 bugs
โข UK Mobile Telco x.xM per month, 3 bugs
โข Travel Aggregator x.xM per month, 5 bugs
โข Gambling / Event site x.xM per month, 11 bugs
โข Total developer time to fix ALL of these = < 5 days!
โข At a cost of ยฃ8000 to find and fix, the ROI was 91,150%
โข In many cases, savings larger than the ENTIRE IT budget
15. @OptimiseOrDie
2. Heat, Scroll and Click Maps
โข All these session recording tools also have heat/click/scroll maps
โข How far do people read down this page? What do they click on? Where did they interact?
16. @OptimiseOrDie
1. Change Growth Trajectory2. Scroll maps โ what?
PROS
โข See how far people scroll down a page
โข Can be used with analytics data to analyse and
optimise read/scrolling behaviour, particularly on
long form pages.
โข Helps spot where you have fold issues with the
line landing on white or solid colour space (users
donโt see cues to scroll)
CONS
โข Biased if people see different grid layouts
โข Needs filtering/segments to bring out power
17. @OptimiseOrDie
1. Change Growth Trajectory2. Scroll maps โ tips?
โข Split by device experience category (mobile, tablet, desktop)
โข Make sure you can split by viewport or resolution (or at
least design breakpoint)
โข If you can integrate your own segmentation, this makes it a
really good tool โ see โhigh spending customersโ segment,
for example.
โข Useful for optimising the bottlenecks and flow problems
sometimes introduced into long page designs.
โข Allows you to focus optimisation on actually where the
eyeball attention is!
18. @OptimiseOrDie
1. Change Growth Trajectory2. Click maps โ what?
PROS
โข Helps you see where people click (that is NOT
ACTUALLY clickable)
โข See broad brush visual representation of data
you can also get from analytics
CONS
โข I find using analytics event tracking to be
easier to get what I want. Looks nice though!
19. @OptimiseOrDie
1. Change Growth Trajectory2. Click maps โ tips?
โข Look for non clickable areas with high volumes of clicks
โข Sort it out!
20. @OptimiseOrDie
1. Change Growth Trajectory2. Heat / Attention maps โ what?
PROS
โข Models where attention is being given on a
page, by deducing this from mouse
movements.
โข Yes โ there is a correlation between mouse
movement and eye gaze (but not big).
CONS
โข There is also a correlation with frustrated
behaviour (wheeling, meerkatting,
dogshaking)
โข Interesting but not as useful as the other two,
IMHO
21. @OptimiseOrDie
1. Change Growth Trajectory2. Heat / Attention maps โ tips?
โข Look for non clickable areas with high volumes of clicks
โข Sort it out!
22. โข One of the most useful yet undervalued parts
of these tools!
โข Insider tip!
โข If you donโt know what to do with this
page/process/step/widget/badly designed
piece of crap โ why not ASK SOMEONE!
โข Polls are one of the easiest ways to remove
bullshit from your organisation.
โข Surveys often turn into a bunfight over how
many useless questions you can add (thatโs a
whole presentation in itself)
โข Snap polls (landing pages, triggers) โ are really
bloody useful.
โข Open ended questions can yield some
surprising stuff
3. VOC / Polls / Surveys โ what?
23. โข Task Gap Survey (above)
โข General site feedback
โข Funnel Abandonment Survey
โข Site Exit Survey
โข NPS and Customer satisfaction
โข Landing Page Polls
3. VOC / Polls / Surveys โ types
Based on
todayโs visit,
how would you
rate your site
experience
overall?
Were you able
to complete
the purpose of
your visit
today?
If NO, why
were you not
able to?
If YES, what
did you value
most about
the website?
Thank You Page survey
โThank you!
Was there one thing that nearly
stopped you from purchasing a flight
today?โ
24. โข If you have integration with Google Analytics (or
another package), tick!
โข If it integrates with your AB testing tool, tick!
โข If it has decent segmentation capabilities, tick!
โข Ask what methods the tool supports (poll overlay,
inpage, survey, exit survey, inline questions etc.)
โข Talk to customers whoโve actually used this part
of the tool
โข Check out my list of qualitative tools at the end of
this deck.
โข Try usabilityhub.com
โข Follow @Contentverve
3. VOC / Polls / Surveys โ tips?
25. @OptimiseOrDie
1. Change Growth Trajectory3. Competitive Advantage
โข When hacking the growth of LOVEFiLM (now Amazon)
โข Regular competitor surveys โ we LOVED this!
โข Survey your OWN customers AND competitors
โข We got a quarterly sample of 1000 surveys from each brand
โข NPS data collected as well as Service Metric Ratings
โข Allowed us to know where competitors were weak
Speed of Email response Question Answered first time
โข Here is one example
โข Spend where it will shift delight
โข Spend where competitors are weak
โข Stop spending where it doesnโt
make a difference
โข Prioritise based on ROI
26. 4. Funnel and Journey Optimisation
โข Some of these tools have very nifty
funnel optimisation capabilities.
โข Too much to cover here, but always
weigh up whether you want to add
meal to your salt (or salt to your
meal).
โข Iโd rather have the tool data in Google
Analytics, to do my analysis there
(using flow reports, ring models,
horizontal funnels, step drops)
โข The data layer collected in your tool
may not be as rich as your analytics
(or collected in the same way) so
decide where you want to integrate!
27. @OptimiseOrDie
5. Form Analytics
โข Where do people pause?
โข What fields or form areas cause
people to abandon you?
โข What errors make them leave?
โข Youโd notice this on a phonecall,
so why not the website?
โข Absolutely vital for maximising
yield of your forms
โข Prioritise your effort around
where the leaks are, not the
whole product!
โThis is so we can call you in case thereโs a
problem delivering your order.โ
29. @OptimiseOrDie
1. What do optimisers find most useful?
1. High fidelity recordings, segmentable, filterable
2. Tactical insights before optimisation โ inspecting small
areas, template groups, funnel steps
3. Polls โ too few people use VOC for instant, lightweight
feedback โ it cuts stupid effort. Bullshit in the office?
โAsk a Customerโ is the cry!
4. Form analytics โ underutilised and yet rich in money
5. Validation and submission errors in forms can be one of
the largest drivers of abandonment.
6. Device experiences = Money
31. @OptimiseOrDie
Summary
โWho cares if your conversion rate is 5%.
You need to care about why the 95% donโt
buy and invest huge effort in understanding
- WHY?โ
@OptimiseOrDie
38. Som, feedback
Crowdsourcing Tools:
Usertesting (P/S)* www.usertesting.com
Userlytics (P/S) www.userlytics.com
Userzoom (S) www.userzoom.com
What Users Do (P) www.whatusersdo.com
Loop11 (S) www.loop11.com
Open Hallway (S) www.openhallway.com
Ethnio (For Recruiting) www.ethnio.com
Feedback on Creatives:
Usability Hub www.usabilityhub.com
Five second test www.fivesecondtest.com
Pidoco www.pidoco.com
Usabilla www.usabilla.com
Conceptshare www.conceptshare.com
* P = Panel, S=Site
38
12. Crowdsource your Research!
Editor's Notes
Iโve been working on this presentation and thinking about it for over two months now. And this is one of the first graphs I wanted to include, because it represents the hype cycle in AB testing.
This is what a lot of companies go through with new technology adoption, so I wanted to show you what the AB testing hype cycle would look like [CLICK]
Iโve been working on this presentation and thinking about it for over two months now. And this is one of the first graphs I wanted to include, because it represents the hype cycle in AB testing.
This is what a lot of companies go through with new technology adoption, so I wanted to show you what the AB testing hype cycle would look like [CLICK]
Iโve been working on this presentation and thinking about it for over two months now. And this is one of the first graphs I wanted to include, because it represents the hype cycle in AB testing.
This is what a lot of companies go through with new technology adoption, so I wanted to show you what the AB testing hype cycle would look like [CLICK]
I once explained to my daughter โ you know, when adults like look really in control and making decisions and appearing not to suffer from indecision? Donโt believe it for a minute โ weโre just better at winging it cause weโre older.
And this is the huge hole thatโs gnawing at the hear of many digital operations. The inability to understand what you can and canโt be confident about โ but nobody wants to admit theyโre guessing a lot of the time.
Hopefully now youโve joined Guessaholics Anonymous from today, you can move on to a data driven and more productive future. Enjoy less wastage in your products, greater efficiency in marketing spend and grow faster than you imagined. All it takes is a bit of love of your customers, your business and some solid ground rules.
Iโve been working on this presentation and thinking about it for over two months now. And this is one of the first graphs I wanted to include, because it represents the hype cycle in AB testing.
This is what a lot of companies go through with new technology adoption, so I wanted to show you what the AB testing hype cycle would look like [CLICK]