1. @OptimiseOrDie
WAW
London Jan
2016

2. • UX and Analytics (1999)
• User Centred Iterative Design (2001)
• Agile, Startups, No budget (2003)
• Funnel optimisation (2004)
• Form analytics (2004)
• Multivariate & A/B testing (2005)
• Conversion Optimisation (2005)
• Persuasive Copywriting (2006)
• Lean UX (2008)
• Holistic Optimisation (2009)
• Psychology and Conversion (2010)
• Velocity & Scaling testing (2011)
• Optimisation Mentoring / Workshops
(2013)
My Timeline
@OptimiseOrDie
Ruuhijarvi

3. @OptimiseOrDie
1. What are these tools we’re talking about?
2. What stuff do they do?
3. What do optimisers find most useful?
4. Summary
Sep 2015, Ruuhijarvi

4. What are these tools
we’re talking about?
@OptimiseOrDie

5. @OptimiseOrDie
1. What are these tools?
Naming these tools is quite hard!
What are the characteristics?
• Session recordings (all of them)
• Attention / Click / Scroll maps (all)
• Polls, Surveys, Feedback (VOC) (most)
• Funnel or segmentation tools (some)
• Form Analytics (some)

6. @OptimiseOrDie
1. Who should you consider?
• Helping you find out WHY!
Decibel Insight www.decibelinsight.com
Hotjar www.hotjar.com
Inspectlet www.inspectlet.com
SessionCam www.sessioncam.com
Clicktale www.clicktale.com
Mouseflow www.mouseflow.com
Ghostrec www.ghostrec.com
Yandex WebVisor (free) metrica.yandex.com/promo/webvisor

7. @OptimiseOrDie
4. If you have 4 hours
PLUS
• Snap interviews (Sales, Customer Services, Tech Support)
• Run a quick poll or survey (See my tools slides)
Less
Bullshit!

8. What stuff do they do?
@OptimiseOrDie

9. @OptimiseOrDie
1. Change Growth Trajectory1. Session Recording – what?
PROS
• Video recordings of people using the site
• Vital to ‘see’ a qualitative layer with data
• Spot form problems, annoyance, friction,
validation issues, errors, repetitive or
confused behaviour.
• Push code live, watch people interacting in
near realtime. Fix stuff – very lean!
CONS
• NOT a substitute for form analytics
• Hard to ‘view’ thousands of recordings
• Will hit site performance!
• It’s always an abstraction of reality*

10. @OptimiseOrDie
1. Change Growth Trajectory1. Session Recording – tips?
• Record only when you need to
• Use it tactically – turn on for the landing page, collect 300
recordings, run them at high speed, crunch in an afternoon
• Recording all users is fine, as long as you accept the drag on
experiences (especially on mobile)
• Use the GA DOM timings report to check the hit (ask me)
• Ensure that the tool records not just mouse but touch, swipe,
zoom layer and the right device groups.
• Filtering is vital – if you have shite filtering of session
recordings, it’s now a shite tool vomiting data.
• JavaScript errors/browser & device problems – use these tools
to mine for large bugs in your device experiences.

11. @OptimiseOrDie
1. Change Growth Trajectory1. Session Recording

12. 1. An Opportunity? Modelling!
22%
65%
13%
92%
6%
2%
65%
34%
61%
37%
2%
100%

13. @OptimiseOrDie
1. Examples
• US Underwear Brand x.xM per month, 2 bugs
• European Travel site x.xM per month, 13 bugs
• UK Mobile Telco x.xM per month, 3 bugs
• Travel Aggregator x.xM per month, 5 bugs
• Gambling / Event site x.xM per month, 11 bugs
• Total developer time to fix ALL of these = < 5 days!
• At a cost of £8000 to find and fix, the ROI was 91,150%
• In many cases, savings larger than the ENTIRE IT budget

14. Browser Checks www.crossbrowsertesting.com
www.browserstack.com
www.spoon.net
www.saucelabs.com
Mobile & Tablet www.appthwack.com
www.deviceanywhere.com
www.opendevicelab.com
Article & Info bit.ly/devicetesting
1. Here is our £84 Million testing rig!

15. @OptimiseOrDie
2. Heat, Scroll and Click Maps
• All these session recording tools also have heat/click/scroll maps
• How far do people read down this page? What do they click on? Where did they interact?

16. @OptimiseOrDie
1. Change Growth Trajectory2. Scroll maps – what?
PROS
• See how far people scroll down a page
• Can be used with analytics data to analyse and
optimise read/scrolling behaviour, particularly on
long form pages.
• Helps spot where you have fold issues with the
line landing on white or solid colour space (users
don’t see cues to scroll)
CONS
• Biased if people see different grid layouts
• Needs filtering/segments to bring out power

17. @OptimiseOrDie
1. Change Growth Trajectory2. Scroll maps – tips?
• Split by device experience category (mobile, tablet, desktop)
• Make sure you can split by viewport or resolution (or at
least design breakpoint)
• If you can integrate your own segmentation, this makes it a
really good tool – see ‘high spending customers’ segment,
for example.
• Useful for optimising the bottlenecks and flow problems
sometimes introduced into long page designs.
• Allows you to focus optimisation on actually where the
eyeball attention is!

18. @OptimiseOrDie
1. Change Growth Trajectory2. Click maps – what?
PROS
• Helps you see where people click (that is NOT
ACTUALLY clickable)
• See broad brush visual representation of data
you can also get from analytics
CONS
• I find using analytics event tracking to be
easier to get what I want. Looks nice though!

19. @OptimiseOrDie
1. Change Growth Trajectory2. Click maps – tips?
• Look for non clickable areas with high volumes of clicks
• Sort it out!

20. @OptimiseOrDie
1. Change Growth Trajectory2. Heat / Attention maps – what?
PROS
• Models where attention is being given on a
page, by deducing this from mouse
movements.
• Yes – there is a correlation between mouse
movement and eye gaze (but not big).
CONS
• There is also a correlation with frustrated
behaviour (wheeling, meerkatting,
dogshaking)
• Interesting but not as useful as the other two,
IMHO

21. @OptimiseOrDie
1. Change Growth Trajectory2. Heat / Attention maps – tips?
• Look for non clickable areas with high volumes of clicks
• Sort it out!

22. • One of the most useful yet undervalued parts
of these tools!
• Insider tip!
• If you don’t know what to do with this
page/process/step/widget/badly designed
piece of crap – why not ASK SOMEONE!
• Polls are one of the easiest ways to remove
bullshit from your organisation.
• Surveys often turn into a bunfight over how
many useless questions you can add (that’s a
whole presentation in itself)
• Snap polls (landing pages, triggers) – are really
bloody useful.
• Open ended questions can yield some
surprising stuff
3. VOC / Polls / Surveys – what?

23. • Task Gap Survey (above)
• General site feedback
• Funnel Abandonment Survey
• Site Exit Survey
• NPS and Customer satisfaction
• Landing Page Polls
3. VOC / Polls / Surveys – types
Based on
today’s visit,
how would you
rate your site
experience
overall?
Were you able
to complete
the purpose of
your visit
today?
If NO, why
were you not
able to?
If YES, what
did you value
most about
the website?
Thank You Page survey
“Thank you!
Was there one thing that nearly
stopped you from purchasing a flight
today?”

24. • If you have integration with Google Analytics (or
another package), tick!
• If it integrates with your AB testing tool, tick!
• If it has decent segmentation capabilities, tick!
• Ask what methods the tool supports (poll overlay,
inpage, survey, exit survey, inline questions etc.)
• Talk to customers who’ve actually used this part
of the tool
• Check out my list of qualitative tools at the end of
this deck.
• Try usabilityhub.com
• Follow @Contentverve
3. VOC / Polls / Surveys – tips?

25. @OptimiseOrDie
1. Change Growth Trajectory3. Competitive Advantage
• When hacking the growth of LOVEFiLM (now Amazon)
• Regular competitor surveys – we LOVED this!
• Survey your OWN customers AND competitors
• We got a quarterly sample of 1000 surveys from each brand
• NPS data collected as well as Service Metric Ratings
• Allowed us to know where competitors were weak
Speed of Email response Question Answered first time
• Here is one example
• Spend where it will shift delight
• Spend where competitors are weak
• Stop spending where it doesn’t
make a difference
• Prioritise based on ROI

26. 4. Funnel and Journey Optimisation
• Some of these tools have very nifty
funnel optimisation capabilities.
• Too much to cover here, but always
weigh up whether you want to add
meal to your salt (or salt to your
meal).
• I’d rather have the tool data in Google
Analytics, to do my analysis there
(using flow reports, ring models,
horizontal funnels, step drops)
• The data layer collected in your tool
may not be as rich as your analytics
(or collected in the same way) so
decide where you want to integrate!

27. @OptimiseOrDie
5. Form Analytics
• Where do people pause?
• What fields or form areas cause
people to abandon you?
• What errors make them leave?
• You’d notice this on a phonecall,
so why not the website?
• Absolutely vital for maximising
yield of your forms
• Prioritise your effort around
where the leaks are, not the
whole product!
“This is so we can call you in case there’s a
problem delivering your order.”

28. What do optimisers find
most useful?
@OptimiseOrDie

29. @OptimiseOrDie
1. What do optimisers find most useful?
1. High fidelity recordings, segmentable, filterable
2. Tactical insights before optimisation – inspecting small
areas, template groups, funnel steps
3. Polls – too few people use VOC for instant, lightweight
feedback – it cuts stupid effort. Bullshit in the office?
“Ask a Customer” is the cry!
4. Form analytics – underutilised and yet rich in money
5. Validation and submission errors in forms can be one of
the largest drivers of abandonment.
6. Device experiences = Money

30. Summary
@OptimiseOrDie

31. @OptimiseOrDie
Summary
“Who cares if your conversion rate is 5%.
You need to care about why the 95% don’t
buy and invest huge effort in understanding
- WHY?”
@OptimiseOrDie

32. 32

34. WE’RE ALL WINGING
IT

35. Guessaholics Anonymous

36. @OptimiseOrDie
Summary
“The greatest enemy of knowledge is
not ignorance, it is the illusion of
knowledge.”
Stephen Hawking

37. @OptimiseOrDie
Email me sullivac@gmail.com
Slides Will be mailed to you!
LinkedIn linkd.in/pvrg14
Mentoring Teams/People - ask!

38. Som, feedback
Crowdsourcing Tools:
Usertesting (P/S)* www.usertesting.com
Userlytics (P/S) www.userlytics.com
Userzoom (S) www.userzoom.com
What Users Do (P) www.whatusersdo.com
Loop11 (S) www.loop11.com
Open Hallway (S) www.openhallway.com
Ethnio (For Recruiting) www.ethnio.com
Feedback on Creatives:
Usability Hub www.usabilityhub.com
Five second test www.fivesecondtest.com
Pidoco www.pidoco.com
Usabilla www.usabilla.com
Conceptshare www.conceptshare.com
* P = Panel, S=Site
38
12. Crowdsource your Research!