The document outlines techniques and strategies for optimizing customer behavior through conversion rate optimization (CRO) and testing in the context of digital marketing. It emphasizes the importance of usability testing, A/B testing, and multivariate testing to increase revenue, enhance customer satisfaction, and improve overall business efficiency. The author shares insights from their experience at Belron, highlighting the testing culture, iterative design processes, and the significant return on investment achieved through these optimized practices.

1. @OptimiseOrDie
Craig Sullivan
Group CX Manager
Conversion Conference London Belron®

2. Obligatory promo slide
@OptimiseOrDie
• Conversion Rate Optimisation • Contact deflection and
• A/B and Multi-variate testing online self service
• Cross channel optimisation • Site search analytics
• User Centered Design • Site, Page and Campaign
optimisation
• Usability testing
• Test design and execution
• Customer Research & Insight
• A long usable site portfolio
• Web Analytics
• ROI on UX improvements
• Browser and email
compatibility • Over 19 years of really
boring meetings
Group CX Manager for Belron® (Autoglass ®)
I get to do it with crowds

3. OPTIMISATION
A set of techniques, implemented in
order to influence customer behaviour
towards these outcomes:
• Increased revenue or profit
• Increasing NPS (Customer Sat) scores
• Lower cost for business or contact centres
• Increasing productivity or labour flexibility
• Simply delighting customers
We may fail with 90% of what we try.
The 10% of winners let us shift
behaviour hugely and measurably.

4. ®
Belron Testing Culture
@OptimiseOrDie

5. If you go to the men's washrooms at the
Schiphol airport in Amsterdam, you may
notice there's a fly in the urinals. It’s
screen printed .
So what do you think most men do? That's
right, they aim at the fly when they
urinate.
They don't even think about it, and they
don't need to read a user's manual; it's
just an instinctive reaction that means
85% less spillage!
The interesting feature of these urinals is
that they're deliberately designed to take
advantage of this inherent human male
tendency.
This is my job.

6. Small team, big challenge
• Team of 3 - 2 Optimisers, 1 x part time designer
• Higher ROI or lower costs, without spending more
• Stuff that doesn’t hinder SEO, PPC, Performance
• Measurable ROI - not ego or opinion
• Fast return and low cost techniques
• Over 26M page tests, 20 international tests live now
• Cost is few hundred K – ROI is > 50 x investment
• Yes, that’s more than 5,000% back.

7. Primer on testing
=

8. A/B test
Control Version A Version B

9. Multi variate Pizza
Ingredients Experiment Recipes
=

10. Month 1 - results
+2% +3% +1% +2%
+9% +4% -2% +1%
+5% +13% -7% -9%

11. So what about ingredients?
+20% -2% +5%
+3% +2% +6%
+6% +10% -18%

12. Weed out some losers…
+2% +3%
+9% +4%
+5% +13%

13. The final…
+10% +13%

14. Our testing program
@OptimiseOrDie

15. Usability & Customers are the glue
• Usability testing – hundreds of participants, 6 countries so far
• Foolproof – you can’t kid a kidder
• Diary studies
• Remote testing
• Analytics data
• Customer insight – contact, customer feedback, research
• Clicktale recordings
• Business owners – market data, customer mix, local knowledge
• Customer contact centre immersion (listening)
• Books, Websites, Tips & Other Optimisers
• Old Test Data results (e.g. Click here, Get started, Fix my glass)
• Making lots of mistakes – be brave, fail faster, learn stuff

16. What’s our testingand SpokeThe traffic picture
Hub mix – model – homepage traffic

17. What’s our testing mix?
• A/B testing – Homepage, Landing pages, Funnels
• MVT testing – high traffic landing or home pages
– Mini (less than 8 variables)
– Midi (8-32)
– Maxi (>32 variables)
And 6 types of specialised testing:
• Funnel testing – bedding in new funnels
• Landing page optimisation
• Cross channel testing
• Iterative feature and product testing
• Micro testing
• Telephony MVT

18. Usability & Optimisation process
Final Usability Final Release Legal review
prototype issues left changes build kickoff
Signoff Cust services
Instrument Marketing
(Legal, Test Plan review
analytics review
Mktng, CCC) kickoff
Instrument
Offline End-End Launch
Contact QA testing
tagging testing 90/10%
Centre
Launch Monitor < 1 Launch
Go live 100% Monitor
50/50% week 80/20%
Analytics Washup and New New test Rinse and
review actions hypotheses design Repeat!

19. International testing examples
@OptimiseOrDie

21. UK Wave 2 – 21/9 to 26/10

27. Another example

28. Testing and UX advantages?
@OptimiseOrDie

29. Your ‘design’ team?
Web/Online
Pesky Meddling Kids?
Marketing
Who
designs this
stuff
anyway?
Brand
IT
THE
CEO CUSTOMER Legal

30. Benefits of UX combined with Testing:
• Reduced ego and opinion (execs + others)
• Measurable ROI
• Faster time to market
• Increased conversion in the pocket early
• Iterative and rapid product lifecycles
• Faster, smaller, focused, measured releases
• Continual testing = continual improvement
• SLED vs. Agile = faster return, better focus
• Less reliance on anecdotal evidence
• Business owners part of testing culture
• Introduces a testing culture
• Large cross silo team now gets involved
• Helps us hit the sweet spot...

31. Designing for multi-platform
@OptimiseOrDie

32. Insight - Inputs Ringback
Phone Email
Retargeting Chat
Mobile
TV handsets
Display Apps
Affiliate
Exploding Tablets
Channels
Desktops
Direct
Social &
SEO Micro
Partners
PPC
DM

33. Insight - Inputs
Forms
Remote Search
analytics Depth
testing analytics
interviews
Eye tracking Market
research
Customer A/B and
contact MVT testing
Social
analytics
Insight Unstructured
data
Web
Clicktale
analytics
Usability Competitor
testing Ops and CS Customer evals
teams Surveys services

34. User Centered Design
Wireframe
Research Concept Prototype
Analyse Test

35. Multi channel and platform
Washup
Mobile
Web 1
1
UX
Insight Washup
Design
Washup App 1

36. What were the benefits?
• Very fast build – post research: • Design product and contact
• 10 weeks to build 3 cross options for revenue
platform booking & sites • Cross channel synergy really helps
• Focused on the user (but not work out issues
driven by them) • Higher overall grasp of service,
• Customer services, the process, lifted barriers, less pain...
the contact channels – a BIG part • Big increase in incremental sales
of the design team effort • To get multi-channel right, you
• Align business and tech capability need to design across them.
with customer ‘models’ • Lo budget testing can start with
• Rapid iterative design and build = paper, webcam, £40!...
fast feedback

37. People images that suck…

38. Photo Guidelines
• Groups, vans, stores, corporate guff = • Female images work best in almost
NO
every country – they slay the guys
• Single person – not groups or even
person + customer • Natural, authentic and not ‘model’
• Open, Friendly, Natural, Smiling,
or ‘stock’ image types work best for
Engaged = YES people
• Uniform – has huge effect, even • Photoshoots and tests now being
wearing branded ball cap improves used to drive advertising strategy
conversion • Pre TV test of potential ‘actors’ on
• Look straight at viewer or slightly website? X factor meets split
towards CTA testing?
• Plain backgrounds only
• We get better images now, but we
• No folded arms, arms on hips, pointing, keep working on the guidelines!
scowling, cheesy smiles, hands behind
back, hanging useless by the side, • We still got a genital framing
clasped by groin, fiddling with hands picture the other week
• Arms with prop instead – hold a
clipboard, rag, tool – I’m working for
*you*

40. Cross channel mini studies
• Mini studies:
– Abandon testing
– Emergency phone switchoff & Overspill systems
– Contact deflection systems and self servicing
– Scripting Experiments
– APIs and other systems integration
– Chat testing
– Phone tracking

41. Abandon testing
Reduce unwanted phone hangups. Test plan
– Remove corporate guffspeak – Two countries, big phone systems,
– Don’t mention the website, esp. – Test variants of messages, music,
if this is the web number (D’oh!) timings, amounts quoted
– Don’t give a generic or corporate – Messages make a big impact –
drone apology message experiment freely with volumes
– Quote real KPIs and timings (ING – Opportunity is large due to high
direct, average queue time) call volumes we have
– Make routing emotionally driven – What people hate personally vs.
– not confusing categories that what those same people have on
map poorly onto customer needs their company lines, lol
– Experiment with queue timing – – On a mission here!
find sweet spot, don’t overquote

42. Switchoff Overspill
If you have spikes, have a throttle: Overspill systems:
– Busiest TV & Radio campaign so far, Q2 – Get a UX designer to make you a fast
2011 for Autoglass® entry system
– Designed a phone ‘switchoff’ system, – Make them simple, easy to learn and
permanently on tap
leverage your web platform
– Routed majority of contact to a web
form – Give API access to overspill
– Prevented a backlog situation from companies
developing – Sometimes heavy IT integration isn’t
– Bought time and still exceeded needed
customer expectations (<4h excellent) – “Nobody needs a freaking training
– Used twice to dampen inbound call course to use facebook” - Make sure
volumes your internal apps and call centre
overspill systems are built for instant
learnability
– This could save millions in process
efficiency. Same as web, really.

43. Contact deflection & self service
• People like immediate gratification! • What about your measurement?
• The iceberg of contact opportunity • Basics – Ratings, Did you find this
• Make people like you for not talking answer helpful?
to them on the phone –double win • Intermediate – Tracking behaviour
• If you stick it in help, will it go away? and contact channels
• No elephant graveyard, please (FAQ) • Add tracking of self service events or
• Instrument for analytics and ROI behaviour
• Set it up to measure and improve • Set up a game or reward structure for
staff
• Gamification of content? Adoption?
• Continually improve and get proof
• LOVEFiLM Help Centre:
• Add phone tracking - make avoidable
– Ratings, Deflection tracking, self contact visible
service tracking
• Remove all barriers to self servicing
– V1 :Please view in your account for high cost/volume contacts
section, V2 : Please visit your
account, V3 : Please check your • Try Amazon help – where is phone?
next billing date, V4 - Final • Help experts? Sticky Content are
version – to show the real info : brilliant (www.stickycontent.co.uk)
“Your next billing date is 1 Feb”

44. Script Optimisation
• How can we optimise phone conversion?
– Dedicated test teams
– Script testing often limited range/shift in companies now
– Now – we can use A/B or MV testing, with API integration
– Try out 3,4,5 different versions of key areas or entire script
– ‘Wrapping’ sends XML message event, triggers analytics tag
– Analyse in the same way as an A/B or multi-variate web test
– Fast optimisation of handling scripts, using a mini lab
– Should yield 3-5% higher performing scripts, minimum
– Shows what variables drive script performance globally
– These emotional and psychological triggers can be mined
– Aim for Q2 2012 for a proof of concept

45. Platform APIs
• Use of web platform APIs now for:
– Contact centre messages
– Facebook app
– Mobile site and apps
– Scheduling
– Analytics tag firing* (soon)
• Just making APIs, automagically makes:
– People use them
– Improvements happen
– New ideas happen, that you didn’t expect
– Incompatible stuff get glued together
– Complex IT integration projects go away

46. Chat Testing Ringback
Contact
Form
Contact
Us
You must measure behaviour
across contact channels
during tests like these
A - Chat B - Chat C - Chat
Split
A B C

47. A - Chat Free B – With Chat
Damage 0844 371 0047 Damage 0844 371 0051
Appt 0844 371 0048 Appt 0844 371 0052
Details 1 0844 371 0049 Details 1 0844 371 0053
Details 2 0844 371 0050 Details 2 0844 371 0054

48. Chat testing
• 1 month trial – properly A/B tested • Customers happy, yes, but not in a
• Measured all the contact channels for good business way
movements – what happened?
• Like putting your customer service
• Over a week, chat saves 17 calls – desk into the street
yay!
• You must measure influences in
• But swapped that for 329 chats @
contact volumes when trialling
8.25 minutes. Ouch.
Chat
• Increased ringback request & Email
form about the same. • Be very careful – prove your ROI
on any pilot.
• No statistically significant impact on
conversion • Only 7-10% use and out of those,
• Assumption was big impact, huge less than 50% are influencable etc.
impact! etc.
• Anecdotally, a wonderful tool. • The productivity Myth of chat
Agreed. • Your situation may differ – please
TEST!

49. Call tracking
How it works:
• Rent a bank of numbers
• Add a dynamic ‘tag’ to your page
• Phone number appears automatically
• Customer calls this number
• Captures the keywords, landing page, converting page, traffic source
• Geo mapping captures customer location
• When the call is hung up, optionally record the outcome code (sale, loss)
• Have a special number for Facebook, SEO, Adwords traffic etc.
• Lets you analyse from web -> phone call -> wait time -> answer ->outcome
• Builds a link between web and call, finally

50. Call tracking
Why you should use it!
• Most people under-report on web driven phone contact or sales
• You can’t work out your advertising effectiveness, if you don’t include
phone
• If you don’t measure phone, your A/B tests may be wrong
• You can capture all your web AND phone, in one easy interface
• Allows you a multi-channel view of all your contact sources
• Lets you record offline campaigns as well with specific numbers (print, TV
etc.)
• You can experiment with routing, messages, on hold music – much more
easily
• If you can’t afford lots of phone numbers, this is an easy way to get
tracking

51. Company Website Coverage
Who?
Mongoose Metrics* www.mongoosemetrics.com UK, USA, Canada
Ifbyphone* www.ifbyphone.com USA
TheCallR* www.thecallr.com USA, Canada, UK, IT, FR, BE, ES, NL
Call tracking metrics www.calltrackingmetrics.com USA
Hosted Numbers www.hostednumbers.com USA
Callcap www.callcap.com USA
UK, SE, FI, NO, DK, LT, PL, IE, CZ,
Freespee* www.freespee.com
SI, AT, NL, DE
Adinsight* www.adinsight.co.uk UK
Infinity tracking* www.infinity-tracking.com UK
Optilead* www.optilead.co.uk UK
Switchboard free www.switchboardfree.co.uk UK
Freshegg www.freshegg.co.uk UK
Avanser www.avanser.com.au AUS
Jet Interactive* www.jetinteractive.com.au AUS
* I read up on these or talked to them. These are my picks .

52. Call tracking
What data can you get?
• Data to keyword, group or traffic source level – for Phone
contact / conversions
• Splits by location, traffic source, time, date
• Data on the call – how long to answer, to abandon, call
duration
• Data post call – was there a sale? (call wrapping)
• Linkage between marketing activity and phone ROI on web
channels (very important)
• Which parts of your site drive contact
• Tagging supports GA, Omniture, Others
• Complete call analytics (including recording, if wanted)

53. Tracking tips
• Make contact points trackable (phone, email, form, chat etc.)
For example:
• Collapsible phone section on Contact Us form – tracks intent
• Phone ‘reveal’ icon, button or widget – tracks intent
• Tap to call wrapper on Phone
• Use a smart email form instead of email address
• Use ‘ringme’ ringback functionality
• If email being generated, tweak subject line
• Integrate call centre events for maximum coverage
• If you can’t buy phone tracking, make your own (mail me!)
• Use phone tracking for major traffic groups, say:
• PPC groups, Organic, Print, PPC, email, TV, Brand keywords
• See your phone traffic in GA, Omniture and most packages

54. Follow the money – ROI example
• 12.4% from Web increase
• 12.5% increase from Multi Variate Testing
• 3.5% increase in downstream conversion rate
• 4% increase in online conversion from mobile optimised site
• Reduced call time (1 second = > 10k per year)
• Faster booking time
• 5.5% increase in NPS (Customer sat) score
• Customer delight – feedback. Less questions and worries
• Total increase in conversion (measured) = 32%
• Actual increase in conversion = 41%
• NO additional marketing spend needed
• Web Channel : ROI positive within 2 weeks

55. More reading. Slides and resources on slideshare.net
Email : sullivac@gmail.com
Twitter : @OptimiseOrDie
: linkd.in/pvrg14
Slideshare : slidesha.re/nlCDm6

56. CRO and Testing resources
• 101 Landing page tips : slidesha.re/8OnBRh
• 544 Optimisation tips : bit.ly/8mkWOB
• 108 Optimisation tips : bit.ly/3Z6GrP
• 32 CRO tips : bit.ly/4BZjcW
• 57 CRO books : bit.ly/dDjDRJ
• CRO article list : bit.ly/nEUgui
• Smashing Mag article : bit.ly/8X2fLk

57. Level 1 Level 2 Level 3 Level 4 Level 5
Starter Level Early maturity Serious testing Core business value You rock, awesomely
Dedicated team Ninja Team
Local Heroes Small team Cross silo team
Culture Volume Testing in the
Chaotic Good Low hanging fruit Systematic tests
opportunities DNA
_____________________________________________________________________________________________ _
Process Ad Hoc Outline process Well developed Streamlined Company wide
_______________________________________________________________________________________________
+Funnel +Cross channel +Spread tool use
Guessing + Multi variate optimisation testing Dynamic adaptive
Testing A/B testing Session replay Call tracking Integrated CRO targeting
focus Basic tools No segments Some segments and analytics Machine learning
Micro testing Segmentation Realtime
________________________________________________________________________________________________
Bounce rates + Funnel analysis + Funnel fixes + offline Multichannel
Analytics Low converting
integration funnels
Big volume Forms analytics
focus landing pages & High loss pages Channel switches
Single channel Cross channel
picture synergy
________________________________________________________________________________________________
+Regular usability + All channel view
Analytics + Customer sat
testing/research +User Centered of customer
Insight Surveys
Prototyping
Design scores tied to UX
Driving offline
Contact Centre Rapid iterative
methods Low budget Session replay
Layered feedback
testing and
using online
usability
Mini product tests design All promotion
Onsite feedback driven by testing
________________________________________________________________________________________________
Continual
Mission Get buyin Prove ROI Scale the testing Mine value improvement