More Related Content

Similar to Conversion conference london nov 2011 - multi channel testing - craig sullivan(20)


More from Craig Sullivan(20)


Conversion conference london nov 2011 - multi channel testing - craig sullivan

  1. @OptimiseOrDie Craig Sullivan Group CX Manager Conversion Conference London Belron®
  2. Obligatory promo slide @OptimiseOrDie • Conversion Rate Optimisation • Contact deflection and • A/B and Multi-variate testing online self service • Cross channel optimisation • Site search analytics • User Centered Design • Site, Page and Campaign optimisation • Usability testing • Test design and execution • Customer Research & Insight • A long usable site portfolio • Web Analytics • ROI on UX improvements • Browser and email compatibility • Over 19 years of really boring meetings Group CX Manager for Belron® (Autoglass ®) I get to do it with crowds
  3. OPTIMISATION A set of techniques, implemented in order to influence customer behaviour towards these outcomes: • Increased revenue or profit • Increasing NPS (Customer Sat) scores • Lower cost for business or contact centres • Increasing productivity or labour flexibility • Simply delighting customers We may fail with 90% of what we try. The 10% of winners let us shift behaviour hugely and measurably.
  4. ® Belron Testing Culture @OptimiseOrDie
  5. If you go to the men's washrooms at the Schiphol airport in Amsterdam, you may notice there's a fly in the urinals. It’s screen printed . So what do you think most men do? That's right, they aim at the fly when they urinate. They don't even think about it, and they don't need to read a user's manual; it's just an instinctive reaction that means 85% less spillage! The interesting feature of these urinals is that they're deliberately designed to take advantage of this inherent human male tendency. This is my job.
  6. Small team, big challenge • Team of 3 - 2 Optimisers, 1 x part time designer • Higher ROI or lower costs, without spending more • Stuff that doesn’t hinder SEO, PPC, Performance • Measurable ROI - not ego or opinion • Fast return and low cost techniques • Over 26M page tests, 20 international tests live now • Cost is few hundred K – ROI is > 50 x investment • Yes, that’s more than 5,000% back.
  7. Primer on testing =
  8. A/B test Control Version A Version B
  9. Multi variate Pizza Ingredients Experiment Recipes =
  10. Month 1 - results +2% +3% +1% +2% +9% +4% -2% +1% +5% +13% -7% -9%
  11. So what about ingredients? +20% -2% +5% +3% +2% +6% +6% +10% -18%
  12. Weed out some losers… +2% +3% +9% +4% +5% +13%
  13. The final… +10% +13%
  14. Our testing program @OptimiseOrDie
  15. Usability & Customers are the glue • Usability testing – hundreds of participants, 6 countries so far • Foolproof – you can’t kid a kidder • Diary studies • Remote testing • Analytics data • Customer insight – contact, customer feedback, research • Clicktale recordings • Business owners – market data, customer mix, local knowledge • Customer contact centre immersion (listening) • Books, Websites, Tips & Other Optimisers • Old Test Data results (e.g. Click here, Get started, Fix my glass) • Making lots of mistakes – be brave, fail faster, learn stuff
  16. What’s our testingand SpokeThe traffic picture Hub mix – model – homepage traffic
  17. What’s our testing mix? • A/B testing – Homepage, Landing pages, Funnels • MVT testing – high traffic landing or home pages – Mini (less than 8 variables) – Midi (8-32) – Maxi (>32 variables) And 6 types of specialised testing: • Funnel testing – bedding in new funnels • Landing page optimisation • Cross channel testing • Iterative feature and product testing • Micro testing • Telephony MVT
  18. Usability & Optimisation process Final Usability Final Release Legal review prototype issues left changes build kickoff Signoff Cust services Instrument Marketing (Legal, Test Plan review analytics review Mktng, CCC) kickoff Instrument Offline End-End Launch Contact QA testing tagging testing 90/10% Centre Launch Monitor < 1 Launch Go live 100% Monitor 50/50% week 80/20% Analytics Washup and New New test Rinse and review actions hypotheses design Repeat!
  19. International testing examples @OptimiseOrDie
  20. UK Wave 2 – 21/9 to 26/10
  21. Another example
  22. Testing and UX advantages? @OptimiseOrDie
  23. Your ‘design’ team? Web/Online Pesky Meddling Kids? Marketing Who designs this stuff anyway? Brand IT THE CEO CUSTOMER Legal
  24. Benefits of UX combined with Testing: • Reduced ego and opinion (execs + others) • Measurable ROI • Faster time to market • Increased conversion in the pocket early • Iterative and rapid product lifecycles • Faster, smaller, focused, measured releases • Continual testing = continual improvement • SLED vs. Agile = faster return, better focus • Less reliance on anecdotal evidence • Business owners part of testing culture • Introduces a testing culture • Large cross silo team now gets involved • Helps us hit the sweet spot...
  25. Designing for multi-platform @OptimiseOrDie
  26. Insight - Inputs Ringback Phone Email Retargeting Chat Mobile TV handsets Display Apps Affiliate Exploding Tablets Channels Desktops Direct Social & SEO Micro Partners PPC DM
  27. Insight - Inputs Forms Remote Search analytics Depth testing analytics interviews Eye tracking Market research Customer A/B and contact MVT testing Social analytics Insight Unstructured data Web Clicktale analytics Usability Competitor testing Ops and CS Customer evals teams Surveys services
  28. User Centered Design Wireframe Research Concept Prototype Analyse Test
  29. Multi channel and platform Washup Mobile Web 1 1 UX Insight Washup Design Washup App 1
  30. What were the benefits? • Very fast build – post research: • Design product and contact • 10 weeks to build 3 cross options for revenue platform booking & sites • Cross channel synergy really helps • Focused on the user (but not work out issues driven by them) • Higher overall grasp of service, • Customer services, the process, lifted barriers, less pain... the contact channels – a BIG part • Big increase in incremental sales of the design team effort • To get multi-channel right, you • Align business and tech capability need to design across them. with customer ‘models’ • Lo budget testing can start with • Rapid iterative design and build = paper, webcam, £40!... fast feedback
  31. People images that suck…
  32. Photo Guidelines • Groups, vans, stores, corporate guff = • Female images work best in almost NO every country – they slay the guys • Single person – not groups or even person + customer • Natural, authentic and not ‘model’ • Open, Friendly, Natural, Smiling, or ‘stock’ image types work best for Engaged = YES people • Uniform – has huge effect, even • Photoshoots and tests now being wearing branded ball cap improves used to drive advertising strategy conversion • Pre TV test of potential ‘actors’ on • Look straight at viewer or slightly website? X factor meets split towards CTA testing? • Plain backgrounds only • We get better images now, but we • No folded arms, arms on hips, pointing, keep working on the guidelines! scowling, cheesy smiles, hands behind back, hanging useless by the side, • We still got a genital framing clasped by groin, fiddling with hands picture the other week • Arms with prop instead – hold a clipboard, rag, tool – I’m working for *you*
  33. Cross channel mini studies • Mini studies: – Abandon testing – Emergency phone switchoff & Overspill systems – Contact deflection systems and self servicing – Scripting Experiments – APIs and other systems integration – Chat testing – Phone tracking
  34. Abandon testing Reduce unwanted phone hangups. Test plan – Remove corporate guffspeak – Two countries, big phone systems, – Don’t mention the website, esp. – Test variants of messages, music, if this is the web number (D’oh!) timings, amounts quoted – Don’t give a generic or corporate – Messages make a big impact – drone apology message experiment freely with volumes – Quote real KPIs and timings (ING – Opportunity is large due to high direct, average queue time) call volumes we have – Make routing emotionally driven – What people hate personally vs. – not confusing categories that what those same people have on map poorly onto customer needs their company lines, lol – Experiment with queue timing – – On a mission here! find sweet spot, don’t overquote
  35. Switchoff Overspill If you have spikes, have a throttle: Overspill systems: – Busiest TV & Radio campaign so far, Q2 – Get a UX designer to make you a fast 2011 for Autoglass® entry system – Designed a phone ‘switchoff’ system, – Make them simple, easy to learn and permanently on tap leverage your web platform – Routed majority of contact to a web form – Give API access to overspill – Prevented a backlog situation from companies developing – Sometimes heavy IT integration isn’t – Bought time and still exceeded needed customer expectations (<4h excellent) – “Nobody needs a freaking training – Used twice to dampen inbound call course to use facebook” - Make sure volumes your internal apps and call centre overspill systems are built for instant learnability – This could save millions in process efficiency. Same as web, really.
  36. Contact deflection & self service • People like immediate gratification! • What about your measurement? • The iceberg of contact opportunity • Basics – Ratings, Did you find this • Make people like you for not talking answer helpful? to them on the phone –double win • Intermediate – Tracking behaviour • If you stick it in help, will it go away? and contact channels • No elephant graveyard, please (FAQ) • Add tracking of self service events or • Instrument for analytics and ROI behaviour • Set it up to measure and improve • Set up a game or reward structure for staff • Gamification of content? Adoption? • Continually improve and get proof • LOVEFiLM Help Centre: • Add phone tracking - make avoidable – Ratings, Deflection tracking, self contact visible service tracking • Remove all barriers to self servicing – V1 :Please view in your account for high cost/volume contacts section, V2 : Please visit your account, V3 : Please check your • Try Amazon help – where is phone? next billing date, V4 - Final • Help experts? Sticky Content are version – to show the real info : brilliant ( “Your next billing date is 1 Feb”
  37. Script Optimisation • How can we optimise phone conversion? – Dedicated test teams – Script testing often limited range/shift in companies now – Now – we can use A/B or MV testing, with API integration – Try out 3,4,5 different versions of key areas or entire script – ‘Wrapping’ sends XML message event, triggers analytics tag – Analyse in the same way as an A/B or multi-variate web test – Fast optimisation of handling scripts, using a mini lab – Should yield 3-5% higher performing scripts, minimum – Shows what variables drive script performance globally – These emotional and psychological triggers can be mined – Aim for Q2 2012 for a proof of concept
  38. Platform APIs • Use of web platform APIs now for: – Contact centre messages – Facebook app – Mobile site and apps – Scheduling – Analytics tag firing* (soon) • Just making APIs, automagically makes: – People use them – Improvements happen – New ideas happen, that you didn’t expect – Incompatible stuff get glued together – Complex IT integration projects go away
  39. Chat Testing Ringback Contact Form Contact Us You must measure behaviour across contact channels during tests like these A - Chat B - Chat C - Chat Split A B C
  40. A - Chat Free B – With Chat Damage 0844 371 0047 Damage 0844 371 0051 Appt 0844 371 0048 Appt 0844 371 0052 Details 1 0844 371 0049 Details 1 0844 371 0053 Details 2 0844 371 0050 Details 2 0844 371 0054
  41. Chat testing • 1 month trial – properly A/B tested • Customers happy, yes, but not in a • Measured all the contact channels for good business way movements – what happened? • Like putting your customer service • Over a week, chat saves 17 calls – desk into the street yay! • You must measure influences in • But swapped that for 329 chats @ contact volumes when trialling 8.25 minutes. Ouch. Chat • Increased ringback request & Email form about the same. • Be very careful – prove your ROI on any pilot. • No statistically significant impact on conversion • Only 7-10% use and out of those, • Assumption was big impact, huge less than 50% are influencable etc. impact! etc. • Anecdotally, a wonderful tool. • The productivity Myth of chat Agreed. • Your situation may differ – please TEST!
  42. Call tracking How it works: • Rent a bank of numbers • Add a dynamic ‘tag’ to your page • Phone number appears automatically • Customer calls this number • Captures the keywords, landing page, converting page, traffic source • Geo mapping captures customer location • When the call is hung up, optionally record the outcome code (sale, loss) • Have a special number for Facebook, SEO, Adwords traffic etc. • Lets you analyse from web -> phone call -> wait time -> answer ->outcome • Builds a link between web and call, finally
  43. Call tracking Why you should use it! • Most people under-report on web driven phone contact or sales • You can’t work out your advertising effectiveness, if you don’t include phone • If you don’t measure phone, your A/B tests may be wrong • You can capture all your web AND phone, in one easy interface • Allows you a multi-channel view of all your contact sources • Lets you record offline campaigns as well with specific numbers (print, TV etc.) • You can experiment with routing, messages, on hold music – much more easily • If you can’t afford lots of phone numbers, this is an easy way to get tracking
  44. Company Website Coverage Who? Mongoose Metrics* UK, USA, Canada Ifbyphone* USA TheCallR* USA, Canada, UK, IT, FR, BE, ES, NL Call tracking metrics USA Hosted Numbers USA Callcap USA UK, SE, FI, NO, DK, LT, PL, IE, CZ, Freespee* SI, AT, NL, DE Adinsight* UK Infinity tracking* UK Optilead* UK Switchboard free UK Freshegg UK Avanser AUS Jet Interactive* AUS * I read up on these or talked to them. These are my picks .
  45. Call tracking What data can you get? • Data to keyword, group or traffic source level – for Phone contact / conversions • Splits by location, traffic source, time, date • Data on the call – how long to answer, to abandon, call duration • Data post call – was there a sale? (call wrapping) • Linkage between marketing activity and phone ROI on web channels (very important) • Which parts of your site drive contact • Tagging supports GA, Omniture, Others • Complete call analytics (including recording, if wanted)
  46. Tracking tips • Make contact points trackable (phone, email, form, chat etc.) For example: • Collapsible phone section on Contact Us form – tracks intent • Phone ‘reveal’ icon, button or widget – tracks intent • Tap to call wrapper on Phone • Use a smart email form instead of email address • Use ‘ringme’ ringback functionality • If email being generated, tweak subject line • Integrate call centre events for maximum coverage • If you can’t buy phone tracking, make your own (mail me!) • Use phone tracking for major traffic groups, say: • PPC groups, Organic, Print, PPC, email, TV, Brand keywords • See your phone traffic in GA, Omniture and most packages
  47. Follow the money – ROI example • 12.4% from Web increase • 12.5% increase from Multi Variate Testing • 3.5% increase in downstream conversion rate • 4% increase in online conversion from mobile optimised site • Reduced call time (1 second = > 10k per year) • Faster booking time • 5.5% increase in NPS (Customer sat) score • Customer delight – feedback. Less questions and worries • Total increase in conversion (measured) = 32% • Actual increase in conversion = 41% • NO additional marketing spend needed • Web Channel : ROI positive within 2 weeks
  48. More reading. Slides and resources on Email : Twitter : @OptimiseOrDie : Slideshare :
  49. CRO and Testing resources • 101 Landing page tips : • 544 Optimisation tips : • 108 Optimisation tips : • 32 CRO tips : • 57 CRO books : • CRO article list : • Smashing Mag article :
  50. Level 1 Level 2 Level 3 Level 4 Level 5 Starter Level Early maturity Serious testing Core business value You rock, awesomely Dedicated team Ninja Team Local Heroes Small team Cross silo team Culture Volume Testing in the Chaotic Good Low hanging fruit Systematic tests opportunities DNA _____________________________________________________________________________________________ _ Process Ad Hoc Outline process Well developed Streamlined Company wide _______________________________________________________________________________________________ +Funnel +Cross channel +Spread tool use Guessing + Multi variate optimisation testing Dynamic adaptive Testing A/B testing Session replay Call tracking Integrated CRO targeting focus Basic tools No segments Some segments and analytics Machine learning Micro testing Segmentation Realtime ________________________________________________________________________________________________ Bounce rates + Funnel analysis + Funnel fixes + offline Multichannel Analytics Low converting integration funnels Big volume Forms analytics focus landing pages & High loss pages Channel switches Single channel Cross channel picture synergy ________________________________________________________________________________________________ +Regular usability + All channel view Analytics + Customer sat testing/research +User Centered of customer Insight Surveys Prototyping Design scores tied to UX Driving offline Contact Centre Rapid iterative methods Low budget Session replay Layered feedback testing and using online usability Mini product tests design All promotion Onsite feedback driven by testing ________________________________________________________________________________________________ Continual Mission Get buyin Prove ROI Scale the testing Mine value improvement