Successfully reported this slideshow.
Your SlideShare is downloading. ×

Emetrics - Oct 19 2011 - New York - X channel optimisation


Check these out next

1 of 56 Ad

More Related Content

Slideshows for you (19)


Similar to Emetrics - Oct 19 2011 - New York - X channel optimisation (20)

More from Craig Sullivan (20)


Recently uploaded (20)

Emetrics - Oct 19 2011 - New York - X channel optimisation

  1. 1. Craig Sullivan The Multi-variate advantage Group CX Manager How testing saved my life Belron® @OptimiseOrDie
  2. 2. Naked self promotion @OptimiseOrDie • Conversion Rate Optimisation • Contact deflection and • A/B and Multi-variate testing online self service • Cross channel optimisation • Site search analytics • User Centered Design • Site, Page and Campaign optimisation • Usability testing • Test design and execution • Customer Research & Insight • A long usable site portfolio • Web Analytics • ROI on UX improvements • Browser and email compatibility • Over 18 years of boring meetings Group CX Manager for Belron® (Safelite) I get to do it with crowds
  3. 3. Outline • What is optimisation? • Belron® Testing Culture • What is MVT • Our testing program • Advantages of testing • The results • Cross channel optimisation • Case study : Chat ROI • Resources
  4. 4. What’s optimisation? @OptimiseOrDie
  5. 5. A set of techniques, implemented in order to influence customer behaviour towards these outcomes: • Increased revenue or profit • Increasing NPS (Customer Sat) scores • Lower cost for business or contact centres • Increasing productivity or labour flexibility • Simply delighting customers We may fail with 90% of what we try. The 10% of winners let us shift behaviour hugely and measurably.
  6. 6. ® Belron Testing Culture @OptimiseOrDie
  7. 7. If you go to the men's washrooms at the Schiphol airport in Amsterdam, you may notice there's a fly in the urinals. It’s screen printed . So what do you think most men do? That's right, they aim at the fly when they urinate. They don't even think about it, and they don't need to read a user's manual; it's just an instinctive reaction that means 85% less spillage! The interesting feature of these urinals is that they're deliberately designed to take advantage of this inherent human male tendency. This is my job.
  8. 8. Small team, big challenge • Team of 3 - 2 Optimisers, 1 x part time designer • Higher ROI or lower costs, without spending more • Stuff that doesn’t hinder SEO, PPC, Performance • Measurable ROI - not ego or opinion • Fast return and low cost techniques • Over 26M page tests, 17 international tests live now • ROI is > 50 x optimisation costs • Total cost = ~800k for two years. ROI > $40M USD
  9. 9. Primer on testing =
  10. 10. A/B test Control Version A Version B
  11. 11. Multi variate Pizza Ingredients Experiment Recipes =
  12. 12. Month 1 - results +2% +3% +1% +2% +9% +4% -2% +1% +5% +13% -7% -9%
  13. 13. So what about ingredients? +20% -2% +5% +3% +2% +6% +6% +10% -18%
  14. 14. Weed out some losers… +2% +3% +9% +4% +5% +13%
  15. 15. The final… +10% +13%
  16. 16. Our testing program @OptimiseOrDie
  17. 17. Insight - Inputs Forms Remote Search analytics Depth testing analytics interviews Eye tracking Market research Customer A/B and contact MVT testing Social analytics Insight Unstructured data Web Clicktale analytics Usability Competitor testing Ops and CS Customer evals teams Surveys services
  18. 18. Killer idea generation sources • Analytics data • Customer insight – contact, customer feedback, research • Usability testing • Diary studies • Clicktale recordings • Business owners – market data, customer mix, local knowledge • Customer contact centre immersion (listening) • Books, Websites, Tips • Other optimisers • Test data – variables plotted on axis • Making lots of mistakes – Fail faster
  19. 19. Typical test lifecycle Other test Gather and results – good Brainstorm & Instrument review inputs variables & Hypotheses learnings Signoff (Legal, Design variables Instrument Test Plan Marketing, & creative analytics Document Brand) elements Next Wave Wave design Results of wave Washup (repeat) Business led Do verification Repeat from reporting (with = wave + 90/10% Declare winner start £x, without = £y) split
  20. 20. What’s our testingand SpokeThe traffic picture Hub mix – model – homepage traffic
  21. 21. What’s our testing mix? • A/B testing – Homepage, Landing pages, Funnels • MVT testing – high traffic landing or home pages – Mini (less than 8 variables) – Midi (8-32) – Maxi (>32 variables) And 5 types of specialised testing: • Funnel testing – bedding in new funnels • Landing page optimisation • Cross channel testing • Iterative feature and product testing • Micro testing • Telephony MVT
  22. 22. Level 1 Level 2 Level 3 Level 4 Level 5 Starter Level Early maturity Serious testing Core business value You rock, awesomely Dedicated team Ninja Team Local Heroes Small team Cross silo team Culture Volume Testing in the Chaotic Good Low hanging fruit Systematic tests opportunities DNA _____________________________________________________________________________________________ _ Process Ad Hoc Outline process Well developed Streamlined Company wide _______________________________________________________________________________________________ +Funnel +Cross channel +Spread tool use Guessing + Multi variate optimisation testing Dynamic adaptive Testing A/B testing Session replay Call tracking Integrated CRO targeting focus Basic tools No segments Some segments and analytics Machine learning Micro testing Segmentation Realtime ________________________________________________________________________________________________ Bounce rates + Funnel analysis + Funnel fixes + offline Multichannel Analytics Low converting integration funnels Big volume Forms analytics focus landing pages & High loss pages Channel switches Single channel Cross channel picture synergy ________________________________________________________________________________________________ +Regular usability + All channel view Analytics + Customer sat testing/research +User Centered of customer Insight Surveys Prototyping Design scores tied to UX Driving offline Contact Centre Rapid iterative methods Low budget Session replay Layered feedback testing and using online usability Mini product tests design All promotion Onsite feedback driven by testing ________________________________________________________________________________________________ Continual Mission Get buyin Prove ROI Scale the testing Mine value improvement
  23. 23. International testing examples @OptimiseOrDie
  24. 24. UK Wave 2 – 21/9 to 26/10
  25. 25. What is the ROI then?
  26. 26. Another example
  27. 27. Testing advantages @OptimiseOrDie
  28. 28. Your ‘design’ team? Web/Online Pesky Meddling Kids? Marketing Who designs this stuff anyway? Brand IT THE CEO CUSTOMER Legal
  29. 29. BENEFITS OF TESTING: • Reduced ego and opinion (execs + others) • Measurable ROI • Faster time to market • Increased conversion in the pocket early • Iterative and rapid product lifecycles • Faster, smaller, focused, measured releases • Continual testing = continual improvement • SLED vs. Agile = faster return, better focus • Less reliance on anecdotal evidence • Business owners part of testing culture • Introduces a testing culture • Large cross silo team now gets involved • Reduces timidity – be brave and curious! • Helps us hit the sweet spot...
  30. 30. Cross Channel Optimisation @OptimiseOrDie
  31. 31. Insight - Inputs Ringback Phone Email Retargeting Chat Mobile TV handsets Display Apps Affiliate Exploding Tablets Channels Desktops Direct Social & SEO Micro Partners PPC DM
  32. 32. TV advertising and Web TV - Off TV - On
  33. 33. People images that suck…
  34. 34. Photo Guidelines • Groups, vans, stores, corporate guff = • Female images work best in almost NO every country – they slay the guys • Single person – not groups or even person + customer • Natural, authentic and not ‘model’ • Open, Friendly, Natural, Smiling, or ‘stock’ image types work best for Engaged = YES people • Uniform – has huge effect, even • We get better images now, but we wearing branded ball cap improves keep working on the guidelines! conversion • We still got a genital framing • Look straight at viewer or slightly picture the other week towards CTA • Photoshoots and tests are now • Plain backgrounds only being used to drive advertising • No folded arms, arms on hips, pointing, strategy scowling, cheesy smiles, hands behind back, hanging useless by the side, • Pre TV test of potential ‘actors’ on clasped by groin, fiddling with hands website. X factor meets split • Arms with prop instead – hold a testing. clipboard, rag, tool – I’m working for *you*
  35. 35. Cross channel mini studies • Mini studies: – Abandon testing – Emergency phone switchoff & Overspill systems – Contact deflection systems and self servicing – Scripting Experiments – APIs and other systems integration – Chat testing – Phone tracking
  36. 36. Abandon testing Reduce unwanted phone hangups. Test plan – Remove corporate guffspeak – Two countries, big phone systems, – Don’t mention the website, esp. – Test variants of messages, music, if this is the web number (D’oh!) timings, amounts quoted – Don’t give a generic or corporate – Messages make a big impact – drone apology message experiment freely with volumes – Quote real KPIs and timings (ING – Opportunity is large due to high direct, average queue time) call volumes we have – Make routing emotionally driven – What people hate personally vs. – not confusing categories that what those same people have on map poorly onto customer needs their company lines, lol – Experiment with queue timing – – On a mission here! find sweet spot, don’t overquote
  37. 37. Switchoff Overspill If you have spikes, have a throttle: Overspill systems: – Busiest TV & Radio campaign so far, Q2 – Get a UX designer to make you a fast 2011 for Autoglass® entry system – Designed a phone ‘switchoff’ system, – Make them simple, easy to learn and permanently on tap leverage your web platform – Routed majority of contact to a web form – Give API access to overspill – Prevented a backlog situation from companies developing – Sometimes heavy IT integration isn’t – Bought time and still exceeded needed customer expectations (<4h excellent) – “Nobody needs a freaking training – Used twice to dampen inbound call course to use facebook” - Make sure volumes your internal apps and call centre overspill systems are built for instant learnability – This could save millions in process efficiency. Same as web, really.
  38. 38. Contact deflection & self service • People like immediate gratification! • What about your measurement? • The iceberg of contact opportunity • Basics – Ratings, Did you find this • Make people like you for not talking answer helpful? to them on the phone –double win • Intermediate – Tracking behaviour • If you stick it in help, will it go away? and contact channels • No elephant graveyard, please (FAQ) • Add tracking of self service events or • Instrument for analytics and ROI behaviour • Set it up to measure and improve • Set up a game or reward structure for staff • Gamification of content? Adoption? • Continually improve and get proof • LOVEFiLM Help Centre: • Add phone tracking - make avoidable – Ratings, Deflection tracking, self contact visible service tracking • Remove all barriers to self servicing – V1 :Please view in your account for high cost/volume contacts secion, V2 : Please visit your account, V3 : Please check your • Try Amazon help – where is phone? next billing date, V4 - Final • Help experts? Sticky Content are version – to show the real info : brilliant ( “Your next billing date is 1 Feb”
  39. 39. Script Optimisation • How can we optimise phone conversion? – Dedicated test teams – Script testing often limited range/shift in companies now – Now – we can use A/B or MV testing, with API integration – Try out 3,4,5 different versions of key areas or entire script – ‘Wrapping’ sends XML message event, triggers analytics tag – Analyse in the same way as an A/B or multi-variate web test – Fast optimisation of handling scripts, using a mini lab – Should yield 3-5% higher performing scripts, minimum – Shows what variables drive script performance globally – These emotional and psychological triggers can be mined – Aim for Q2 2012 for a proof of concept
  40. 40. Platform APIs • Use of web platform APIs now for: – Contact centre messages – Facebook app – Mobile site and apps – Scheduling – Analytics tag firing • Just making APIs, automagically makes: – People use them – Improvements happen – New ideas happen, that you didn’t expect – Incompatible stuff get glued together – Complex IT integration projects go away
  41. 41. Chat Testing Ringback Contact Form Contact Us You must measure behaviour across contact channels during tests like these A - Chat B - Chat C - Chat Split A B C
  42. 42. A - Chat Free B – With Chat Damage 0844 371 0047 Damage 0844 371 0051 Appt 0844 371 0048 Appt 0844 371 0052 Details 1 0844 371 0049 Details 1 0844 371 0053 Details 2 0844 371 0050 Details 2 0844 371 0054
  43. 43. Chat testing • 1 month trial – properly A/B tested • Customers happy, yes, but not in a • Measured all the contact channels for good business way movements – what happened? • Like putting your customer service • Over a week, chat saves 17 calls – desk into the street yay! • You must measure influences in • But swapped that for 329 chats @ contact volumes when trialling 8.25 minutes. Ouch. Chat • Increased ringback requests? Email form about the same. • Be very careful – prove your ROI on any pilot. • No statistically significant impact on conversion • Only 7-10% use and out of those, • Assumption was big impact, huge less than 50% are influencable etc. impact! etc. • Anecdotally, a wonderful tool. • The productivity Myth of chat Agreed. • Your situation may differ – please TEST!
  44. 44. Call tracking How it works: • Rent a bank of numbers • Add a dynamic ‘tag’ to your page • Phone number appears automatically • Customer calls this number • Captures the keywords, landing page, converting page, traffic source • Geo mapping captures customer location • When the call is hung up, optionally record the outcome code (sale, loss) • Have a special number for Facebook, SEO, Adwords traffic etc. • Lets you analyse from web -> phone call -> wait time -> wrapping • Builds a link between web and call, finally
  45. 45. Call tracking Why you should use it! • Most people under-report on web driven phone contact or sales • You can’t work out your advertising effectiveness, if you don’t include phone • If you don’t measure phone, your A/B tests may be wrong • You can capture all your web AND phone, in one easy interface • Allows you a multi-channel view of all your contact sources • Lets you record offline campaigns as well with specific numbers (print, TV etc.) • You can experiment with routing, messages, on hold music – much more easily • If you can’t afford lots of phone numbers, this is an easy way to get tracking
  46. 46. Company Website Coverage Who? Mongoose Metrics* UK, USA, Canada Ifbyphone* USA TheCallR* USA, Canada, UK, IT, FR, BE, ES, NL Call tracking metrics USA Hosted Numbers USA Callcap USA UK, SE, FI, NO, DK, LT, PL, IE, CZ, SI, Freespee* AT, NL, DE Adinsight* UK Infinity tracking* UK Optilead* UK Switchboard free UK Freshegg UK Avanser AUS Jet Interactive* AUS * I read up on these or talked to them. These are my picks .
  47. 47. Call tracking What data can you get? • Data to keyword, group or traffic source level – for Phone contact / conversions • Splits by location, traffic source, time, date • Data on the call – how long to answer, to abandon, call duration • Data post call – was there a sale? (call wrapping) • Linkage between marketing activity and phone ROI on web channels (very important) • Which parts of your site drive contact • Tagging supports GA, Omniture, Others • Complete call analytics (including recording, if wanted)
  48. 48. Tracking tips • Make contact points trackable (phone, email, form, chat etc.) For example: • Collapsible phone section on Contact Us form – tracks intent • Phone ‘reveal’ icon, button or widget – tracks intent • Tap to call wrapper on Phone • Use a smart email form instead of email address • Use ‘ringme’ ringback functionality • If email being generated, tweak subject line • Integrate call centre events for maximum coverage • If you can’t buy phone tracking, make your own (mail me!) • Use phone tracking for major traffic groups, say: • PPC groups, Organic, Print, PPC, email, TV, Brand keywords • See your phone traffic in GA, Omniture and most packages
  49. 49. Follow the money – ROI example • 12.4% from Web increase • 12.5% increase from Multi Variate Testing • 3.5% increase in downstream conversion rate • 4% increase in online conversion from mobile optimised site • Reduced call time • Faster booking time • 5.5% increase in NPS (Customer sat) score • Customer delight – feedback. Less questions and worries • Total increase in conversion (measured) = 32% • Actual increase in conversion = 41% • NO additional marketing spend needed • Web Channel : ROI positive within 2 weeks
  50. 50. More reading. Slides and resources on Email : Twitter : @OptimiseOrDie : Slideshare :
  51. 51. CRO and Testing resources • 101 Landing page tips : • 544 Optimisation tips : • 108 Optimisation tips : • 32 CRO tips : • 57 CRO books : • CRO article list : • Smashing Mag article :