Agenda – Conversion Manager Course#1 The Optimisers Toolkit    The best tools recommended by CRO practitioners#2 CRO Proje...
Conversion Heroes!Meet @OptimiseOrDieReason for being : I Optimise StuffPast : John Lewis, Lovefilm, Belron, PwCNow : Dire...
"If you go to the mens washrooms atthe Schiphol airport in Amsterdam,you may notice theres a fly in theurinals. It’s scree...
Is there a way to fix this then?                                   20 Mar 2013
Agenda#1 The Optimisers Toolkit    The best tools recommended by CRO practitioners#2 CRO Project Styles    How does CRO fi...
#1   The Optimisers Toolkit
The Optimisers Toolkit#1   Session Replay#2   Browser & Email testing#3   VOC & Feedback tools#4   Guerrilla Usability#5  ...
#1 : Session Replay• 3 kinds of tool :  Client side• Normally Javascript based• Pros : Rich mouse and click data,         ...
#1 : Session Replay•   Vital for optimisers & fills in a ‘missing link’ for insight•   Rich source of data on visitor expe...
#2 : Feedback / VOC tools•   Anything that allows immediate realtime onpage feedback•   Comments on elements, pages and ov...
#2a : Survey ToolsI’ve used• Surveymonkey      www.surveymonkey.com   (1/5)• Zoomerang         www.zoomerang.com      (3/5...
#3 – Testing toolsEmail testing     www.litmus.com                  www.returnpath.com                  www.lyris.comBrows...
#4 : Guerrilla Usability Testing• All you need is a device, time and people!• Use one of these tools for session recording...
#5 : Productivity tools
#5 Join.me             20 Mar 2013
#5 Pivotal Tracker                     20 Mar 2013
#5 Basecamp              20 Mar 2013
#5 Google Docs•   Seriously wasting your time doing manual Excel?•   Fed up doing stuff that takes hours?•   Use the Googl...
#5 Cloud Collaboration • LucidChart                         20 Mar 2013
#5 Cloud Collaboration • Webnotes                         20 Mar 2013
#5 Cloud Collaboration • Protonotes                         20 Mar 2013
#5 Cloud Collaboration • Conceptshare                         20 Mar 2013
#6 – Split testing tools – Cheap!• Google Content Experiments  bit.ly/Ljg7Ds• Multi Armed Bandit Explanation  bit.ly/Xa80O...
#7 Performance                 • Google Site Speed                 • Webpagetest.org                 • Mobitest.akamai.org
Remote tests : iPhoneSite                   Size   Requests  www.coop.se         1200k     63  m.ikea.com/se/sv/    684k  ...
Show your boss this split test:         Slides : slidesha.re/PDpTPD
Show the e-com director this one:
#7 – UX and Crowd tools Som, feedbackRemote UX tools (P=Panel, S=Site recruited, B=Both)Usertesting (B)                   ...
#8 : Web Analytics Love•   Properly instrumented analytics•   Investment of 5-10% of developer time•   Add more than you n...
#8 : Tap 2 Call trackingStep 1 : Add a unique phone number on ALL channels         (or insert your own dynamic number)Step...
10.0                                                         15.0                                                         ...
What about desktop?Step 1 : Add ‘Click to reveal’• Can be a link, button or a collapsed section• Add to your analytics sof...
What about desktop?                                                     Onbound Call analytics:                           ...
So what does phone tracking get you?• You can do it for free on your online channels• If you’ve got any phone sales or con...
Company                                         Website                    CoverageWho?Mongoose Metrics*                  ...
#9 : Web Analytics Love•   UX of web analytics tools and reports•   People, Process, Human problems•   Make the UI force d...
CRO TOOLS – ANY QUESTIONS?• Break!
#2 CRO Project Styles
What’s the problem?• #1 User Experience and Conversion  Optimisation are not a checkbox or a  step in the process – it nee...
Also, the dial won’t turn anymore        PPC             SEOWith thanks to @morys
Why is this happening?• PPC changes• Advertising models flattening – i.e. mobile Google costs• Comprehensive SEO changes• ...
So what do people do?•   They throw tools at the problem•   They try usability testing and research•   They generate more ...
Skinner’s Pigeon Experiment• Participants invited into a room with objects• Told to score 100 points within 30 minutes• Pa...
Lean UX“The application of UX design methods into productdevelopment, tailored to fit Build-Measure-Learn cycles.”     Pos...
Agile UX / UCD / Collaborative Design“An integration of User Experience Design and Agile*Software Development Methodologie...
CRO
Lean Conversion Optimisation   “A blend of User Experience Design, Agile PM, Rapid Lean   UX Build-Measure-Learn cycles, t...
Lean CRO                           Inspection              Instrument                Immersion      Learn                 ...
Triage and Triangulation    “This is where the smarts of CRO are – in identifying the    easiest stuff to test or fix that...
The Bucket Methodology“Helps you to stream actions from the insights and prioritisation work.Forces an action for every is...
How is it working out?• Methodologies are not Real Life ™• It’s mainly about the mindset of the team and  managers, not th...
Questions ?
#3   2013 Tips roundup!!!!!!!
Analytics Genius Tips#1   Performance tune-ups#2   Browser money#3   Keyboard shortcuts#4   Ranking data#5   Content engag...
#1 : Performance Tune-ups• With thanks to @Danbarker•   Add “_gaq.push([_setSiteSpeedSampleRate, 100]);”•   Amps up sampli...
#2 : Browser money•   Create a desktop only segment *Exclude “mobile including tablets” +•   Create a mobile only segment ...
#2 : Browser money•   You should see something like this:
#3 : GA Keyboard Shortcuts• Thanks to Farid Alhadi and @fastbloke  dt        Set date range to TODAY  dy        Set date r...
#4 : Ranking data•   Someone searches on Google for ‘Term’•   Clicks on a link to your site•   What was the keyword rank f...
#5 : Content Engagement•   Get a better bounce rate metric•   See more detailed engagement metrics•   Measure scrolling an...
#6 : Enhanced Page Analyticsvar _gaq = _gaq || [];var pluginUrl =//www.google-analytics.com/plugins/ga/inpage_linkid.js;_g...
#7 : Duplicate transactions in GA• Thanks to Matt Clarke and @timlb• Stop skewing the data with duplicates/reloads• Custom...
#8 : Event tracking• Thanks to #Measurecamp – check the stream• A beginners guide : http://bit.ly/13RFoJs• Some great idea...
#9 : Google API and more• Use the Google API to get super custom reports• You can fetch different data types (on the fly a...
#10 : Microdata (a.k.a rich snippets)•   http://www.seoskeptic.com/beyond-rich-snippets-semantic-web-technologies-for-bett...
#10 : Microdata – SERPS UX•   Reviews – huge increases in CTR and Conversion•   People (Authors)•   Products•   Businesses...
#11 : Measure viewport size•   Thanks to @Beantin and others!•   Measure the viewport size, not the resolution•   Why?•   ...
Questions   ?
#4 Best Practice• There is no such thing as ‘easy best practice’ in  design or conversion optimisation• There are patterns...
#1 - What is optimisation?My definition:“A set of techniques, implemented inorder to influence customer behaviourtowards t...
Hijack App Video
#2 - What is a leak model?• Very like a funnel diagram, except• We analyse what happens to the losses• Then we assign new ...
Leak Model Detail
Pipe diagram
The Marketing Bath                     Acquisition = Water flow                     Holes = Conversion loss               ...
Leak surprises!•   When I use a leak model – 30-70% is not a loss•   A loss is sometimes a channel or contact switch•   Th...
#3 - Where are the best places to test?•   High traffic pages of influence•   Landing pages – Organic and PPC•   Home page...
#4 How do I identify issues?•   Lab Usability testing•   Remote Usability testing•   Guerrilla Usability testing•   Voice ...
#5 How do I quantify & prioritise?• Triangulate with data sources• Score each issue found by:   –   Effort to test or fix ...
#5 How do I quantify & prioritise? Opportunity               Effort
#5 Prioritisation• Allocate an outcome bucket• For an issue, cluster or page   (See bucket slides)• You now have outcomes:...
#6 How long will a test take?• Very simple  – 2 business cycles minimum  – Always test complete cycles  – Usually a week, ...
#7 What prep do I need for each issue?•   Usability test insights & Personas•   Analytics review data•   CRO review data• ...
#8 – Brainstorming the hypotheses• Not a guide to brainstorming but for optimal  CRO workshops:• Look at all the insight &...
#9 – What should I test on a page?•   Work those value propositions•   Titles, Subheadings, Copy•   Call to action buttons...
#10 – What kind of test should I run?• A/B testing – good for:   –   A single change of content or design layout   –   A g...
Netherlands A/B Shift ExamplePrevious winner    +7.25%                                 +8.19% lift
#10 – What kind of test should I run?• Micro testing (tiny change) – good for:    –   Proving to the boss that testing wor...
#11 – How do I optimise a funnel?• This is worth a workshop on just this topic!• Firstly, watch for my Funnel Testing reso...
#12 – What master planning document is needed?• Create one document. It should contain:   – The test configuration & flow ...
#13 – Should I run an A/A test first?• No – and this is why:   – It’s a waste of time   – It’s easier to test and monitor ...
#14 – What QA testing should I do?•   Cross Browser Testing•   Testing from several locations•   Test IP filtering•   Test...
#15 – What’s on the pre-flight checklist?•   KPI success measures agreed•   Analytics tags firing correctly?•   Final test...
#16 – Early stage – what to watch for?•   Ignore the graphs. Don’t draw conclusions.•   Get a feel for the test but don’t ...
#17 – What happens when a test flips?          •   Something like this can happen:•   Check your sample size. If it’s stil...
#18 – What happens if a test is still moving around?• There are three reasons it is moving around    – Your sample size (o...
#19 – How do I know when it’s ready?• The hallmarks of a cooked test are:   – It’s done at least 1 or 2 (preferred) cycles...
#20 – What happens if it’s inconclusive?• Analyse the segmentation• One or more segments will be over and under• See if th...
#21 – What happens if it fails?•   Analyse the segmentation•   One or more segments will be over and under•   Check for va...
#22 – What was driving my test (segments)?• To understand beyond the headline  average, you need to segment your data.• If...
#23 – Why are these numbers fuzzy?
#24 – What about the words – what help do I need?•   Employ a dedicated copywriter – period•   Find someone who understand...
5 Persuasive Copywriting Levers         Key insights from user testing                @OptimiseOrDie
Copy testing mission – Carglass Worldwide• Identify techniques through testing and user insight, that will:• Reflect, assi...
Emotion & Persuasion - leversCustomers do not know what to do and need support and advice•   Emphasize the fact that you u...
#1 “Customers do not know what to do                       and need support and advice”1. Emphasize the fact that you unde...
#1 “Customers do not know what to do                      and need support and advice”1. Emphasize our expertise and leade...
#1 “Customers do not know what to do                       and need support and advice”Explain what they will need to do o...
#2 “Customers do not feel confident                          in assessing the damage”Emphasize the fact that you will help...
#3 “Customers need to understand                        the benefits of booking online”Emphasize that the online booking s...
#4 “Customers mistrust insurers and find        dealing with their insurance situation very frustrating”1. Show that you u...
#5 “Some customers cannot be bothered                   to take action to fix their car glass ”Emphasize the consequences ...
SummaryI’m stressed out – reassure me. Take the pain and worry away.Make it safe – I’m worried it will fall in, crack or c...
Worldwide testing : Culture?• If you operate internationally, use each country results to  inform or drive tests in other ...
Copy optimisation – short summary• Think of the optimisation process as a scent trail  you’re laying• Take away words, the...
Copy optimisation – short summary• Value proposition – where  is it? Is there one?• Thinking about goal and  emotionally o...
Better ad copy - spot the winner
Lowest                Cost per   Lowest CPAHighest Click   Click                   HighestThrough Rate                    ...
#25 – Sharing the test results – the best way?•   Take a look at www.whichtestwon.com•   Aim for the same thing internally...
#26 – Springing to the next test• So you’ve completed your test and published the results• Do you (a) Bask in glory at how...
#27 – How can I test *more* stuff than I do now?•   What are your constraints?•   Most common problem – Time to launch•   ...
#28 – What’s the best kept secret in CRO?• That 12.5% test you ran 3 months ago?• Happy at the results? Still on your slid...
#29 – What is a good conversion rate?•   Higher than the one you had last month!•   Use benchmarking services (usually irr...
#30 – What are the key success factors?•   Number of tests completed a month•   Implementing changes directly, in batches ...
BEST PRACTICE – QUESTIONS?
Unexpected CRO
#30 – What part do images play?                             FRANCE                             +8% over control           ...
GERMANY+12% over control99% confidence
SPAIN+22% over control99% confidence
Make sure you use real,                             authentic, smiling,                             friendly, approachable...
Take the fake smile test : bbc.in/5rtnv
Images that Suck!
Image Guidelines for Belron    Photo Guidelines•   Groups, vans, stores, corporate guff =    •   Female images work best i...
“We alwayshad a plan for pulling out”
“We alwayshad a plan for pulling out”
• Make sure you use                                        real, authentic, smiling, frien                                ...
Evil CRO• Optimisation can be used for evil!• Either by accident (just stupid)• Or by design, to trick peoplewiki.darkpatt...
Sneaky fold
Sneaky basket
Hidden opt-out
Ripped
Evil genius award?
So how do I become an Optimiser?•   It’s a state of mind, not just a course!•   You need to read lots, test loads and watc...
Don’t give up!• Start with the customer at the heart of everything• It’s liberating!• Use low budget solutions, cheap or f...
HOMEWORK 1•   I’d like you to look at how unconscious action is part of your life every week.•   Several times a day, you’...
HOMEWORK 23.   The Furnishings. Take a look at the door furniture and signs:      •    Is there any other stuff apart from...
Collecting the Evidence   Apps• https://itunes.apple.com/au/app/this-is-note-calendar-+-  photoalbums/id403746123?mt=8• ht...
More reading. Slides and resources on slideshare.net                              Email : sullivac@gmail.com              ...
#5 Site Reviews!
Obligatory promo slide @OptimiseOrDie• Conversion Rate               • Contact deflection and  Optimisation               ...
CRO and Testing resources•   101 Landing page tips   :   slidesha.re/8OnBRh•   544 Optimisation tips   :   bit.ly/8mkWOB• ...
Level 1             Level 2             Level 3              Level 4               Level 5              Starter Level     ...
Conversionista : Conversion manager course  - Stockholm 20 march 2013
Conversionista : Conversion manager course  - Stockholm 20 march 2013
Conversionista : Conversion manager course  - Stockholm 20 march 2013
Conversionista : Conversion manager course  - Stockholm 20 march 2013
Conversionista : Conversion manager course  - Stockholm 20 march 2013
Conversionista : Conversion manager course  - Stockholm 20 march 2013
Conversionista : Conversion manager course  - Stockholm 20 march 2013
Upcoming SlideShare
Loading in...5
×

Conversionista : Conversion manager course - Stockholm 20 march 2013

6,299

Published on

Published in: Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
6,299
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
44
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide
  • “A piece of paper with your design mockup. A customer in a shop or bookstore. Their finger is their mouse, the paper their screen. Where would they click? Do they know what these labels mean? Do they see the major routes out of the page? Any barriers.Congratulations, you just got feedback on your design, before writing a single freaking line of code or asking your developers to keep changing stuff.”
  • “A piece of paper with your design mockup. A customer in a shop or bookstore. Their finger is their mouse, the paper their screen. Where would they click? Do they know what these labels mean? Do they see the major routes out of the page? Any barriers.Congratulations, you just got feedback on your design, before writing a single freaking line of code or asking your developers to keep changing stuff.”
  • Create a suck index = pageviews * load time.
  • Here I show you some examples of well known brands, some of whom should know better. The larger the size of the page, the longer it will take to download and render on the device, especially when you don’t have perfect data conditions. The numberof requests also makes a difference, as it’s inefficient on mobile to open lots of connections like this. In short, the smaller the pagesize and number of requests you can aim for, the better. I’m patient with bad data connections but do people have the tolerance for 10-15 seconds on mobile? No – it has to happen much faster.
  • These are the results of a live test on a site, where an artificial delay is introduced in the performance testing. I’ve done some testing like this myself on desktop and mobile sites and confirm this is true – you’re increasing bounce rate, decreasing conversion, site engagement…It doesn’t matter what metric you use, performance equals MONEY or if not measured, a HUGE LOSS.
  • Performance also harms the lifeblood of e-commerce and revenue generating websites – repeat visitors! The gap here in one second of delay is enormous over time. You’re basically sucking a huge portion of potential business out of your site, with every additional bit of waiting time you add.
  • Add unique phone numbers to all your mobile sites and apps. That’s for starters.Then configure your analytics to collect data when people Click or Tap to make a phone call.Make sure you add other events like ringbacks, email, chat – any web forms or lead gen activity too.
  • So what does this graph say? That I have a long tail thing I want to talk to you about?No – this shows how much the ratio of phone to online conversion we have, by keyword.Some keywords generate nearly 25 times the call volume of others, which is a huge differential.This means that if you thought you got ‘roughly’ the same proportion of phone calls for different marketing activity, you are wrong.What this graph tells me is that the last 2 years of my stats are basically a big dog poo.
  • Add unique phone numbers to all your mobile sites and apps. That’s for starters.Then configure your analytics to collect data when people Click or Tap to make a phone call.Make sure you add other events like ringbacks, email, chat – any web forms or lead gen activity too.
  • Add unique phone numbers to all your mobile sites and apps. That’s for starters.Then configure your analytics to collect data when people Click or Tap to make a phone call.Make sure you add other events like ringbacks, email, chat – any web forms or lead gen activity too.
  • Phone tracking costs you nothing – you can add it in a few minutes to your app or mobile website, by changing your analytics tracking.Now you can see exactly which bits of inbound marketing are driving telephone and other contact channelsIf you have any sort of phone component in your service or support, the insight could be vitalYou can take traffic by keyword, source, campaign or advert creative and work out the TRUE mix of conversion activityAnd all this is also available on Desktop too – by using dynamic numbers, we can track exactly the same stuff.Talk to this company : www.infinity-tracking.com
  • So what does this graph say? That I have a long tail thing I want to talk to you about?No – this shows how much the ratio of phone to online conversion we have, by keyword.Some keywords generate nearly 25 times the call volume of others, which is a huge differential.This means that if you thought you got ‘roughly’ the same proportion of phone calls for different marketing activity, you are wrong.What this graph tells me is that the last 2 years of my stats are basically a big dog poo.
  • “A piece of paper with your design mockup. A customer in a shop or bookstore. Their finger is their mouse, the paper their screen. Where would they click? Do they know what these labels mean? Do they see the major routes out of the page? Any barriers.Congratulations, you just got feedback on your design, before writing a single freaking line of code or asking your developers to keep changing stuff.”
  • Conversionista : Conversion manager course - Stockholm 20 march 2013

    1. 1. Agenda – Conversion Manager Course#1 The Optimisers Toolkit The best tools recommended by CRO practitioners#2 CRO Project Styles How does CRO fit in with Lean UX, Agile, User Centred, Waterfall?#3 Analytics Genius Tips Top tips from 2013 – from around the world#4 Best Practice Answers The key learnings from 6 years of optimisation#5 Site Reviews Let me give you feedback or advice on your site or project
    2. 2. Conversion Heroes!Meet @OptimiseOrDieReason for being : I Optimise StuffPast : John Lewis, Lovefilm, Belron, PwCNow : Director of Optimisation, Rush Email : sullivac@gmail.com Twitter : @OptimiseOrDie : linkd.in/pvrg14 Slides : slidesha.re/nlCDm6
    3. 3. "If you go to the mens washrooms atthe Schiphol airport in Amsterdam,you may notice theres a fly in theurinals. It’s screen printed .So what do you think most men do?Thats right, they aim at the fly whenthey urinate.They dont even think about it, andthey dont need to read a usersmanual; its just an instinctivereaction that means 85% less spillage!The interesting feature of theseurinals is that theyre deliberatelydesigned to take advantage of thisinherent human male tendency.“This is my job.
    4. 4. Is there a way to fix this then? 20 Mar 2013
    5. 5. Agenda#1 The Optimisers Toolkit The best tools recommended by CRO practitioners#2 CRO Project Styles How does CRO fit in with Lean UX, Agile, User Centred, Waterfall?#3 Analytics Genius Tips Top tips from 2013 – from around the world#4 Best Practice Answers The key learnings from 6 years of optimisation#5 Site Reviews Let me give you feedback or advice on your site or project
    6. 6. #1 The Optimisers Toolkit
    7. 7. The Optimisers Toolkit#1 Session Replay#2 Browser & Email testing#3 VOC & Feedback tools#4 Guerrilla Usability#5 Productivity tools#6 Split testing#7 Performance#8 Crowdsourcing#9 Analytics Love
    8. 8. #1 : Session Replay• 3 kinds of tool : Client side• Normally Javascript based• Pros : Rich mouse and click data, errors, forms analytics, UI interactions.• Cons : Dynamic content issue, Performance hit Server side• Black Box -> Proxy, Sniffer, Port copying device• Pros : Gets all dynamic content, fast, legally tight• Cons : No client side interactions, Ajax, HTML5 etc. Hybrid• Clientside and Sniffing with central data store
    9. 9. #1 : Session Replay• Vital for optimisers & fills in a ‘missing link’ for insight• Rich source of data on visitor experiences• Segment by browser, visitor type, behaviour, errors• Forms Analytics (when instrumented) are awesome• Can be used to optimise in real time!Session replay tools• Clicktale (Client) www.clicktale.com• Mouseflow (Client) www.mouseflow.com• Ghostrec (Client) www.ghostrec.com• Tealeaf (Hybrid) www.tealeaf.com• Usabilla (Client) www.usabilla.com• UserReplay (Server) www.userreplay.com• SessionCam (Client) www.sessioncam.com
    10. 10. #2 : Feedback / VOC tools• Anything that allows immediate realtime onpage feedback• Comments on elements, pages and overall site & service• Can be used for behavioural triggered feedback• Tip! : Take the Call Centre for beers• Kampyle www.kampyle.com• Qualaroo www.qualaroo.com• 4Q 4q.iperceptions.com• Usabilla www.usabilla.com Any more?
    11. 11. #2a : Survey ToolsI’ve used• Surveymonkey www.surveymonkey.com (1/5)• Zoomerang www.zoomerang.com (3/5)• SurveyGizmo www.surveygizmo.com (5/5)• Read anything by and follow @cjforms• Any others?
    12. 12. #3 – Testing toolsEmail testing www.litmus.com www.returnpath.com www.lyris.comBrowser testing www.browsercam.com (BOING!) www.crossbrowsertesting.com www.cloudtesting.com www.multibrowserviewer.com www.saucelabs.comMobile devices www.perfectomobile.com www.deviceanywhere.com www.mobilexweb.com/emulators www.opendevicelab.com
    13. 13. #4 : Guerrilla Usability Testing• All you need is a device, time and people!• Use one of these tools for session recording:CamStudio (free) www.camstudio.orgMediacam AV (cheap) www.netu2.co.ukSilverback (Mac) www.silverbackapp.comScreenflow (Mac) www.telestream.netUX Recorder (iOS) www.uxrecorder.com
    14. 14. #5 : Productivity tools
    15. 15. #5 Join.me 20 Mar 2013
    16. 16. #5 Pivotal Tracker 20 Mar 2013
    17. 17. #5 Basecamp 20 Mar 2013
    18. 18. #5 Google Docs• Seriously wasting your time doing manual Excel?• Fed up doing stuff that takes hours?• Use the Google API to roll your own reports straight into Big G• Lots of good articles but ask for advice from: @danbarker @timlb #measurecamp• Google Analytics + API + Google docs integration = A BETTER LIFE!• Hack your way to having more productive weeks• Learn how to do this and to have fun with GA custom reports• Ask me about the importance of training 20 Mar 2013
    19. 19. #5 Cloud Collaboration • LucidChart 20 Mar 2013
    20. 20. #5 Cloud Collaboration • Webnotes 20 Mar 2013
    21. 21. #5 Cloud Collaboration • Protonotes 20 Mar 2013
    22. 22. #5 Cloud Collaboration • Conceptshare 20 Mar 2013
    23. 23. #6 – Split testing tools – Cheap!• Google Content Experiments bit.ly/Ljg7Ds• Multi Armed Bandit Explanation bit.ly/Xa80O8• Optimizely www.optimizely.com• Visual Website Optimizer www.visualwebsiteoptimizer.com
    24. 24. #7 Performance • Google Site Speed • Webpagetest.org • Mobitest.akamai.org
    25. 25. Remote tests : iPhoneSite Size Requests www.coop.se 1200k 63 m.ikea.com/se/sv/ 684k 14 High St Retailer 307k 43 Department Store 100k 18 Newspaper 195k 35 Supermarket 125k 14 Auto Sales 151k 47 Autoglass 25k 10
    26. 26. Show your boss this split test: Slides : slidesha.re/PDpTPD
    27. 27. Show the e-com director this one:
    28. 28. #7 – UX and Crowd tools Som, feedbackRemote UX tools (P=Panel, S=Site recruited, B=Both)Usertesting (B) www.usertesting.comUserlytics (B) www.userlytics.comUserzoom (S) www.userzoom.comIntuition HQ (S) www.intuitionhq.comMechanical turk (S) www.mechanicalturk.comLoop11 (S) www.loop11.comOpen Hallway (S) www.openhallway.comWhat Users Do (P) www.whatusersdo.comFeedback army (P) www.feedbackarmy.comUser feel (P) www.userfeel.comEthnio (For Recruiting) www.ethnio.comFeedback on Prototypes / MockupsPidoco www.pidoco.comVerify from Zurb www.verifyapp.comFive second test www.fivesecondtest.comConceptshare www.conceptshare.comUsabilla www.usabilla.com
    29. 29. #8 : Web Analytics Love• Properly instrumented analytics• Investment of 5-10% of developer time• Add more than you need• Events insights• Segmentation• Call tracking love!
    30. 30. #8 : Tap 2 Call trackingStep 1 : Add a unique phone number on ALL channels (or insert your own dynamic number)Step 2 : For phones, add “Tap to Call” or “Click to Call”• Add Analytics event or tag for phone calls!• Very reliable data, easy & cheap to do• What did they do before calling?• Which page did they call you from?• What PPC or SEO keyword did they use?• Incredibly useful – this keyword level call data• What are you over or underbidding for?• Will help you shave 10, 20%+ off PPC• Which online marketing really sucks?
    31. 31. 10.0 15.0 20.0 25.0 0.0 5.0 safe lit windshield chip repair safelite windshield autoglass auto window replacement safelight auto auto glass repair windshield replacement costs safelite auto glass safe auto glass safelite repair windshield autoglass replacement car glass safelite locations auto glass repair quotes Phone to Booking Ratio windshield repair mobile windshield replacement replace windshield car windshield repair new windshield costauto glass windshield replacement car window repair cost safe gl auto glass repair houston tx windshield crack
    32. 32. What about desktop?Step 1 : Add ‘Click to reveal’• Can be a link, button or a collapsed section• Add to your analytics software• This is a great budget option!Step 2 : Invest in call analytics• Unique visitor tracking for desktop• Gives you that detailed marketing data• Easy to implement• Integrates with your web analytics• Let me explain…
    33. 33. What about desktop? Onbound Call analytics: Split testing Priority routing Wait time VIP teams Call centre Abandon A 222 1234 Call length music On hold 0800 222 1234 0800 Operator outcome Menu Sales value 0800 222 1245 Messages Ratings Call recording B 222 1263 0800 Offline promotions! 0800 222 1299 Profit line! C Web Analytics Telephony Cloud
    34. 34. So what does phone tracking get you?• You can do it for free on your online channels• If you’ve got any phone sales or contact operation, this will change the game for you• For the first time, analytics for PHONE for web to claim• Optimise your PPC spend• Track and Test stuff on phones, using web technology• The two best tests?• Talk to Someone today ->
    35. 35. Company Website CoverageWho?Mongoose Metrics* www.mongoosemetrics.com UK, USA, Canada Ifbyphone* www.ifbyphone.com USA TheCallR* www.thecallr.com USA, Canada, UK, IT, FR, BE, ES, NL Call tracking metrics www.calltrackingmetrics.com USA Hosted Numbers www.hostednumbers.com USA Callcap www.callcap.com USA UK, SE, FI, NO, DK, LT, PL, IE, CZ, Freespee* www.freespee.com SI, AT, NL, DE Adinsight* www.adinsight.co.uk UK Infinity tracking* www.infinity-tracking.com UK Optilead* www.optilead.co.uk UK Switchboard free www.switchboardfree.co.uk UK Freshegg www.freshegg.co.uk UK Avanser www.avanser.com.au AUS Jet Interactive* www.jetinteractive.com.au AUS * I read up on these or talked to them. These are my picks .
    36. 36. #9 : Web Analytics Love• UX of web analytics tools and reports• People, Process, Human problems• Make the UI force decisions!• Playability and exploration• Skunkworks project time (5-10%)• Give it love, time, money and iteration• Spend to automate, get more time
    37. 37. CRO TOOLS – ANY QUESTIONS?• Break!
    38. 38. #2 CRO Project Styles
    39. 39. What’s the problem?• #1 User Experience and Conversion Optimisation are not a checkbox or a step in the process – it needs to be integrated in everything you do.• #2 This work isn’t a one off exercise either – it’s an ongoing continuous improvement process – like Kaizen• #3 Usability testing isn’t enough – other UX factors like the visceral and behavioural emotional responses we have to products need tuning too.• #4 It’s not just about the user!• #5 It’s usually Self Centred Design driven by Ego, Opinion, Assumption
    40. 40. Also, the dial won’t turn anymore PPC SEOWith thanks to @morys
    41. 41. Why is this happening?• PPC changes• Advertising models flattening – i.e. mobile Google costs• Comprehensive SEO changes• Competition increasing• Fleetness of foot – Asos in Australia• Entry costs are lower now• New entrants compete without the cruft• Startups are using better ‘build and optimise’ methodologies than nearly all corporates• The old way of doing things is going to die• The new way of doing things is your only survival ticket
    42. 42. So what do people do?• They throw tools at the problem• They try usability testing and research• They generate more data to look at• They make changes without measuring or testing• They hope to randomly create the optimal system• They get an expensive agency to help them• They push their team harder, like galley slaves• They experiment with riskier advertising models• They wonder why they’re burning rubber• They try more random things• Then they call a CRO person and say “We’ve tried everything. It isn’t working. Help!”
    43. 43. Skinner’s Pigeon Experiment• Participants invited into a room with objects• Told to score 100 points within 30 minutes• Participants moved objects around, made noise, jumped around, tried anything to make a counter increase the points score.• They got horribly confused• They then created convincing lies for themselves, to explain what they thought was working. They became superstitious and made rituals.• The points allocation was made randomly by a goldfish, swimming back and forward in a tank.• Know any marketing departments like this?• I’ve seen this a lot – and it paralyses companies• We need a better way. A methodology?
    44. 44. Lean UX“The application of UX design methods into productdevelopment, tailored to fit Build-Measure-Learn cycles.” Positive – Lightweight and very fast methods – Realtime or rapid improvements – Documentation light, value high – Low on wastage and frippery – Fast time to market, then optimise – Allows you to pivot into new areas Negative – Often needs user test feedback to steer the development, as data not enough – Bosses distrust stuff where the outcome isn’t known
    45. 45. Agile UX / UCD / Collaborative Design“An integration of User Experience Design and Agile*Software Development Methodologies” *Sometimes Research Positive – User centric – Goals met substantially – Rapid time to market (especially when using Agile iterations) Wireframe Negative Concept Prototype – Without quant data, user goals can drive the show – missing the business sweet spot – Some people find it hard to integrate with siloed teams Analyse Test – Doesn’t’ work with waterfall IMHO
    46. 46. CRO
    47. 47. Lean Conversion Optimisation “A blend of User Experience Design, Agile PM, Rapid Lean UX Build-Measure-Learn cycles, triangulated data sources, triage and prioritisation.” Positive– A blend of several techniques– Multiple sources of Qual and Quant data aids triangulation– CRO analytics focus drives unearned value inside all products Negative– Needs a one team approach with a strong PM who is a Polymath (Commercial, Analytics, UX, Technical)– Only works if your teams can take the pace – you might be surprised though!
    48. 48. Lean CRO Inspection Instrument Immersion Learn Identify Triage & Measure Triangulate Outcome Streams
    49. 49. Triage and Triangulation “This is where the smarts of CRO are – in identifying the easiest stuff to test or fix that will drive the largest uplift.”• Starts with the analytics data• Then UX and user journey walkthrough from SERPS -> key paths• Then back to analytics data for a whole range of reports:• Segmented reporting, Traffic sources, Device viewport and browser, Platform (tablet, mobile, desktop) and many more• We use other tools or insight sources to help form hypotheses• We triangulate with other data where possible• We estimate the potential uplift of fixing/improving something as well as the difficulty (time/resource/complexity/risk)• A simple quadrant shows the value clusters• We then WORK the highest and easiest scores by…• Turning every opportunity spotted into an OUTCOME
    50. 50. The Bucket Methodology“Helps you to stream actions from the insights and prioritisation work.Forces an action for every issue, a counter for every opportunity being lost.” Test If there is an obvious opportunity to shift behaviour, expose insight or increase conversion – this bucket is where you place stuff for testing. If you have traffic and leakage, this is the bucket for that issue. If an issue is placed in this bucket, it means we need to beef up the Instrument analytics reporting. This can involve fixing, adding or improving tag or event handling on the analytics configuration. We instrument both structurally and for insight in the pain points we’ve found. Hypothesise This is where we’ve found a page, widget or process that’s just not working well but we don’t see a clear single solution. Since we need to really shift the behaviour at this crux point, we’ll brainstorm hypotheses. Driven by evidence and data, we’ll create test plans to find the answers to the questions and change the conversion or KPI figure in the desired direction. Just Do It JFDI (Just Do It) – is a bucket for issues where a fix is easy to identify or the change is a no-brainer. Items marked with this flag can either be deployed in a batch or as part of a controlled test. Stuff in here requires low effort or are micro-opportunities to increase conversion and should be fixed. Investigate You need to do some testing with particular devices or need more information to triangulate a problem you spotted. If an item is in this bucket, you need to ask questions or do further digging.
    51. 51. How is it working out?• Methodologies are not Real Life ™• It’s mainly about the mindset of the team and managers, not the tools or methodologies they play with• Not all my clients have all the working parts• You should not be a methodology slave• Feel free to make your own or flexibly adapt• Use some, any techniques instead of ‘guessing’• Blending lean and agile with conversion optimisation outcomes is my critical learning of the last 5 years• Doing rapid cycles of this outcome driven work for Belron:• World Conversion Rate Increase: 2009 +5%, 2010 +10%, 2011 +15%, 2012 +25%• If you’d like to develop a good one for your company, talk to me first!• Don’t over complicate it.
    52. 52. Questions ?
    53. 53. #3 2013 Tips roundup!!!!!!!
    54. 54. Analytics Genius Tips#1 Performance tune-ups#2 Browser money#3 Keyboard shortcuts#4 Ranking data#5 Content engagement#6 Enhanced In page#7 Duplicate transactions#8 Event tracking#9 Google API + more
    55. 55. #1 : Performance Tune-ups• With thanks to @Danbarker• Add “_gaq.push([_setSiteSpeedSampleRate, 100]);”• Amps up sampling for small & medium websites• Use the distribution report - % of pages < 3 seconds• DOM timings vital – let me explain• Avg. Document Content Loaded Time (sec)• This data is very accurate and helps conversion – pretty vital for landing pages, where I find lots of stuff• Make yourself a [pageview * content load time] report• This is called a ‘Suck Index’• Work your way down from the top• Mobile speed doesn’t count Safari – please be careful!• Read more at: http://p.barker.dj/sitespeedtips
    56. 56. #2 : Browser money• Create a desktop only segment *Exclude “mobile including tablets” +• Create a mobile only segment [ For GA, Include Mobile (including tablet) & Exclude Tablet = yes]. 3 segments Desktop, Tablet, Mobile only• Now start segmenting like this: Browser Segment Conv rate Quote the Safari Mobile Traffic 0.79% Internet Explorer Desktop only 1.34% opportunity! Chrome Desktop only 1.01% IE8 = 1.41% revenue Safari Tablet Traffic 1.00% Safari Desktop only 1.28% IE8 converts at 20% Firefox Desktop only 1.20% of IE9 and IE10 Android Browser Mobile Traffic 0.31% Some nasty bugs Safari (in-app) Mobile Traffic 0.69% cause the problem Chrome Mobile Traffic 0.62% Safari (in-app) Tablet Traffic 0.89% Worth fixing? Chrome Tablet Traffic 0.84% Opera Desktop only 0.10% That 1.41% problem Android Browser Tablet Traffic 0.71% is worth nearly 6% Mozilla Compatible Agent Mobile Traffic 0.51% more checkouts!
    57. 57. #2 : Browser money• You should see something like this:
    58. 58. #3 : GA Keyboard Shortcuts• Thanks to Farid Alhadi and @fastbloke dt Set date range to TODAY dy Set date range to YESTERDAY dw Set date range to LAST WEEK dm Set date range to LAST MONTH dc Toggle date comparison mode (to the previous period of whatever you are looking at. Example, if you’re looking at 6 days, this will compare it to the 6 days before it) dx Toggle date comparison mode (to the previous year of the period you are looking at) ? Open keyboard shortcut help h Search help center a open account panelshift + a Go to account list s/ Search reportsshift + d Go to the default dashboard of the current profile
    59. 59. #4 : Ranking data• Someone searches on Google for ‘Term’• Clicks on a link to your site• What was the keyword rank for that term?• Reverse engineer the actual rankings from users machines• More accurate than some SEO tools (IMHO)• Two articles to read: http://bit.ly/Vaisno http://bit.ly/13lmYF2
    60. 60. #5 : Content Engagement• Get a better bounce rate metric• See more detailed engagement metrics• Measure scrolling and reading activity• Came from @fastbloke but originally Justin Cutroni• Measure your scrolling and exit points from long form content• Very nice technique – read more at: http://bit.ly/13lmYF2 http://goo.gl/1AZZb
    61. 61. #6 : Enhanced Page Analyticsvar _gaq = _gaq || [];var pluginUrl =//www.google-analytics.com/plugins/ga/inpage_linkid.js;_gaq.push([_require, inpage_linkid, pluginUrl]);_gaq.push([_setAccount, UA-XXXXXX-Y]);_gaq.push([_trackPageview]);• Correct link attribution for in-page Analytics• Very nice
    62. 62. #7 : Duplicate transactions in GA• Thanks to Matt Clarke and @timlb• Stop skewing the data with duplicates/reloads• Custom report to check if you’re affected http://techpad.co.uk/content.php?sid=247• I’ve seen this a few places so worth checking, particularly if figures don’t tally!• Read more at : http://bit.ly/13lmYF2
    63. 63. #8 : Event tracking• Thanks to #Measurecamp – check the stream• A beginners guide : http://bit.ly/13RFoJs• Some great ideas here : http://bit.ly/UCcptx• Don’t go for ‘Event Blizzard’• Focus on specific areas where insight is needed• Choose your naming structure carefully: http://bit.ly/WJ4R4c• Read this complete guide : http://bit.ly/VmFSJ4
    64. 64. #9 : Google API and more• Use the Google API to get super custom reports• You can fetch different data types (on the fly as well as pre-calculated)• Automate a lot of Excel work – more later…• @timlb recorded all the tips for #Measurecamp• We then had an unconference• He recorded a special session for us• Listen to great tips and more: www.youtube.com/watch?v=JWXg1_4quwU www.youtube.com/watch?v=z41Cdg9xbk0• Roundup : www.measurecamp.org/aftermath/
    65. 65. #10 : Microdata (a.k.a rich snippets)• http://www.seoskeptic.com/beyond-rich-snippets-semantic-web-technologies-for-better-seo/• http://www.slideshare.net/ismepete/maximising-your-serp-potential-enhance-your-listings-with-rich-snippets 20 Mar 2013
    66. 66. #10 : Microdata – SERPS UX• Reviews – huge increases in CTR and Conversion• People (Authors)• Products• Businesses and Organisations• Recipes• Events• Music• Local• Video Helps to:• Dominate the page• Push other stuff down• Makes it more persuasive• The conversion journey starts here!
    67. 67. #11 : Measure viewport size• Thanks to @Beantin and others!• Measure the viewport size, not the resolution• Why?• Toolbars, chrome and setup varies• UK average is ~2 toolbars installed• Code example here : http://bit.ly/4xaNYK• A common conversion issue• Your desk vs. Users = different• Turn off the wifi, reduce the viewport• Cheap solution – use a toilet roll!
    68. 68. Questions ?
    69. 69. #4 Best Practice• There is no such thing as ‘easy best practice’ in design or conversion optimisation• There are patterns - but the context varies• The answer is always, “it depends” ;-)• It starts with your customers, your site, your data, your insights – not an article online!• Ensure they’re involved – that’s BP• So what? I need some advice!• How about answers to questions?• The top 30 questions for testing• Today is an exclusive!• Steal and use my slides .com .kp .se
    70. 70. #1 - What is optimisation?My definition:“A set of techniques, implemented inorder to influence customer behaviourtowards these outcomes” :• Increased revenue or profit• Removing cost from the business or contact center• Increasing productivity or labor flexibility• Increasing NPS (Customer Sat) scores• Simply delighting customers All work should be challenged on this basis! Would you do it in a retail store or shop window? Is all that stuff on the page earning rental income?
    71. 71. Hijack App Video
    72. 72. #2 - What is a leak model?• Very like a funnel diagram, except• We analyse what happens to the losses• Then we assign new outcomes: – Used chat – Sent an email / contact form – Visited Contact Us and Exited – Called us on the phone – Requested a ringback – Subscribed to a newsletter – Completed on mobile / desktop / tablet• Let’s see an example
    73. 73. Leak Model Detail
    74. 74. Pipe diagram
    75. 75. The Marketing Bath Acquisition = Water flow Holes = Conversion loss Plughole = Retention Floor water = Retargeting
    76. 76. Leak surprises!• When I use a leak model – 30-70% is not a loss• A loss is sometimes a channel or contact switch• This is the key insight!• UX issues or missing information contributes hugely to the real ‘total leavers’ figure• If you do any phone business, you MUST track• If you are multi-platform & channel, you MUST track• If you decide based on loss rates alone, you suck• If you don’t segment data, you’re missing out
    77. 77. #3 - Where are the best places to test?• High traffic pages of influence• Landing pages – Organic and PPC• Home page (if high traffic)• Category pages• Product/Content pages and controls• Pre-Funnel pages• Funnel steps and elements• Payment pages• Poor Navigation or Search• Hook or Trigger pages• Anywhere with a large leak %• Anything with traffic and a CTA!
    78. 78. #4 How do I identify issues?• Lab Usability testing• Remote Usability testing• Guerrilla Usability testing• Voice of Customer• Session Replay• Key Journey walkthroughs• CRO expert review• Analytics review• Leak Modelling• Call tracking• Asking people questions• The Call Centre – get them drunk!
    79. 79. #5 How do I quantify & prioritise?• Triangulate with data sources• Score each issue found by: – Effort to test or fix (time/resource) – Volume of people impacted – Severity of issue – Lasting impact of issue – Type of issue - Persuasion, UX, Device – Opportunity (to shift behaviour) – Risk or Difficulty• I normally use effort vs. opportunity• Plot them on a Quadrant diagram
    80. 80. #5 How do I quantify & prioritise? Opportunity Effort
    81. 81. #5 Prioritisation• Allocate an outcome bucket• For an issue, cluster or page (See bucket slides)• You now have outcomes: – By activity stream – By priority and opportunity• Time to start work!
    82. 82. #6 How long will a test take?• Very simple – 2 business cycles minimum – Always test complete cycles – Usually a week, 2 weeks, Month• Second part – You need 250 outcomes – Ideally 350 or more – In each outcome bucket – If you test 4 recipes, that’s 1400 outcomes – In my case above, it’s 2 weeks for 400 outcomes – Work out how long you want it to take! – Make a note of your minimum ‘length’ for 350 outcomes – If you segment, you’ll need more data
    83. 83. #7 What prep do I need for each issue?• Usability test insights & Personas• Analytics review data• CRO review data• Surveys or VOC feedback• Social analysis – common issues• Call centre top issues• Any additional data sources• Previous test results• Articles on this site area – as ideas• Pattern libraries• Competitor analysis (Don’t copy!)• Test run length estimates• Assemble the brainstorming team
    84. 84. #8 – Brainstorming the hypotheses• Not a guide to brainstorming but for optimal CRO workshops:• Look at all the insight & data• Invite everyone: Developers, UX people, Designers, Operations or Call centre staff, Marketing, Business owners, Copywriters, PPC and SEO people – one team!• Show the page, process or element (The What)• Explain the analytics data (The How Much)• Explain the UX or CRO issue (The Why)• Brainstorm solutions or behaviour shifting• Identify the best issues• Cut the tests to fit the run length!• Keep what’s left for later tests
    85. 85. #9 – What should I test on a page?• Work those value propositions• Titles, Subheadings, Copy• Call to action buttons, placement, size, design• Hero or product images• Supporting text or intro copy• Text size, decoration and formatting (scanning)• Design & Layout (paying attention to the fold but also utilising scroll behaviour)• Visual hierarchy and separation• Removing stuff• Simplifying things• Chris Goward will cover this superbly• +1 - I highly recommend his book.
    86. 86. #10 – What kind of test should I run?• A/B testing – good for: – A single change of content or design layout – A group of related changes (e.g. payment security) – Finding a new and radical shift for a template design – Lower traffic pages or shorter test times• Multivariate testing – good for: – Higher traffic pages – Groups of unrelated changes (e.g. delivery & security) – Multiple content or design style changes – Finding specific drivers of test lifts – Testing multiple versions (e.g. click here, book now, go)
    87. 87. Netherlands A/B Shift ExamplePrevious winner +7.25% +8.19% lift
    88. 88. #10 – What kind of test should I run?• Micro testing (tiny change) – good for: – Proving to the boss that testing works – Demonstrating to IT that it works without impact – Showing the impact of a seemingly tiny change – Proof of concept before larger test• Funnel testing – good for: – Checkouts – Lead gen Vs. – Forms processes – Quotations – Any multi-step process with data entry• Fake it and Build it – good for: – Testing new business ideas – Trying out promotions on a test sample – Estimating impact before you build – Helps you calculate ROI
    89. 89. #11 – How do I optimise a funnel?• This is worth a workshop on just this topic!• Firstly, watch for my Funnel Testing resource pack• This is a specialised area but here are some examples: – One text change in Carglass.de & fr = +8% – Optimising address lookup = +2.5% (one field) – Cross Browser issues = 10, 14, 18% increase (8M)• Remember – these are very warm leads• Small errors or messaging issues = -$$$• Security, Privacy, Trust, Clarity, Simplicity, Explanations = +$$$• Forms validation causes huge problems – every error = -$$$• It’s all about Humanising often robotic forms• Removing, simplifying, reassuring• If you have one, let me review it later!
    90. 90. #12 – What master planning document is needed?• Create one document. It should contain: – The test configuration & flow – All assets and recipes for the testing – Analytics tagging or instrumentation – IP addresses of all ‘non customer’ sites – Success Measure & KPI definitions – Any exceptions or exclusions – Estimated test run time minimum
    91. 91. #13 – Should I run an A/A test first?• No – and this is why: – It’s a waste of time – It’s easier to test and monitor instead – You are eating into test time – Also applies to A/A/B/B testing – A/B/A running at 25%/50%/25% is the best• Read my post here : http://bit.ly/WcI9EZ
    92. 92. #14 – What QA testing should I do?• Cross Browser Testing• Testing from several locations• Test IP filtering• Test analytics firing tags correctly• Test repeat visitor and session timeouts• Cross check figures from 2+ sources
    93. 93. #15 – What’s on the pre-flight checklist?• KPI success measures agreed• Analytics tags firing correctly?• Final test document signed off• Test estimates issued• QA cross browser and location testing complete• IP addresses excluded from test• Checked the marketing diary again?• Legal, Brand, IT -> signoff complete
    94. 94. #16 – Early stage – what to watch for?• Ignore the graphs. Don’t draw conclusions.• Get a feel for the test but don’t do anything yet!• Remember – in A/B - 50% of returning visitors will see a new shiny website!• Until your test has had at least 1 business cycle and 250-350 outcomes, don’t bother drawing conclusions or getting excited!• You’re looking for anything that looks really odd – your analytics person should be checking all the figures until you’re satisfied• All tests move around or show big swings early in the testing cycle. Here is a very high traffic site – it still takes 10 days to start settling. Lower traffic sites will stretch this period further.
    95. 95. #17 – What happens when a test flips? • Something like this can happen:• Check your sample size. If it’s still small, then expect this until the test settles.• If the test does genuinely flip – and quite severely – then something has changed with the traffic mix, the customer base or your advertising. Maybe the PPC budget ran out? Seriously!• To analyse a flipped test, you’ll need to check your segmented data. This is why you have a split testing package AND an analytics system.• The segmented data will help you to identify the source of the shift in response to your test. I rarely get a flipped one and it’s always something changing on me, without being told. The heartless bastards.
    96. 96. #18 – What happens if a test is still moving around?• There are three reasons it is moving around – Your sample size (outcomes) is still too small – The external traffic mix, customers or reaction has suddenly changed or – Your inbound marketing driven traffic mix is completely volatile (very rare)• Check the sample size• Check all your marketing activity• Check the instrumentation• If no reason, check segmentation
    97. 97. #19 – How do I know when it’s ready?• The hallmarks of a cooked test are: – It’s done at least 1 or 2 (preferred) cycles – You have at least 250-350 outcomes – It’s not moving around hugely at creative or segment level performance – The test results are clear – even if the precise values are not – The intervals are not overlapping (much) – If a test is still moving around, you need to investigate – Always declare on a business cycle boundary – not the middle of a period (this introduces bias) – Don’t declare in the middle of a limited time period advertising campaign (e.g. TV, print, online) – Always test before and after large marketing campaigns (one week on, one week off)
    98. 98. #20 – What happens if it’s inconclusive?• Analyse the segmentation• One or more segments will be over and under• See if they are cancelling out• The average is a lie• Don’t think of the average – think of the customer segments inside• If you genuinely have a test which failed to move any segments, it’s a crap test• This usually happens when it isn’t bold or brave enough in shifting away from the original design, particularly on lower traffic sites• Get testing again!
    99. 99. #21 – What happens if it fails?• Analyse the segmentation• One or more segments will be over and under• Check for varied performance• Now add the failed element or test to your Knowledge Base. Mark with a warning sign.• Look at it carefully – what does the failure tell you? Which element do you think drove the failure?• If you know what failed (e.g. making the price bigger) then you have very useful information• You turned the handle the wrong way• Now brainstorm a new test
    100. 100. #22 – What was driving my test (segments)?• To understand beyond the headline average, you need to segment your data.• If you have 350 outcomes and segment, you will now have much smaller samples – be very very careful! One of the key mistakes people make here.• Segment by: – New or Returning visitor – Logged in vs. Guest – Traffic source and medium – Landing page – Keyword – Day of week / Time of day – Browser, Device, Platform (mobile, tablet, desktop) – Depth of engagement (how far they got)
    101. 101. #23 – Why are these numbers fuzzy?
    102. 102. #24 – What about the words – what help do I need?• Employ a dedicated copywriter – period• Find someone who understands persuasive writing• If you can’t hire someone, read lots of books!• Start with Robert Cialdini, Bryan Eisenberg• Get a course from Sticky Content (for your team)• It’s not expensive to get specialist help• Always involve in the brainstorming• This is just selling – using persuasion and psychology• It isn’t new – read ‘Scientific Advertising’ from 1923• Most websites don’t practice this• It makes a huge difference in testing• Use customer goals and insight to drive levers• Here is an example!
    103. 103. 5 Persuasive Copywriting Levers Key insights from user testing @OptimiseOrDie
    104. 104. Copy testing mission – Carglass Worldwide• Identify techniques through testing and user insight, that will:• Reflect, assist, aid comprehension of and provide clarity of message, value proposition and overall comprehension – all blended together to get higher conversion!• Where is the f***ing value proposition? Make sure it’s always there!• Do more to reflect the emotional and persuasive elements of writing for our customers. It’s not for you – it’s for them.• Encourage testing of small or large scale copy changes• Drive copy from customer insight, not assumption – from targeted hypotheses instead of randomly trying stuff and do it from the heart!• 50-70% of the lift of some tests – is *just* from the copy changes.• Leverage this in all of our work.
    105. 105. Emotion & Persuasion - leversCustomers do not know what to do and need support and advice• Emphasize the fact that you understand that their situation is stressful• Emphasize your expertise and leadership in vehicle glazing and will help them get the best solution for their situation• Explain what they will need to do online and during the call-back so that they know what the next steps will be• Explain that they will be able ask any other questions they might have during the call-backCustomers do not feel confident in assessing the damage• Emphasize the fact that you will help them assess the damage correctly onlineCustomers need to understand the benefits of booking online• Emphasize that the online booking system is quick, easy and provides all the information they need in regards with their appointment and general cost informationCustomers mistrust insurers and find dealing with their insurance situation very frustrating• Where possible communicate the fact that the job is most likely to be free for insured customers, or good value for money for cash customers• Show that you understand the hassle of dealing with insurance companies – emphasise that you will help with their insurance paperwork for them, freeing them of this burdenSome customers cannot be bothered to take action to fix their car glass• Emphasize the consequences of not doing anything, e.g. „It‟s going to cost you more if the chip develops into a crack‟
    106. 106. #1 “Customers do not know what to do and need support and advice”1. Emphasize the fact that you understand that their situation is stressful2. Take advantage of this in copy terms, by stressing their urgency, the need to have this resolved and that they are worrying about it.“Damaged windscreen? Let’s get you back on the road in no time!”“Worried about that chip becoming a crack? Book a repair online in 5 minutes or less”“Urgent glass repair? We can get you booked in today. Free ringback service.”“Let’s take away the worry of driving with a chip. Our repairs are lifetime guaranteed”“Cracked windscreen? Worry less with a top quality replacement from Autoglass”“If you’re off the road, let’s get your windscreen repaired fast, at any location round the UK”
    107. 107. #1 “Customers do not know what to do and need support and advice”1. Emphasize our expertise and leadership in vehicle glazing and that we will help them get the best solution for their situation2. Take advantage of this in copy terms, by emphasising call centre level of training vs. average, our range of glass, our huge supply chain, value to the consumer in innovation, cool stuff: “Our staff train continually, just on glass. Call us on xxx so we can find the bit you need.” “We stock over 17,815 kinds of windscreens. We’re sure to have yours covered.” “We fit over x,000 windscreens a week. We know how to fit, clean up and recycle, 500 times a day” “Online booking in 5 minutes. Point out the damage and we’ll call you back free – and yes, we work with most insurers. ” “No matter the size, type, tint or place on your car – we’re sure to have the glass you need as we have over x,000 in stock, every day, within the UK.” “We test over x,000 repairs a year – to make sure we only give you the best” “We work with all the major insurers, as we do 100,000 repairs a year like yours” “Find your Nearest Autoglass with GPS. Visit our mobile site to search” “We even have support staff in our call centres. So you don’t have to go on hold”
    108. 108. #1 “Customers do not know what to do and need support and advice”Explain what they will need to do online and during the call-back so thatthey know what the next steps will be“Online booking, free ringback service, book online in as little as 5 minutes.”“Calling from a mobile? Use our free ringback service or book online”“Windscreen repair? Book online and let us find *your* glass, out of the 17,000 kinds ourcustomers ask us to stock.”“Our online booking is fast but we always make a courtesy call to check you’re happy”
    109. 109. #2 “Customers do not feel confident in assessing the damage”Emphasize the fact that you will help them assess the damage correctlyonline“Damaged windscreen? Use our damage wizard to get you booked in fast.”“Cracked windshield? Pick out the damage and get booked in easily and quickly”“Job done. Get that pesky chip repair sorted and booked today.”“Still putting it off? Let us worry about your repair by booking online”“Phone us, get a free ringback or use our online booking. Our trained glass experts will talkyou through your options”“Need a windscreen repair? Point out the damage online and we’ll have your appointmentbooked today..”
    110. 110. #3 “Customers need to understand the benefits of booking online”Emphasize that the online booking system is quick, easy and provides allthe information they need in regards with their appointment and generalcost information“5 minutes is all you need… Today you can get your appointment booked”“Our online booking is simple, easy to use and doesn’t take forever to fill out. Get finished inas little as 5 minutes.”“Worried about windscreen repair? Our online guides, damage picker and helpful hints willset you straight.”“Repair or replace? Want to know your options? Try our damage checker or read our onlineguides”“Don’t worry. We take all your details and damage online and work with most majorinsurers”“We test our websites on 1,000s of customers a year, so you don’t have to. 5 minutebooking”
    111. 111. #4 “Customers mistrust insurers and find dealing with their insurance situation very frustrating”1. Show that you understand the hassle of dealing with insurance companies – emphasise that you will help with their insurance paperwork for them, freeing them of this burden2. Where possible communicate the fact that the job is most likely to be free for insured customers, or good value for money for cash customers“Windscreen chip repair? If you’re insured, it’s usually free.”“Hate doing paperwork? We work with most insurers to do all that for you. Call us today on xxxx”“Insurer approved. Most major insurers work with us, so you can book online direct, today.”“Competitive quotes. For the glass you need, you’ll great service and the best price withoutcompromising on quality or guarantees.“It may cost less than you think. We work with most insurers to get a quality windscreen fitted. ”“Hate paperwork? So do we. Call us direct and we’ll work with most insurers to get your appointmentbooked in today”.“You pay nothing but your excess. We bill directly to most insurance companies so we can save you thepaperwork”
    112. 112. #5 “Some customers cannot be bothered to take action to fix their car glass ”Emphasize the consequences of not doing anything, e.g. ‘It’s going to costyou more if the chip develops into a crack’“Chip turning into a crack? Stop it today. Book online with Autoglass and get your slotreserved today.”“Chipped windscreen? It could cost more on your insurance to leave it for later”“Chipped windscreen? Leaving it late could cost you more money or bonus”“Cracked windscreen? Stop worrying today and get it replaced, at a convenient time for yourdiary. Book online and get your slot booked today.”“If your chip turns into a crack, it may trigger your insurance excess. Let’s take the worryaway by booking today. Autoglass.”
    113. 113. SummaryI’m stressed out – reassure me. Take the pain and worry away.Make it safe – I’m worried it will fall in, crack or chop my head off – humour me.Expertise – I don’t know how you’ll diagnose it. Looks unique to me, like a squiggle. I’m worried you’ll getthe wrong glass and it will all go wrong from there. How are you going to do this online? Are your peopletrained or just call centres that can’t do anything and don’t know anything. How do I know all of this? Areyou going to help me or jerk me around?Explain and Goal orient – Explain what’s going to happen – an outcome, a good thing, a contact, anevent, a saving of time, a safer car, a finished repair. Take them in a time machine to when it’s done.Assess damage – How will you do this online? How do you work out where it is on my screen? What ifI’m not sure. Ooops – I think I chose the wrong option. I’m worried about picking these options – how areyou going to help me here? Is it really easy to use or complicated like lots of car stuff?Contact and service – emphasise contact and service, particularly if you can quote KPIsBenefits – What’s the value proposition here? What does the customer get? What’s the impliedemotional content of the message? What will it trigger in people who read it, in the context of the keywordsearch they just made?Online USP – Why not just phone you eh? Probably going to ask me the questions all over again. Howcan I know that it will be fast, easy and painless? How do I trust you here not to make it hard and longand boring.Insurers – Often seen as the Cerberus dog by customers, will often call them for anything car related.We need to emphasise we can do this direct. Strongest and best message is around avoiding paperwork.Be careful about messages here that imply bad things about insurers (legal).Free – Find ways to say free, nearly free, almost free. Free for most people. Find a way to say free –because if you are broadly free for something, just say it. This works, the F word.Hurry up – Some customers will coast, not getting anything done. Time to rock their world, at least interms of the video they play in their heads after you’ve written some good copy here.
    114. 114. Worldwide testing : Culture?• If you operate internationally, use each country results to inform or drive tests in other markets• We don’t see cultural artefacts in testing – there isn’t a ‘German’ or ‘Dutch’ cultural response, unless the test itself exposes this.• We *do* see market, insurance, driver differences• We *don’t* see huge variances in how techniques work elsewhere – unless the browser model is different (e.g. Right to left reading)• That offers a great opportunity for international testing• Don’t be afraid of ‘click here’ either• Haz Clic Aqui, Klik Hier, Click here, Cliquez ici• Key learnings – goal oriented keywords, simplicity, less words, benefit driven, appeals to their mental model
    115. 115. Copy optimisation – short summary• Think of the optimisation process as a scent trail you’re laying• Take away words, then more again.• Use Front loaded, very plain – lower educational level Language• Persuasive, meaning laden, paints a picture with few words• Customer words, not corporate ‘drone speak’ or ‘our language’• Simplicity – less words, better chosen for meaning• Ask how the customer interprets – not you
    116. 116. Copy optimisation – short summary• Value proposition – where is it? Is there one?• Thinking about goal and emotionally oriented copy, not clicks• Be honest, not woolly• Stop using things that don’t push customer buttons (24/7 – for example)• The emotional message – the Tennis Meeting example
    117. 117. Better ad copy - spot the winner
    118. 118. Lowest Cost per Lowest CPAHighest Click Click HighestThrough Rate Conversion Rate
    119. 119. #25 – Sharing the test results – the best way?• Take a look at www.whichtestwon.com• Aim for the same thing internally• Get people to gamble on tests you’re running• Have a sweepstake or bet on the results• When the result is declared, talk lots about it• Circulate the results and add to a database• Use the old test results to train new people• Put all your failures in there• Write a short blog post about your tests• Run this like a commercial blog or website• Get people interested, involved and losing money!• Even the best UX and CRO people only get 60+% right.• Flip a coin, anyone?
    120. 120. #26 – Springing to the next test• So you’ve completed your test and published the results• Do you (a) Bask in glory at how great you are (b) Stop testing (c) Wait a while or (d) Declare your work is done?• Always aim to run the next test immediately after the previous one, whilst the knowledge is fresh• Operate a continual testing program on key pages• Take all your results, insights and your brainstorming team• Rework your hypotheses based on test results, new data, insights or segmentation.• Your mission is to drive another shift or turn a failure or inconclusive test into a win• DO NOT test in a vacuum – ALWAYS consider history• If you tested subtleties or the result was small – be bolder!
    121. 121. #27 – How can I test *more* stuff than I do now?• What are your constraints?• Most common problem – Time to launch• Second common problem – Not bold enough• Third common problem – Test takes too long to finish• All of these push down the key metric – number of tests *completed* per month – not started• Are you running multiple streams of testing?• Are you working on new tests as current tests run?• Is your vendor or IT department slowing things down?• Lean Methodologies = cutting waste from the process• Make sure you know where the delays and wastage are• Find a way to interleave or run multiple test patterns
    122. 122. #28 – What’s the best kept secret in CRO?• That 12.5% test you ran 3 months ago?• Happy at the results? Still on your slide deck?• The lift is probably not the same anymore• If you retest, you’ll get a different result!• Traffic, marketing, response and customers -> change• My tip – leave 5-10% running on the old test• Helps you prove the ROI continues• However – don’t rely on payback from one test• Run a continual program• You’re only as good as your next test• Shorter test run lengths are usually better – as they drive shorter insight payback. MVT is still great for complex or multiple variable monitoring• Much more impressive to quote long term lifts
    123. 123. #29 – What is a good conversion rate?• Higher than the one you had last month!• Use benchmarking services (usually irrelevant)• Use hitwise – to see competitor traffic and conversion• Analyse competitor weaknesses with a CRO or funnel review.• Continual improvement, test throughput and velocity of change will make it rise every month• Don’t trade on your test results – trade on the annual rise in conversion. Try to beat your own conversion rate – as this is what you have control over! Don’t assume your competition gets better conversion.• This is a long term game but most clients of mine get around 15- 40% improvement in core conversion within 3-6 months.• Belron has compounded growth over 4 years of 66% higher core conversion – growing the business hugely.• I can tell you that their core conversion is > 8%. Does this help?
    124. 124. #30 – What are the key success factors?• Number of tests completed a month• Implementing changes directly, in batches (JFDI)• Usability testing and customer input (better tests)• The throughput of your IT team and test vendor• The quality of your QA checking (wasted tests)• Instrumenting for the split testing tool AND analytics• Segmentation insights• Learning from every test + embedding this in the team• Being bold and brave – button colour test if you’re google!• Declaring test results only when they are ready• Running multiple fix/test/analyse/investigate – activity streams
    125. 125. BEST PRACTICE – QUESTIONS?
    126. 126. Unexpected CRO
    127. 127. #30 – What part do images play? FRANCE +8% over control 99% confidence
    128. 128. GERMANY+12% over control99% confidence
    129. 129. SPAIN+22% over control99% confidence
    130. 130. Make sure you use real, authentic, smiling, friendly, approachable people. Don’t use stock images for testing or on your site. Low Q video and images are also fine – they reinforce authenticity.www.headsethotties.com www.awkwardstockphotos.com
    131. 131. Take the fake smile test : bbc.in/5rtnv
    132. 132. Images that Suck!
    133. 133. Image Guidelines for Belron Photo Guidelines• Groups, vans, stores, corporate guff = • Female images work best in almost NO every country – they slay the guys• Single person – not groups or even person + customer • Natural, authentic and not „model‟ or• Open, Friendly, Natural, Smiling, Engag „stock‟ image types work best for ed = YES people• Uniform – has huge effect, even • Smile MUST involve the cheeks and wearing branded ball cap improves eyes must crinkle! Take the smile test conversion here to see how good you are• Look straight at viewer or slightly towards CTA button – don‟t ignore the • Photo shoots and tests can be used to visitor! drive advertising strategy• Plain backgrounds only • Pre Test your TV advertising before• No folded arms, arms on you shoot the video! hips, pointing, hands behind back, hanging useless by the • We get better images now, but we side, clasped by groin, fiddling with keep working on the guidelines! hands • We once got a genital framing• Arms with prop instead – hold a clipboard, rag, tool – I‟m working for picture… *you*
    134. 134. “We alwayshad a plan for pulling out”
    135. 135. “We alwayshad a plan for pulling out”
    136. 136. • Make sure you use real, authentic, smiling, frien dly, approachable people. Real smiles work best so make people laugh! • Don’t use stock images for testing or on your site. • Where models or actors eyes are looking affects conversion. True for single models and groups. • Low Q video and images are also fine – they can reinforce authenticity.Read this book : t.co/xYW6izLI8n and visit www.photoux.co.ukRead everything by Susan Weinschenk (@TheBrainLady)
    137. 137. Evil CRO• Optimisation can be used for evil!• Either by accident (just stupid)• Or by design, to trick peoplewiki.darkpatterns.org/wiki/Home
    138. 138. Sneaky fold
    139. 139. Sneaky basket
    140. 140. Hidden opt-out
    141. 141. Ripped
    142. 142. Evil genius award?
    143. 143. So how do I become an Optimiser?• It’s a state of mind, not just a course!• You need to read lots, test loads and watch customers even more• Ask someone to mentor you• Go to conferences, events, find people online• Don’t say “I think” – say “I’ll find out” or “I’ll test that”• Volunteer for shit jobs and do them well• Be the glue between people – volunteer for cross team work• Fix bad things so your team loves you• Always connect others with outcomes• Hack your job to a better life• Make yourself more valuable, not less• Challenge fucking everything• Be annoying and never give up!
    144. 144. Don’t give up!• Start with the customer at the heart of everything• It’s liberating!• Use low budget solutions, cheap or free tools to get business cases made or gain insight• Money talks so even micro testing will illustrate ROI• Invest in people who care about customers, as well as conversion• Invest in analytics talent and instrumentation• If you can make a 10% shift in a key metric (CTR, conversion), just from smart thinking, then what would a team of 5 get you?• If you want to hire people, ask me for advice!
    145. 145. HOMEWORK 1• I’d like you to look at how unconscious action is part of your life every week.• Several times a day, you’ll use a door. There are many different interfaces for doors like handles, knobs, buttons, push plates, levers and more.• We all go through every day using these things and don’t consciously think about what we’re doing. You’ll use them at work, at home, when you travel, shop or use the loo!• There are several things you’ll spot if you keep a door diary for a few days. Let’s give you 3 weeks to finish, to give you time to fit this in. Here is your work:1. Explore. See how many different types of door interface you can spot. Take pictures of them and add notes on your phone or using an app. Take notes if you like with a notepad and pencil. Photographs are especially useful for showing examples.2. Patterns and Groups. Do these door interfaces have a pattern? Do they fit into groups? What would you call these groups?
    146. 146. HOMEWORK 23. The Furnishings. Take a look at the door furniture and signs: • Is there any other stuff apart from the handle that you look at? • What signs or stuff are plastered on the door? • Are there any messages telling you stuff?4. Error with door. What happens when it goes wrong?. This is really hard to catch but if you keep trying for a week, you’ll spot a few. What happens when the door interface goes wrong? You get it the wrong way, curse to yourself and then do something different. What do you notice about when this happens?5. What caused it? What was it, when it all went wrong, that led you to ‘get the door wrong’ and have to try again. What went wrong that didn’t happen with all the other doors? If you watch the door, does it happen to other people?6. Summary So keep a log if you can (scribbled notes, mobile phone app, photos) and look at the five things I’ve listed. There might be more stuff than I’m hinting at so observe closely. • How many different ‘kinds’ of door interface can you spot? • Are there groups of them – similar kinds? What would you call these? • Catch yourself when it goes wrong • Watch other people when it goes wrong • Why did it go wrong
    147. 147. Collecting the Evidence Apps• https://itunes.apple.com/au/app/this-is-note-calendar-+- photoalbums/id403746123?mt=8• https://itunes.apple.com/us/app/awesome-note-+to-do- calendar/id320203391?mt=8• http://www.blurb.com/mobile Inbox or Stream based• http://www.memonic.com/tour#web-clipper• https://launch.unifiedinbox.com/
    148. 148. More reading. Slides and resources on slideshare.net Email : sullivac@gmail.com Twitter : @OptimiseOrDie : linkd.in/pvrg14 Slideshare : slidesha.re/nlCDm6
    149. 149. #5 Site Reviews!
    150. 150. Obligatory promo slide @OptimiseOrDie• Conversion Rate • Contact deflection and Optimisation online self service• A/B and Multi-variate • Site search analytics testing • Site, Page and Campaign• Cross channel optimisation optimisation• User Centered Design • Test design and execution• Usability testing • A long usable site• Customer Research & Insight portfolio• Web Analytics • ROI on UX improvements• Browser and email • Over 19 years of really compatibility boring meetings Director of Optimisation, RUSH Hair Optimiser for Hire
    151. 151. CRO and Testing resources• 101 Landing page tips : slidesha.re/8OnBRh• 544 Optimisation tips : bit.ly/8mkWOB• 108 Optimisation tips : bit.ly/3Z6GrP• 32 CRO tips : bit.ly/4BZjcW• 57 CRO books : bit.ly/dDjDRJ• CRO article list : bit.ly/nEUgui• Smashing Mag article : bit.ly/8X2fLk
    152. 152. Level 1 Level 2 Level 3 Level 4 Level 5 Starter Level Early maturity Serious testing Core business value You rock, awesomely Dedicated team Ninja Team Local Heroes Small team Cross silo team Culture Volume Testing in the Chaotic Good Low hanging fruit Systematic tests opportunities DNA_____________________________________________________________________________________________ _ Outline process Process Ad Hoc Well developed Streamlined Company wide_______________________________________________________________________________________________+ Multi variate Guessing +Funnel optimisation +Cross channel testing +Spread tool use Dynamic adaptive Testing A/B testing Session replay Call tracking Integrated CRO targeting focus Basic tools No segments Some segments and analytics Machine learning Micro testing Segmentation Realtime________________________________________________________________________________________________Funnel analysis Analytics Bounce rates + Low converting + Funnel fixes + offline integration Multichannel funnels Big volume Forms analytics focus landing pages & High loss pages Channel switches Single channel Cross channel picture synergy________________________________________________________________________________________________ Analytics +Regular usability + Customer sat + All channel view testing/research +User Centered of customer Insight Surveys Prototyping Design scores tied to UX Driving offline Contact Centre Rapid iterative methods Low budget Session replay Layered feedback testing and using online usability Mini product tests design All promotion Onsite feedback driven by testing________________________________________________________________________________________________ Prove ROI Mission Get buyin Scale the testing Mine value Continual improvement
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×