E Metrics Summit May 2006


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • . [Next]
  • [Next]
  • E Metrics Summit May 2006

    1. 1. Form improvements Patricia Gildea, e-Delivery Manager, npower.com E-metrics Summit, London, May 5, 2006 e-DeliveryTeam@npower.com 05-May-06 Web analytics in every decision: from micro to macro
    2. 2. <ul><li>npower - one of top UK utility companies </li></ul><ul><li>Serving residential & business customers </li></ul><ul><li>Website content & functionality includes: </li></ul><ul><ul><li>Brand engagement, sponsorship, etc. </li></ul></ul><ul><ul><li>Corporate info </li></ul></ul><ul><ul><li>Marketing & sales </li></ul></ul><ul><ul><li>Customer service </li></ul></ul><ul><ul><li>Social action programmes, education, etc. </li></ul></ul>Overview e-DeliveryTeam@npower.com 05-May-06
    3. 3. <ul><li>Over 5 year life of this brand & site, we ’ve moved from: </li></ul><ul><ul><li>Log files, to </li></ul></ul><ul><ul><li>Basic web trends package, to </li></ul></ul><ul><ul><li>[unnamed] analytics package to </li></ul></ul><ul><ul><li>Red Eye managed service </li></ul></ul>npower & web analytics e-DeliveryTeam@npower.com 05-May-06
    4. 4. <ul><li>Why a managed service? </li></ul><ul><ul><li>Very small team at the time </li></ul></ul><ul><ul><li>Little expertise in e-metrics </li></ul></ul><ul><ul><li>Business required extensive support in learning curve and ongoing reporting </li></ul></ul><ul><li>New vendor selected with managed service one year ago </li></ul><ul><ul><li>Red Eye been instrumental in moving us forward </li></ul></ul>npower & web analytics e-DeliveryTeam@npower.com 05-May-06
    5. 5. npower & web analytics <ul><li>So how is it used now? </li></ul><ul><ul><li>On average 3-10x week </li></ul></ul><ul><ul><li>Daily micro-decisions by web delivery team </li></ul></ul><ul><ul><li>Meso-design considerations on site journeys, sections, commercials </li></ul></ul><ul><ul><li>Macro-decisions on strategy & site structure </li></ul></ul><ul><ul><li>All examples here will be residential </li></ul></ul>e-DeliveryTeam@npower.com 05-May-06
    6. 6. At the micro level e-DeliveryTeam@npower.com 05-May-06 <ul><li>Examples - micro-decisions using analytics: </li></ul><ul><ul><li>- Prioritising bug-fixing </li></ul></ul><ul><ul><li>- Prioritising browser support </li></ul></ul><ul><ul><li>- Retiring v. updating pages </li></ul></ul><ul><ul><li>- Navigation exposure </li></ul></ul>
    7. 7. At the micro level <ul><li>Circular journeys, frustrated feedback </li></ul><ul><ul><li>Case study: Contact Us </li></ul></ul><ul><li>Minor text changes for increase in conversion rates </li></ul><ul><ul><li>Case study: “just skip it” </li></ul></ul>e-DeliveryTeam@npower.com 05-May-06
    8. 8. Case study: Contact Us <ul><li>Noticed increase in website feedback asking for information that was already in Contact Us section of site. </li></ul><ul><li>Analysed most popular paths - found circularities </li></ul><ul><li>Then identified key area of user confusion </li></ul>e-DeliveryTeam@npower.com 05-May-06
    9. 9. Case study: Contact Us <ul><li>Before </li></ul><ul><ul><li>“ Electricity and Gas contacts” link not highly used, but should be </li></ul></ul><ul><ul><li>Details unintentionally buried one level down </li></ul></ul>e-DeliveryTeam@npower.com 05-May-06
    10. 10. Case study: Contact Us <ul><li>Redesign: </li></ul><ul><ul><li>Move these contact details up a level </li></ul></ul><ul><ul><li>Reorder link lists and hierarchy of customer service section to reflect most common areas of usage </li></ul></ul><ul><ul><li>New wireframe prepared, pages rebuilt </li></ul></ul><ul><ul><li>Section streamlined </li></ul></ul>e-DeliveryTeam@npower.com 05-May-06
    11. 11. Case study: “just skip it” <ul><li>Focus on attrition rates through application form ahead of planned significant increase in e-marketing spend </li></ul><ul><li>Plug holes in “leaky bucket” </li></ul><ul><li>Application form: 7 steps </li></ul><ul><li>Largest page-to-page attrition: step 4 to 5 </li></ul>e-DeliveryTeam@npower.com 05-May-06
    12. 12. Case study: “just skip it” e-DeliveryTeam@npower.com 05-May-06 Supply Number is not mandatory. Meter point reference number is not mandatory.
    13. 13. Case study: “just skip it” e-DeliveryTeam@npower.com 05-May-06 Added: “ Don’t know this? Just skip it. ” Added: “ Don’t know this? Just skip it.”
    14. 14. Case study: “just skip it” e-DeliveryTeam@npower.com 05-May-06 <ul><li>Results </li></ul><ul><li>3% improvement on this step alone due to just this tiny change </li></ul><ul><li>Cautionary note! </li></ul><ul><ul><li>Use judiciously as similar use may increase deletions, churn, increase back-office costs. </li></ul></ul>
    15. 15. At the meso-design level <ul><li>Case study: connecting journeys </li></ul><ul><ul><li>Core acquisition journey for residential supply signups </li></ul></ul><ul><ul><li>Savings calculator compares npower prices against existing supplier </li></ul></ul><ul><ul><li>Application form (electronic contract) </li></ul></ul>e-DeliveryTeam@npower.com 05-May-06
    16. 16. Case study: connecting journeys <ul><li>Part of attrition rates study ahead of increase in e-marketing spend </li></ul><ul><li>Proposal: Connect application form to savings calculator </li></ul><ul><li>Purpose: reduce attrition, increase conversion through reduced user inputs, reduced opportunity to exit journey </li></ul>e-DeliveryTeam@npower.com 05-May-06
    17. 17. Case study: connecting journeys e-DeliveryTeam@npower.com 05-May-06 % Users moving between: After connection Step 1 to 2 Increase of 28.84% Step 2 to 3 3.72% Step 3 to 4 2.13% Step 4 to 5 20.98% Step 5 to 6 2.54% Step 6 to 7 3.09%
    18. 18. Case study: connecting journeys <ul><li>Looks pretty good, right? </li></ul><ul><li>Sales crashed by over 50%! </li></ul><ul><li>Why?? </li></ul><ul><li>Two reasons: price and required data </li></ul>e-DeliveryTeam@npower.com 05-May-06
    19. 19. Case study: connecting journeys <ul><li>Price issues: </li></ul><ul><li>All users then forced through calculator </li></ul><ul><li>At that time, we were not aggressively competitive on price in this channel </li></ul><ul><li>Therefore all users (= prospects) were exposed to pricing strategy </li></ul><ul><li>Only small numbers of areas/payment methods/consumption journeys completed </li></ul>e-DeliveryTeam@npower.com 05-May-06
    20. 20. Case study: connecting journeys <ul><li>Required data issues: </li></ul><ul><li>All users forced through calculator </li></ul><ul><li>Therefore, to sign up, user (= prospects) must now have to know current supplier, current tariff, current spend/consumption </li></ul><ul><li>Also, users/prospects who wanted to sign up not on savings but values, brand, sponsorship were forced through irrelevant savings journey </li></ul>e-DeliveryTeam@npower.com 05-May-06
    21. 21. Case study: connecting journeys <ul><li>Since then, Sign Online tariff was introduced (very competitive) </li></ul><ul><li>Journey options further developed where benefit of connection maintained for reduced burden on user, but calculator usage not forced </li></ul>e-DeliveryTeam@npower.com 05-May-06
    22. 22. Case study: connecting journeys e-DeliveryTeam@npower.com 05-May-06 Increase in % Users moving between: Connect Disconnect Step 1 to 2 28.84% 3.17% Step 2 to 3 3.72% -1.56% Step 3 to 4 2.13% -1.85% Step 4 to 5 20.98% -2.15% Step 5 to 6 2.54% -0.26% Step 6 to 7 3.09% 2.76%
    23. 23. Case study: connecting journeys <ul><li>Total volume of sales increased </li></ul><ul><li>Decrease in user complaints about wanting to sign up on brand/values (e.g. green) but being forced to calculate savings </li></ul><ul><li>Also decrease in complaints about not having arcane details to hand (e.g. tariff) </li></ul>e-DeliveryTeam@npower.com 05-May-06
    24. 24. At the macro-analysis level <ul><li>Largest ‘leaky bucket’ holes plugged (ongoing development project) </li></ul><ul><li>Time to start pumping volumes into site </li></ul><ul><li>E-marketing campaigns: banners & skyscrapers, email and PPC </li></ul><ul><li>First significant campaigns launched </li></ul>e-DeliveryTeam@npower.com 05-May-06
    25. 25. Case study: early campaigns <ul><li>Typical banner campaign set-up </li></ul><ul><li>Commercial success measurement of Cost per Contract (CPC) plus volume </li></ul><ul><li>Results include low cost per arrival, high click-through to first step of calculator or application form (disconnected journeys) BUT poor CPC and volumes </li></ul><ul><li>Anecdotal evidence - post-impression issues </li></ul>e-DeliveryTeam@npower.com 05-May-06
    26. 26. Case study: later campaigns <ul><li>Subsequent campaign trials included post-impression, post-session behaviour measurement </li></ul><ul><li>How do consumers actually buy electricity & gas online? </li></ul><ul><li>The “considered purchase” debate </li></ul>e-DeliveryTeam@npower.com 05-May-06
    27. 27. Case study: later campaigns <ul><li>Banners </li></ul><ul><ul><li><1% of ads served resulted in a click. <1% of arrival converted. </li></ul></ul><ul><ul><li>However , ‘post impression’ customer acquisition increases by 500% </li></ul></ul><ul><li>PPC </li></ul><ul><ul><li>3.7% of arrivals result in a contract </li></ul></ul><ul><ul><li>However , the lifetime of the visit result in a 4.5% conversion rate and increased sales of 22% </li></ul></ul>e-DeliveryTeam@npower.com 05-May-06
    28. 28. Case study: later campaigns <ul><li>Now challenges are to further understand measuring post session behaviour </li></ul><ul><li>Issue of integrating RedEye metrics with multiple other campaign vendor tagging & measurements (e.g. MSN) </li></ul>e-DeliveryTeam@npower.com 05-May-06
    29. 29. Case study: homepage strategy <ul><li>npower.com serves B2C, B2C and corp </li></ul><ul><li>Homepage had become a free-for-all; no clear strategy, no clear priorities in use of real estate </li></ul><ul><li>“ Squeaky-wheel” design </li></ul>e-DeliveryTeam@npower.com 05-May-06
    30. 30. e-DeliveryTeam@npower.com 05-May-06
    31. 31. Case study: homepage strategy <ul><li>Strategy project affirmed: </li></ul><ul><ul><li>npower.com is retail-level asset </li></ul></ul><ul><ul><li>Homepage need balance & simplification in structure </li></ul></ul><ul><ul><li>‘ Challenger’ brand campaign required re-branding </li></ul></ul><ul><ul><li>Significantly reduced real-estate allocations required set of decision-rules to manage </li></ul></ul>e-DeliveryTeam@npower.com 05-May-06
    32. 32. e-DeliveryTeam@npower.com 05-May-06
    33. 33. Case study: homepage strategy <ul><li>Decision-rules </li></ul><ul><ul><li>Every campaign, new product, promotion or initiative should have a predicted NPV and predicted web usage (from business case). </li></ul></ul><ul><ul><li>Replacement rules are based on a comparison of predicted NPV and usage with adjusted NPV and usage, and then against predicted NPV and usage of the new initiative. </li></ul></ul>e-DeliveryTeam@npower.com 05-May-06
    34. 34. Case study: homepage strategy <ul><ul><li>For example: </li></ul></ul><ul><ul><li>Campaign A has an NPV = 10 & usage =1000/wk. </li></ul></ul><ul><ul><li>>Launched on day 1. </li></ul></ul><ul><ul><li>+10 days, Campaign B briefed in for launch on day 30. </li></ul></ul><ul><ul><li>Campaign A ’s NPV and usage is then adjusted based </li></ul></ul><ul><ul><li>on actuals from day 1 to 15, projected to day 30 and </li></ul></ul><ul><ul><li>compared against predicted NPV and usage for B. </li></ul></ul><ul><ul><li>To replace A, B must be predicted to outperform A. </li></ul></ul><ul><ul><li>If yes, then B replaces A. However, B is monitored and if B doesn ’t outperform the predicted A metrics, then B could be pulled and replaced with A. </li></ul></ul>e-DeliveryTeam@npower.com 05-May-06
    35. 35. Case study: homepage strategy <ul><li>Decision-rules </li></ul><ul><ul><li>Web analytics critical to analysis on predicted and adjusted usage </li></ul></ul><ul><ul><li>Web analytics become the lens on reality, combating “audience of one” decision-making </li></ul></ul><ul><li>Measuring success of new homepage </li></ul><ul><ul><li>Benchmarks and comparison reports of before and after </li></ul></ul>e-DeliveryTeam@npower.com 05-May-06
    36. 36. Case study: homepage strategy <ul><li>We will be measuring: </li></ul><ul><ul><li>Immediate exits from homepage including duration </li></ul></ul><ul><ul><li>Top 10 journeys completion rate </li></ul></ul><ul><ul><li>Customer frustration level </li></ul></ul><ul><ul><li>“ Wandering journeys” </li></ul></ul><ul><ul><li>Split click-throughs of B2C v. B2B </li></ul></ul>e-DeliveryTeam@npower.com 05-May-06
    37. 37. The future <ul><li>Challenges will include </li></ul><ul><ul><li>Measuring segmented journeys with targeted content: “Conversion Enhancement” </li></ul></ul><ul><ul><li>Converting to dynamic content management system and measuring dynamic pages </li></ul></ul><ul><ul><li>Measuring multi-channel experiences & journeys </li></ul></ul>e-DeliveryTeam@npower.com 05-May-06