20. 2015
March April May June July August
Low Funnel
Optimize/Convert
View All
Plans
Enrollment
Funnel
Tabbed #2
All steps (1 test per week)
Usability
#2
New
Design
Integrate
Winners
New
Design #2
Super
Test Optimize (1 test/week)
The ‘Super Test’
21. Medicare View All Plans – Removing the ‘Quote Start’ page – Test ID: MED.005.14
Medicare Advantage landing page Quote start page View all plans
Projected Annual ROI Impact:
+17.32%
Removing the ‘Quote start page’ from the Medicare quoting funnel increased the number of
consumers who make it to the ‘View all plans’ page by 17.32%.
22. Medicare Quoting: View all Plans Modal – Test ID: MED.059.14
Funnel Pages Treatment Lift
(1) Getting Started 5.6%
(2) Your Info 8.5%
(3) Enrollment 8.4%
(4) Thank you 7.9%
+7.89%
A redesigned ‘View all Plans’ page Medicare modal that
displays when visitors need to choose a county, is increasing
traffic to the Thank You page by +7.89%.
Control Treatment
23. Medicare Quoting: View All Plans – Test ID: MED.057.14
Control Treatment 1
Including value prop messaging on the View
All Plans page does not increase traffic to the
next step of the funnel, but does increase
process-level value. This improves completed
application rate by +11.13%.
+11.13%
Treatment 2
24. Medicare Enrollment: ‘Your Information’ Page – Test ID: MED.050.14
Control Treatment
Projected Annual ROI Impact:Including a picture of a Medicare card, with sections needed to fill out our application
clearly outlined, improved clicks by +1.5% on the Medicare Enrollment ‘Your Information’
page, and increased completed applications by +6.65%.
25. Medicare Enrollment: ‘Enrollment Review’ Page – Test ID: MED.060.14
Control Treatment
Projected Annual ROI Impact:
A simplified version of the Medicare Enrollment ‘Review’ page had no significant effect on
performance.
26. Medicare Enrollment: Getting Started Page – Test ID: MED.061.14
Control Treatment 1
A simplified ‘Getting Started’ page in the
Medicare Enrollment funnel increased the
number of consumers going to the next step
by +6.98%, and completed applications by
+5.88%.
+5.88%
27. +5.85%
Clicks to the next step of the enrollment funnel
Experiment Summary
Simplified experience improved the
percentage of consumers making it to
the next step of the enrollment funnel
by +5.85%.
ID: MED.011.15 Name: Medicare Getting Started
Control Treatment
28. ID: MED.008.15 Name: Medicare Select Payment
+10.6%
Increase in completed
applications
Experiment Summary
Control Winning Treatment
Changes Tested
• Refreshed page style and clearer navigation elements
• Removed unnecessary copy to simplify and unclutter the
page
• ‘Financial help’ copy placed behind an expander
URL: https://www.humana.com/medicare-secured/enroll/payment-details
29. 92%
In completed applications
Experiment Summary
• A simplified page increased
engagement with most
form fields by +29% -
+217%
• A lack of auto-formatting in
the tricky ‘Medicare
Information’ section caused
anxiety
• Respondents to the next
page decreased by -8%,
Completed applications
down by -92%
ID: MED.012.15 Name: Medicare Personal Information
URL: https://www.humana.com/medicare-secured/enroll/your-information
Control Treatment
30. 11.39%
An ‘accordion’ layout had no effect
on entrants into the quoting funnel.
It did decrease completed
applications by 11.39%.
Experiment Summary
Plan Summary
Headline
Personalize Your Results
Plan Comparison
Plan Headers
Plan display layout
URL: https://www.humana.com/medicare-secured/view-all-plans
ID: MED.002.15 Name: Medicare View Plans “Accordion” Layout
Control Treatment
31. Treatment
A tabbed layout was being utilized,
though did not result in increased
applications. This was due to a sizable
percentage of users getting distracted by
an ‘Additional Resources’ section.
Experiment Summary ID: MED.003.15 Name: Medicare View Plans Tab Layout
Control
URL: https://www.humana.com/medicare-secured/view-all-plans
32. Experiment Summary
• Page layout utilizing tabs for plan types
• Less prominence of the ‘Additional
Resources’ section, (to decrease
distractions)
• Improved entrants into the enrollment
funnel by +11%
• Improved completed applications by
+12.75%
URL: https://www.humana.com/medicare-secured/view-all-plans
ID: MED.007.15 Name: Medicare View Plans “Tabbed” Layout v2
Control Treatment
33. +8.43%
Increase in completed
Medicare applications
Experiment Summary
• Simplify above-the-fold content
with plan ‘tiles’
• remove large ‘life blocks’ that
competed with the CTA
ID: MED.004.15 Name: Medicare View Plans “Stacked” Layout
Control
URL: https://www.humana.com/medicare-secured/view-all-plans
Treatment
34. ID: MED.014.15 Name: Medicare View Plans Tabbed LayoutExperiment Summary
Changes Tested
• Refreshed page style and clearer
navigation elements
• Removed unnecessary copy to
simplify and unclutter the page
• Used tabs to display different plan
types
URL: https://www.humana.com/medicare-secured/view-all-plans
TreatmentControl
13.69%
Fewer applications
35. Interpretation
• There was an 18% spread in getting to the next step when ‘Enroll now’ was the main CTA vs ‘Learn more’
• There was a 6% spread in completed apps with ‘Learn more’ getting more apps
• Declines of: 50% in ‘add pharmacy’, 35% in ‘add prescriptions’, 30% in ‘add doctor’ (FIX)
• ‘Additional resources’ clicks up 150% (FIX) (Baseline %, variance)
• ‘Change location’ up 90% from the control = 1% additional fallout (?)
• Compare plans decline of 40% (FIX)
• Filter usage decline of 98% (Not a factor)
• 14.87% easier to find plan details when closer to the main CTA (OK)
• HUGE decline in MED Advantage (FIX)
• HUGE increase in SUPP and PDP (OK)
Recommendations
• Re-test ‘learn more’ vs ‘enroll now’ when the rest of the page is fixed
• Make ‘Narrow your results’ more visible, add in plans section, and change the wording/description in the title to clarify
• Make other fixes as outlined above
Next Steps
• FIX and Re-test
Interpretation and Recommendations
37. Homepage overview
• Homepage current state composed of:
– Main header and navigation (mega menu)
– Carousel promoting LOB and brand campaign
– Products and Find a Doctor area
– Content Area: Vitality and RightSource
– Promo area
– Footer
There is a lot of low engagement content
on the homepage. Data and trends
indicate we need to simplify and target
what users want to do from the
homepage, what we want them to do and
align with our overall goals.
38. • Members are finding sign-in easily. It’s the homepage’s #1 clicked
element.*
• Low bounce rate (visitors who enter site on this page and exit)
• High percentage of new visitors (new members and prospects)
• Good engagement on Browse Products area and Find a Doctor
What’s Working
Primary User Actions
• Sign in
• Find a Doctor (#2 clicked element)
• Self-Selection (excluding Employer)
• Products
Need to align with
popular onsite
search terms
*LIVEView and Analytics dashboard data.
39. Promotion Impressions Clicks Click-Through Rate
Total 22,406,557 146,227 0.65%
Medicare_Shop_Plans_Homepage_Banner 2,314,494 43,195 1.87%
Apple_Healthkit_Integration_Homepage_Banner 1,866,870 1,988 0.11%
Medicare_Preview_Plans_Homepage_Banner 1,693,038 24,299 1.44%
ClosetheGap_Homepage_Banner 1,504,939 2,199 0.15%
Coverage_Questions_Homepage_Banner 1,377,281 1,171 0.09%
Medicare_Get_Started_Now_Homepage_Banner 44,688 918 2.05%
Discover_Rightsource_Mobile_App 31,544 910 2.88%
Carousel engagement is low (seasonal sales see increased CTR)*
What’s NOT Working
RS, Vitality and Promo content largely ignored
Vitality CTA = 0.15%
RightSource = 0.12% (.05% and .07%)
Flu promo = 0.03%
Walmart Promo = 0.1%
New to Humana Promo = 0.03%
TOTAL CLICKS = 45k ( < 0.4% of page clicks) for Q1 2015
*From Analytics report for Oct 2014
40. • Navigation menu items go largely ignored, with 5 links
making up the majority of clicks.
• Most clicked navigation elements are the main links at top
• Homepage has > 170 links
What’s NOT Working
No top actions, no tagline, no branding.
• Decide what actions we want our ideal customers to take.
• Best is a single action or a single emphasized action with
additional de-emphasized actions.
41. 41
Treatment 4
Experiment Side-by-Side
Control
• Product focus aligns with Enterprise Goal #1.
• Simplified product links.
• Simpler treatment for Doctor and Medication
finders
• Less imagery overall to test if a cleaner, text
link layout increases engagement.
42. 42
Treatment 5
Experiment Side-by-Side
Control
• Bolder design to emphasize branding
• Product focus to align with Enterprise #1
• Prominent Doctor and Medication finders
• Less distracting content overall to streamline
what we know the user wants to do and what
we want them to do.
43. 78%
In page CTR
Product links and
RighSource received
the largest lift.
Treatment 3
Increased Engagement
A) Audience Links ↑ 60%
D) Banner Links ↑ 41%
E) Product Links ↑ 239%
G) RightSource Promo ↑ 165%
Decreased Engagement
F) Find a Doctor ↓ - 16%
H) Vitality Content ↓ - 28%
I) Promo Content ↓ - 56%
J) Footer Links ↓ - 16%
D
M
G
K
E
F
L
J
I
B
C
H
A
44. The homepage funnel remained steady throughout the test. It has
never been a strong or wide funnel, though emphasizing products
did increase entrants to respective funnels.
Test ID: HHP.003.15
Test Name: Humana Homepage Redesign
URL: www.humana.com
• All treatments increased or had little impact to funnel traffic for Medicare, HumanaOne and
RightSource.
Experiment Results – Funnels
45. +240% in Products/Banner CTR
+78% in total page CTR
Test ID: HHP.003.15
URL: www.humana.com
• All treatments increased engagement with the products,
banners and RightSource content.
• Find a Doctor clicks decreased on all treatments (though
not significantly) except Treatment 5.
• Find a Medication received significantly less clicks than
Find a Doctor.
• Although footer links on the treatments were reduced in
number, the drop in clicks were low, indicating a less
complicated footer helps visitors find what they need.
• Humana at Home blurb and link received lowest
engagement, indicating lack of awareness and/or interest
among site visitors.
• Vitality treatments failed to increase engagement.
Experiment Highlights
Treatment 3 saw the highest increase in
overall clicks, mainly on the Products area.
