3. Consumer AB Test Performance3
0
12.5
25
37.5
50
Very
Poor
Poor Neutral Good Very
Good
0%
70%
14%
18%
Conversation Rate Lift Performance
(≤5%) (-5% to -1%) (-1% to -1%) (1% to 5%) (>5%)
The majority of
ideas are neutral
or negative
PercentageofTotalTests
56%
7%5%
A histogram of all the tests performed by the consumer optimization team reveals that the vast majority of
ideas proposed to improve the site were either neutral or had a negative impact on conversion.
4. Defining an evaluation framework
4
88%
The majority of people who indicated some interest in mobile devices never
found a product or category page.
Fallout from Awareness to
Consideration
5. Pepsi had a similar problem
Making small incremental changes can result in a big
impact on conversion
100%What looks best is not necessarily
what’s most effective in achieving
business.
Increase in
conversion rate
8. 8
Business Objectives
Grow Revenue
• Customer acquisition
• Frequency of purchase
• Increase basket size
Reduce Costs
• Reduce customer acquisition
cost
• Loyalty / return engagement
• Advocacy / referral
First Step: Define Business Objectives
Strategy
• Return engagement
• Content messaging
• Learn / test / implement
Tactics
• AB testing
• Segmentation – learn
• RFM analytics
• Promote
• Gamification
Next, develop a strategy to achieve business objectives. Then use tactics to implement the strategy.
9. Awareness
Retargeting, email, search, display…
Consideration
• Landing pages
• Product category pages
• Content based algorithms
Purchase Intent
• Shopping cart
• Collaborative based
Loyalty
• Post checkout
• Email reminders
At Samsung we learned the that primary variable for
predicting purchase intent was contract renewal.
We could then target the right customer with the right
message at the right time.
Each step in the customer journey will have specific triggers that will drive customer behavior.
Samsung Shopper Marketing Journey Map
Purchase
• Shopping cart
10. How to implement a
strategy:
Target & tailor communication to customers:
identify need and problem
Virtuous Cycle: Learn, Test, Implement, an
iterative Agile Framework
Learn, Test, Implement are the tactics:
This is how you develop, test, and refine
strategy.
Short iterative cycles mitigate risk. These
tactics are used to make process more
efficient. Ultimately, you build a better
product.
13. WHO ARE OUR CUSTOMERS
• Types
• Product Seekers
• Browsers
• Bargain Hunters
• Researchers
• One Time or First time shoppers
• Personas
• Hard Core
• Casual
• Gift Givers or
• Collectors
14.
15. • Conversion from consideration to purchase intent
• Coop
• Triad Top Blade
• Hero Banner
• Trade Center – second most viewed page
• Pre-owned –
• Assortment – 2,500 under $20
• Quality – guarantee 7 days
• value – 10% and 20%
• Awareness and consideration of new products (merchandising)
• New products – games and collectibles
• Rediscover old favorites
• Discover classic games sales and promotional products
• Loyalty
• membership / Registration
• Point generation
• Point consumption
• Benefits / Exclusive
• In-store Conversion
• Find a store (Store Locator)
• Hold for Pick Up in Store (HOPS and BOPS)
• Preorder
• Existing inventory
Business objectives:
16. HIGH LEVEL STRATEGY
• Single brand voice representing all business objectives.
• Leveraging a hierarchy of anticipated customer needs
• A simple unified experience seamless integrating all content and messaging
• Expose undiscovered categories to surprise and delight (Turning Product Seekers in to
Browsers)
• Contextual messaging based on product timing and relevance. (coming soon vs available
now).
18. People are seeing their points disappear and
they don’t know why
This person was auto-issued but not activated so they do not have
access to account details or their certificates and the stores are not
making them aware of the certificates on purpose.
19. These are the
steps that Michael
Hagarty must go
through get his
certificate if he
lost the email
27. ANDY’S UX HEURISTICS OF DESIGN
1. TRANSPARENCY, TRUST, AND CREDIBILITY VS. MANIPULATION
2. EMOTIONAL NOT COGNITIVE
3. HIERARCHY OF INFORMATION BASED ON CUSTOMER NEEDS
4. DISCOVERY THROUGH SURPRISE AND DELIGHT
5. AESTHETICALLY PLEASING, SIMPLE, MINIMALIST
6. RECOGNITION RATHER THAN RECALL DON’T BREAK
EXPECTATIONS
7. TIMELINESS AND APPROPRIATENESS
8. NO DARK PATTERNS
9. TASK ORIENTATION
10. PROGRESSION
28. 1. TRANSPARENCY, TRUST, AND CREDIBILITY VS.
MANIPULATION
1. Make a distention between promotional content and editorial content
2. Layout
3. Accessibility
29. 1. TRANSPARENCY AND TRUST AND CREDIBILITY VS. MANIPULATION
Make a distention between promotional
content and editorial content
30. 1. TRANSPARENCY AND TRUST AND CREDIBILITY VS.
MANIPULATION
• Are standard elements (such as page titles, website navigation, page navigation and privacy
policy) easy to locate?
• Is there a good balance between information density and white space?
• Does the website have a consistent and clearly recognizable look and feel that will engage
users?
layout and visual design
31. ACCESSIBILITY
• Is the color contrast across the website enough to make all of the content accessible?
• Does the website work comfortably at lower resolutions (e.g. 1024 × 768 pixels)?
• Does the CSS validate with the W3C’s validation services?
1. Transparency and Trust and credibility
vs. Manipulation
34. 2. EMOTIONAL NOT COGNITIVE
1. Lead with the emotion drill into the reason
2. Concrete value proposition rather than abstract
3. Don’t use marketing speak use the words that people use with each other.
35. Lead with the emotion drill into the reason
There are no product details on the
category page. You only see the images
meant to evoke an emotional response.
36. Very simple visual categorical
navigation structure that guides
customer through a decision tree.
Lead with the emotion drill
into the reason
37. 3. HIERARCHY OF INFORMATION BASED ON CUSTOMER
NEEDS
1. More space to that which is more important to the user
2. Lead with consumer needs not business objectives
38. Hierarchy
Once on the PDP page there is a strong
focus on the product itself.
Next most important element is add to cart
CTA.
Supporting test and other description is
clearly secondary.
Clear and easy to see recommendations that
are outside the category of the product itself.
39. 4. DISCOVERY THROUGH SURPRISE AND
DELIGHT
1. Common to others uncommon to me
2. User control and freedom – lead them down a path don’t force me around obstacles
3. Floccus on what customers what not what the bra
40. 1. Common to others uncommon to me
2. User control and freedom – lead them down a path don’t force me around obstacles
3. Floccus on what customers what not what the bra
41. 5. AESTHETICALLY PLEASING, SIMPLE, MINIMALIST
DESIGN
1. Balance between density and negative space
2. Creating just enough tension to create engagement
3. Challenge audience just enough
42. Tiles with asymmetric break points
However above and below this component there are
solid breakpoints that are about the same size of one
screen. If you do have sections make them smaller
then one full screen.
43. Tiles with long vertical images break up
the grid and leads the user down the page
. This achieves the effect within the
context of a standard grid pattern.
49. 6. RECOGNITION RATHER THAN RECALL – USE
PATTERNS THAT ARE FAMILIAR TO PEOPLE
• Use patterns that are familiar to people
50.
51.
52. 7. TIMELINESS AND APPROPRIATENESS
1. Only ask me for information when there is a clear benefit to me
1. Sign up to see if you qualify for savings
2. Organization and categorization
1. Coming soon vs available now
2. If you show inventory that is not available online display that info up front
3. Show me what I need when I need it
53. 8. NO DARK PATTERNS
• Bait and switch
• Disguised ads
• Friend Spam
• Hidden Costs
• Misdirection
• Road Blocks
• Sneak into Basket
• Trick Questions
http://darkpatterns.org/
54. WFSB.com, a TV news station
based in Connecticut, USA has
an unfortunate set of disguised
ads on their article pages
56. 9. TASK ORIENTATION GIVE ME THE TOOLS TO
DO WHAT I CAME HERE TO DO
• search and taxonomy match mental map
• Lead them down a path but show them where they are going and where they have been
• There should be no surprises
62. 10. PROGRESSION
• Only expose new users to basic functionality and reveal new functionality as they
progress through the experience.
• Only ask for info when clearly beneficial to the end user. Timeliness and
relevance.
63. PROGRESSION
Apple starts out with
the “Experience” that is
very aspirational and
emotions then moves
the user to “Specs” and
then “Buy”
70% (the red combined) of ideas brought to the consumer group have zero (or negative) impact to the business.
While this is taken from consumer, the process we’re using is the same, it’s subjective, decisions are made without visibility.
A data driven framework for analysis & development (that is objective in nature) will force the correct code to be written – yielding code that positively impacts business.
Create a framework that standardizes a methodology for opportunities.
Foundation for testing opportunities.
Decide where to test. Seeing that if customers cannot find the product page, don’t test there. Remove adhawk decision making.
Define the testing strategy, introducing analysis of where and how strategy should be implemented.
Testing to for discovery, to understand customer behavior at a high altitude.
Result of AB testing call to action above the fold.
Tried and true framework used with success.
Always test for a conversion rate, NOT total numbers. Always test for a %, test for engagement, time on site, any kind of metric that you can be normalized. Normalized goal metrics.
You may not have enough customers to test for conversion rates, and your goal may not be conversion rate. Enterprise goal should be return engagement….Their conversion rate is for lead generation. Goes back to business objectives. If Business objectives are to support sales channels, then metrics that ladder up to engagement is what’s being tested.
20% increase
.5 to 1 = 100% increase in conversion rate
10% to 30% = is a 20% increase in conversion rate.
It’s a tactic that supports the strategic roadmap.
With problem statement, we need to understand the framework for how and why it is deployed. We need to first understand business objective, then tactics to implement strategy.
Testing is an integral part to deploying a strategy.
***KD note: I changed employed to deployed. Is this right?
Top 5 "Actionable“ Items to Drive CSAT
Usability: Ease of Use, Learnability, and Delightful
NPS (Net Promoter Score)
Minimal Touch
Simplicity
Empowerment
Take the framework previously define and apply to any channel.
Virtuous loop versus a loop that degrades overtime, virtuous cycle
Graph has components of actual tactics.
Learn Test Implement are the tactics:
This is how you develop, refine, and test strategy
Short cycles allow for implementation of learning. Use to make process more efficient, ultimately you build a better product, the success or failure of the product is determined by previous release.
It’s a tactic that supports the strategic roadmap.
With problem statement, we need to understand the framework for how and why it is deployed. We need to first understand business objective, then tactics to implement strategy.
Testing is an integral part to deploying a strategy.
***KD note: I changed employed to deployed. Is this right?
Virtuous loop versus a loop that degrades overtime, virtuous cycle
Graph has components of actual tactics.
Learn Test Implement are the tactics:
This is how you develop, refine, and test strategy
Short cycles allow for implementation of learning. Use to make process more efficient, ultimately you build a better product, the success or failure of the product is determined by previous release.
Education around trade Half price books
Opportunistic without the intent they just want to save money
Value shopper – I have $40 what can I buy – not looking for anything – just browsing and
Price too close to new – lack of value
Bad experience with pre-owned and immediately stopped
It’s a tactic that supports the strategic roadmap.
With problem statement, we need to understand the framework for how and why it is deployed. We need to first understand business objective, then tactics to implement strategy.
Testing is an integral part to deploying a strategy.
***KD note: I changed employed to deployed. Is this right?