Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Presentation Name
Speaker’s Name
Title
Ecommerce Test Ideation: What to Optimize and Why
• Optimizely's top solutions partner for enterprise brands & retailers
• Partner to over 30 clients including Adidas, Tory...
so, you want
test ideas.
hypotheses learnings investmentsproblemsgoals
wouldn’t you rather a repeatable framework for
prod...
the ideas will come.
but, let’s start with the framework.
goals problems hypotheses learning
UX
product
merchandising
mark...
unifying.
wait like this applies to everything?
#1 Everything is an experiment
Banner/hero changes for new
seasons/campaig...
how to
communicate goals
good example:
In 2016, we want to increase
smart phone purchase
conversion rates by 10% y/o/y.
ba...
how to
capture problems
good example:
Our customers have a problem
understanding the difference
between and value proposit...
how to
articulate a hypothesis
good example:
I believe that if we add geo-targeted
shipping commitments on the shipping
pa...
Be consistent about requiring
all changes get mapped to
problems & hypotheses.
2
what do you do once
it’s all mapped?
Dete...
I believe that if we remove the “New
Customers” section on the interstitial
page between the ShoppingBag and
checkout, we ...
surface product options earlier
so let’s givesome real examples
Primary Test KPI
Conversion to Order
Data Story Metrics
PL...
I believe that if we offer ShopRunner’s
(SR) free two-day shippingservice to
visitors, then we will increase the
incentive...
I believe if we placed an inline email
signup in the header of our desktop
site, we will materially increase email
signups...
test idea process x
SPRING INTO ACTION
goals & problems
established
Do start with the critical
problems you want to solve
...
key
takeaways
The magic of optimization & personalization is not simply
creating experiments or campaigns. It is the magic...
pages.optimizely.com/spring-into-action
Upcoming Spring Into Action Events
-webinar-series
I
Upcoming SlideShare
Loading in …5
×

Ecommerce Test Ideation: What to Optimize and Why

1,444 views

Published on

Test ideation is at the core of growing your company’s online sales, especially as consumers’ expectations for ease of use and personalized experiences continue to climb. This session will help you uncover tests to increase key metrics, like conversion rates and average order value, and make shopping on your site a delightful experience for every customer.

Published in: Retail
  • Hey guys! Who wants to chat with me? More photos with me here 👉 http://www.bit.ly/katekoxx
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Ecommerce Test Ideation: What to Optimize and Why

  1. 1. Presentation Name Speaker’s Name Title Ecommerce Test Ideation: What to Optimize and Why
  2. 2. • Optimizely's top solutions partner for enterprise brands & retailers • Partner to over 30 clients including Adidas, Tory Burch, BluDot, Whole Foods, Cost Plus World Market, Express, Patagonia & many more • Proud winner of 2015 Opticon Partner of the Year & Most Dramatic Business Impact hi. i’m matty. founder & ceo at SPRING INTO ACTION Matty Wishnow & Leo Wishnow • 15+ years testing & validating as: • SVP, Direct to Consumer at Warner Music Group • Founder & CEO of Insound.com • Loves “Less and More” design principles & advanced baseball stats • Hates dabbling
  3. 3. so, you want test ideas. hypotheses learnings investmentsproblemsgoals wouldn’t you rather a repeatable framework for producing & prioritizing great test ideas? SPRING INTO ACTION
  4. 4. the ideas will come. but, let’s start with the framework. goals problems hypotheses learning UX product merchandising marketing prioritized by data validated by data investments 1 2 3 Ps pS ps PS 2 3 4 1 2 3 4 1 SPRING INTO ACTION
  5. 5. unifying. wait like this applies to everything? #1 Everything is an experiment Banner/hero changes for new seasons/campaigns. Merchandising changes. Copy changes. Redesigns & replatforms. New SaaS additions. Everything. #2 Questions to live by: What problems are the changes solving? How will you know if the changewas successful? SPRING INTO ACTION
  6. 6. how to communicate goals good example: In 2016, we want to increase smart phone purchase conversion rates by 10% y/o/y. bad example: We want to improve the mobile user experience. Ask yourself: • Are they widely understood? • Are they actionable? • Are they measurable? • Do they have refined baselines/targets? • Are they time-based? SPRING INTO ACTION
  7. 7. how to capture problems good example: Our customers have a problem understanding the difference between and value proposition of products on PLPs. bad example: I think we need a responsive website. Ask yourself: • Why is it a problem? • How does the problem impact the business or end user? • Is it a root problem? • Do you have data to confirm that it is a significant problem? SPRING INTO ACTION
  8. 8. how to articulate a hypothesis good example: I believe that if we add geo-targeted shipping commitments on the shipping page of checkout, customers will have more confidence in ordering from us. If I am right we will see increased conversion & reduced abandonment at the shipping page of checkout. bad example: I think we need to redesign checkout. Try to: • Be specific about the change you are contemplating. • Describe what you expect the result of that change to be. • Clearly define the metric you can most confidently measure success by. • Start with: • I believe that… • If I am right then… SPRING INTO ACTION
  9. 9. Be consistent about requiring all changes get mapped to problems & hypotheses. 2 what do you do once it’s all mapped? Determine if additional problem research is needed. 1 hypotheses learnings investmentsproblemsgoals Determine what should be tested vs “just do it”. 3 Review & iterate regulary.4 SPRING INTO ACTION
  10. 10. I believe that if we remove the “New Customers” section on the interstitial page between the ShoppingBag and checkout, we will reduce friction to continue to checkout for non-logged-in users. If I am right, order conversions will increase overall with minimal impact to new account sign-up rates. Primary Test KPI Conversion to Order Data Story Metrics New Account Creation Problem Currently, new customers can create a new account either on the My Account interstitial page, or on the Order Confirmation page. I believe having the “New Customers” section at the beginning of the checkout funnel is overwhelming to visitors overall, as they have to decide between logging into an account, checking out as a guest, and registering a new account. reducing checkout friction so let’s givesome real examples SPRING INTO ACTION
  11. 11. surface product options earlier so let’s givesome real examples Primary Test KPI Conversion to Order Data Story Metrics PLP Bounce Rate Problem Previously, we have tried showing every color as its own product on the PLP grid (thus creating many PLPs to scroll through) or shown only the “parent product” in a single color on the grid (thus requiring people to click into PDPs for more color selections). Neither solutions have been satisfactory for our business of our customers. I believe that if we denote which additional colors are available for each “parent product” on PLPs, customers will be more likely to identify a product color selection that appeals to them earlier in the funnel. If I am right, we will get more people to PDPs and see increased conversion rates. SPRING INTO ACTION
  12. 12. I believe that if we offer ShopRunner’s (SR) free two-day shippingservice to visitors, then we will increase the incentive to buy when able to get items faster at no additional cost. If I am right, we will see an increase in overall order conversion rates and revenue, driven by users taking advantage of the ShopRunner option. testing saas benefit claim so let’s givesome real examples Primary Test KPI Conversion to Order Data Story Metrics Bounce at Point of ShopRunner Option Problem Currently, we do not have ShopRunner as a shipping optionon the site. We offer free 4-7 day shipping on all orders (no threshold). Visitors can upgrade to faster shipping, but it comes as an additional cost. SPRING INTO ACTION
  13. 13. I believe if we placed an inline email signup in the header of our desktop site, we will materially increase email signups. If I am right, we will increase email signups and no decrease in conversion rate. inline email sign-up so let’s givesome real examples Primary Test KPI Email Newsletter Sign-Up Rate Data Story Metrics Conversion to Order Problem Currently, we have links to sign-up for newsletters at the header & footer of our websites. These links go to a second page with several forms to fill in. The extra steps, we believe, create friction for something that can/should be very simple. SPRING INTO ACTION
  14. 14. test idea process x SPRING INTO ACTION goals & problems established Do start with the critical problems you want to solve for your customers, mapped to the measurable goals that define success. Do not sneak solutions into your goal or problem statements. hypothesis generation hypothesis prioritization customer service data video session recordings drop off analysis web analytics customer service feedback customer survey data 1 2 3 4 3 1 2 effort to test potential impact likelihood belief is true ability to adopt and more X
  15. 15. key takeaways The magic of optimization & personalization is not simply creating experiments or campaigns. It is the magic of solving a customer experience or business problem. Metrics rise in correlation to your ability to solve problems. 1 Don’t lead with solutions. Don’t chase ideas simply because others have succeeded with it or because you are able to test the change. Prioritize your experiments based, initially, on the problems you expect them to solve. 2 Develop a unifying problem & solution mapping framework to ensure that you are working in concert to solve the biggest problems and that you are not dabbling. Getting winning tests is not winning if the biggest problems go unsolved and the biggest hypotheses unvalidated. 3 SPRING INTO ACTION
  16. 16. pages.optimizely.com/spring-into-action Upcoming Spring Into Action Events -webinar-series I

×