Customer Researchers as Health Professionals How eBay Uses Research to Improve Product Health Christian Rohrer Director, Customer Intelligence – Product Rian Van Der Merwe Manager, Customer Intelligence – Product Customer Intelligence eBay, Inc. © 2007 eBay Inc. All rights reserved. eBay and the eBay logo are registered trademarks of eBay Inc. Other trademarks are the property of their respective owners. Please don’t take our picture or record this class without first obtaining permission.

Context at eBay

Context at eBay

The elements of user experience Marketing campaigns It starts by being useful… Functionally, people must be able to use it… The way it looks must be pleasing … This plays a major role in creating the overall brand experience Executing well on all of these areas is what creates a great user experience. Research is needed for each. Brand experience My overall feeling about a brand/product (in the abstract) is good. I like the way the product looks and feels. Desirability Usability I am able to use the product easily. Utility It is useful to me. It meets my needs.

User Experience Research and Market Research User Experience Research Market Research - How to get people to come to the product - Who we want to target - What do people want? - Attitudes and potential buying habits Once people are at the product, do they understand what to do? - Motivations and potential usage patterns - User requirements: goals & behaviors - How should the product be designed? - Who should we build this for? - User insights throughout the product development process - Buying triggers - Industry trends - What should we build? - Competitive user experience designs & usability

Once people are at the product, do they understand what to do?

- User requirements: goals & behaviors

- User insights throughout the product development process

- Buying triggers

- Competitive user experience designs & usability

Product Development Stages and Classes of Research Role: New design ideas, discovery, product or business opportunities Innovation & strategy METHODS Ethnography: user needs (tacit & explicit) Competitive analysis Feature/task analysis Concept development & assessments Role: Improve, refine, and the design Design, reduce risk METHODS Usability inspections Participatory design Rapid, iterative design & testing (RITE method, paper prototyping) Desirability studies Usability (lab) studies Field studies Role: Measure or compare against self or competition Primarily quantitative METHODS Product Health/Tracker Online User Experience Assessments (Keynote) Data mining analysis Live (A/B) Testing Product surveys Customer Satisfaction Other Trackers STRATEGIC (Inspire) DESIGN RESEARCH (Inform) TRACK (Assess) Understand Conceive Design Develop Launch

Role: New design ideas, discovery, product or business opportunities

Innovation & strategy

METHODS

Ethnography: user needs (tacit & explicit)

Competitive analysis

Feature/task analysis

Concept development & assessments

Role: Improve, refine, and the design

Design, reduce risk

METHODS

Usability inspections

Participatory design

Rapid, iterative design & testing (RITE method, paper prototyping)

Desirability studies

Usability (lab) studies

Field studies

Role: Measure or compare against self or competition

Primarily quantitative

METHODS

Product Health/Tracker

Online User Experience Assessments (Keynote)

Data mining analysis

Live (A/B) Testing

Product surveys

Customer Satisfaction

Other Trackers

Business Levers Available to eBay “ 2007 is the year of user experience” – Meg Whitman Longest to implement Complex effects Has the most untapped potential Site improvements Generates more transactions that feed pricing lever Product (website) Seasonal or based on current needs Immediate effect, but fast decay Internet Marketing Direct Marketing Brand campaigns (TV, radio, print) Marketing Periodic revisions based on pricing research Immediate effect once implemented Positive & negative effect Fee structure Pricing changes Based on revenue model Pricing Usage and Results Description Business Lever

“ 2007 is the year of user experience” – Meg Whitman

Longest to implement

Complex effects

Has the most untapped potential

Site improvements

Generates more transactions that feed pricing lever

Seasonal or based on current needs

Immediate effect, but fast decay

Internet Marketing

Direct Marketing

Brand campaigns (TV, radio, print)

Periodic revisions based on pricing research

Immediate effect once implemented

Positive & negative effect

Fee structure

Pricing changes

Based on revenue model

Org Changes: Customer Intelligence - Product Customer Intelligence Market Research Analytics

Product Health Product Health Strategy Overview Product Health Synthesis Beyond the slides: driving product strategy

Product Health

Product Health Strategy Overview

Product Health Synthesis

Beyond the slides: driving product strategy

Product Health Strategy Overview What is Product Health? A suite of user experience research projects that, as a whole, provides an ongoing, holistic assessment of all different areas of our site, with specific focus on the key flows. Why are we doing it? We believe there is a genuine need for a holistic, cross-GXT view of our product to help guide us as a company to focus our development efforts for existing products and new ideas on those areas that will fulfill the most important unmet user needs. What is this Product Health Synthesis report about? Five key guiding principles for product development based on key insights from the Product Health components along with a visual model that maps these principles to the eBay site experience. These insights also help us create concrete examples of what products might look like when we apply these guiding principles What should all of this be used for? Each individual research component should be used by GXT leaders and their teams to develop and prioritize their product roadmaps. The guiding principles from the Product Health synthesis should be used by executives to provide focus and clarity to the holistic site experience design and development process Product Health drives global product strategy and resource allocation through insights from comprehensive, ongoing, methodologically robust user assessments of our site experience.

What is Product Health?

A suite of user experience research projects that, as a whole, provides an ongoing, holistic assessment of all different areas of our site, with specific focus on the key flows.

Why are we doing it?

We believe there is a genuine need for a holistic, cross-GXT view of our product to help guide us as a company to focus our development efforts for existing products and new ideas on those areas that will fulfill the most important unmet user needs.

What is this Product Health Synthesis report about?

Five key guiding principles for product development based on key insights from the Product Health components along with a visual model that maps these principles to the eBay site experience.

These insights also help us create concrete examples of what products might look like when we apply these guiding principles

What should all of this be used for?

Each individual research component should be used by GXT leaders and their teams to develop and prioritize their product roadmaps.

The guiding principles from the Product Health synthesis should be used by executives to provide focus and clarity to the holistic site experience design and development process

Product Health Communication Strategy Detailed Findings Detailed Findings Detailed Findings Detailed Findings Detailed Findings Detailed Findings Product Health Synthesis 5 guiding principles for a desirable site experience Product Tracker Key Flow Exit Surveys True Intent Research In-Page Diagnostics Open Web Research Technical Issues Core Product Health research components Distribute findings Distribute findings Executives Etc SYI & Seller Tools Conversion Shipping Finding E2E Desirable site experience GXT Teams Product Health work sessions with GXTs Resource allocation Overall product focus Product strategy Product roadmap

Resource allocation

Overall product focus

Product strategy

Product roadmap

Product Health Strategy Overview Product Health drives global product strategy and resource allocation Product Health Studies Product Tracker Tracks users’ attitudes and usage. Key Flow Exit Surveys Captures reasons why users exit eBay. Qualitative Based Insights Provides the “why” behind the quant data. True Intent Research Maps actual usage against intended usage. Analytics Provides key metrics for tracking eBay performance. Technical Issues Research Uncovers technical issues experienced by users. Real-Time Online Qual Captures in-the-moment feedback. In-Page Diagnostics Pin points problem areas on eBay. What users say What users do Targeted decisions Long term strategy What it’s best used for Type of data

Product Health Studies

Product Tracker

Tracks users’ attitudes and usage.

Key Flow Exit Surveys

Captures reasons why users exit eBay.

Qualitative Based Insights

Provides the “why” behind the quant data.

True Intent Research

Maps actual usage against intended usage.

Analytics

Provides key metrics for tracking eBay performance.

Technical Issues Research

Uncovers technical issues experienced by users.

Real-Time Online Qual

Captures in-the-moment feedback.

In-Page Diagnostics

Pin points problem areas on eBay.

Product Health Synthesis: Sample Results

Product Health Synthesis:

Sample Results

eBay Site Experience Model Search results View item Auction + Fixed Price: BUY/SELL PLATFORMS LIST & PROMOTE BUY Visitor, Buyer path Seller path Express Home Core site Register SYI Seller tools Check out Bid Buy it now Watch & track Watch & track Stores Community hub Help hub R&G The typical paths of Sellers and of Visitors & Buyers are shown below. Page sizes indicate relative volume of traffic. B S FIND My eBay V KEY: | Seller path | Visitor, Buyer path | Page size = traffic | Shipping issues appear

eBay Site Experience Model Search results View item Auction + Fixed Price: BUY/SELL PLATFORMS LIST & PROMOTE BUY Visitor, Buyer path Seller path Express Home Core site Register My eBay SYI Seller tools Check out Bid Buy it now Watch & track Watch & track Stores Help hub R&G My eBay is a major nexus that provides a personal space from which to interact with eBay’s marketplace. My eBay is often the starting point for experienced users. V B S FIND Community hub KEY: | My eBay connections | Seller path | Visitor, Buyer path | Page size = traffic | Shipping issues appear

eBay Site Experience Model Search results View item Auction + Fixed Price: BUY/SELL PLATFORMS LIST & PROMOTE BUY Visitor, Buyer path Seller path FIND Express Home Core site Register SYI Seller tools Check out Bid Buy it now Watch & track Watch & track Stores Help hub R&G The combined Buy/Sell Platforms make up another major nexus where users connect to each other, as well as eBay’s marketplace. B S My eBay Community hub V KEY: | Buy/Sell connections | My eBay connections | Seller path | Visitor, Buyer path | Page size = traffic | Shipping issues appear

eBay Site Experience Model Search results View item Auction + Fixed Price: BUY/SELL PLATFORMS LIST & PROMOTE BUY Visitor, Buyer path Seller path Express Home Core site Register SYI Seller tools Check out Bid Buy it now Watch & track Watch & track Stores Help hub R&G Other pages have the potential to play a more significant role in the relationships among Visitors, Buyers, Sellers and eBay. B S FIND My eBay Community hub V KEY: | Buy/Sell connections | My eBay connections | Seller path | Visitor, Buyer path | Page size = traffic | Shipping issues appear

eBay Site Experience Model Search results View item Auction + Fixed Price: BUY/SELL PLATFORMS LIST & PROMOTE BUY Visitor, Buyer path Seller path Express Home Core site Register SYI Seller tools Check out Bid Buy it now Watch & track Watch & track Stores Help hub R&G This model will be used throughout this report to illustrate insights. V B S FIND My eBay Community hub KEY: | Buy/Sell connections | My eBay connections | Seller path | Visitor, Buyer path | Page size = traffic | Shipping issues appear

Product Health Synthesis | 5 Guiding Principles for Product Development Guiding principles Guiding principles to provide focus and clarity to the site experience design and development process Build Trust Be responsive to customer needs for security and service. Ensure the quality and accuracy of the information available on the site. Build responsibility and accountability in the eBay community. Simplify Streamline and clarify processes, navigation, site performance issues and fees that make the customer’s experience unnecessarily complex and impact commerce. Be Relevant Design an experience around how people naturally explore, evaluate and purchase items. Provide relevant, quality information that supports this experience. Provide value Align buyers and sellers with a common means of determining item value and shipping fees. Treat sellers as “paying customers” who deserve value-added service for the fees they pay. Connect people through commerce Support sellers in using eBay to market their items more effectively to reach their prospective buyers. Leverage My eBay and other areas on the site to connect people with common interests through commerce.

Build Trust

Be responsive to customer needs for security and service.

Ensure the quality and accuracy of the information available on the site.

Build responsibility and accountability in the eBay community.

Simplify

Streamline and clarify processes, navigation, site performance issues and fees that make the customer’s experience unnecessarily complex and impact commerce.

Be Relevant

Design an experience around how people naturally explore, evaluate and purchase items.

Provide relevant, quality information that supports this experience.

Provide value

Align buyers and sellers with a common means of determining item value and shipping fees.

Treat sellers as “paying customers” who deserve value-added service for the fees they pay.

Connect people through commerce

Support sellers in using eBay to market their items more effectively to reach their prospective buyers.

Leverage My eBay and other areas on the site to connect people with common interests through commerce.

Product Health Synthesis | Executive Summary Other sites Search engines Build Trust Connect People Through Commerce Be Relevant Provide quality info on eBay to keep users from going to other sites to do research. Simplify Clarify which selling formats are appropriate for Sellers’ businesses. Provide Value Align Buyers and Sellers on the benefits and value of different selling formats. Connect People Through Commerce Help Sellers and Buyers connect to those with similar interests. Be Relevant Improve the relevancy of Search Results, and design an experience that maps to users’ natural ways of searching and browsing. Simplify Provide Value Be relevant Streamline the Home Page experience so Buyers and Visitors are not overwhelmed. Build Trust Provide Value Provide proactive, responsive customer support, useful help content and better guidance through key flows. Build Trust Provide a clear, trustworthy registration process. Simplify Streamline the registration process. Connect People Through Commerce Leverage the popularity of My eBay to connect users to the larger eBay community. Leverage the user community and partner assets (e.g., ePinions) to provide trustworthy, relevant, quality product information. Leverage the user community to help regulate the site. Provide simple, clear fee structures packaged with useful, easy-to-use selling tools; specific focus needed on shipping costs Simplify Show buyers the information they need to see to make the best buying decision; don’t overload the page

Other sites

Search engines

Be Relevant

Provide quality info on eBay to keep users from going to other sites to do research.

Simplify

Clarify which selling formats are appropriate for Sellers’ businesses.

Provide Value

Align Buyers and Sellers on the benefits and value of different selling formats.

Connect People Through Commerce

Help Sellers and Buyers connect to those with similar interests.

Be Relevant

Improve the relevancy of Search Results, and design an experience that maps to users’ natural ways of searching and browsing.

Simplify

Provide Value

Be relevant

Streamline the Home Page experience so Buyers and Visitors are not overwhelmed.

Build Trust

Provide Value

Provide proactive, responsive customer support, useful help content and better guidance through key flows.

Build Trust

Provide a clear, trustworthy registration process.

Simplify

Streamline the registration process.

Connect People Through Commerce

Leverage the popularity of My eBay to connect users to the larger eBay community.

Leverage the user community and partner assets (e.g., ePinions) to provide trustworthy, relevant, quality product information.

Leverage the user community to help regulate the site.

Provide simple, clear fee structures packaged with useful, easy-to-use selling tools; specific focus needed on shipping costs

Simplify

Show buyers the information they need to see to make the best buying decision; don’t overload the page

Beyond the slides Driving product strategy

Beyond the slides

Driving product strategy

Beyond the slides – Driving Product Strategy Each individual Product Health component is communicated extensively Specific recommendations for product changes Start with leaders of functional teams responsible for change Communicate with each of their teams separately Product Health Synthesis is communicated from the top down and teams are immersed in it Provides guidelines for Product design Start with SVP and above – owners of Product Strategy Communicate with functional teams Workshops and brainstorming sessions to develop product solutions based on product design guidelines Principles for effective communication and change Bring vendors into your circle – in-person work sessions are essential Be partners with internal teams, not consultants coming in to “set them straight” Tell stories, don’t give presentations Get executive buy-in without alienating functional teams

Each individual Product Health component is communicated extensively

Specific recommendations for product changes

Start with leaders of functional teams responsible for change

Communicate with each of their teams separately

Product Health Synthesis is communicated from the top down and teams are immersed in it

Provides guidelines for Product design

Start with SVP and above – owners of Product Strategy

Communicate with functional teams

Workshops and brainstorming sessions to develop product solutions based on product design guidelines

Principles for effective communication and change

Bring vendors into your circle – in-person work sessions are essential

Be partners with internal teams, not consultants coming in to “set them straight”

Tell stories, don’t give presentations

Get executive buy-in without alienating functional teams

For more information Christian Rohrer: [email_address] Rian Van Der Merwe: [email_address] Vendor partnerships Hall & Partners Cheskin Avenue A | Razorfish iModerate Keynote SirValUse

For more information

Christian Rohrer: [email_address]

Rian Van Der Merwe: [email_address]

Vendor partnerships

Hall & Partners

Cheskin