SlideShare a Scribd company logo
1 of 39
Download to read offline
Market Research
A self-help guide
# To provide the key sponsors of market research (i.e.,
Product Managers) with the tools and understanding to
initiate and largely manage their own research projects
# To educate potential consumers of research results
about the issues and methodologies involved in market
research
Note: this document should be used in conjunction with
the Primary Research RFP Template in order to initiate a
new primary research project
Purpose of this document
# The market research process described
# Developing the research plan
# When to do research
# Key highlights of specific approaches
# The sampling plan
# Budgeting and global issues
# Recommended vendors for RFPs
# Role of MS&I
# Recommended reading
Overview
# Market research has been defined as "the systematic
design, collection, analysis, and reporting of data and
findings relevant to a specific situation facing the
company" (Kotler)
< Secondary research involves the compilation and analysis of
data and intelligence that have already been collected from
another source, either external or internal
< Primary research involves specialized methods for the direct
collection of custom data
Secondary research sources often use primary research
methodologies to compile their data
Market research definition
Market research process
Define the
problem and
research
objectives
Develop the
research
plan
Collect the
information
Analyze the
information
Present the
findings
Potential level of outside vendor involvement
Low Medium High High Medium
Research Tool: Key
Secondary Research S
Observation O
Customer Visits / 1on1's C
Focus Groups F
Survey Research R
Choice models M
Experiments E
When to do research
MRD Review
Product
Focus
Alpha
Beta
End Of Life
GM
Concept exploration (S/O/C/F)
Market segmentation and profiling (S/F/R)
Competitive analysis (S)
Market opportunity sizing/forecasting (S/R/M/E)
Milestones:
Customer profiling (O/C/F/R)
Workflow validation (O/C/F)
Feature validation (O/F/S/M)
Positioning and messaging (F/R)
Packaging and collateral (F)
Pricing (S/C/E)
Customer/product satisfaction (C/F/S)
Marketing effectiveness (F/S)
Market share tracking (S/R)
The tools listed on the
previous slide should
be used in different
ways, at different
times in the product
lifecycle
# The problem in question must be clearly thought-through and stated
< "A problem well defined is half solved."
# Just as with marketing in general, the more focused (targeted) the
problem definition, the greater the chance of success
# The cost of the research project must be less than the cost of not
solving the problem!
< Put another way: the net profit resulting from achieving the research objective
must be greater than the total cost of the research
< Remember: time is also money
< Therefore: don't do the research if you are not planning on taking significant
action based on the results
# Critical success factor: build consensus among all affected parties
surrounding the problem statement and key objectives
Defining the problem/objective
Verb Possible Objects
Identify Problems, opportunities, choice criteria
Define Concept, design, potential
Describe Decision process, usage, work environment
Explore Perceptions, reactions, remedies
Generate Hypotheses, alternatives, explanations
Evaluate Feasibility, attractiveness, potential
Select Product, concept, ad execution
Test Preference, direction, profitability
Measure Growth, size, frequency
Prioritize Segments, needs, opportunities
Monitor Trends, competition, events
Track Spending, satisfaction, awareness
# After a problem statement is created, specific research
objectives need to be outlined
# Hint: use the verb/object approach and pick from this list:
Remember to STAY FOCUSED!
Examples of research objectives
# The research plan calls for decisions in the following
areas:
< Data sources
– Secondary data, primary data
< Research approaches
– Observation, customer visits/1on1's, focus groups, surveys, choice
models, experiments
< Research instruments
– Specific techniques within each approach
< Sampling plan
– Decide who will be the subject of the study
– Arrive at the sample size and procedure for generating respondents
Developing the research plan
# The data necessary to achieve research objectives can
come from either secondary or primary sources, or a
combination
# Secondary research is usually much quicker and
cheaper than engaging in a primary research project
< For assistance in locating relevant secondary market research,
contact the MS&I team
Watch for an announcement in early 2003 regarding the new
MS&I Portal, enabling more self-service secondary market data
Research plan:
Data sources
# Carpenters know that selecting the right tool is critical
to doing the best job - the same holds true for market
research
# Research techniques are generally divided into the
following categories:
< Exploratory: gather preliminary data; suggest hypotheses or
new ideas
– Common tools: observation; customer visits/1on1's; focus groups
< Descriptive: ascertain magnitudes; frequencies; percentages
– Common tools: surveys
< Causal: test cause-and-effect relationships
– Common tools: choice models; experiments
Research plan:
Approaches and
instruments
Identify Describe Monitor Generate Define Explore Test Evaluate Prioritize Select Measure Track
Research Tool:
Secondary Research 1 1 1 2 2 2 2 2 2 2 1 1
Observation 1 1 1 1 3 2 X 3 X 3 X 3
Customer Visits / 1on1's 1 1 1 1 1 1 X X 3 X X 3
Focus Groups 2 2 3 1 1 1 X X 3 X X 3
Survey Research 2 1 2 X 2 X 1 1 1 1 1 1
Choice models X X X 3 3 X 1 1 1 1 X X
Experiments X X 3 X X X 1 1 1 1 1 2
Legend: 1 = Primary means of achieving the objective
2 = Contributes secondarily to that objective
3 = Minimal fit with this objective
X = Potentially misleading or dangerous if applied to this objective
Evaluate Success
Stages and Objectives:
Scan Environment Generate Options Select an Option
# Use the following table as a guideline to tool selection:
Note that sometimes more than one tool is needed!
Matching the tool to the job
# The following slides highlight key aspects of the major
research approaches
# The term "researcher" in these slides is used loosely to
indicate whoever it is that is actually managing the
research project. The "researcher" could be:
< A Product Manager
< Someone in Sales
< A member of the Market Strategy & Intelligence group
< A contracted market research vendor
< Etc.
Highlights of major approaches
Description Unobtrusive observation of the purchase or usage behaviors of a target group. Observational market
research - with its basis in the science of anthropology - is gaining in popularity in technology companies in
recent years. Usability research is closely related.
General
Procedure
1) Set objectives.
2) Decide upon location for observation and target sample.
3) Hire vendor if necessary.
4) Carry out observation.
5) Compile and report results.
Sample
Projects
- Observation of purchase behaviors at retail.
- Analyzing Website click-stream data.
- Record usage behaviors and stumbling blocks for users of a new product.
Result Format Qualitative
Strengths - Enables unbiased view of actual customer behaviors - who can quarrel with reality?
- No artificial boundaries or preconceived notions cloud the results.
Weaknesses - Samples are generally small and non-representative of the population.
- Difficult for the observer not to impact the observed.
- Researcher bias may enter into the recording or reporting of results.
Timeframe Varies widely. Simple projects can be carried out in hours or days. Complex anthropological projects
involving the hiring of a vendor can take months to complete.
Possible Cost Widely varying cost, from $0 to $100k+.
Approaches:
Observation
Description A researcher interacts with a current or potential customer, either at the customer's worksite or at a
separate location. Visits may be ad-hoc (as with a sales call) or programmatic (part of a predetermined
market research strategy).
General
Procedure
1) Set objectives.
2) Select a sample.
3) Select team to conduct visit - the best teams are cross-functional.
4) Devise a discussion guide.
5) Conduct the interviews.
6) Debrief the teams.
7) Analyze and report results.
Sample
Projects
- Assess initial reaction to new product concept.
- Discover customer pain points.
Result Format Qualitative
Strengths
- Richest of all communication modes - getting out of the office and talking to customers is always good.
- Best way to get "into the customer's mind".
- Information is all first-hand and does not suffer from a researcher's or analysts' interpretation.
Weaknesses - Huge potential for interviewer bias.
- Instability and imprecision of small sample sizes - do not use customer visits to estimate statistics.
Timeframe Can often be carried out quickly.
Possible Cost - Generally low in dollar terms. Usually limited to travel costs for the researcher's staff. Cost can escalate if
a list of potential customers needs to be purchased and/or extensive travel is necessary.
- But can be one of the most time-intensive forms of market research for the researcher in terms of their
personal time.
Approaches:
Customer visits/1on1's
Description A group interview, usually involving 6 to 10 participants who share common characteristics, who meet with
a skilled moderator to discuss a particular subject.
General
Procedures 1) Set objectives.
2) Send out RFP to vendor detailing kind of participants to recruit and number of groups to hold.
3) Edit "screener" which will be used to recruit and filter appropriate participants. Recruiting then begins.
4) Meet with moderator to develop discussion guide - a loose "script" for the moderator.
5) Attend focus groups (1.5 to 2 hours each).
6) Vendor prepares report. Video tapes and other records of events may also be produced.
Sample
Projects
- Preliminary testing of a new product concept.
- Explore and identify new features for existing products.
- Feel customer pain with current solutions.
- Testing of messaging and marketing collateral.
- As preparation for more extensive/expensive survey research to ensure that money is well-spent.
Result Format Qualitative
Strengths - Significant capacity to produce surprise and a fresh perspective.
- Synergistic effect when customers are brought in contact with one another.
- Possible to obtain the most frank and candid assessments when done anonymously.
- Results (if videotaped) can provide a powerful tool for exposing company to the views of customers.
- Useful for putting future surveys into the "language of the customer".
Weaknesses - Reliance on small, non-random sample. Never extrapolate a percentage or frequency from a focus group
experience.
- Vulnerability to dominant and shy individuals over/underparticipating. Thus the need for a competent
moderator and more than one group.
- Moderator is not likely to be an expert in the particular industry or area of technology.
Timeframe Working through all the stages of setting up and conducting a series of focus groups will take one to two
months.
Possible Cost $5k to $10k per group plus any travel costs. This price includes paying participants for their time. A
minimum of three groups is recommended due to the significant possibility that any single group could get
overly focused on a particular topic or participant - three groups ensures that a wide enough range of
opinions is heard.
Approaches:
Focus groups
Description A situation where a fixed set of questions is asked of a sample of respondents. Answer categories are
usually specified in advance. Can be administered in-person (rare), by telephone, mail, or e-mail/Web.
General
Procedures
1) Prepare a Request For Proposal (RFP) that defines the problem and objectives; characteristics of the
sample; proposed methodology.
- Note that research vendors are a valuable source of insight into proper methods for achieving objectives.
Not everything needs to be figured out before approaching a vendor but, at a minimum, the problem and
objectives should be clear.
2) Vendor will work with you to create the screener and the questionnairre. The screener is made up of
introductory questions in order to ensure that the correct respondents are completing the survey.
3) The size, characteristics, and source of the sample are agreed upon. More on this later in this
document.
4) Survey is pre-tested on a small sample. This step is critical. Pretesting identifies critical flaws in the
survey that will prevent wasting the entire survey budget. Resist the significant temptation to skip
pretesting.
5) The survey is administered (fielded). The vendor should provide continuous feedback on issues and
problems that arise.
6) Results are compiled, analyzed, and reported.
Sample
Projects
- Customer satisfaction measures.
- Market segmentation studies.
- Product usage and ownership.
- Purchase intention.
- Brand image and perceptions. Advertising tracking.
Result Format Quantitative (usually) or Qualitative
Approaches:
Surveys (1 of 2)
Strengths
- Ability to deliver precise numerical estimates of frequency and magnitude.
- Superior objectivity to more "exploratory" methods.
- Ability to apply a wealth of statistical techniques: measure if differences in segments are significant or not.
- Can be repeated reliably over time to create longitudinal analyses.
- Individual instrument strengths: Telephone surveys provide the most control over sample bias and
have the highest response rates. Mail surveys are useful for complex surveys that include long lists of
choice options. E-mail/Web surveys are quick to administer and easy to compile results for.
Weaknesses - Tells you "what" but not "why". Can be overcome if exploratory techniques are used in conjunction.
- Not good at revealing "what you did not know you did not know". Surveys are a precision tool, not a
discovery tool.
- Rely on self-reported data. Often customers do not recall, cannot verbalize what they do recall, lie, and
are unreliable in projecting future behaviors.
- Survey fatigue. People are generally over-surveyed and the quality of respondents is a major concern.
- Does not deal well with group decision making processes - especially relevant in B2B markets.
- Individual instrument weaknesses: Telephone surveys are the most expensive to administer and are
seen as most intrusive by respondents. Mail surveys are slow to administer and suffer from low response
rates. Due to the anonymity of the Web, E-mail/Web surveys can suffer from significant sample bias (more
on this in the sampling section of this document).
Timeframe Generally 1.5 to three months from start to finish.
Possible Cost - Varies widely, but $50 to $100 per respondent is not an unreasonable starting point for budget
estimations.
- The extent and complexity of reporting and analysis required will drive cost up. Simple results in tabular
format are less expensive (but may be less impactful).
- Open-ended (free text) questions that need to be "coded" drives up cost somewhat.
- The cost of good contact lists can vary dramatically. In general, more senior level decision makers cost
more to contact.
- Cost also varies with survey type: E-mail/Web surveys are the least expensive to administer, followed by
mail surveys, with phone being the most expensive.
Approaches:
Surveys (2 of 2)
Description A procedure that attempts to analyze how different factors combine to influence the choice of one product
over another. Choice models provide an estimate of the importance or weight of each attribute of a whole
product. Optimal combinations can be derived and "what-if" scenarios modeled. So-called "conjoint
analysis" and "direct attribute weighting" are two approaches to choice modeling.
General
Procedures
Choice modeling is a new and somewhat complex form of survey research. The assistance of an
experienced vendor is critical in carrying out this type of research. To engage in this type of research, the
basic steps are:
1) Agree internally upon problem and objectives.
2) Identify product attributes (i.e., price) that need to be studied.
3) Identify sample.
4) Create RFP and submit to vendor.
5) Iterate on all of the above with research vendor.
6) Vendor does fieldwork, compiles data, and presents results.
Sample
Projects
- Price sensitivity research.
- Analysing effect on market share of adding/removing features.
Result Format Quantative
Strengths - Ability to deal with complex issues and problems in a rational manner.
- Does not require potential customers to guess about future behaviors. Future behaviors are derived from
this research, but knowledge of them is not explicitly required.
- Can deal with a large number of product attribute combinations.
Weaknesses - One of the most expensive and time consuming forms of market research.
- Difficult to implement for products purchased through a group decision making process.
- Results are very susceptible to improper selection of product attributes. Garbage in, Garbage out. That is
why the use of customer visits and other exploratory techniques is often used at the front end.
Timeframe - Two to three months. Expect the "analysis and reporting" phase to take longer than with a standard
survey.
Possible Cost $100 to $150 per respondent is a good ballpark estimate to start with.
Approaches:
Choice models
Description An artificial situation constructed by the researcher. Can take place in a laboratory, but often takes place in
the field (in the market). As with any experimental process, the idea is to arrange conditions in such a way
so that one can infer causality from the results.
General
Procedures
The "scientific method" is the general procedure to be used for an experiment. This method is used
extensively in the CPG industry but is underused in technology circles. In brief:
1) Observe some aspect of the market (i.e., when price goes down, volume goes up).
2) Invent a hypothesis related to what was observed (i.e., lowering my product's price will increase net
profit).
3) Use the hypothesis to make predictions (i.e., if I lower the price by $100 then net profit will go up by at
least 10%).
4) Create an experiment to test the hypothesis. Use an experimental group and compare this with a control
group who experience otherwise similar market conditions.
5) Report findings (i.e., lowering the price by $100 resulted in a net profit increase of 12%).
Sample
Projects
- Selecting the most effective advertisement or direct mail piece.
- Choosing an optimal price point.
- Decision on whether or not to release a line extension.
Result Format Quantitative or Qualitative. Measurement criteria is often in terms of dollars earned or units sold. Resulting
decision is often simply "go" or "no go".
Strengths - Good for comparing a small number of alternatives in order to select the best.
- Simple and straightforward to design.
- Offers empirical evidence that is convincing and drives decision making.
- Useful for forecasting.
Weaknesses - The structure of the market may make it difficult to separate the experimental and control groups.
- Only a limited number of options can generally be explored with an experiment. Choice modeling is a
better choice if a large number of possibilities need to be explored.
Timeframe Varies widely. Experiments are often long term in nature since it takes time for market dynamics to unfold
in response to the experiment.
Possible Cost Experiment cost is mostly about opportunity cost. In the above example, if the experiment to lower the
price by $100 failed to prove the hypothesis then the company would have lost the revenue that it gave up
by reducing the price by $100 for the experimental group.
Approaches:
Experiments
# A three step process:
< 1. Devise major content areas
– I.e.: Screening questions -> current satisfaction and usage ->
purchase intent -> demo/firmographics
< 2. Develop specific questions
< 3. Edit for length and clarity
# Questions should be targeted specifically at addressing
the key objectives of the research study
< Shorter surveys increase response rates - very critical
# The research vendor will work with you to properly
word and format questions based on the objectives
< Sample questionnaires are available from MS&I upon request
Questionnaire development
# Sometimes it is possible to survey the entire
population, but usually this isn't feasible or necessary
# The sample is the lynch-pin of all market research. The
quality of the results is directly related to the
appropriateness of the sample. Therefore:
< Spend considerable time thinking about the nature of the
sample required
< If you do not feel good about the sample, do not do the
research. It will be a waste of time and money
# It is also critical to consider the sample when reviewing
published secondary market research
< Most secondary research has its basis in primary research
techniques - don’t take published research at face value
# If the sample is not valid then neither is the research
#
Research Plan:
Sampling plan
# Sampling unit:
< Who is to be surveyed?
# Sample size:
< How many people should be surveyed?
# Sampling procedure:
< How should the respondents be chosen?
Each of these will be dealt with separately in the
following slides
Sampling plan:
Three elements of a
sampling plan
# The "population" or "universe" is the complete group to
which the market research is directed
# The first critical step is to be as clear about defining this
universe as possible
< If you are not sure what the universe is, then err on the side of a
universe with a larger scope, rather than smaller. Why?
< Because the danger of excluding appropriate respondents from
the sample is likely to be greater than the cost of surveying a
larger sample
# The goal should be to create a sample which reflects
the universe (as defined) as accurately as possible
< Tools and data from the U.S. Bureau of Labor Statistics and
other sources are available to help compare samples with
populations. Refer to MS&I for assistance with this
Sampling plan:
Sampling unit
# Who should you really speak to?
< The end user?
< The economic buyer (sponsoring line-of-business manager)?
< The technical/infrastructure buyer (CIO)?
# Depends on the product’s TALC stage
< Early market: (visionary) economic buyers
< Chasm/bowling alley: economic buyer
< Tornado: technical buyer (just ship)
< Main street: end user
Sampling plan:
Sampling unit (cont’d)
# Two judgmental decisions are necessary to select an
acceptable sample size
# Margin of error:
< Accuracy of results +/- a certain percentage
< Lower margin of error means that it is easier to compare
results. Example:
– If the margin of error in a study is +/- 10%, then importance ratings
on two key attributes of 47% and 65% are not statistically different
# Level of confidence:
< Measures how confident we are that the sample is accurate
within the margin of error. Example:
– A 95% confidence level means that we can assume the results
within the margin of error (from above) 19 times out of 20. One time
out of 20, the results will not be within the margin of error
Sampling plan:
Sample size (1 of 3)
80% 90% 95% 99%
+/- 1% 4,096 6,766 9,604 16,590
+/- 2% 1,024 1,692 2,401 4,148
+/- 3% 456 752 1,068 1,844
+/- 4% 256 423 601 1,037
+/- 5% 164 271 385 664
+/- 10% 41 68 97 166
+/- 15% 19 31 43 74
+/- 20% 11 17 25 42
Level of Confidence
MarginofError
Samplesizerequired
# The random sample size required is directly related to
the acceptable margin of error and level of confidence
as per the following table:
95% confidence +/-5% margin of error is an often-used
standard for sample size
Sampling plan:
Sample size (2 of 3)
# Issues with sample size
< The previous table considers the case of a single survey question. If the survey
asks 20 questions, then at a 95% confidence level there is a very high
probability that at least one of the questions will be outside the specified margin
of error
< The level of confidence and margin of error are computed separately for each
sub-segment in the sample. For example, from the previous table, a question
asked to all 400 respondents is accurate to +/-5% 19 times out of 20. But if the
respondents are then sub-segmented into small groups, the responses of
these groups have a much lower level of confidence and higher margin of
error. The table can be used to estimate this effect
MOST IMPORTANT: The previous table assumes that the
sample was generated randomly from the population. In
reality, most samples are not truly random.
Always be cautious about interpreting/comparing results
Sampling plan:
Sample size (3 of 3)
# Samples are derived in different ways
< The "best" way depends on the characteristics of the market, the research
objective and budget available. Consult with MS&I and the vendor for further
advice as to appropriate technique
< Probability samples are necessary for the calculation of proper confidence
limits (as per the previous slides)
# Probability sampling techniques
< Simple random sample: Every member of the population has a known and
equal chance of selection
< Stratified random sample: The population is divided into mutually exclusive
groups (i.e., by industry) and random samples are drawn from each group
# Non-probability sampling techniques
< Convenience sample: The most accessible population members are chosen
< Judgment sample: Researcher judgment is used to select population members
who are good prospects for accurate information
< Quota sample: The researcher finds and interviews a prescribed number of
people in each of several categories
Sampling plan:
Sampling procedure
# For market sizing and market share work it is critical that a
probability sampling technique is used. Issues relevant to the
different measures of market size include:
# TPM - Total Potential Market
< TPM accounts for all current and potential users of a product
< To use primary research to ascertain the size of the TPM, ensure that the
universe definition is set as wide/general as possible
# TAM - Total Addressable Market
< TAM accounts for all current users of a product in the marketspace under
consideration
< Therefore, the sample would have to include users of competitive products and
possibly direct substitutes (like homegrown solutions)
# TCM - Total Captured Market
< TCM accounts for Corel's market share, therefore Corel's customer list might
be used to generate this sample
< But beware of sample bias due to low/uneven product registration rates
Sampling plan:
TPM/TAM/TCM and
sampling
# Response rates
< The lower the response rate, the greater the likelihood of response bias
< Response bias = a skewing of the results based on the possibility that the
actual respondents within the sample do not represent the universe to be
measured
< If possible, try to ascertain how respondents differ from non-respondents so
that hypotheses may be generated about the effect on the survey results
# Cost
< The larger the sample the higher the cost (but at a declining rate)
< The greater the efforts taken to approximate a random sample, the higher the
cost
Once again, sampling issues are highly critical to any
research project. If the sample is poor, then everything
else is meaningless at best (or misleading at worst)
Sampling plan:
Other issues with
sampling
# Approximate costs for project budgeting can be found earlier in this
presentation
< However, actual costs will vary widely depending on all of the issues that we
have identified previously
# A key decision is whether or not to state your available project
budget to the research vendor in the RFP. Guidelines:
< If you feel that your available budget meets or exceeds the cost estimates in
this document, then do not inform the vendor of your budget. Sending out
multiple RFPs will result in the best price with minimal padding
< If you feel that your available budget may be insufficient relative to the
guidelines presented here, then it is a good idea to inform the research vendor
ahead of time of the limit on what you can spend. This will enable the research
vendor to specify possible approaches and alternatives that may allow the
project to succeed within your budget
Always allow some room for unforseen costs -
they are inevitable
Budgeting
# Market research outside of North America, both primary
and secondary, is at an earlier stage than within North
America
# As a general rule: multiply both cost and time estimates
by 1.5 for Tier 1 countries (UK, France, Germany, Japan)
and 2x for other geos
# Privacy laws, especially in the EU, may also
significantly impact the ability to collect data
< Consult with your research vendor for relevancy to your specific
project
# Also note that June/July/August can be difficult months
to carry out a research project in Europe
Global issues
#
MS&I has placed the following vendors on its “short list” to receive
primary market research RFPs from Corel
< Vision Research
< Phase 5
# Overviews of these vendors appear on the following slides
# Certain specialized projects may require particular vendors. MS&I
will help you assess your needs
# Note that this list will change from time to time. Please refer to the
latest version of this slide deck for the most recent list
# Before initially contacting any vendor, all new RFPs should be
reviewed by MS&I
Recommended vendors for RFPs
# Comments:
< Small operation, office in Ottawa
< Conducts all types of qualitative and quantative research
< Has worked for Corel in the past. Most recently: Vector
Illustration survey in the U.S., UK, Germany, and France
< Likely to be the “low cost” vendor for a project. Do not rely on
them for extensive strategic analysis and implications
# Contact info:
< Candace Fedoruck, Executive Vice President
< Phone: 613 594 8282 / 1 888 310 6062
< E-mail: candace@delta-media.com
Vision Research
Recommended vendors (cont’d)
# Comments:
< Offices in Ottawa and Toronto
< Targets technology companies: high Web and technology IQ
< Ani Ieroncig, former Corel Market Research Director is our main
contact there - providing some knowledge of issues facing Corel
< Phase 5 key value-add is the strategic element of projects. They
are unlikely to be the low-cost vendor, but for projects that require
complex setup or detailed analysis, they may be a good choice
# Contact info:
< Ani Ieroncig, Senior Manager
< Phone: 613 241 7555 ext.108
< E-mail: anii@phase-5.com
Phase 5
Recommended vendors (cont’d)
# Comments:
< A third recommended vendor will be inserted here when MS&I
has identified an appropriate candidate
# Contact info:
< TBD
TBD
Recommended vendors (cont’d)
# Market Strategy & Intelligence (MS&I) is available to
assist you in identifying and locating relevant
secondary research resources
< In early 2003, Corel will launch a Market Intelligence Portal
which will enable more of a self-service approach
# The role of MS&I in product-specific market research
projects is to act primarily as a consultant, assisting
your team in scoping projects and in selecting vendors
< The bandwidth is currently not available to directly manage
every project that takes place
# Review the latest “RFP Template” which is meant to be
used in conjunction with the content in this slide deck
Role of MS&I in Corel’s
market research
# The Market Research Toolbox - A Concise Guide for
Beginners
< Edward F. McQuarrie, 1996
# An excellent, non-technical introduction and guide to
market research written for managers (not market
research professionals)
< A good two-night read: available for borrow from Blaine
Recommended reading

More Related Content

What's hot

Market Research END-TO-END Benefits
Market Research END-TO-END BenefitsMarket Research END-TO-END Benefits
Market Research END-TO-END BenefitsEminenture
 
Writing a research brief
Writing a research briefWriting a research brief
Writing a research briefKKylle
 
What Are Problem Statements
What Are Problem StatementsWhat Are Problem Statements
What Are Problem StatementsStewart Rogers
 
Research brief
Research briefResearch brief
Research briefiain bruce
 
Market research
Market researchMarket research
Market researchjeetgk
 
Chapter 1,2,3 marketing research
Chapter 1,2,3   marketing researchChapter 1,2,3   marketing research
Chapter 1,2,3 marketing researchr2sa
 
DEFINING THE MARKETING RESEARCH PROBLEM AND DEVELOPING AN APPROACH
DEFINING THE MARKETING RESEARCH PROBLEM AND DEVELOPING AN APPROACHDEFINING THE MARKETING RESEARCH PROBLEM AND DEVELOPING AN APPROACH
DEFINING THE MARKETING RESEARCH PROBLEM AND DEVELOPING AN APPROACHShashank Kapoor
 
detailed notes AMR.pdf
detailed notes AMR.pdfdetailed notes AMR.pdf
detailed notes AMR.pdfRameshG767908
 
Chapter 4&amp;5 mr
Chapter 4&amp;5   mrChapter 4&amp;5   mr
Chapter 4&amp;5 mrr2sa
 
Questionnaire desiging
Questionnaire desigingQuestionnaire desiging
Questionnaire desigingKritika Jain
 
Chapter 3. marketing research
Chapter 3. marketing researchChapter 3. marketing research
Chapter 3. marketing researchJags Jagdish
 
Market research-from research problem to research question
Market research-from research problem to research questionMarket research-from research problem to research question
Market research-from research problem to research questionSarah Sloan
 
Problem Definition in research design
Problem Definition in research designProblem Definition in research design
Problem Definition in research designAbu Bashar
 
Marketing research process
Marketing research processMarketing research process
Marketing research processkarthik v
 
MARKETING RESEARCH AND PROCESS BY JEET PAREKH IIT BHU
MARKETING RESEARCH AND PROCESS BY JEET PAREKH IIT BHUMARKETING RESEARCH AND PROCESS BY JEET PAREKH IIT BHU
MARKETING RESEARCH AND PROCESS BY JEET PAREKH IIT BHUSameer Mathur
 
The types of Business Research, Business Reasearch process flowchart and steps
The types of Business Research, Business Reasearch process flowchart and stepsThe types of Business Research, Business Reasearch process flowchart and steps
The types of Business Research, Business Reasearch process flowchart and stepsNeazMujeri
 
Business Research process
Business Research processBusiness Research process
Business Research processSandeep Bhat
 

What's hot (20)

Market Research END-TO-END Benefits
Market Research END-TO-END BenefitsMarket Research END-TO-END Benefits
Market Research END-TO-END Benefits
 
Writing a research brief
Writing a research briefWriting a research brief
Writing a research brief
 
What Are Problem Statements
What Are Problem StatementsWhat Are Problem Statements
What Are Problem Statements
 
Research
ResearchResearch
Research
 
Research brief
Research briefResearch brief
Research brief
 
Market research
Market researchMarket research
Market research
 
Chapter 1,2,3 marketing research
Chapter 1,2,3   marketing researchChapter 1,2,3   marketing research
Chapter 1,2,3 marketing research
 
DEFINING THE MARKETING RESEARCH PROBLEM AND DEVELOPING AN APPROACH
DEFINING THE MARKETING RESEARCH PROBLEM AND DEVELOPING AN APPROACHDEFINING THE MARKETING RESEARCH PROBLEM AND DEVELOPING AN APPROACH
DEFINING THE MARKETING RESEARCH PROBLEM AND DEVELOPING AN APPROACH
 
detailed notes AMR.pdf
detailed notes AMR.pdfdetailed notes AMR.pdf
detailed notes AMR.pdf
 
Chapter 4&amp;5 mr
Chapter 4&amp;5   mrChapter 4&amp;5   mr
Chapter 4&amp;5 mr
 
Questionnaire desiging
Questionnaire desigingQuestionnaire desiging
Questionnaire desiging
 
Research design
Research designResearch design
Research design
 
Chapter 3. marketing research
Chapter 3. marketing researchChapter 3. marketing research
Chapter 3. marketing research
 
Market research-from research problem to research question
Market research-from research problem to research questionMarket research-from research problem to research question
Market research-from research problem to research question
 
Problem Definition in research design
Problem Definition in research designProblem Definition in research design
Problem Definition in research design
 
Marketing research process
Marketing research processMarketing research process
Marketing research process
 
MARKETING RESEARCH AND PROCESS BY JEET PAREKH IIT BHU
MARKETING RESEARCH AND PROCESS BY JEET PAREKH IIT BHUMARKETING RESEARCH AND PROCESS BY JEET PAREKH IIT BHU
MARKETING RESEARCH AND PROCESS BY JEET PAREKH IIT BHU
 
Market research
Market research Market research
Market research
 
The types of Business Research, Business Reasearch process flowchart and steps
The types of Business Research, Business Reasearch process flowchart and stepsThe types of Business Research, Business Reasearch process flowchart and steps
The types of Business Research, Business Reasearch process flowchart and steps
 
Business Research process
Business Research processBusiness Research process
Business Research process
 

Similar to Tech Market Research Guide

Chapter-3-Lesson Getting to know the market.pptx
Chapter-3-Lesson Getting to know the market.pptxChapter-3-Lesson Getting to know the market.pptx
Chapter-3-Lesson Getting to know the market.pptxAshlyMarieDiongco1
 
marketing research design-ppt new.pptx
marketing research design-ppt new.pptxmarketing research design-ppt new.pptx
marketing research design-ppt new.pptxDiksha Vashisht
 
Steps in marketing research process
Steps in marketing research processSteps in marketing research process
Steps in marketing research processHimanshi Gupta
 
Rsearch process indetail
Rsearch process indetailRsearch process indetail
Rsearch process indetailAnkit2709
 
Rsearch process indetail
Rsearch process indetailRsearch process indetail
Rsearch process indetailAnkit2709
 
Rsearch process indetail
Rsearch process indetailRsearch process indetail
Rsearch process indetailAnkit2709
 
Introduction to business research copy
Introduction to business research   copyIntroduction to business research   copy
Introduction to business research copyAnjuPrakash7
 
PRINCIPLES OF MARKETING MODULE 3.2.pptx
PRINCIPLES OF MARKETING MODULE 3.2.pptxPRINCIPLES OF MARKETING MODULE 3.2.pptx
PRINCIPLES OF MARKETING MODULE 3.2.pptxYatogami27
 
Market research ppt @ mba bec doms
Market research ppt @ mba bec domsMarket research ppt @ mba bec doms
Market research ppt @ mba bec domsBabasab Patil
 
Mba ii rm unit-2.1 research process a
Mba ii rm unit-2.1 research process aMba ii rm unit-2.1 research process a
Mba ii rm unit-2.1 research process aRai University
 
MARKET RESEARCH WEEK 1.pptx
MARKET RESEARCH WEEK 1.pptxMARKET RESEARCH WEEK 1.pptx
MARKET RESEARCH WEEK 1.pptxPreciousChanaiwa
 
MARKET RESEARCH WEEK 1 LESSON OVERVIEW.pptx
MARKET RESEARCH WEEK 1 LESSON OVERVIEW.pptxMARKET RESEARCH WEEK 1 LESSON OVERVIEW.pptx
MARKET RESEARCH WEEK 1 LESSON OVERVIEW.pptxPreciousChanaiwa
 
Marketing Research Assignment
Marketing Research AssignmentMarketing Research Assignment
Marketing Research AssignmentMaha H
 
Research Marketing Ch3 Edited.powerpoint
Research Marketing Ch3 Edited.powerpointResearch Marketing Ch3 Edited.powerpoint
Research Marketing Ch3 Edited.powerpointcjoypingaron
 
Introduction to Market Research
Introduction to Market ResearchIntroduction to Market Research
Introduction to Market ResearchKan K.
 
Chapter 2 Consumer Reserch
Chapter 2 Consumer ReserchChapter 2 Consumer Reserch
Chapter 2 Consumer ReserchAvinash Kumar
 
CHAPTER-4-CONDUCT-MARKETING-RESEARCH.pptx
CHAPTER-4-CONDUCT-MARKETING-RESEARCH.pptxCHAPTER-4-CONDUCT-MARKETING-RESEARCH.pptx
CHAPTER-4-CONDUCT-MARKETING-RESEARCH.pptxJenecelleCabanagEnci
 
Consumer behaviour research
Consumer behaviour researchConsumer behaviour research
Consumer behaviour researchPrerna Gaur
 
Market Research WEEK 6 LESSON PLAN .pptx
Market Research WEEK 6 LESSON PLAN .pptxMarket Research WEEK 6 LESSON PLAN .pptx
Market Research WEEK 6 LESSON PLAN .pptxPreciousChanaiwa
 

Similar to Tech Market Research Guide (20)

Chapter-3-Lesson Getting to know the market.pptx
Chapter-3-Lesson Getting to know the market.pptxChapter-3-Lesson Getting to know the market.pptx
Chapter-3-Lesson Getting to know the market.pptx
 
marketing research design-ppt new.pptx
marketing research design-ppt new.pptxmarketing research design-ppt new.pptx
marketing research design-ppt new.pptx
 
Steps in marketing research process
Steps in marketing research processSteps in marketing research process
Steps in marketing research process
 
Rsearch process indetail
Rsearch process indetailRsearch process indetail
Rsearch process indetail
 
Rsearch process indetail
Rsearch process indetailRsearch process indetail
Rsearch process indetail
 
Rsearch process indetail
Rsearch process indetailRsearch process indetail
Rsearch process indetail
 
Introduction to business research copy
Introduction to business research   copyIntroduction to business research   copy
Introduction to business research copy
 
PRINCIPLES OF MARKETING MODULE 3.2.pptx
PRINCIPLES OF MARKETING MODULE 3.2.pptxPRINCIPLES OF MARKETING MODULE 3.2.pptx
PRINCIPLES OF MARKETING MODULE 3.2.pptx
 
Market research ppt @ mba bec doms
Market research ppt @ mba bec domsMarket research ppt @ mba bec doms
Market research ppt @ mba bec doms
 
Mba ii rm unit-2.1 research process a
Mba ii rm unit-2.1 research process aMba ii rm unit-2.1 research process a
Mba ii rm unit-2.1 research process a
 
MARKET RESEARCH WEEK 1.pptx
MARKET RESEARCH WEEK 1.pptxMARKET RESEARCH WEEK 1.pptx
MARKET RESEARCH WEEK 1.pptx
 
MARKET RESEARCH WEEK 1 LESSON OVERVIEW.pptx
MARKET RESEARCH WEEK 1 LESSON OVERVIEW.pptxMARKET RESEARCH WEEK 1 LESSON OVERVIEW.pptx
MARKET RESEARCH WEEK 1 LESSON OVERVIEW.pptx
 
Marketing Research Assignment
Marketing Research AssignmentMarketing Research Assignment
Marketing Research Assignment
 
Research Marketing Ch3 Edited.powerpoint
Research Marketing Ch3 Edited.powerpointResearch Marketing Ch3 Edited.powerpoint
Research Marketing Ch3 Edited.powerpoint
 
Introduction to Market Research
Introduction to Market ResearchIntroduction to Market Research
Introduction to Market Research
 
Chapter 2 Consumer Reserch
Chapter 2 Consumer ReserchChapter 2 Consumer Reserch
Chapter 2 Consumer Reserch
 
Attachments 2012 06_18
Attachments 2012 06_18Attachments 2012 06_18
Attachments 2012 06_18
 
CHAPTER-4-CONDUCT-MARKETING-RESEARCH.pptx
CHAPTER-4-CONDUCT-MARKETING-RESEARCH.pptxCHAPTER-4-CONDUCT-MARKETING-RESEARCH.pptx
CHAPTER-4-CONDUCT-MARKETING-RESEARCH.pptx
 
Consumer behaviour research
Consumer behaviour researchConsumer behaviour research
Consumer behaviour research
 
Market Research WEEK 6 LESSON PLAN .pptx
Market Research WEEK 6 LESSON PLAN .pptxMarket Research WEEK 6 LESSON PLAN .pptx
Market Research WEEK 6 LESSON PLAN .pptx
 

Tech Market Research Guide

  • 2. # To provide the key sponsors of market research (i.e., Product Managers) with the tools and understanding to initiate and largely manage their own research projects # To educate potential consumers of research results about the issues and methodologies involved in market research Note: this document should be used in conjunction with the Primary Research RFP Template in order to initiate a new primary research project Purpose of this document
  • 3. # The market research process described # Developing the research plan # When to do research # Key highlights of specific approaches # The sampling plan # Budgeting and global issues # Recommended vendors for RFPs # Role of MS&I # Recommended reading Overview
  • 4. # Market research has been defined as "the systematic design, collection, analysis, and reporting of data and findings relevant to a specific situation facing the company" (Kotler) < Secondary research involves the compilation and analysis of data and intelligence that have already been collected from another source, either external or internal < Primary research involves specialized methods for the direct collection of custom data Secondary research sources often use primary research methodologies to compile their data Market research definition
  • 5. Market research process Define the problem and research objectives Develop the research plan Collect the information Analyze the information Present the findings Potential level of outside vendor involvement Low Medium High High Medium
  • 6. Research Tool: Key Secondary Research S Observation O Customer Visits / 1on1's C Focus Groups F Survey Research R Choice models M Experiments E When to do research MRD Review Product Focus Alpha Beta End Of Life GM Concept exploration (S/O/C/F) Market segmentation and profiling (S/F/R) Competitive analysis (S) Market opportunity sizing/forecasting (S/R/M/E) Milestones: Customer profiling (O/C/F/R) Workflow validation (O/C/F) Feature validation (O/F/S/M) Positioning and messaging (F/R) Packaging and collateral (F) Pricing (S/C/E) Customer/product satisfaction (C/F/S) Marketing effectiveness (F/S) Market share tracking (S/R) The tools listed on the previous slide should be used in different ways, at different times in the product lifecycle
  • 7. # The problem in question must be clearly thought-through and stated < "A problem well defined is half solved." # Just as with marketing in general, the more focused (targeted) the problem definition, the greater the chance of success # The cost of the research project must be less than the cost of not solving the problem! < Put another way: the net profit resulting from achieving the research objective must be greater than the total cost of the research < Remember: time is also money < Therefore: don't do the research if you are not planning on taking significant action based on the results # Critical success factor: build consensus among all affected parties surrounding the problem statement and key objectives Defining the problem/objective
  • 8. Verb Possible Objects Identify Problems, opportunities, choice criteria Define Concept, design, potential Describe Decision process, usage, work environment Explore Perceptions, reactions, remedies Generate Hypotheses, alternatives, explanations Evaluate Feasibility, attractiveness, potential Select Product, concept, ad execution Test Preference, direction, profitability Measure Growth, size, frequency Prioritize Segments, needs, opportunities Monitor Trends, competition, events Track Spending, satisfaction, awareness # After a problem statement is created, specific research objectives need to be outlined # Hint: use the verb/object approach and pick from this list: Remember to STAY FOCUSED! Examples of research objectives
  • 9. # The research plan calls for decisions in the following areas: < Data sources – Secondary data, primary data < Research approaches – Observation, customer visits/1on1's, focus groups, surveys, choice models, experiments < Research instruments – Specific techniques within each approach < Sampling plan – Decide who will be the subject of the study – Arrive at the sample size and procedure for generating respondents Developing the research plan
  • 10. # The data necessary to achieve research objectives can come from either secondary or primary sources, or a combination # Secondary research is usually much quicker and cheaper than engaging in a primary research project < For assistance in locating relevant secondary market research, contact the MS&I team Watch for an announcement in early 2003 regarding the new MS&I Portal, enabling more self-service secondary market data Research plan: Data sources
  • 11. # Carpenters know that selecting the right tool is critical to doing the best job - the same holds true for market research # Research techniques are generally divided into the following categories: < Exploratory: gather preliminary data; suggest hypotheses or new ideas – Common tools: observation; customer visits/1on1's; focus groups < Descriptive: ascertain magnitudes; frequencies; percentages – Common tools: surveys < Causal: test cause-and-effect relationships – Common tools: choice models; experiments Research plan: Approaches and instruments
  • 12. Identify Describe Monitor Generate Define Explore Test Evaluate Prioritize Select Measure Track Research Tool: Secondary Research 1 1 1 2 2 2 2 2 2 2 1 1 Observation 1 1 1 1 3 2 X 3 X 3 X 3 Customer Visits / 1on1's 1 1 1 1 1 1 X X 3 X X 3 Focus Groups 2 2 3 1 1 1 X X 3 X X 3 Survey Research 2 1 2 X 2 X 1 1 1 1 1 1 Choice models X X X 3 3 X 1 1 1 1 X X Experiments X X 3 X X X 1 1 1 1 1 2 Legend: 1 = Primary means of achieving the objective 2 = Contributes secondarily to that objective 3 = Minimal fit with this objective X = Potentially misleading or dangerous if applied to this objective Evaluate Success Stages and Objectives: Scan Environment Generate Options Select an Option # Use the following table as a guideline to tool selection: Note that sometimes more than one tool is needed! Matching the tool to the job
  • 13. # The following slides highlight key aspects of the major research approaches # The term "researcher" in these slides is used loosely to indicate whoever it is that is actually managing the research project. The "researcher" could be: < A Product Manager < Someone in Sales < A member of the Market Strategy & Intelligence group < A contracted market research vendor < Etc. Highlights of major approaches
  • 14. Description Unobtrusive observation of the purchase or usage behaviors of a target group. Observational market research - with its basis in the science of anthropology - is gaining in popularity in technology companies in recent years. Usability research is closely related. General Procedure 1) Set objectives. 2) Decide upon location for observation and target sample. 3) Hire vendor if necessary. 4) Carry out observation. 5) Compile and report results. Sample Projects - Observation of purchase behaviors at retail. - Analyzing Website click-stream data. - Record usage behaviors and stumbling blocks for users of a new product. Result Format Qualitative Strengths - Enables unbiased view of actual customer behaviors - who can quarrel with reality? - No artificial boundaries or preconceived notions cloud the results. Weaknesses - Samples are generally small and non-representative of the population. - Difficult for the observer not to impact the observed. - Researcher bias may enter into the recording or reporting of results. Timeframe Varies widely. Simple projects can be carried out in hours or days. Complex anthropological projects involving the hiring of a vendor can take months to complete. Possible Cost Widely varying cost, from $0 to $100k+. Approaches: Observation
  • 15. Description A researcher interacts with a current or potential customer, either at the customer's worksite or at a separate location. Visits may be ad-hoc (as with a sales call) or programmatic (part of a predetermined market research strategy). General Procedure 1) Set objectives. 2) Select a sample. 3) Select team to conduct visit - the best teams are cross-functional. 4) Devise a discussion guide. 5) Conduct the interviews. 6) Debrief the teams. 7) Analyze and report results. Sample Projects - Assess initial reaction to new product concept. - Discover customer pain points. Result Format Qualitative Strengths - Richest of all communication modes - getting out of the office and talking to customers is always good. - Best way to get "into the customer's mind". - Information is all first-hand and does not suffer from a researcher's or analysts' interpretation. Weaknesses - Huge potential for interviewer bias. - Instability and imprecision of small sample sizes - do not use customer visits to estimate statistics. Timeframe Can often be carried out quickly. Possible Cost - Generally low in dollar terms. Usually limited to travel costs for the researcher's staff. Cost can escalate if a list of potential customers needs to be purchased and/or extensive travel is necessary. - But can be one of the most time-intensive forms of market research for the researcher in terms of their personal time. Approaches: Customer visits/1on1's
  • 16. Description A group interview, usually involving 6 to 10 participants who share common characteristics, who meet with a skilled moderator to discuss a particular subject. General Procedures 1) Set objectives. 2) Send out RFP to vendor detailing kind of participants to recruit and number of groups to hold. 3) Edit "screener" which will be used to recruit and filter appropriate participants. Recruiting then begins. 4) Meet with moderator to develop discussion guide - a loose "script" for the moderator. 5) Attend focus groups (1.5 to 2 hours each). 6) Vendor prepares report. Video tapes and other records of events may also be produced. Sample Projects - Preliminary testing of a new product concept. - Explore and identify new features for existing products. - Feel customer pain with current solutions. - Testing of messaging and marketing collateral. - As preparation for more extensive/expensive survey research to ensure that money is well-spent. Result Format Qualitative Strengths - Significant capacity to produce surprise and a fresh perspective. - Synergistic effect when customers are brought in contact with one another. - Possible to obtain the most frank and candid assessments when done anonymously. - Results (if videotaped) can provide a powerful tool for exposing company to the views of customers. - Useful for putting future surveys into the "language of the customer". Weaknesses - Reliance on small, non-random sample. Never extrapolate a percentage or frequency from a focus group experience. - Vulnerability to dominant and shy individuals over/underparticipating. Thus the need for a competent moderator and more than one group. - Moderator is not likely to be an expert in the particular industry or area of technology. Timeframe Working through all the stages of setting up and conducting a series of focus groups will take one to two months. Possible Cost $5k to $10k per group plus any travel costs. This price includes paying participants for their time. A minimum of three groups is recommended due to the significant possibility that any single group could get overly focused on a particular topic or participant - three groups ensures that a wide enough range of opinions is heard. Approaches: Focus groups
  • 17. Description A situation where a fixed set of questions is asked of a sample of respondents. Answer categories are usually specified in advance. Can be administered in-person (rare), by telephone, mail, or e-mail/Web. General Procedures 1) Prepare a Request For Proposal (RFP) that defines the problem and objectives; characteristics of the sample; proposed methodology. - Note that research vendors are a valuable source of insight into proper methods for achieving objectives. Not everything needs to be figured out before approaching a vendor but, at a minimum, the problem and objectives should be clear. 2) Vendor will work with you to create the screener and the questionnairre. The screener is made up of introductory questions in order to ensure that the correct respondents are completing the survey. 3) The size, characteristics, and source of the sample are agreed upon. More on this later in this document. 4) Survey is pre-tested on a small sample. This step is critical. Pretesting identifies critical flaws in the survey that will prevent wasting the entire survey budget. Resist the significant temptation to skip pretesting. 5) The survey is administered (fielded). The vendor should provide continuous feedback on issues and problems that arise. 6) Results are compiled, analyzed, and reported. Sample Projects - Customer satisfaction measures. - Market segmentation studies. - Product usage and ownership. - Purchase intention. - Brand image and perceptions. Advertising tracking. Result Format Quantitative (usually) or Qualitative Approaches: Surveys (1 of 2)
  • 18. Strengths - Ability to deliver precise numerical estimates of frequency and magnitude. - Superior objectivity to more "exploratory" methods. - Ability to apply a wealth of statistical techniques: measure if differences in segments are significant or not. - Can be repeated reliably over time to create longitudinal analyses. - Individual instrument strengths: Telephone surveys provide the most control over sample bias and have the highest response rates. Mail surveys are useful for complex surveys that include long lists of choice options. E-mail/Web surveys are quick to administer and easy to compile results for. Weaknesses - Tells you "what" but not "why". Can be overcome if exploratory techniques are used in conjunction. - Not good at revealing "what you did not know you did not know". Surveys are a precision tool, not a discovery tool. - Rely on self-reported data. Often customers do not recall, cannot verbalize what they do recall, lie, and are unreliable in projecting future behaviors. - Survey fatigue. People are generally over-surveyed and the quality of respondents is a major concern. - Does not deal well with group decision making processes - especially relevant in B2B markets. - Individual instrument weaknesses: Telephone surveys are the most expensive to administer and are seen as most intrusive by respondents. Mail surveys are slow to administer and suffer from low response rates. Due to the anonymity of the Web, E-mail/Web surveys can suffer from significant sample bias (more on this in the sampling section of this document). Timeframe Generally 1.5 to three months from start to finish. Possible Cost - Varies widely, but $50 to $100 per respondent is not an unreasonable starting point for budget estimations. - The extent and complexity of reporting and analysis required will drive cost up. Simple results in tabular format are less expensive (but may be less impactful). - Open-ended (free text) questions that need to be "coded" drives up cost somewhat. - The cost of good contact lists can vary dramatically. In general, more senior level decision makers cost more to contact. - Cost also varies with survey type: E-mail/Web surveys are the least expensive to administer, followed by mail surveys, with phone being the most expensive. Approaches: Surveys (2 of 2)
  • 19. Description A procedure that attempts to analyze how different factors combine to influence the choice of one product over another. Choice models provide an estimate of the importance or weight of each attribute of a whole product. Optimal combinations can be derived and "what-if" scenarios modeled. So-called "conjoint analysis" and "direct attribute weighting" are two approaches to choice modeling. General Procedures Choice modeling is a new and somewhat complex form of survey research. The assistance of an experienced vendor is critical in carrying out this type of research. To engage in this type of research, the basic steps are: 1) Agree internally upon problem and objectives. 2) Identify product attributes (i.e., price) that need to be studied. 3) Identify sample. 4) Create RFP and submit to vendor. 5) Iterate on all of the above with research vendor. 6) Vendor does fieldwork, compiles data, and presents results. Sample Projects - Price sensitivity research. - Analysing effect on market share of adding/removing features. Result Format Quantative Strengths - Ability to deal with complex issues and problems in a rational manner. - Does not require potential customers to guess about future behaviors. Future behaviors are derived from this research, but knowledge of them is not explicitly required. - Can deal with a large number of product attribute combinations. Weaknesses - One of the most expensive and time consuming forms of market research. - Difficult to implement for products purchased through a group decision making process. - Results are very susceptible to improper selection of product attributes. Garbage in, Garbage out. That is why the use of customer visits and other exploratory techniques is often used at the front end. Timeframe - Two to three months. Expect the "analysis and reporting" phase to take longer than with a standard survey. Possible Cost $100 to $150 per respondent is a good ballpark estimate to start with. Approaches: Choice models
  • 20. Description An artificial situation constructed by the researcher. Can take place in a laboratory, but often takes place in the field (in the market). As with any experimental process, the idea is to arrange conditions in such a way so that one can infer causality from the results. General Procedures The "scientific method" is the general procedure to be used for an experiment. This method is used extensively in the CPG industry but is underused in technology circles. In brief: 1) Observe some aspect of the market (i.e., when price goes down, volume goes up). 2) Invent a hypothesis related to what was observed (i.e., lowering my product's price will increase net profit). 3) Use the hypothesis to make predictions (i.e., if I lower the price by $100 then net profit will go up by at least 10%). 4) Create an experiment to test the hypothesis. Use an experimental group and compare this with a control group who experience otherwise similar market conditions. 5) Report findings (i.e., lowering the price by $100 resulted in a net profit increase of 12%). Sample Projects - Selecting the most effective advertisement or direct mail piece. - Choosing an optimal price point. - Decision on whether or not to release a line extension. Result Format Quantitative or Qualitative. Measurement criteria is often in terms of dollars earned or units sold. Resulting decision is often simply "go" or "no go". Strengths - Good for comparing a small number of alternatives in order to select the best. - Simple and straightforward to design. - Offers empirical evidence that is convincing and drives decision making. - Useful for forecasting. Weaknesses - The structure of the market may make it difficult to separate the experimental and control groups. - Only a limited number of options can generally be explored with an experiment. Choice modeling is a better choice if a large number of possibilities need to be explored. Timeframe Varies widely. Experiments are often long term in nature since it takes time for market dynamics to unfold in response to the experiment. Possible Cost Experiment cost is mostly about opportunity cost. In the above example, if the experiment to lower the price by $100 failed to prove the hypothesis then the company would have lost the revenue that it gave up by reducing the price by $100 for the experimental group. Approaches: Experiments
  • 21. # A three step process: < 1. Devise major content areas – I.e.: Screening questions -> current satisfaction and usage -> purchase intent -> demo/firmographics < 2. Develop specific questions < 3. Edit for length and clarity # Questions should be targeted specifically at addressing the key objectives of the research study < Shorter surveys increase response rates - very critical # The research vendor will work with you to properly word and format questions based on the objectives < Sample questionnaires are available from MS&I upon request Questionnaire development
  • 22. # Sometimes it is possible to survey the entire population, but usually this isn't feasible or necessary # The sample is the lynch-pin of all market research. The quality of the results is directly related to the appropriateness of the sample. Therefore: < Spend considerable time thinking about the nature of the sample required < If you do not feel good about the sample, do not do the research. It will be a waste of time and money # It is also critical to consider the sample when reviewing published secondary market research < Most secondary research has its basis in primary research techniques - don’t take published research at face value # If the sample is not valid then neither is the research # Research Plan: Sampling plan
  • 23. # Sampling unit: < Who is to be surveyed? # Sample size: < How many people should be surveyed? # Sampling procedure: < How should the respondents be chosen? Each of these will be dealt with separately in the following slides Sampling plan: Three elements of a sampling plan
  • 24. # The "population" or "universe" is the complete group to which the market research is directed # The first critical step is to be as clear about defining this universe as possible < If you are not sure what the universe is, then err on the side of a universe with a larger scope, rather than smaller. Why? < Because the danger of excluding appropriate respondents from the sample is likely to be greater than the cost of surveying a larger sample # The goal should be to create a sample which reflects the universe (as defined) as accurately as possible < Tools and data from the U.S. Bureau of Labor Statistics and other sources are available to help compare samples with populations. Refer to MS&I for assistance with this Sampling plan: Sampling unit
  • 25. # Who should you really speak to? < The end user? < The economic buyer (sponsoring line-of-business manager)? < The technical/infrastructure buyer (CIO)? # Depends on the product’s TALC stage < Early market: (visionary) economic buyers < Chasm/bowling alley: economic buyer < Tornado: technical buyer (just ship) < Main street: end user Sampling plan: Sampling unit (cont’d)
  • 26. # Two judgmental decisions are necessary to select an acceptable sample size # Margin of error: < Accuracy of results +/- a certain percentage < Lower margin of error means that it is easier to compare results. Example: – If the margin of error in a study is +/- 10%, then importance ratings on two key attributes of 47% and 65% are not statistically different # Level of confidence: < Measures how confident we are that the sample is accurate within the margin of error. Example: – A 95% confidence level means that we can assume the results within the margin of error (from above) 19 times out of 20. One time out of 20, the results will not be within the margin of error Sampling plan: Sample size (1 of 3)
  • 27. 80% 90% 95% 99% +/- 1% 4,096 6,766 9,604 16,590 +/- 2% 1,024 1,692 2,401 4,148 +/- 3% 456 752 1,068 1,844 +/- 4% 256 423 601 1,037 +/- 5% 164 271 385 664 +/- 10% 41 68 97 166 +/- 15% 19 31 43 74 +/- 20% 11 17 25 42 Level of Confidence MarginofError Samplesizerequired # The random sample size required is directly related to the acceptable margin of error and level of confidence as per the following table: 95% confidence +/-5% margin of error is an often-used standard for sample size Sampling plan: Sample size (2 of 3)
  • 28. # Issues with sample size < The previous table considers the case of a single survey question. If the survey asks 20 questions, then at a 95% confidence level there is a very high probability that at least one of the questions will be outside the specified margin of error < The level of confidence and margin of error are computed separately for each sub-segment in the sample. For example, from the previous table, a question asked to all 400 respondents is accurate to +/-5% 19 times out of 20. But if the respondents are then sub-segmented into small groups, the responses of these groups have a much lower level of confidence and higher margin of error. The table can be used to estimate this effect MOST IMPORTANT: The previous table assumes that the sample was generated randomly from the population. In reality, most samples are not truly random. Always be cautious about interpreting/comparing results Sampling plan: Sample size (3 of 3)
  • 29. # Samples are derived in different ways < The "best" way depends on the characteristics of the market, the research objective and budget available. Consult with MS&I and the vendor for further advice as to appropriate technique < Probability samples are necessary for the calculation of proper confidence limits (as per the previous slides) # Probability sampling techniques < Simple random sample: Every member of the population has a known and equal chance of selection < Stratified random sample: The population is divided into mutually exclusive groups (i.e., by industry) and random samples are drawn from each group # Non-probability sampling techniques < Convenience sample: The most accessible population members are chosen < Judgment sample: Researcher judgment is used to select population members who are good prospects for accurate information < Quota sample: The researcher finds and interviews a prescribed number of people in each of several categories Sampling plan: Sampling procedure
  • 30. # For market sizing and market share work it is critical that a probability sampling technique is used. Issues relevant to the different measures of market size include: # TPM - Total Potential Market < TPM accounts for all current and potential users of a product < To use primary research to ascertain the size of the TPM, ensure that the universe definition is set as wide/general as possible # TAM - Total Addressable Market < TAM accounts for all current users of a product in the marketspace under consideration < Therefore, the sample would have to include users of competitive products and possibly direct substitutes (like homegrown solutions) # TCM - Total Captured Market < TCM accounts for Corel's market share, therefore Corel's customer list might be used to generate this sample < But beware of sample bias due to low/uneven product registration rates Sampling plan: TPM/TAM/TCM and sampling
  • 31. # Response rates < The lower the response rate, the greater the likelihood of response bias < Response bias = a skewing of the results based on the possibility that the actual respondents within the sample do not represent the universe to be measured < If possible, try to ascertain how respondents differ from non-respondents so that hypotheses may be generated about the effect on the survey results # Cost < The larger the sample the higher the cost (but at a declining rate) < The greater the efforts taken to approximate a random sample, the higher the cost Once again, sampling issues are highly critical to any research project. If the sample is poor, then everything else is meaningless at best (or misleading at worst) Sampling plan: Other issues with sampling
  • 32. # Approximate costs for project budgeting can be found earlier in this presentation < However, actual costs will vary widely depending on all of the issues that we have identified previously # A key decision is whether or not to state your available project budget to the research vendor in the RFP. Guidelines: < If you feel that your available budget meets or exceeds the cost estimates in this document, then do not inform the vendor of your budget. Sending out multiple RFPs will result in the best price with minimal padding < If you feel that your available budget may be insufficient relative to the guidelines presented here, then it is a good idea to inform the research vendor ahead of time of the limit on what you can spend. This will enable the research vendor to specify possible approaches and alternatives that may allow the project to succeed within your budget Always allow some room for unforseen costs - they are inevitable Budgeting
  • 33. # Market research outside of North America, both primary and secondary, is at an earlier stage than within North America # As a general rule: multiply both cost and time estimates by 1.5 for Tier 1 countries (UK, France, Germany, Japan) and 2x for other geos # Privacy laws, especially in the EU, may also significantly impact the ability to collect data < Consult with your research vendor for relevancy to your specific project # Also note that June/July/August can be difficult months to carry out a research project in Europe Global issues
  • 34. # MS&I has placed the following vendors on its “short list” to receive primary market research RFPs from Corel < Vision Research < Phase 5 # Overviews of these vendors appear on the following slides # Certain specialized projects may require particular vendors. MS&I will help you assess your needs # Note that this list will change from time to time. Please refer to the latest version of this slide deck for the most recent list # Before initially contacting any vendor, all new RFPs should be reviewed by MS&I Recommended vendors for RFPs
  • 35. # Comments: < Small operation, office in Ottawa < Conducts all types of qualitative and quantative research < Has worked for Corel in the past. Most recently: Vector Illustration survey in the U.S., UK, Germany, and France < Likely to be the “low cost” vendor for a project. Do not rely on them for extensive strategic analysis and implications # Contact info: < Candace Fedoruck, Executive Vice President < Phone: 613 594 8282 / 1 888 310 6062 < E-mail: candace@delta-media.com Vision Research Recommended vendors (cont’d)
  • 36. # Comments: < Offices in Ottawa and Toronto < Targets technology companies: high Web and technology IQ < Ani Ieroncig, former Corel Market Research Director is our main contact there - providing some knowledge of issues facing Corel < Phase 5 key value-add is the strategic element of projects. They are unlikely to be the low-cost vendor, but for projects that require complex setup or detailed analysis, they may be a good choice # Contact info: < Ani Ieroncig, Senior Manager < Phone: 613 241 7555 ext.108 < E-mail: anii@phase-5.com Phase 5 Recommended vendors (cont’d)
  • 37. # Comments: < A third recommended vendor will be inserted here when MS&I has identified an appropriate candidate # Contact info: < TBD TBD Recommended vendors (cont’d)
  • 38. # Market Strategy & Intelligence (MS&I) is available to assist you in identifying and locating relevant secondary research resources < In early 2003, Corel will launch a Market Intelligence Portal which will enable more of a self-service approach # The role of MS&I in product-specific market research projects is to act primarily as a consultant, assisting your team in scoping projects and in selecting vendors < The bandwidth is currently not available to directly manage every project that takes place # Review the latest “RFP Template” which is meant to be used in conjunction with the content in this slide deck Role of MS&I in Corel’s market research
  • 39. # The Market Research Toolbox - A Concise Guide for Beginners < Edward F. McQuarrie, 1996 # An excellent, non-technical introduction and guide to market research written for managers (not market research professionals) < A good two-night read: available for borrow from Blaine Recommended reading