The document provides an overview of Lesson 2 from a short course on Market Research. The lesson covers quantitative research methods, including defining quantitative research, sources of quantitative data, quantitative modes of data collection, population and sampling techniques, and significance testing. It also discusses writing questionnaires, including linking questions to research objectives, types of survey questions, considerations for survey length and question wording, and testing questionnaires.

1. A short course in Market Research
with Ray Poynter
(English language)
Lesson 2
Tuesday, 8 July
• Ch. 3, Quantitative Research
• Ch. 5, Writing Questionnaires
@RayPoynter
ray.poynter@thefutureplace.com

2. Dates and Modules
Thu 3 July
Introduction
The context for market research
Communicating results
Tue 8 July
Quantitative research
Writing questionnaires
Thu 10 July
Qualitative research
Analysing qualitative data
Tue 15 July
Major applications of research
Mobile market research
Thu 17 July
Emerging research methods
Communities
Social media research
Tue 22 July
Fri 25 July
How to analyse quantitative data
Quantitative analysis techniques
Pricing research
Thu 24 July
B2B (business to business)
International research
Political polling
Tue 29 July
Research ethics, Guidelines and laws
Current areas of sensitivity
Questions from new researchers

3. QUANTITATIVE RESEARCH
Part A

4. What is Quantitative Research?
• Measuring things to:
– Describe
– Monitor
– Explain
– Predict
• Seeks to generalise the measurements to a wider
context
• Assumes things can be measured with numbers
– How many children do you have?
– How much do you love your children?

5. Sources of Quantitative Data
• Surveys
• Store audit data
• Diary data
• Web analytics
• Transactional data (e.g. phone usage,
purchases, downloads)
• Meters (e.g. TV viewing meters)
• Social media data

6. Quantitative Modes
Global Japan
Online 35% 52%
Telephone 17% 2%
Face-to-face 16% 13%
Postal 1% 12%
Other
(mostly non-survey)
31% 21%
Source: ESOMAR Global Market Research Report, 2013

7. Population and Sample
Population
– Everybody we are interested in
• All adults in Japan
• Everybody with a Nissan car
• Every male over 40 who buys whiskey every week
Sample
– Some of the people from a population
– Used to estimate what the answer would
have been if we had interviewed everybody
If we interview everybody then it is a census

8. Reasons why the Sample Might
NOT Match the Population
1. The list of the population may be incomplete
2. The method of selecting people might not be
‘statistically fair’
3. Some people may decline to take part
4. Random chance

9. Significance Testing
• Significance test looks at the odds that something
is just chance, i.e. sampling error / noise
• Significance testing helps us understand if
differences are big enough to be interested in
• If a result is ‘significant’
– It MIGHT be a genuine finding
• If a result is NOT significant
– Perhaps there is no difference
– Or, the difference is small
– Or, the sample was too small to be sure about the
difference

10. Imagine Coin Game – You want a head
TAIL 50% chance it was bad luck, 50% chance coin is not fair
TAIL 25% chance it was bad luck, 75% chance coin is not fair
TAIL 12.5% chance it was bad luck, 87.5% chance coin is not fair
TAIL
6.3% chance it was bad luck, 93.7% chance coin is not fair
Significant at 90% level to market researchers
TAIL
3.1% chance it was bad luck, 96.9% chance coin is not fair
Significant at 95% level to market researchers

11. Coin Game
1. Everybody toss a coin 5 times and write down
how many tails
2. Then, do it again, 2 numbers per person
– To pretend we have twice as many people
3. How many people have 5 tails
– If you had 5 tails both times, put both hands up

12. Types of Sample
Random probability sample
– Everybody in the population has a known
probability of being selected
Quota sample – controls the outcome
– E.g. 50% male, 50% female, 25% North, 75% South
Convenience
– People who can easily be sampled
Most online access panels are a mixture of 2 and 3

13. How Big Should the Sample Be?
• Size is not everything
– Quality can be important
• There is a theory – read it in the books
• N = (z2σ2) / e2
• Three factors
– Custom and practice
– The size of the smallest cell (usually 50 or 100)
– The budget
Sample size calculator at http://www.maritzresearch.co.uk/insights/maritz-stats.aspx

14. 1936
US Presidential Election
• American magazine, Literary Digest, wanted to
predict the Presidential election
• Mailed 10 million questionnaires
– 2 million replies – in 1936
• Predicted Landon would beat Roosevelt
– But Roosevelt won
• Why?
– Sample phone owners, car owners, buyers of the
magazine
– During an economic recession

15. Predicting the Population
Two key problems
1. Were the respondents representative?
2. Were respondents willing and able to tell the
truth?
E.g. “How many kilos of rice will you buy next
month if the price goes up by 10%?”
Main link between survey data and the real
world is modelling:
• Benchmarks
• Regression
• Econometrics

16. Weighting the Data
Weighting Example Target Actual Weight Report
Men 450 500 0.9 450
Women 450 400 1.125 450
Total 900 900 900
Weighting Example Target Actual Weight Report
Men 450 500 0.9 450
Women 450 400 1.125 450
Total 900 900 900
Weighting Example Target Actual Weight Report
Men 450 500 0.9 450
Women 450 400 1.125 450
Total 900 900 900

17. Weighting the Data
Weighting Example Target Actual Weight Report
Men 450 500 0.9 450
Women 450 400 1.125 450
Total 900 900 900
2 Main Reasons
• To make the data closer to expectations
• To remove sample effects
• Too many men might result in different answers

18. The Quiz
• The quizzes are mostly to check the
communication.
• If the communication has been clear
– Most students should score 100% or close to
100%.
• If people do not score 100%
– We will try to explain the material in a different
way.
• Take Quiz 02, Part A (and then a short break)

19. Part B
Any questions before we re-start?

20. WRITING QUESTIONNAIRES
Part B

21. Two Tips
1. Choose one textbook and stick to it
2. Watch Pete Cape’s video on NewMR
Ian Brace
Questionnaire Design
Published by Kogan Page
http://www.koganpage.com/editions/questionnaire-design/9780749467791
http://newmr.org/play-again/a194cb09/

22. Two Tips
1. Choose one textbook and stick to it
2. Watch Pete Cape’s video on NewMR
Ian Brace
Questionnaire Design
Published by Kogan Page
http://www.koganpage.com/editions/questionnaire-design/9780749467791
http://newmr.org/play-again/a194cb09/

23. Linking Questions to the Objectives
Objective 1
Objective 2
Objective 3
Question 1
Question 2
Question 3
Question 4
Question 5
Question 6

24. Main Types of Survey Questions
• Open / Closed
• Single / Multi
• Nominal, Ordinal, or Cardinal
• Special questions
– Timed questions, eye tracking, geolocation etc

25. Open/Closed Questions
Open
How long does it take to get home, in minutes?
(Please type your answer in the box)
Closed
How long does it take to get home, in minutes?
(Please select one answer)
 Less than 10 minutes
 10 to 25 minutes
 26 to 40 minutes
 41 minutes to 60 minutes
 More than one hour

26. Single / Multi
Single
Which is your favourite drink?
(Please select one answer)
 ビール
 ワイン
 ブランデ一
 None of these
Multi
Which of these do you ever drink?
(Please select all that apply)
 ビール
 ワイン
 ブランデ一
 None of these

27. Nominal, Ordinal, Cardinal
Nominal
Which do you regularly eat?
(Please select all that apply)
 ご飯
 パスタ
 うどん
 None of these
Ordinal
Please rank these.
1=most favourite
2=next favourite, 3=least favourite.
 ご飯
 パスタ
 うどん
Cardinal
Rate how much you like these.
10 = very much, 0 = do not like at all
ご飯 0 1 2 3 4 5 6 7 8 9 10
パスタ 0 1 2 3 4 5 6 7 8 9 10
うどん 0 1 2 3 4 5 6 7 8 9 10

28. How Long Should the Survey Be?
As short as possible
– But no shorter
Longer surveys lead to:
– More people dropping out
– More people not wanting to do surveys
– Satisficing
Satisficing
– Speeding
– Straight lining
– Skipping questions

29. When to Use Don’t Know,
NA, None of These … ?
When respondents might want to pick them
Which is your favourite drink?
(Please select one answer)
 ビール
 ワイン
 ブランデ一
 None of these
What is your favourite drink?
(Please select one answer)
 ビール
 ワイン
 ブランデ一
 Other
How long does it take to travel from your home by bus to the
nearest major train station?
 Less than 30 minutes
 30 to 60 minutes
 More than one hour
 Don’t know

30. Key Rules for Survey Questions
1. Respondents should understand what they
are supposed to do.
2. Questions should NOT be ambiguous
o Was the train on time and clean today?
o Questions should minimise bias.
Do you sometimes drink
too much wine or beer?
 Yes
 No
How often do you drink more
than 4 glasses of wine or beer?
 More than 10 times a month
 6 to 10 times a month
 1 to 5 times a month
 Less often
 Never

31. Making Surveys More Engaging
1. Choose the right style for your audience
2. Narrative flow
3. Don’t annoy the respondents
– Grids
– Long lists
– Long introductions
– Asking the same question 2 or more times

32. Gamification?
• Lots of interest
• But! Not proven, still experimental
http://www.researchthroughgaming.com

33. How to Test Questionnaires
1. Compare with Research Objectives
2. Spelling and grammar
3. Software checks
– Routing
– Auto-responses / dummy respondents
4. Comparing a printed copy with the screen
5. Soft launch
– 20 to 50 completes
– Check responses, drop-outs, missing data etc

34. Questions
And Then Quiz B

35. Feedback for the next lessons?
• Feedback now, GREAT!
• Or,
– Email it to ray.poynter@thefutureplace.com