Masters in Creative Technologies Research Methods

1. Research Methods:
Qualitative vs Quantitative
Creative Technologies Masters
(MA/MSc) Programme
November 2015
Dr Tracy Harwood, Senior Research Fellow
Institute of Creative Technologies / Usability Lab @ De Montfort University
E tharwood@dmu.ac.uk

5. Qual or quant…
depends what your research aims are
what method you use
what your data set comprises
how you analyse data
your ability to interpret findings!

6. overview
™ Designing research – qual vs quant
™ Doing data collection
™ Analysing data
™ Using findings
™ Contemporary issues
™ multichannel marketing
™ netnography and infinite reality
™ neurosciences
™ mixed methods (transdisciplinarity)

7. Problem definition
Interest in product ‘x’ in the South East is declining
™ shrinking market
™ loss of market share
™ poor quality
™ expensive
™ awareness
™ reputation

8. Designing research
validity
An interactive model of research design (Maxwell, 1996)
methods
research
questions
conceptual
context
purposes

9. Research questions
™ experimental + exploratory = qualitative
™ descriptive + explanatory = quantitative
?

10. Research questions
™ experimental + exploratory = qualitative
™ descriptive + explanatory = quantitative
?

11. Research context
market growth
consumer
behaviour
R&D

12. quantitative design
™ existing data set interrogation
™ + readily available and cost effective
™ - historical
™ observation
™ + non-intrusive(?) and response rates
™ - interpretation
™ survey
™ + straight forward and easily analysed
™ - sample bias, response rates and cost

13. Types of sampling - quantitative
™ census > rigour vs impractical
™ probability > rigour vs costs
™ random > error rate vs impractical
™ quota (stratified) > coverage vs bias
™ judgment > selective criteria vs bias
™ convenience > access vs bias
™ accidental > mmm?
Available free download at
www.nao.org.uk/ publications/
Samplingguide.pdf

14. Example - recommended
sample size (NAO) for analysis
18-25
years
26-35
years
36-45
years
46-55
years
Totals
Male 50 50 50 50 200
Female 50 50 50 50 200
Totals 100 100 100 100 400
Min = 50 – 100 for each group or sub-group of interest

15. Different survey contact methods (Jobber, 2001)
Questionnaire
Face to
Face Telephone Mail Internet
Use of open ended
questions High Medium Low Low
Ability to probe High Medium Low Low
Use of visual aids High Poor High High
Sensitive questions Medium Low High Low
Resources
Cost High Medium Low Low
Sampling
Widely dispersed
populations Low Medium High High
Response rates High Medium Low Low
Experimental control High Medium Low Low
Interviewing
Control of who completes
q/naire High High Low Low/High
Interviewer bias Possible Possible Low Low

16. Developing a questionnaire
Planning
Stage
Planning
Pilot
Stage
Definition of the
research problem
Exploratory research
Information required
Definition of
population
Target groups
Survey method
Ordering of topics, type of question
wording and instructions
Layout Scaling
Probes and prompts Coding
Pilot testing
Redesign
Final questionnaire
Source: Jobber (2001)

17. Qualitative design
™ understanding meaning and/
or context
™ identifying unanticipated
phenomena
™ understanding process
™ developing causal
explanations
Miles & Huberman, 1994

18. Types of sampling - qualitative
™ homogenous
™ maximum variation (heterogeneous)
™ critical case
™ confirming/disconfirming case
™ snowball/chain
™ typical case
™ political
™ purposeful (random)
™ convenience

19. Focus group definition
“interview generally involves 8-12 individuals who discuss a
particular topic under the direction of a moderator”
Stewart & Shamdasani (1990)
“a number of respondents gathered together to generate
ideas through the discussion of, and reaction to, specific
stimuli. Under the steerage of a moderator, focus groups
are often used in exploratory work or when the subject
matter involves social activities, habits and status” MRS
(2008)
“discussion is focussed on a particular topic and group
dynamics assist in data generation” Cattarall and
Maclaran (1997)

20. FG components for success
™ interview
™ number of participants
™ moderator
™ roles
™ discussion
™ stimuli
™ duration

21. Discussion process
™ group dynamics
(Hess, 1968)
™ stimulation
™ security
™ spontaneity
™ synergism
™ snowballing

22. FGs - advantages
™ in-depth data possible
™ participants ‘own words’
™ speed of data collection process
™ use of recording equipment
™ no technical knowledge of research method
required by participants
™ flexible and responsive to interesting /
unanticipated answers
™ moderator
™ ability to reframe questions to enhance
understanding
™ training - Market Research Society guidelines

23. FGs - disadvantages
™ Sample size
™ non-random, often self-selecting (strong
opinions)
™ representativeness > generalisability
™ moderator!
™ potential bias from moderator in framing
questions
™ participants!
™ subjectivity
™ ‘social posturing’ within group
™ dominance
™ ‘groupthink’
™ costs

24. FGs - overcoming disadvantages
™ multiple focus groups + collect other data (triangulate)
™ use purposive sample ie., knowledge of individuals
™ record interviews to evaluate moderator bias
™ use a question framework to minimise group problems
™ ask individuals to introduce themselves
™ single speaker turn rules
™ seek opinions from all participants
™ use experienced moderator!
™ costs… use another method

25. Neurosciences – qual / quant
™ human factor analysis
™ psychology
™ physiology
™ behaviour

26. Internet infinite reality – qual / quant
™ ‘virtual living’
™ identity
™ social identity
™ co-creation with
environment provider
™ gamified experience

27. validity
™ influence of medium >
truthfulness
™ ‘we think’ > social dynamics
™ attention > preparedness to
participate
™ experience > ability to elucidate

28. Doing data collection
™ legals
™ privacy in communications
™ data protection > opt-in, opt out
™ Market Research Society
™ ethics
™ codes of conduct
™ practical data management

29. MRS Code principles
Researchers shall
1. ensure that participation in their activities is
based on voluntary informed consent
2. be straightforward and honest in all their
professional and business relationships
3. be transparent as to the subject and purpose
of data collection
4. respect the confidentiality of information
collected in their professional activities
5. respect the rights and well being of all
individuals

30. MRS Code principles
6. ensure that respondents are not harmed or
adversely affected by their professional
activities
7. balance the needs of individuals, clients, and
their professional activities
8. exercise independent professional judgement in
the design, conduct and reporting of their
professional activities
9. ensure that their professional activities
are conducted by persons with appropriate
training, qualifications and experience
10. protect the reputation and integrity of the
profession

31. MRS Code principles
6. ensure that respondents are not harmed or
adversely affected by their professional
activities
7. balance the needs of individuals, clients, and
their professional activities
8. exercise independent professional judgement in
the design, conduct and reporting of their
professional activities
9. ensure that their professional activities
are conducted by persons with appropriate
training, qualifications and experience
10. protect the reputation and integrity of the
profession
goods or services, or
vouchers to purchase client
goods or services, must not
be used as incentives in a
research project

32. Analysing data - quantitative
™ descriptive analysis
™ summary stats – frequency analysis
™ cross tabulation
™ coding open ended responses
™ differences between subjects
™ significance tests
™ correlations between variables
™ independent vs dependent
™ regression
™ error estimation
™ sample representation (size)
™ confidence
SurveyMonkey
xls

33. Analysing data - qualitative
™ transcription and extraction
™ emergent themes > events,
descriptions, comments,
behaviour
™ coding categories > content
analysis, compare
consistencies, differences,
patterns
™ discourse / narrative analysis
> critical incidents
™ sense-making
™ theory formed = inductive process
™ using existing theory = deductive process

34. Using findings
™ adage > rubbish in, rubbish out
™ inform strategy development
™ planning requires decision-making which requires
research
™ embed within product and service, promotions,
distribution, pricing, etc developments
™ incremental vs radical
™ underpinning for future research > comparative
analysis

35. Conclusion
™ Aims and objectives inform research design
™ Mixed methods improve quality of findings
™ Good practice essential in data collection
™ Avoid paralysis by analysis > any amount of data
will not tell you what to do
™ Acknowledge the role of theory and secondary
data

36. Selected sources
Any
qual
or
quant
text
will
be
useful!
Bryman
&
Bell,
Business
research
methods,
Oxford
Walford,
Tucker
&
Viswanathan,
Handbook
of
measurement,
Sage
Maxwell,
QualitaEve
research
design,
Sage
Miles
&
Huberman,
QualitaEve
data
analysis,
Sage
Kozinets,
Netnography,
Sage
Hine,
Virtual
methods,
Berg
Margolis
&
Pauwels,
Visual
research
methods,
Sage