Session 2 Methods qualitative_quantitative


Published on

Qualitative vs Quantitative research methods - the right tools for the right job. New Media Research Methods.

Published in: Education, Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Session 2 Methods qualitative_quantitative

  1. 1. New Media Research MethodsPart 1 – How researchmethods relate to theresearch questionPart 2- Qualitative andQuantitativePart 3 – Data collectionPart 4 – Presentation andanalysis
  2. 2. New Media Research Methods- Part 2Focus on the qualitative and quantitative methods:The right tools for the right job.Gosia Kwiatkowska
  3. 3. Recap from last week
  4. 4. New Media Research processAnalysisDesignImplementation
  5. 5. Qualitative Quantitative• Qualitative - deepunderstanding of theexperience. Noteverything can bereduced to numbers.• E.G.Observation, ethnography• Quantitative –objective,measurable. Helps toestablish patternsand relationships.• E.G. Survey
  6. 6. Which one to use?
  7. 7. Qualitative Method?– User perspectives– Questions: how and why?– Captures perceptions, judgements, meanings,processes and reasons– Open ended questions, checklist of topics– Hypothesis and follow up questions are generatedduring data collection and analysis– Subjective – not easy to objectively verify– Interviews allow to probe– Provides deeper insight– Broader understanding and explanations, views
  8. 8. Qualitative Method?• Strengths– Participatory– Rich, detailed data– Considers users perspectives and the context for theirbehaviours• Weaknesses– Hard work– Time consuming– Smaller sample of users– Not easily verifiable– Not easy to group your responses/categorised
  9. 9. Quantitative Methods– Designed to ensure objectivity, reliability and ability togeneralise– Test predetermined hypothesis – explanations– Who, how much, how many?– Closed questions– Short answers– Averages, percentages, ranges, means, frequencies– Can generate accurate and precise data– Can test statistical relationship between variable– Can prove whether or not a particular problem exists– Can identify specific characteristics of a population
  10. 10. Quantitative Methods• Strengths– Robust– Objective– Verifiable• Weaknesses– Out of context – human behaviour, real worldsettings are not considered– Any variables left out of data collection are notused in analysis
  11. 11. Quantitative or Qualitative or Both?• Quantitative methods can highlight an issuesthat could then be studied in depth usingqualitative methods.• Qualitative methods might be used at thebeginning of a study to help the researcher todecide what closed questions could be used inthe bigger quantitative survey orquestionnaire.
  12. 12. New Media ResearchInterviews• Purpose• Characteristics• Advantages• Limitations• Process/Stages/Questions
  13. 13. Definitions / purposes“Qualitative research [such as interviewing]attempts to understand meanings thatpeople give to their deeds, experiences, or toother social phenomena”Silverman D (1997)‘The purpose of interviewing is to find out whatis in and on someone else’s mind. We findout from them those things which we cannotdirectly observe’May T (1997)
  14. 14. Advantages / limitations• Your views …
  15. 15. Advantages• Rich data - excellent for complex subjects• Meanings / understandings / perceptions explored• Powerful when territory unfamiliar• Unforeseen issues / experiences elicited• Clarification / Follow up Q&As possible• Captive subjects• Certainty over who is responding (unlike surveys)• Non-verbal communication / Observational opportunities
  16. 16. Disadvantages (1)Subjects may:• Conform to expectations – (social desirability)• try to be rationalInterviewer may:• Be inconsistent• (Unwittingly) bias respondents’ answers• Mis-interpret answers
  17. 17. Disadvantages (2)• Difficult to get quantitative data• Data analysis difficult / complicated• Unrepresentative - reliance on key figures• Time consuming / costly• Stressful for interviewer (& interviewee?)• Low validity / reliability
  18. 18. Reliability and validity• Validity: "By validity, I mean truth: interpreted as theextent to which an account accurately represents thesocial phenomena to which it refers. " (Hammersley,1992). Pg. 57. (e.g. does the data-gathering measurewhat you want it to measure?)• Reliability: the degree of consistency with whichinstances are assigned to the same category bydifferent observers or by the same observer ondifferent occasions". (Hammersley, 1990). Pg. 67 (e.g.does the data-gathering produce the same results ifrepeated?) .
  19. 19. Validating interviews• Triangulation• Purposive sampling - focus on specific population• Choose deviant case• Member check (refer findings back to subjects)• Co-researcher re-coding• Researcher ‘reflexivity’• ‘Fair dealing’ – ensure all viewpoints mentioned
  20. 20. But don’t forget …One case may be enough(!):‘What is wrong with samples of one? Why shouldresearchers have to apologise for them? ShouldPiaget apologise for studying his own children, aphysicist for splitting only one atom?’Mintzberg H (1973) The nature of managerial work NY:Harper & Row
  21. 21. Preparing the interview (1)Getting ethical approval:“The (UH) Ethics Committee will … carefully evaluate thefollowing aspects of your application:• the validity of the research• the welfare of the research subjects• the dignity of the research subjects• the ability of the researcher/team/supervisor to conduct theresearch”[see:]
  22. 22. Preparing the interview (2)• Aims/objectives of the interview• Where these fit with overall research study• Interview style:– Structured unstructured– Formal informal– Open closed• Choose physical setting (if possible)• Atmosphere / environment
  23. 23. Preparing the interview (3)Individual questions/areas/themesFact• objective information (e.g., age, gender, education, behaviour,experience)Opinion / Preference / attitude / feelings• evaluative (e.g., satisfaction, agreement, likedislike)Intended Behaviour• motivation or intention (e.g., likeliness, willingness)
  24. 24. Recording the interviewTo tape or not to tape?? Consider:• Effect on interviewee• Listening / transcribing time• Reliability of machine / recordingBut also…• Scribbling whilst listening• Making sense of notes
  25. 25. Conducting the interview• Thank interviewee• Set the scene (why / how / where of study)• Give idea of question areas• Give ground rules (can refuse to answer/canterminate interview, anonymity etc.)• Start with demographic and ID questions• (usually) start with general question and thenfunnel
  26. 26. Types of question (1)Open• Opening stages in line of questioning (funnel)• Invites opinion, general knowledge.• Can cover areas where interviewer’s own knowledgelacking• No presumption about responseClosed• Elicit hard facts• Control pace/direction of interview
  27. 27. Types of question (2)Probing• Extracts more depth• Maintains line of enquiryLeading• Confirm interviewee’s answer• Help interviewee, by rephrasing answer• Bring a line of questioning to an end (summarising)
  28. 28. New Media ResearchQuestionnaires/Surveys• Purpose• Characteristics• Advantages• Limitations• Process/Stages/Questions
  29. 29. Purpose of questionnairesTo survey large number of people, todescribe/explain characteristics/opinion of apopulation, usually through a representativesample.Questionnaires measure generalities / theextent to which groups of people behave orthink in certain ways.
  30. 30. Characteristics of questionnaires• Cross-sectional• Mainly quantitative• ‘Snapshot’ in time• Qs and As standardised
  31. 31. Advantages / limitationsYour views …
  32. 32. Advantages• Reaches large / dispersed populations• Can generalise results• Provides quantitative, authoritative (?) data• Appears easy– work done by respondent• Relatively cheap and quick (per unit)• Removes personal influence• Respondent works in own time• Provides structure for report• Replicable
  33. 33. Limitations (1):Questionnaire construction• Low response rate (5-25%?)• Respondents may differ from non-respondents• No way to adapt add/remove questions• Little opportunity for respondents to explain• Can over-simplify issues• Respondents ‘shoehorned’
  34. 34. Limitations (2):Respondent issuesRespondents may:• Take middle position in polar responses• Lack recall / rely on recent experience• Want to please researcher by being:– socially responsible– compliant
  35. 35. Process/stages• Formulate study aims• Identify objectives that address aims• Decide what information is required, and from whom• Decide sampling frame (elements making up population)• Research for similar questionnaires• Formulate appropriate questions• Decide distribution method– Postal; Telephone; Clip board/street; Web/Internet; Email;f2f• Pilot (see next slide) and get feedback• Amend• Distribute / administer
  36. 36. The pilot• Use more “open ended” questions than in finalproduct (often interviews are undertaken first)• For “Pilot” responses look for:– Variation in type of answer– Redundancy – areas of no/limited response– Evidence of ambiguity etc.– Acquiesce• Amend questionnaire as appropriate
  37. 37. Question areasFact• objective information (e.g., age, gender, education,behaviour)Opinion / Preference / attitude• evaluative (e.g., satisfaction, agreement, likedislike)Intended Behaviour• motivation or intention (e.g., likeliness, willingness)
  38. 38. Question types (1)• Open questions– For further information (‘tell us more ‘)– For new information (‘what are your view on …’)• Multiple choice checklist– One answer only– All that apply (inc. Guttman scale)• Binary (yes/no)– Good for filtering• Rank order
  39. 39. Question types (2)• Likert scale– Agreement (strongly agree, agree, …)– Frequency (always, frequently, …)– Importance (very important, important …)– Quality (excellent, good, average …)– Likelihood (definitely, probably …)• Semantic differentialReactions to stimulus words / concepts in terms of ratings on‘bipolar’ scale with contrasting adjectives at each end:– Excellent ………………terrible– Helpful …………………unhelpful
  40. 40. Closed v open questions (1)closed …Advantages• Quick & easy for respondents• Less articulate for disadvantaged• Fewer irrelevant answers• Easy to code and analyseDisadvantages• Responses suggest ideas (e.g., shoehorning)• Frustrates respondents if categories not exhaustive• Misinterpretation goes unnoticed• Complex issues forced into simple categories
  41. 41. Closed v open questions (2)Open …Advantages• Permits detail, clarification• Unanticipated answers• Reveals the logic behind responseDisadvantages• Generalisation or comparison difficult• Coding and statistical analysis difficult• Irrelevant answers possible• Bias towards educated• Time consuming for respondents & researcher
  42. 42. Sampling
  43. 43. Good practice• Explain purpose• Catch interest early• Question sequence logical / helpful - broad tospecific• Closed questions need "complete" set of responsealternatives• Use appropriate language• Include clear instructions• Use clear tick boxes• Keep short
  44. 44. Increasing response rate• Name recipient if possible• SAE• Prize (?)• Offer copy of the report• Good design/construction• Engender involvement/interest• Chase non-respondents• Target ignored/committed groups
  45. 45. Issues to considerQuestion wordingQuestion orderQuestion types
  46. 46. Question wordingWhat’s wrong with these questions:Do you like using the Internet and playing onlinegames?Do you agree with most people that onlinegaming is becoming more popular?
  47. 47. Question orderWhat’s wrong with these questions:1) Do you have any children?2) How old are you?3) Are you married?4) How long have you been studying at theUniversity?
  48. 48. Question types (1)Leading questions:• Why do you think the PC’s are worse than Apple computers?Social pressure:• Would you prefer to have more money spent on improving theeducation or more tax cuts?Presuming questions• How often do you search the Internet?
  49. 49. Question types (2)Ambiguous/imprecise questions• How much time to you spend playing online games?Double-barreled questions• How long have you been using computers and the Internet?Too much knowledge dependent questions• Do you agree with the … theory?
  50. 50. Question types (3)Memory dependent• How many times have you played a computer game in the lastmonth?Wish List/hypothetical questions• Would you like better computer games to be designed?
  51. 51. Match tools to your questions• For questions on factualclarification• Use polls and surveys• For questions on opinion• Use surveys, interviews, focusgroups• For questions on experience• Interviews, focus groups,observations, user testing, andethnographies• For questions on concepts• surveys, interviews,ethnographies, and user testing• For questions on emotions• Surveys, interviews, focus groups,and observations
  52. 52. You can mix tools• You need to map out each step you will take inyour research so that things follow in a logicalorder…• This is your method!
  53. 53. Spend the rest of the time thinking throughyour proposed question• Independent Variable– What is tested, measured or manipulated• Dependent Variable– What is observed – the outcome
  54. 54. References• Curasi CF (2001) A critical exploration of face-to-face interviewing interviewing International Journal of MarketResearch 43(4)pp361-375• Hammersley, M. (1992). What’s wrong with Ethnography:Methodological Exploration, London: Routledge.• May T (1997) Social research: issues, methods, progress Buckingham:OUP• Rugg, D. (1941) Experiments in wording question. II Public OpinionQuarterly, 5:91-92.• Silverman D (1997) Qualitative research: theory, method,practiceLondon: Sage• Silverman D (2004) Doing Qualitative Research: A Practical HandbookLondon: Sage