Media - quantitative and qualitative research 2012

34,644 views

Published on

Validating quantitative and qualitative research

Published in: Technology, Business

Media - quantitative and qualitative research 2012

  1. 1. RESEARCH RESEARCH METHODS + THEORY QUANTITATIVE QUALITATIVE
  2. 2. RESEARCHPART1RESEARCH PARADIGMS: The Basics
  3. 3. RESEARCH METHODS AND THEORY The Research There are many ways to conduct a research. You often have to adjust your Process objectives to the field you’re working in and to the environments and  Introduction people you are working with. The field you’re working in could be “website  usability” and the specific environments and persons (informants) you are working with could be an office environment and its staff.  During this lesson you will work theoretically and practically with two main types of research processes:  The quantitative research process + the qualitative research process Literature:  This lesson is based on Klaus Bruhn Jensen’s (et al) (2005): A Handboook of Media and Communication Research. Routledge. “The Quantitative Research Process” by Barrie Gunter (chapter 13, p. 209–234).  “The Qualitative Research Process” by Klaus Bruhn Jensen (chapter 14, p. 235–253). 
  4. 4. RESEARCH METHODS AND THEORY Secondary Research Desk Research Desk research relies on existing data and information published on the net,  Secondary data in printed magazines or any other valid source (!) TIP: Reading economy articles, trend news, articles based on web surveys and  reports from cultural and national organisations gives you a good picture of your target group based on desk research ...
  5. 5. RESEARCH METHODS AND THEORY Primary Basic research is also referred to as field research. Research Field Research Field research refers to the collection of new data through primary research.  Primary data That means direct contact with people through interviews, focus groups and  surveys. Bigger and complex surveys are often done by bying this expertise from  companies specialized in conducting effective, reliable surveys.
  6. 6. RESEARCH METHODS AND THEORY Research Often there is a mix between the two paradigms: paradigms Introduction to the Basic  PR I M A RY DATA S ECO N DA RY DATA Concepts Qualitative • Observations • Documents Data • Interviews (open questions) • Notes (from secondary source) • Movie recording (actively) • Letters • Think aloud test • Sound and movie recordings (other’s material) • Artifacts (to be interpreted) Quantitative • Closed questions • Articles and pictures, etc. Data • Surveys (to be interpreted) • Clearly defined objectives in observations. • Statistics • Page traffic • Registrations Artifacts like for example letters and movies etc. can be used to support  primary data for some purposes. Statistics can also support qualitative arguments i user research.
  7. 7. RESEARCH METHODS AND THEORY Research SCOPE QUANTITATIVE METHOD QUALITATIVE METHOD Paradigms Examples of investigations Surveys by mail, online or handout Casestudy with interview Introduction to the Basic  General approach Precision: an exact mapping of the quantitative variation Empathy: the best understanding of the qualitative variation Concepts General perspective Width: seek information about as many Depth: seek as much information as  quantifiable units as possible possible on a few qualitative units The purpose of the research Average: seek common,  Specific: find out what can be said representative features to be unique and special. Criteria of methodology Representative: Chosen informants  Relevanse: Informants are relevant in  must represent the target group. relation to problems and meaning Design of method Systematic: Survey with closed questions Fleksibilitet: Interview with no clear  answers, dialogue, interaction. Level of structuration High: The possibilities of answers are laid Low: Open answers and a variety of  out for response. Low flexibility. interpretations. High flexibility. Key concepts of methodology Explanation: How. How many.  Understanding: Why. Who thinks what Who does what. Causal explanations. about ... Meanings and attitudes Example of web research How many are using the site? What motivates the target group to visit  and possible methods/tools What are the user patterns?:  the website and buy products? What are Google analytics combined with a survey their preferences for colour, menus etc?:  designed for representative users. Think aloud tests and focus groups.
  8. 8. RESEARCH METHODS AND THEORY QUANTITATIVE RESEARCH Research The Basic Concepts in Quantitative Research Paradigms Introduction The Quantitative approach to research is based on the scientific tradition  to the Basic of studying aspects of human reality with empirical proof.  Concepts Research in anthropology, economics, geography, linguistics, history, polital science etc. can often be done by initially measuring hard facts. Quantitative research often aims at closed questions—questions your respondents can answer yes or no to; questions that are very narrow defined.
  9. 9. RESEARCH METHODS AND THEORY QUANTITATIVE RESEARCH Quantitative Examples Research Introduction Numbers: You can measure how many women there are in a room to the Basic and you can measure how many men there are.  Concepts Specific actions: You can measure people’s actions (but not why they do it!)  Opinions: You can measure people’s opinions by asking closed questions:  “Do you think our prime minister will win the next election?”  This question would be followed up by additional (anonymous) information  from the informant: gender, age, city etc. You can use these data as valuable information in your research objective. Assignment See usibilia.com … What is this? See surveymonkey.com … What is this?
  10. 10. RESEARCH METHODS AND THEORY QUALITATIVE RESEARCH Research The Basic Concepts in Qualitative Research Paradigms Introduction The qualitative approach to conducting research has for many years been to the Basic  the ‘soft’ area of science.  Concepts The qualitative aspects deals also with observable facts, but not always the  same ‘hard’, measurable emperical facts that is the focus for quantitative research.  Qualitative reseach investigates the concept of meaning, its embedding in and orientation of social actions. It is the connection between meaning and  action—for example as performed inside media contexts—that tells us how we are, how we think and act in a realistic context.
  11. 11. RESEARCH METHODS AND THEORY QUALITATIVE RESEARCH Qualitative Examples: Designing Qualitative Research: Interviews Research Introduction To design an emperical study is to identify and delimit a portion of reality  to the Basic (Jensen 2005: 237). Ask into why instead of what. Concepts Respondent interviews: The informant is percieved as a representative of   a social and cultural category. This can be a key to the decoding of the user‐ pattern in relation to user behaviour (think aloud test for example). Group interviews: In order to explore what goes on in a more or less naturalistic social setting, a group can form the basis for discussions, that explore aspects of a product credibity, of lifestyles, advertisements etc. Focus groups: Gathering a specific type of users, this kind of interviewing is  often based on the strategy of getting to know attitudes toward a product or  political and ethical issues. Again, it’s why instead of what.
  12. 12. RESEARCH METHODS AND THEORY Qualitative Designing for Research Research Introduction to the Basic Assignment Concepts See http://en.wikipedia.org/wiki/Online_focus_group … What is this? What are the pros and cons in online and offline qualitative surveys? Is a think aloud test a qualitative or a quantitative survey method?  Or can it be both? Why?
  13. 13. RESEARCHPART2RESEARCH PARADIGMS: Variables and Concepts
  14. 14. RESEARCH METHODS AND THEORY Research Variables Variables and Concepts The basic part of the research process is the notion of the variable.   Variables are in this context the emperical representation of a concept. Man, woman, number of …, high/low, happy, not happy etc. Variables provide operational measures that can be quantified and manipulated by researchers. The concepts of gender, age, economics, and personal behaviour are variables that you can measure. These variables must be described in  further detail as concepts and/or constructs (see the next slides)
  15. 15. RESEARCH METHODS AND THEORY Research Variables and Concepts Variables and Concepts A concept represents an abstract idea that embodies the nature of  observable phenomenon, or an interpretation of why such phenomea occur. For example, individuals may be differentiated in terms of their use of  media: What you define as a “active user” could be distinguished from what you define as a “passive user” (in questions based on how often a specific media or an application is used). Media usage can be linked to explain different behavioual patterns:  Media usage becomes an explanatory concept (Jensen 2005: 210).
  16. 16. RESEARCH METHODS AND THEORY Research Single‐Concept Example (Media Usage as Behavioural Patterns) Variables and Concepts The observable phenomenon:  More and more people over the age of 65 use FaceBook (FB). source/desk research:  http://www.insidefacebook.com/2009/12/11/facebook‐a‐top‐destination‐for‐users‐over‐65/ The concept I want to use: What is it to be “user”—it’s a bit too vague a concept! Just because you have a  FB account you’re not always active. I find it interesting to investigate the  single concept of the “active user” in the age group 65+.  First, I define the concept of a “user” by describing precisely what this implies in relation to skills in the use of FB. Second, I suggest the number of visits on FB per day in order for one to be a “active user” of FB.  Objective: How many “active users” in the age group 65+ are there on FB in  Denmark alone. I base this on my concept, my problem statement and on a  sampling of informants (say, 1000 FB‐users in the age group 65+)
  17. 17. RESEARCH METHODS AND THEORY Research Constructs (Combined Concept Example) Variables and Concepts A construct comprises a combination of concepts. (Constructs) This term can be used as a way of defining characteristics or actions  of individuals that are associated with their personality type.  For example, one personality type is defined as  a range of sensation‐ seeking individuals: High‐sensation seekers generally need higher levels of  environmental stimulation than low‐sensation seekers.  High‐sensation seekers may be described by a series of other concepts as: sociability, tolerance for strong stimulation, risk‐taking etc.  Constructs have a dimensional quality, so that individuals may be classified (in this example) as high or low on the personality dimension  of sensation‐seeking (Jensen 2005: 210).
  18. 18. RESEARCH METHODS AND THEORY Research Variables, Concepts and Constructs: Example Variables and Concepts In other words: The variables get more substantial when they are applied (Constructs) to a) a concept and b) a construct. Example: Gender is interesting because of the construct of  the differences  between certain actions of men and women, say, buying actions. It’s not just a  single concept you can establish without any goals for your research.
  19. 19. RESEARCH METHODS AND THEORY Research Independent and dependent variables Variables and Concepts Variables can be further defined in terms of their relationship with each other.   Before the research: The independent variable (concept/construct) can be manipulated by the researcher—it is meant to produce some measurable response or outcome. You design your variables (concept/construct/attributes) to fit your objective, for  example light user, active user etc. After the research: The dependent variable is the measure of the response or outcome. It is the obtained data which is treated as information.
  20. 20. RESEARCH METHODS AND THEORY Research Reliability and validity Variables and Concepts Another important aim of a research is to determine its reliability. To meet the demands for reliability and validity, you can …  repeate evidence/tests to show similar results over time  or in another context.  differentiate between the respondent (age, gender, education, etc.)  explain accurately how and why you conduct your research.   use good internal validity: the design of the research process must be free from theoretical and methodological errors. Use validated theory. The reliability concerns the dependability and consistency of the relationship in one or between more variables. The validity indicates whether a measure properly captures the meaning of the concept or construct it represents.  (Jensen 2005: 212).
  21. 21. RESEARCHPART3RESEARCH PARADIGMS: More on Methods
  22. 22. RESEARCH METHODS AND THEORY Research The Survey: Field Research Survey Methods A descriptive survey simply attempts to document current conditions.  Public opinion polls, for example, can rovide information about people’s present attitudes on a specified topic. The concept of “fear of terrorism” can be constructed through a variety of constructs from “no fear” to “very afraid,”  and the survey can gain answers from different age groups. The outcome is  descriptive and can of course be used as an initial research narrowing down a  target group. Analytical surveys also collect descriptive data, but attempt to go on to  examine relationships among variables in order to test research hypotheses.  A survey may assess the impact of an advertising campaign on public  awareness of a brand and changes in the market’s share of a product. Such explantory surveys can also research social effects in‐ and because of media. 
  23. 23. RESEARCH METHODS AND THEORY Research The Survey: Field Research, Observing and Coding Survey Methods Thick description: The thick description means that when you observe a  situation, you use means of gathering data that accounts for everyhing.  You could use movie‐recording devices and then later describe artefacts as  well as actions. This can lead to a very detailed description of an  environment and the interpretation of the actions. Coding: The coding means fixating and capturing certain qualities of the  units, i.e. person/persons, texts, events (or other unit), you are observing. For example can the thematic coding (concept variable) function as an  important objective to make way for interpretations and for applying abstract theory of culture and perception.
  24. 24. RESEARCH METHODS AND THEORY Research The Survey: Field Research Survey Methods Telephone interviews: You can accomplish the data very quickly and directly. It is cheap to conduct The respondents can be reached globally. Face‐to‐face interviews: Short interviews as well as longer interviews can be conducted,  i.e. in a shopping mall or in a home. Video‐ and audio techniques can be used. Better personal credibility can be achieved by personal interview. Web‐/mail‐/online‐ or paper questionnaire: Simple forms can be filled out by respondents. By using web or mail the  statistical answers will be easy to monitor quickly afterwards.
  25. 25. RESEARCH METHODS AND THEORY Quantitative Describing Data  Research The Survey Links: See more on percentage calculation here … http://www.math.com/everyone/calculators/calc_source/percent.htm http://www.easycalculation.com/statistics/statistics.php http://www.euromonitor.com/  Don’t panic!   If you use an online survey tool all the calculations are done for you.
  26. 26. RESEARCH METHODS AND THEORY Research 1. The hypothesis Propose that if an independent variable is assumed to have a certain strength, it may be expected to exert a measurable impact on a designated dependent variable. Then it can be An Overview tested, if this hypothesis can be proved or disproved. (Jensen 2005: 212). Qualitative and/or quantitative research design 2. The Variable … is gender, age, nationality etc. … is the empirical representation of a phenomenon (like trends) …Before the research 3. Independent Variables Concept and a… Construct (combination of concepts) … can be manipulated by the  A concept represents A combination of concepts used to define the  researcher.  an abstract idea that characteristics of the individual users as  … means the way the research  embodies the nature of  grouped in concepts (in the example). process is constructed to measure  observable phenomena,  Heavy users may be described by such a response. (Jensen, 2005) or an interpretation of  concepts as sociability, tolerance for strong why such phenomea stimulation, risk‐taking etc. (Jensen, 2005). 4. Dependent Variables occur. (Jensen, 2005)After the research … is the measure of the  Example: Individuals outcome of the research  are defined in terms of  based on its construct and its their use of a media.  dependent variables, i.e. it is  You define the notions the information created by the  of a “light user” and a  variables and the hypothesis. “heavy user” based on  (Jensen, 2005) media types, levels of  defined usage etc. (Jensen, 2005)
  27. 27. RESEARCH METHODS AND THEORY Research Example The Survey Let’s visit http://www.youtube.com/watch?v=MFbmHDs7TmI … … and have a talk about this type of survey and its components. Variables? Concepts? Independent variables/dependent variables? Qualitative, quantitative, mix, goal, interpretation? 
  28. 28. RESEARCH METHODS AND THEORY Assignment USABILITY TESTS—design a survey of a user experience TESTING 1. Make a test of a website or a mobile application: Heuristics. 2. What is your hypothesis as a media designer, based on your initial test,  on usability in relation to the concepts: navigation, readability and design? 3. Work with these valid concepts in relation to: the attitudes toward the visual design (qualitative method)  the ability to read the text (quantitative method). Interview two, three or more people (Who? / why?) 4. Record, write down, observe etc. (how? / why?) Present according to the template on the next slide →
  29. 29. RESEARCH METHODS AND THEORY Assignment USABILITY TESTS—design a survey of a user experience R E P O R T I N G   (validating your results) Source: Munk & Mørk (2002): Brugervenlighed på internettet.. Samfundslitteratur. 1. Summary of the main results with conclusion. 2. Introduction: hypothesis, problems and research questions, concepts in use. 3. Describtion of method/methods: A mix of quantative and qualitative approaches? How? Focus? Independent and dependent variables? Your unique research design? 4. Observations and results, focus points.  Pie chart templates may be used. 5. Conclusion (short, recap on 1) 6. Recommendations for improvements of media.

×