More Related Content
Measurement
- 1. Conceptual and Operational Linkage Dr. Serena
- 2. Operationalization Choices •Conceptualization is the refinement and specification of abstract concepts • Then we operationalize (developing rules for inclusion and exclusion) based on previous definitions or based on concepts in conceptual definition
- 3. Measurement • Theassignment of numerals to objects or events according to rules
- 4. Empiricism • Whatto observe • Whom to observe • How to observe • When to observe • How to analyze the data
- 5. Indicators and Dimensions • Indicator (variable) – An observation that we choose to consider as a reflection of a variable we wish to study. • Dimension – A specifiable aspect of a concept.
- 6. Stevens • Distinguished4 types of “measurement scales” – Nominal – Ordinal – Interval – Ratio • Theoretical structures are represented by data structures – Data helps to focus theory
- 7. Levels of Measurement– Nominal • Variables who attributes have only the characteristics of exhaustiveness and mutually exclusiveness – Examples: gender, religious affiliation, college major, hair color, birthplace, nationality
- 8. Levels of Measurement– Ordinal • Variables with attributes we can logically rank order – Examples: socioeconomic status, level of conflict, prejudice, conservativeness, ranking of topics,
- 9. Levels of Measurement– Interval • Variables for which the actual distance between attributes has meaning. – Temperature, IQ score
- 10. Levels of Measurement– Ratio • Variables whose attributes meet the requirements of a interval measure, and has a true zero point. –Age, length of time, number of organizations, number of groups
- 12. Judge the relativesuccess (or failure) in measuring various concepts • Reliability (consistency of measurement) • Validity (confidence in measures)
- 13. Reliability • Consistencyof measurement – Reproducibility over time – Consistency between different coders – Consistency among indicators
- 14. Measurement Validity •Are we really measuring the concept we defined? – Is it a valid way to measure the concept? • Many approaches to validation
- 15. Key to Reliabilityand Validity • Concept explication • Conceptual definition • Operational definition
- 16. Four Aspects ofReliability 1. Stability 2. Reproducibility 3. Homogeneity 4. Accuracy
- 17. 1. Stability •Consistency across time – Repeating a measure at a later time to measure consistency – Compare Time 1 and Time 2
- 18. 2. Reproducibility •Consistency between observers • Equivalent application of measuring device – Do observers reach the same conclusion?
- 19. 3. Homogeneity •Consistency between different measures of the same concept
- 20. Alpha – Self-efficacy • Even when things are tough, I can perform quite well. • I will be able to successfully overcome many challenges. • I don’t achieve most goals that I have set for myself. • I like ice cream.
- 21. 4. Accuracy •Mistakes in measurement • Observers must have sufficient: – Training – Motivation – Concentration
- 22. Increasing Reliability •General – Training coders/interviewers/lab personnel – More careful concept explication • Survey – Increase number of items on scale – Weed out bad items • Content Analysis – Improve definitions of content categories – Remove bad coders
- 23. 1. Face Validity • Subjective evaluation from experts • Compare items to conceptual definition
- 24. 2. Content Validity (Face Validity) • Subjective agreement about what is not there • Start with conceptual definition of each dimension – It is operated at an operational definition – Over- or under-represented • Example: Civic participation – Did you vote in the last election? – Do you belong to civic groups? – Have you ever attended a city council meeting?
- 25. 3. Predictive validity(criterion) • The extent to which one measure predicts the values of another measure
- 26. 4. Convergent Validity • Measuring a concept with different methods – If different methods yield the same results, convergent validity is supported
Editor's Notes
- #2 Two levels of investigations: conceptual and empirical. Representational theory is about providing concrete structure to our theory. What are the features that we can observe to translate concepts into reality. When you represents something, there are intentions. We assign numbers to properties. For example, numbers can represent the length of physical objects. Numbers give the object greater meaning. Since the early 1900s, social scientists have been trying to measure reality. We measure so we can manipulate the universe. We translate our observations into numerical form, but our measures have to measure what they intend to measure
- #3 Identifying the concepts in the conceptual definitions is key to conceptualizing Observations are checked against data Operationalization is the development of specific research procedures that will result in empirical observations representing those concepts in the real world.
- #4 Remember in any conceptual definition, we want to highlight the concepts and think about what they mean. For concept definition exercise, I want you to look at concepts. Stevens in the 1950s. Endorsement within social sciences. He believe any psychological state. Objects and events that are measured, not magnitudes. Careful, deliberate observations of the real world we can describing objects and events. They are most often described in terms of the attributes…. Objects(room) and events (rooms changing color) … attributes of objects (length of a room) and attributes of events (time it takes to paint a room). His definition focuses on objects, however it is really attributes of objects that are being measured. Numerals, not numbers. Numerals - a figure, symbol, or group could be denote a number… meaning that numerals could be assigned to anything. Assignment (representation) and rules…. Methods are the set of rules that you learn. As we learned last class, we should be concerned with how we make those decisions. Numerical representation. Map measurement.
- #5 Making planned observations Boredom in the Classroom Students- elementary Should we interview them or observe them Friday afteroon may be a difficult time Percentage bored? Percentage participating. Correlate boredom with other variables
- #7 Representational theory tie to “scales of measurement”… to encourage consistent rules when observing objects or events. Interval measurement is most commonly applied. This is going to affect how operationalize a variable. Attributes are defined by levels of variability You must think about level of measurement when deciding… because it will influence that statistics you choose to analyze your data. Social attitudes are at ordinal
- #8 Categorical variables: male/female…. Dichomous variables
- #10 Most people view scores on intelligence tests are ordinal because scores have not been shown to correspond to equal differences. Intervals have equal distances but do not have an absolute zero
- #13 Validity also extends to: Precision in the design of the study – ability to isolate causal agents while controlling other factors (Internal Validity) Ability to generalized from the unique and idiosyncratic settings, procedures and participants to other populations and conditions (External Validity) \
- #14 Statistical coefficients that tell use how consistently we measured something Reliability – That quality of measurement method that suggests that the same data would have been collected each time in repeated observations of the same phenomenon. Reliability is not the same as accuracy.
- #15 Judgmental as well as empirical aspects
- #16 Meaning analysis Defining what a concept means Spelling out how we are going to measure concept
- #18 Computer self-efficacy: I have the ability to describe how a computer works and I have the ability to install software
- #19 If we don’t get the same results, what are we measuring? Intercoder reliability. Photograph – visual presentation of self--- visual content cu of person smilling. Lack of reliability can compromise validity
- #20 Different items used to tap a given concept show similar results – ex. open-ended and closed-ended questions Internal consistency -
- #21 The extent to which items are highly correlated with one another – research self-efficacy Number of dimensions influence alpha levels (Cortina, 1993) Cronbach’s Alpha Reliability Coefficients 0.90 Excellent higher suggests item redundancy 0.80 Very Good 0.70 Adequate (Above recommended) .60 (exploratory research – Hair, et al., 2010) 0.50 Poor
- #22 Increased by clear, defined procedures Reduce complications that lead to errors Contnet
- #23 Specification of procedures/rules…Reduce subjectivity (room for interpretation) Specification at a manifest leve;
- #24 Make judgments about superficial appearance. Subjective judgment of experts about: “what’s there” Do the measures make sense? That quality of an indicator that makes it seem a reasonable measure of some variable.
- #25 Make judgments about appropriateness of content Content Validity – The degree to which a measure coves the range of meanings included within a concept. Content validity refers to whether the items on your test actually test what you're looking at --- do your measures actually measure what you say that they are meausirng – factor analysis Construct Validity – The degree to which a measure relates to other variables as expected within a system of theoretical relationships If current indicators are insufficient: develop and add more indicators What about “protest participation” or “online organizing”? Operational level technique of conceptualization Factor analysis
- #26 So lets say for example, I am looking for a doctoral student for my next research project. I ask for samples of their writing. Writing samples show some good and poor writing. I am looking at their writing as a predictor as research success. Communication apprehensiveness – student participation (negative relationship)
- #27 Correlation between different methods measuring the same trait