Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (12)
Similar to TSchehr_Senarios_AIR08Forum
Similar to TSchehr_Senarios_AIR08Forum (20)
Recently uploaded
Recently uploaded (20)
TSchehr_Senarios_AIR08Forum
- 1. Using Scenarios to Assess Student Perspectives on Diversity and Understand Social Desirability Prepared by: Terra Schehr Director of Institutional Research Loyola College in Maryland For: AIR Forum, Seattle May 2008
- 2. Schehr, AIR 2 Overview Measuring perspectives on diversity Institutional context Scenario development Social desirability experiment Key data and outcomes
- 3. Schehr, AIR 3 Approaches to Measuring Perspectives on Diversity Bogardus’s social distance scale (1959) NORC’s general social survey (1972) Social networks Intergroup relations Multiculturalism Prejudice University of Arizona, Rethinking diversity: The millennial student project http://mass.arizona.edu/millennial/aboutstudy.html Dynamic Diversity Paradigm CIRP, YFCY, CSS, NSSE
- 4. Schehr, AIR 4 Commonly Used Diversity Items - Behaviors CIRP (2006) – 1.5 items such as best guess that will “socialize with someone of another racial/ethnic group” NSSE (2007)– 4 items such as frequency of “had serious conversations with students of a different race or ethnicity than your own”
- 5. Schehr, AIR 5 Commonly Used Diversity Items - Opinions CIRP (2006) – 6.5 items such as agreement with the statement that “same sex couples should have the right to legal marital status” NSSE (2007)– 0 items
- 6. Schehr, AIR 6 Examples of Previously Collected Data on Diversity Issues 59% reported “frequently” socializing with someone of a different race/ethnicity before coming to college 60% said there was a “very good chance” they would socialize with someone of a different race/ethnicity while in college 74% said there was a “very good chance” they would socialize with someone of a different socioeconomic status while in college Source: 2006 CIRP (third bullet is from an institutional specific item in 2005)
- 7. Schehr, AIR 7 Where is the Diversity? N = 735 A white B not first-generation college going C parent’s income is $100k+ D Catholic % of total (% in area of total) A B C D 89 % 92 % 63 % 73 % 9 (1 %) 21 (3 %) 1 (0 %) 4 (1 %) 55 (7 %) 1 (0 %) 23 (3 %) 18 (2 %) 15 (2 %) 1 (0 %) (12 %) 87 (19 %) 142 (2 %) 12 (2 %) 18323 (44 %) 5 1 % ~85% went to a “mostly” or “completely white” high school ~87% grew up in a “mostly” or “completely white” neighborhood Source: 2006 CIRP
- 8. Schehr, AIR 8 Institutional Context Medium sized comprehensive institution ~3,500 undergraduates ~2,500 graduate students Undergraduates: ~11% are students of color ~9% are on Pell grants ~1% are gay, lesbian, or bisexual ~66% come from the Bos-Wash corridor
- 9. Schehr, AIR 9 Why a New Approach was Needed Diversity is important part of College’s educational mission “ . . . the College will inspire students to learn, lead and serve in a diverse and changing world.” (Mission) “. . . prepare students to dedicate themselves to diversity that values the richness of human society . . .” (Core Values) “. . . awareness of and sensitivity toward differences among people and cultures.” (Learning Aims) Data being collected lacked consequential or pedagological validity
- 10. Schehr, AIR 10 New Approach to Investigating Perspectives on Diversity CIRP survey with follow-ups on YFCY and CSS (planned) CIRP survey administered on paper during fall orientation 2006 N=859 (91% of the incoming class) 12 diversity scenarios included in the institution specific items
- 11. Schehr, AIR 11 Scenario Development Collaboration with student development professionals - 12 scenarios, 2 versions Scenarios covering issues of: race/ethnicity (6), class (1), gender (1), disability (1), sexual orientation (1), and image (2) No “right” or “wrong” answer
- 12. Schehr, AIR 12 Scenario Example Student A says that white students should try to learn about the challenges that students of color face. Student B disagrees and says if you got into _________ you’re pretty much the same and that students of color and white students don’t really face different kinds of challenges. Which student do you agree with more? a. Completely agree with Student A b. Generally agree with Student A c. Mixed – somewhat agree with both d. Generally agree with Student B e. Completely agree with Student B a. Student A Or b. Student B
- 13. Schehr, AIR 13 Social Desirability Experiment “They’re just telling you what they think you want to hear” Split-half experiment 48% had the two response option format 52% had the five-point scale response option Comparing the data from the two formats allows us to identify topics where we can expect students to respond in a socially desirable manner
- 14. Schehr, AIR 14 The Movement Toward the Middle If you got into Loyola you’re prettymuch the same and that students of color and white students don’t reallyface different kinds of challenges Mixed White students should try to learn about the challenges that students of color face 34% 66% 22% 43% 35% 0% 10% 20% 30% 40% 50% 60% 70% 80% Social Desirability 23% 12%
- 15. Schehr, AIR 15 What Did The Experiment Tell Us? Knowing that students tend to respond in socially desirable ways when asked about certain issues provides a context for understanding students’ expressed opinions on diversity issues SD No SD Race 1 3 Ethnicity 1 1 Class 1 0 Gender 1 0 Disability 1 0 Sexual orientation 0 1 Image 2 0 http://www.loyola.edu/IR/Students/Students “Views on Diversity Class of 2010”
- 16. Schehr, AIR 16 Data Reduction? Factor Analysis? Not conducted for conceptual and practical reasons Conceptual – Scenarios were purposefully designed to have no clear “right” or “wrong” answer Practical – CIRP items do not meet the level of measurement assumptions of factor analysis Cluster Analysis? Not conducted for similar reasons above
- 17. Schehr, AIR 17 Reliability Analysis CIRP Items – Cronbach’s Alpha = .41 Scenarios - Cronbach’s Alpha = .57 Racial discrimination is no longer a major problem in America Whites students have an easier time getting into their first-choice school (Pro-Affirmative Action) Through hard work, everybody can succeed in American society Slavery horrible time & still impacts today Undocumented immigrants should be denied access to public education White students should try to learn about challenges that students of color face Affirmative action in college admissions should be abolished Racial/Ethnic profiling is unfair If people feel more comfortable speaking in their native language they should be able to
- 18. Schehr, AIR 18 Presenting the Data to the Campus Data collected on the standard CIPR items were displayed Inconsistencies in student opinions were discussed Data were disaggregated by key demographics Scenario items – displayed in the aggregate in a section on the social desirability experiment – were discussed in relation to the CIRP items Data from the scenarios were disaggregated by key demographics
- 19. Schehr, AIR 19 Standard CIRP Items – Opinions About Race/Ethnicity 31% 21% 23%14% 45% 31% 34%49% 20% 36% 36% 34% 4% 13% 7% 2%0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Racial discrimination is no longer a major problem in America (worded in the negative) Through hard work, everybody can succeed in American society Undocumented immigrants should be denied access to public education Affirmative action in college admissions should be abolished Agree strongly Agree somewhat Disagree somewhat Disagree strongly
- 20. Schehr, AIR 20 Standard CIRP Items – Opinions About Race/Ethnicity – Disaggregated Racial discrimination is a problem Through hard work everybody can succeed Undocumented immigrants should be denied public education Affirmative action in college admissions should be abolished Gender Women 35% 28% 19% 19% Men 33% 38% * 26% * 31% * Ethnicity Students of color 51% * 25% 14% 7% White 31% 32% 22% * 25% * Family Income <$100,000 40% * 33% 24% 20% $100,000-199,999 30% 31% 20% 25% $200,000+ 30% 32% 21% 25% Religion Catholic 33% 32% 22% 26% * Non-Catholic 37% 30% 20% 16% Political Leaning Conservative 25% 40% * 32% * 31% * Middle of the road 31% 31% 19% 22% Liberal 49% * 23% 12% 14% Percents reflect the proportion of people in each group on the rows who indicated that they “strongly agree.” Because it was worded in the negative, in the case of “racial discrimination is a problem” the percent reflects those who “strongly disagreed” to the statement that it is “no longer a major problem . . .” *Indicates a significant difference from other groups within a category.
- 21. Schehr, AIR 21 Scenario Items – Opinions About Race – Disaggregated White students Students of color Programs such as affirmative action force white students to work even harder to get admitted to schools Because they are presumed to be more intelligent, White students have an easier time getting into their first-choice school White students should try to learn about the challenges that students of color face If you got into Loyola you’re pretty much the same and that students of color and white students don’t really face different kinds of challenges Slavery was horrible but the issues that were present back then do not have a substantial impact on race relations now Slavery was horrible but the issues that were present back then do not have a substantial impact on race relations now Your race has a major impact on how you interact with other people Your race does not impact how you experience the world Completely Agree Generally Agree Mixed Completely Agree Generally Agree
- 22. Schehr, AIR 22 Outcomes Much richer and more nuanced picture of student views on diversity Faculty and student development staff have engaged in discussions of the data Student GLBT group hosted an event called “that’s so gay, the impact of language on the community” Data will be used as one means of assessing impact of the diversity course requirement Data will be used in the comprehensive diversity assessment of the campus
- 23. Schehr, AIR 23 Prepared by: Terra Schehr Director of Institutional Research Loyola College in Maryland www.loyola.edu/ir tschehr@loyola.edu
Editor's Notes
- Bogardus Social Distance Scale – asked to rank a series of social types (race or ethnic groups) on social distance scale from “acceptance to close kinship through marriage” to “exclude from my country” NORC (National Opinion Research Center @ U Chicago) – general social survey started in 1972 conducted nearly annually with N~1,500 Items both about behaviors and opinions Dynamic Diversity Paradigm - seeks to observe perspectives and behavior in regard to diversity. Although the model assumes that the development of knowledge regarding diversity issues is linear, one’s perspective and level of openness to diversity is dynamic and can be influenced by multiple factors CIRP, YFYC Also include both behavior and opinion items
- CIRP - Have (1) and will (.5) “socialize with someone of another racial/ethnic group” NSSE – “included diverse perspectives (different races, religions, genders, political beliefs, etc.) in class discussions and writing assignments” “had serious conversations with students of a different race or ethnicity than your own” “had serious conversations with students who are very different from you in terms of their religious beliefs, political opinions, or personal values” “tried to better understand someone else’s views by imagining how an issue looks from his or her perspective”
- CIRP- Self-rating of ability “understanding others” “it is important to have laws prohibiting homosexual relationships” “racial discrimination is no longer a major problem in America” “same-sex couples should have the right to legal marital status” “affirmative action in college admissions should be abolished” “undocumented immigrants should be denied access to public education”
- That is to say it was neither useful in the general sense (consequential) nor did it help improve teaching (pedagological)
- Identified who had what question set by using different colored paper for the supplemental questions with the first supplemental question being what color is the paper.
- The dark bars in the background are the responses of students who were given only two options. The white bars in the foreground show the responses of students who were given the 5-point scale. For purposes of analysis the “generally agree” and “completely agree” responses have been aggregated. Each chart is oriented so that the more socially desirable position is on the right. When given the option of simply agreeing with one actor or the other in each scenario many students selected the actor who was stating the socially desirable or “politically correct” position. When given a range of response options, however, the proportion of students endorsing these politically correct opinions is lower as many students move into the “mixed—somewhat agree with both” category. Items where the difference between the dark bar and white bar on the socially desirable option is larger than the difference between the responses on the other end of the spectrum are areas where students are more likely to respond in socially desirable ways when asked about these issues at Loyola.
- Lack of social desirability bias on particular issues does not indicate a sophisticated understanding of diversity issues.
- Still, did want to take a look at how the items hang together Internal consistency reliability Throw CIPR and Scenarios together and get .42
- Key demographics – Gender Race/Ethnicity (students of color vs. white) Family Income Religion Political leaning
- Inconsistencies: Interestingly, 83% indicated that racial discrimination is a problem in America, but large portions of students also agreed with the statements: “through hard work everybody can succeed” (76%), “undocumented immigrants should be denied access to public education” (52%), and that “affirmative action in college admissions should be abolished” (57%). These findings were also evident in student responses to the scenarios in which 50% of students indicated that “. . .programs such as affirmative action force white students to work even harder. . .” while 50% also indicated that the history of slavery “. . .still impacts racial issues today.”
- There are two items that are particularly interesting in Figure 6. - in all but one scenario white students and students of color were similar in the direction of their feelings - students of color did not endorse the position that “. . .race has a major impact on how [they] interact with others.”