3. Overview of Presentation
• Background of this Project
• Current Study
Who? What? When? Where? & How?
• Analysis
Outcome: Civic Mindedness (CMG Short Form)
Descriptives & Comparing Means
Regression analysis
• Now What
Closing the Assessment Loop
4. The Background
• Pilot Study
• What we do know
Civic Mindedness
Value of Service to Organization
• What we are wondering about
Institutional, Student & Other Characteristics
Design
• Intentional Learning Outcomes
Civic Mindedness
6. Outcome: CMG Domains
• Knowledge:
Volunteer Opportunities: understanding of ways to contribute to society, particularly
through voluntary service, and including knowledge of nonprofit organizations.
Academic Knowledge and Technical Skills: understanding of how knowledge and skills in
at least one discipline are relevant to addressing issues in society.
Contemporary Social Issues: understanding of current events and the complexity of issues
in modern society locally, nationally, or globally.
• Skills:
Communication and Listening: ability to communicate (written and oral) with others, as
well as listen to divergent points of view.
Diversity: understanding the importance of, and the ability to work with, others from
diverse backgrounds; also appreciation of and sensitivity to diversity in a pluralistic
society.
Consensus-Building: ability to work with others, including those with diverse opinions, and
work across differences to come to an agreement or solve a problem.
• Dispositions:
Valuing Community Engagement: understanding the importance of serving others,
and being actively involved in communities to address social issues.
Self-Efficacy: having a desire to take personal action, with a realistic view that the action
will produce the desired results.
Social Trustee of Knowledge: feeling a sense of responsibility and commitment to use the
knowledge gained in higher education to serve others.
• Behavioral Intentions: A stated intention to be personally involved in community
service in the future.
7. Outcome: CMG Items on Survey (SF)
(α=.905)
• As a result of participating in today's service activities
(i.e., orientation, reflection, service, keynote address)....
(N=870)
I have a better understanding of how organizations are
working to improve societal issues (M=4.67, SD=1.06)
I am more confident that I can contribute to improving life in
my community (M=4.97, SD=0.98)
I have gained more knowledge to plan or help implement an
initiative that improves the community (M=4.61, SD=1.18)
I am more knowledgeable about opportunities to get
involved in the community (M=4.97, SD=1.01)
I have a better sense of who I am, which now includes a
sincere desire to be of service to others (M=4.56, SD=1.28)
I am more aware of a number of community issues that
need to be addressed (M=4.81, SD=1.11)
8. Current Study
• What?
Two different online surveys: one for participants
one for program coordinators.
• Who?
37 different institutions: convenience sampling
• When?
Survey Administration: Directly after 2015 Martin
Luther King Jr., Day of Service (MLK DOS)
• How?
DOS volunteers received survey from their
campus; program coordinators received survey
from researchers.
9. Participating Institutions
• Appalachian State University
• Baldwin Wallace University
• Bowling Green State University
• College of William and Mary
• Concordia University
• Drexel University
• Eastern Illinois University
• George Fox University
• George Washington University
• Grand Valley State University
• IUPUI
• Kennesaw State University
• La Salle University
• Lawrence University
• Lewis & Clark College
• Lipscomb University
• Marquette University
• Mt Hood Community College
• North Dakota State University
• Old Dominion University
• Penn State - New Kensington
• Portland Community College
• Portland State University
• Southern Methodist University
• SUNY, Geneseo
• SUNY, The College at Brockport
• University of Portland
• University of Central Florida
• University of Central Oklahoma
• University of Chicago
• University of Nevada, Reno
• University of Northern Iowa
• University of Oregon
• University of Texas-Pan American
• Warner Pacific College
• William Paterson University
• Williamette University
• Winthrop University
Response
to Survey
Participation
in DOS
Response
Rate
Mean
CMG
SD
Total 940 5,795 16% 4.76 0.90
11. Select Institutional Characteristics:
Compare Means
CE Classification N CMG Mean SD
Yes 399 4.84 0.87
No 541 4.71 0.92
There is a statistically significant difference in mean score based on CE classification
(p< 0.05).
NASPA LEAD
Institution
N CMG Mean SD
Yes 444 4.91 0.90
No 496 4.63 0.88
There is a statistically significant difference in mean score based on NASPA LEAD
institution characteristic (p< 0.00).
CC Member N CMG Mean SD
Yes 691 4.78 0.87
No 249 4.70 0.97
There is NOT a statistically significant difference in mean score based on Campus
Compact membership, but this could be skewed by the distribution of reporting
participants from CC institutions. It is not a very good comparative group.
12. Institutional Characteristics:
Pearson Correlations
Are these institutional characteristics correlated
with civic mindedness?
CMG
Carnegie Elective
CE Classification NASPA Lead
Campus
Compact
CMG 1.00
Carnegie Elective CE
Classification
-0.07** 1.00
NASPA Lead -0.15*** 0.15*** 1.00
Campus Compact -0.04 0.23*** -0.03 1.00
* p< 0.05
** p< 0.01
*** p< 0.001
13. Institutional Characteristics:
ANOVA
Is there a linear relationship between CMG and
these selected institutional characteristics?
Model
Sum of
Square
Degrees of
Freedom Mean Square F
Regression 20.31 3 6.77 8.58***
Residual 738.50 936 0.79
Total 758.78 939
*** p< 0.001
The estimation of the variance explained (R2) was 0.02, indicated that the relationship
was very small.
14. Institutional Characteristics:
Regression Model
Are these institutional characteristics significantly
correlated with civic mindedness?
Model
Unstandardized
Coefficients
Standardized
coefficients
Lower
CI
Upper
CI
B Std.
Error
Beta
(Constant) 5.37*** 0.14 5.09 5.65
Carnegie Elective CE Class. -0.76 0.61 -0.42 --0.20 0.44
NASPA Lead -0.27*** 0.06 -0.15 -0.38 -0.15
Campus Compact -0.07 0.07 -0.03 -0.20 0.07
***p< 0.001
15. DISCUSSION
Select Institutional Characteristics
& Civic Mindedness
REMEMBER: CE elective class & CMG, Campus
Compact & CE elective class significantly
correlated, but not so in the regression model.
These designations as they relate to
a culture for civic mindedness?
What do these institutional
characteristics mean for our
students CMG (& success)?
16. VALUE OF SERVICE TO THE
ORGANIZATION &
SELECTED INSTITUTIONAL
CHARACTERISTICS
Second round of analysis
17. Rate the perceived value of your service
to the organization: ANOVA
Value N CMG Mean SD
Not valuable 21 3.32 1.43
Somewhat
Valuable
402 4.38 0.85
Very Valuable 518 5.11 0.73
The ANOVA of CMG scores on the “Value” scores produced a
statistically significant F ratio, indicating that here was a linear
relationship between CMG and the population (F= 251.53, df= 940;
p<0.000).
The estimate of variance explained (R2) was 0.21, indicating there
was a strong relationship between the outcome and the
independent variable.
18. Value & Institutional Characteristics:
Pearson Correlations
Are these independent variables correlated with
civic mindedness?
CMG Value
Carnegie
Elective CE
Classification
NASPA
Lead
Campus
Compact
CMG 1.00
Value 0.46*** 1.00
Carnegie Elective
CE Classification
0.07** -0.02 1.00
NASPA Lead 0.15*** -0.13*** 0.15*** 1.00
Campus Compact 0.04 0.03 0.23*** 0.03 1.00
* p< 0.05
** p< 0.01
*** p< 0.001
19. Value & Institutional Characteristics:
ANOVA
Is there a linear relationship between CMG and
these independent variables?
Model
Sum of
Square
Degrees of
Freedom Mean Square F
Regression 170.35 4 42.59 67.67***
Residual 588.43 935 0.63
Total 758.78 939
*** p< 0.001
The estimation of the variance explained (R2) was 0.22, indicating a strong
relationship.
20. Value & Institutional Characteristics:
Regression Model
Are these independent variables significantly
correlated with civic mindedness?
Model
Unstandardized
Coefficients
Standardized
coefficients
Lower
CI
Upper
CIB Std. Error Beta
(Constant) 3.35*** 0.18 2.99 3.71
Value 0.74*** 0.05 0.45 0.65 0.84
Carnegie Elective CE Class. -0.07 0.06 -0.04 -0.17 0.04
NASPA Lead -0.16** 0.05 -0.09 -0.27 -0.06
Campus Compact -0.09 0.06 -0.05 -0.22 0.02
** p< 0.01
***p< 0.001
21. Discussion
Value of service to organization &
Select institutional characteristics,
related to civic mindedness
REMINDER: Value itself is significant at a
high power (R2= .21). Which is what the
literature says… and even when controlling
for select institutional characteristics it
remains significant.
How can we ensure that the value
of the experience is “high”? What
else can we ask to further
understand this as it relates to civic
mindedness?
23. DOS Characteristics: Compare Means
Community
Partners as
co-educators N % CMG Mean
Standard
Deviation
Strongly Agree &
Agree
675 87 4.73 0.90
Disagree & Strongly
Disagree
101 13 4.94
0.81
There is a statistically significant difference in CMG mean score, p<0.05.
LO Created N % CMG Mean
Standard
Deviation
Yes 299 36 4.91 0.84
No 532 64 4.69 0.92
There is a statistically significant difference in CMG mean score, p<0.001.
Reflection N % CMG Mean
Standard
Deviation
Yes 726 89 4.77 0.87
No 86 11 4.77 1.12
There is no statistically significant difference in CMG mean score.
24. DOS Characteristics: Pearson
Correlations
Are these DOS characteristics correlated with
civic mindedness?
CMG
Carnegie Elective
CE Classification NASPA Lead
Campus
Compact
CMG 1.00
CP as co-educators 0.08** 1.00
Reflection 0.04 0.10** 1.00
LO Created 0.14*** 0.02 0.21*** 1.00
** p< 0.01
*** p< 0.001
25. DOS Characteristics: ANOVA
Is there a linear relationship between CMG and
these DOS characteristics?
Model
Sum of
Square
Degrees of
Freedom Mean Square F
Regression 15.41 3 5.14 6.55***
Residual 605.21 772 0.78
Total 620.62 775
*** p< 0.001
The estimation of the variance explained (R2) was 0.02, indicating that the
relationship was very small.
26. DOS Characteristics: Regression
Model
Are these DOS characteristics significantly
correlated with civic mindedness?
Model
Unstandardized
Coefficients
Standardized
coefficients
Lower
CI
Upper
CI
B Std.
Error
Beta
(Constant) 4.83*** 0.16 4.52 5.14
CP as co-educators 0.22* 0.09 0.08 0.40 0.03
Reflection 0.02 0.13 0.01 0.24 0.28
LO Created 0.25*** 0.07 0.13 0.12 0.38
* p< 0.05
*** p< 0.001
27. Discussion
Day of Service (DOS) characteristics:
reflection ; learning outcomes ;
community partners as co-educators
related to civic mindedness
REMINDER: Learning outcomes & CP as
co-ed are always significant; reflection
never is statistically significant.
If reflection is a key theoretical and
practical piece of learning when
participating in community-based
experiences, why isn’t it significant?
Importance of Learning Outcomes &
perception of CP as co-educators.
29. DOS & Institutional Characteristics:
Pearson Correlations
Are these independent variables correlated with
civic mindedness?
CMG
LO
Created
CP as
co-ed Reflect
Elective
CE
Class
NASPA
Lead
Campus
Compact
CMG
1.00
LO Created
0.14*** 1.00
CP as co-
educators 0.08* 0.02 1.00
Reflection
0.04 .021*** 0.10** 1.00
Elective CE
Class 0.09** 0.15*** 0.46*** 0.11** 1.00
NASPA Lead
0.20*** 0.39*** 0.12*** 0.29*** 0.01 1.00
Campus
Compact 0.08* 0.17*** 0.20*** 0.38*** 0.13*** 0.15*** 1.00
* p< 0.05
** p< 0.01
*** p< 0.001
30. DOS & Institutional Characteristics:
ANOVA
Is there a linear relationship between CMG and
these independent variables?
Model
Sum of
Square
Degrees of
Freedom Mean Square F
Regression 42.64 6 7.11 9.46***
Residual 577.98 769 0.75
Total 620.62 775
*** p< 0.001
The estimation of the variance explained (R2) was 0.10, indicating a small
relationship.
31. DOS Characteristics & Select
Institutional Characteristics :
Regression Model
Are these DOS & institutional characteristics
significantly correlated with civic mindedness?
Model
Unstandardized
Coefficients
Standardized
coefficients
Lower
CI
Upper
CI
B Std.
Error
Beta
(Constant) 5.64*** 0.23 5.20 6.09
LO Created 0.15 0.80 0.08 0.01 0.31
CP as co-educators 0.3 0.11 0.01 0.02 0.24
Reflection 0.13 0.16 0.04 0.19 0.44
Carnegie Elective CE Class. 0.17* 0.07 0.10 0.31 0.03
NASPA Lead 0.35*** 0.08 0.20 0.50 0.20
Campus Compact 0.22** 0.09 0.10 0.39 0.05
* p< 0.05
** p< 0.01
*** p< 0.001
32. Discussion
Day of Service (DOS) & select
institutional characteristics, related to
civic mindedness
REMINDER: In the regression model all of
our DOS characteristics lost their
significance when being controlled for
selected institutional characteristics.
Further, in our first model, Carnegie CE
class & Campus Compact was not
significant but now it is?
33. Now What:
Closing the Assessment Loop
How can we use what we know here to improve
practices?
What if DOS were intentionally designed around
a civic outcome?
34. Q & A
THE DATA, ANALYSIS, SURVEY, FUTURE PLANS,
ETC.
Ask us anything…
35. Multi-Campus Assessment
of Civic Outcome
Intentionally Designed
Events & Programs
for Students’ Civic Learning
INTERESTED? Need More Info?
Kristin Norris
norriske@iupui.edu
Anne Weiss
haweiss@iupui.edu
Tom Hahn
tomhahn@iupui.edu
THANK YOU!
