Creating Engaed Communities: The Role of Service-Learning conference May 23, 2011 "Using Assessment Tools to Gauge Service-Learning Outcomes". Prof. Isolde Mueller & Beth Knutson-Kolodzne, St. CLoud State Universitl

1. Using Standardized and Locally Created
Assessment Tools to Gauge Student
Service-Learning Outcomes
St. Cloud State University Presenters:
Beth Knutson-Kolodzne–
Coordinator-Volunteer Connection
Prof. Isolde Mueller –
Faculty Liaison for Service-Learning
Kim Oren –
Director of Institutional Research

2. Brief History of Service-Learning &
Community Engagement at SCSU…
• 2003 AmeriCorps Grant (4 yrs.)
SL Coord. engages faculty in using SL in courses.
• 2005 Faculty use of SL in courses = up 2x since 2003 ;
number of students engaged = up 5x; number of
hours reported = up 10x.
• 2006 Began surveying service- learning students about their
experience.
• 2007 Grant ends
• Faculty Liaison for SL begins 25% reassigned time
• New President of SCSU
• Secured 10 hr./wk. GA for SL

3. Brief History of Service-Learning &
Community Engagement at SCSU…
• 2008 Faculty Liaison begins 50% reassigned time
• Began assessment work with Dir. of IR
• Adapted student survey instrument
• 2009 Measured variety of forms of community-based learning
among students in courses.
• Increased University focus on overall community
engagement within and outside of courses
• 2010 Engagement continues to remain stable or grow slightly
in some areas.
• 2011 University reorganization plans underway.
• Position for a Faculty Scholar in the new University
College
• Position for Assoc. Director for Civic Engagement–who is
a liaison with the Faculty Scholar.

4. Outcomes of Service-Learning
Taken from: Where’s the Learning in Service-Learning?
Eyler & Giles, 1999
• Tolerance
• Personal Development
• Interpersonal Development
• Understanding & Applying Knowledge
• Problem Solving/Critical Thinking
• Perspective Transformation
• Citizenship

5. Two data sources
SCSU Student Community-Based Learning Questionnaire
Locally developed. Administered Spring to students after
their community engagement experiences (2009, Fall
2009, Spring 2010, and Fall 2010). Copy of survey
instrument is in handout.
National Survey of Student Engagement (NSSE).
Administered at SCSU in spring semesters of 2001, 2003,
2005, 2007, and 2009. Copy of survey instrument is in
handout.

6. National Survey of Student Engagement
(NSSE or “Nessie.”)
Derived from vast research literature
on college student outcomes.

8. National Survey of Student Engagement
Measures:
x
• Students’ involvement (or engagement) in activities
xxknown to facilitate their growth and development.
x
• Institutional conditions that are conducive to such
xxengagement.

10. Numbers of SCSU students who completed the NSSE, by year
2001 2003 2005 2007 2009 Total
First year 199 234 161 490 458 1542
Seniors 217 240 199 730 651 2037
Total 416 474 360 1220 1109 3579

11. Started with NSSE question 1k:
During the current school year,
about how often have you
participated in a community-based project
(e.g., service learning)
as part of a regular course?
Very often, Often, Sometimes, or Never?

12. NSSE item 1k: During the current school year, about how
often have you participated in a community-based project
(e.g., service learning) as part of a regular course?
First year students Seniors
All combined All combined
N % N %
Never 917 62 Never 926 47
Sometimes 390 27 Sometimes 703 35
Often 125 8 Often 245 12
Very often 39 3 Very often 107 5
Total 1471 100 Total 1981 100

13. NSSE item 1k: During the current school year, about how
often have you participated in a community-based project
(e.g., service learning) as part of a regular course?
First year students First year students
All 5 years All 5 years
N % N %
Never 917 62 Did not participate 917 62
Sometimes 390 27 Participated 554 38
Often 125 8 Total 1471 100
Very often 39 3
Total 1471 100

14. NSSE item 1k: During the current school year, about how
often have you participated in a community-based project
(e.g., service learning) as part of a regular course?
Seniors Seniors
All 5 years All 5 years
N % N %
Never 926 47 Did not participate 926 47
Sometimes 703 35 Participated 1055 53
Often 245 12 Total 1981 100
Very often 107 5
Total 1981 100

15. First year students: Participated in a community-based project
(e.g., service learning) as part of a regular course.
100 Did not participate
Particpated
79
80 72
62 62
60 53 52
47 48
38 38
40
28
21
20
0
Sp 2001 Sp 2003 Sp 2005 Sp 2007 Sp 2009 Total

16. Seniors: Participated in a community-based project
(e.g., service learning) as part of a regular course.
80 Did not participate
Particpated
61 59
58 58
60 55 53
45 47
42 41 42
39
40
20
0
Sp 2001 Sp 2003 Sp 2005 Sp 2007 Sp 2009 Total

17. First year students
All 5 years
N %
Did not participate 917 62
Participated 554 38
Total 1471 100
Seniors
All 5 years
N %
Did not participate 926 47
Participated 1055 53
Total 1981 100

18. 15 NSSE breakout questions. Compared Participants versus Non-participants (from question 1k) for each.
6e How often? Tried to better understand someone else's views by imagining how an issue looks from his or her perspective
1t How often? Discussed ideas from your readings or classes with others outside of class (students, family members, co-workers, etc.).
1u How often? Had serious conversations with students of a different race or ethnicity than your own.
1v How often? Had serious conversations with students who are very different from you in terms of their religious beliefs, political opinions, or personal values.
7b Community service or volunteer work. Done or plan to do?
11b SCSU contribution: Acquiring job or work-related knowledge and skills
11e SCSU contribution: Thinking critically and analytically
11h SCSU contribution: Working effectively with others
11i SCSU contribution: Voting in local, state (provincial), or national (federal) elections
11k SCSU contribution: Understanding yourself
11l SCSU contribution: Understanding people of other racial and ethnic backgrounds
11m SCSU contribution: Solving complex real-world problems
11o SCSU contribution: Contributing to the welfare of your community
8a Quality of relationships with other students
8b Quality of relationships with faculty members

19. Tried to better understand someone else's views
by imagining how an issue looks from his or her perspective
(Sps 01, 03, 05, 07, & 09 combined).
First year**
Often or Very often Total
Comm proj in course N % Total
Did not participate 272 48 566
Participated 254 59 430
All respondents 526 53 996
Seniors**
Often or Very often Total
Comm proj in course N % N
Did not participate 324 51 638
Participated 581 69 845
All respondents 905 61 1483
* Chi-square significant at p < 0.01
** Chi-square significant at p < 0.001
Response options: Very often, Often, Sometimes, or Never

20. ea r
s
ior
st y
Summary of comparisons on 15 NSSE breakout questions
Sen
Fir
1t Discussed ideas outside of class X X
1u Serious conversations with different races or ethnicities X X
1v Serious conversations with different religions, politics, or values X X
6e Tried to understand someone else's perspective X X
7b Community service or volunteer work X X
11b SCSU contribution: Job or work-related knowledge and skills X X
11e SCSU contribution: Thinking critically and analytically X
11h SCSU contribution: Working effectively with others X X
11i SCSU contribution: Voting in elections X X
11k SCSU contribution: Understanding yourself X X
11l SCSU contribution: Understanding other racial and ethnic X X
backgrounds
11m SCSU contribution: Solving complex real-world problems X X
11o SCSU contribution: Contributing to welfare of community X X
8a Quality of relationships with other students X
8b Quality of relationships with faculty members X
Xs = Statistically significant difference between Participants and Non-participants in the predicted direction (27
of 30 comparisons). That is, Participants were more highly engaged than Non-participants.

21. First year students: SCSU contribution composite
Job or work-related knowledge/skills. Understanding yourself.
Thinking critically and analytically. Understanding other races/ethnicities.
Working effectively with others Solving complex real-world problems.
Voting in local/state/national elections. Contributing to welfare of community.

22. Seniors: SCSU contribution composite
Job or work-related knowledge/skills. Understanding yourself.
Thinking critically and analytically. Understanding other races/ethnicities.
Working effectively with others Solving complex real-world problems.
Voting in local/state/national elections. Contributing to welfare of community.

23. First year students: Frequency of effective behaviors composite
Tried to better understand someone else's views.
Discussed ideas from your readings or classes with others outside class.
Serious conversations with students of different races/ethnicities.
Serious conversations with different religions/politics/values.

24. Seniors: Frequency of effective behaviors composite
Tried to better understand someone else's views.
Discussed ideas from your readings or classes with others outside class.
Serious conversations with students of different races/ethnicities.
Serious conversations with different religions/politics/values.

25. Outcomes of Service-Learning
Taken from: Where’s the Learning in Service-Learning?
Eyler & Giles, 1999
• Tolerance
• Personal Development
• Interpersonal Development
• Understanding & Applying Knowledge
• Problem Solving/Critical Thinking
• Perspective Transformation
• Citizenship

26. Results from Local Survey Data
Student Community-Based Learning
Questionnaire (2009-2010)
• Demographics
• Form of Community–Based Learning (CBL)
• Mandatory or optional?
• Number of hours.
• Value.
• Learning.
• Connection to community.
• Awareness of community needs.

27. Local Survey results (page 1 of 5)
Class status.
N %
Senior 671 31
Junior 502 23 Form of Community-Based Learning (CBL).
Freshman 445 20
N %
Sophomore 444 20
Service learning 1115 60
Other 113 5
Total 2175 100 Volunteering 421 23
Mis s ing 16 Field experience 130 7
Community-based research 105 6
Other 87 5
Total 1858 100
Missing 333

28. Local Survey results (page 2 of 5)
Had you done prior CBL?
N %
No 674 52
Yes 507 39 Was CBL mandatory for the class?
Uncertain 107 8
N %
Total 1288 100
Mandatory 1876 87
Missing 2
Note: Asked only in Fall 2009 and Fall 2010.
Optional 291 13
Total 2167 100
Missing 24

29. Local Survey results (page 3 of 5)
# of hours engaged in CBL for class.
N % How CE affected my values and ideas.
11 to 20 710 33 N %
1 to 10 695 32 Continued to see value in CE
1094 51
21 to 30 460 21 I already possessed
More than 40 180 8 Saw value in CE I did not
836 39
31 to 40 135 6 possess before
No effect on my ideas about
Total 2180 100 202 9
CE
Missing 11
Negatively affected my ideas
13 1
about CE
Other 11 1
Total 2156 100
Missing 35

30. Local Survey results (page 4 of 5)
How CE affected my learning.
N %
Helped me learn more than I
1802 83
thought I would
Plan to engage in subsequent
Did not affect my learning 297 14 Community Service.
N %
Learned less than I thought I
55 3 Yes 1497 69
would
Undecided 550 25
Other 18 1
No 125 6
Total 2172 100 Total 2172 100
Missing 19
Missing 19

31. Local Survey results (page 5 of 5)
Feel a greater connection to community
due to CBL in class.
N %
Yes 1574 72
More aware of community needs
No 349 16
due to CBL in class.
Undecided 249 11
N %
Total 2172 100
Yes 1785 83
Missing 19
Undecided 192 9
No 178 8
Total 2155 100
Mis s ing 36

32. Effect on learning by hours.
1 to 10 11 to 20 21 to 30 31 to 40 More than 40 Total
N % N % N % N % N % N %
Helped me learn more than
546 79 576 81 400 88 116 87 162 91 1800 83
I thought I would.
Did not affect my learning. 129 19 106 15 38 8 13 10 10 6 296 14
Learned less than I thought
14 2 21 3 13 3 2 2 4 2 54 2
I would.
Other. 3 0 5 1 5 1 2 2 2 1 17 1
Total 692 100 708 100 456 100 133 100 178 100 2167 100

34. 1k. Participated in a community-based project (e.g., service learning) as
part of a regular course.
20
16
% "Often" or "Very often"
15
13 13 13
12 First year students
13 13 13
10
10 10 11 SCSU
8 10 10 10 Peers
Carn
7
5 Nat
5 6
Region
MnSCU
0
2001 2003 2005 2007 2009
Response options: Very often, Often, Sometimes, or Never.

35. 1k. Participated in a community-based project (e.g., service learning) as
part of a regular course.
25
21
20 20
% "Often" or "Very often"
20 20 20
18 19
19 Seniors
18
17
17
15 14 SCSU
15 Peers
13 14
Carn
12 Nat
10
Region
9 MnSCU
8
5
2001 2003 2005 2007 2009
Response options: Very often, Often, Sometimes, or Never.

36. 7b. Community service or volunteer work.
85
80
80
% "Done" or "Plan to do."
78 79
77
79 79 79
75 75 First year students
75 75
73
74 74 SCSU
71 73
Peers
70
Carn
69 69
Nat
65 Region
MnSCU
64
60
2001 2003 2005 2007 2009
Response options: Done, Plan to do, Do not plan to do, or Have not decided.

37. 7b. Community service or volunteer work.
80
77
76
75
75 73
% "Done" or "Plan to do."
75 73 73
71
72 Seniors
70 71
70
69 SCSU
66
Peers
65 64
63 Carn
63 Nat
61 62 Region
60
MnSCU
55
2001 2003 2005 2007 2009
Response options: Done, Plan to do, Do not plan to do, or Have not decided.

38. Q & A…
• History
• Data Sources
• Outcomes
• Etc…..