This document summarizes a webinar about assessing the educational practices and outcomes of the SAGE 2YC project. It introduces the project goals of implementing practices to support student success, broaden participation, and facilitate pathways in geoscience. An educational practices inventory and common outcomes measures are described to evaluate the impacts. Outcomes related to access, participation, course success, and pathway progression will be tracked over time and by student subgroups. Questions about implementing the assessment plan are invited.

1. Educational Practices and
Outcomes Assessment of SAGE 2YC
Connect to the webinar:
http://serc.adobeconnect.com/sage2yc/
Please use phone or skype for audio

2. Educational Practices and
Outcomes Assessment of SAGE 2YC
November, 2016
Heather Macdonald, College of William & Mary
Debra Bragg and Heather McCambly, Bragg & Associates
Pamela Eddy, College of William & Mary
Ellen Iverson, SERC

4. Webinar Goals and Overview
• Review SAGE 2YC project, goals, and theory of change
• Introduce plan for collecting data that will allow us to measure
project outcomes
– Educational practices inventory
– Outcomes assessment, common outcomes measures
• Q & A about the plan

5. SAGE 2YC Project Team
Evaluation & Research Project Leaders SERC Staff
Debra Bragg Pamela Eddy
Heather McCambly Ellen Iverson
Heather Macdonald Eric Baer
Jan Hodder Norlene Emerson
Carol Ormand John McDaris

6. SAGE 2YC Teams and Their Work
• SAGE 2YC Teams: 24 Faculty Change
Agents in 10 regions
– Administrators
– Institutional Research
• Action plans developed at June 2016
workshop considering three project
strands
– Individual
– Geoscience program/department
– Regional workshop
Project strands
• Supporting academic
success of all students
• Broadening
participation
• Facilitating students’
professional pathways

7. SAGE 2YC Goals
Faculty learn from
research
Faculty make changes
in their own practice
Faculty share what
they are learning with
the geoscience
education community
• Build a national network of 2YC geoscience
faculty and administrators led by 2YC faculty
leaders, called Change Agents, who catalyze
change at multiple levels
• Implement high-impact, educational
practices that:
•Support the academic success of all students
•Broaden participation
•Facilitate professional pathways into the
geosciences
• Investigate professional development models
that support the cycle of innovation to
improve geoscience education
Cycle of Innovation

8. Theory of Change Logic Model
Outcomes of
Educational Practice
Changes to 2YC
Geoscience Education
in:
• Access
• Participation
• Course Success
• Pathway Progression
+
Educational
Practices
SAGE 2YC
Leadership Team
Support
+
Change Agents
Change Agent
Action Plans
SAGE 2YC
Professional
Development
Cycle of Innovation
Cycle of Innovation

9. Educational Practices Inventory
• Research-based practices related to the project goals
– Academic success of all students
– Broadening participation in the geosciences
– Facilitating students’ professional pathways
• Two inventories of faculty practices:
– What individual faculty do
– What faculty do collectively, in the context of their department or
program

10. Outcomes Assessment Goals
• Assess implementation of educational practices
• Apply outcomes assessment to your geoscience program(s)
and course(s)
• Link outcomes assessment to the three strands of SAGE 2YC:
–Academic success
–Broadening participation
–Pathways progression – transfer and career

11. Participation
Geoscience Course
Enrollment
Broadening Participation in
Geoscience Courses
Course Success Outcomes
Success in Geoscience
Courses
Geoscience Course Success
by Student Sub-group
Pathways Progression Outcomes
Enrollment in Multiple
Geoscience Courses
Major in Geoscience Program
Snapshot of Outcomes Assessment Measures

12. Approach
• Use Outcomes Assessment Worksheets, with instructions provided by
the SAGE 2YC evaluators, Debra Bragg & Heather McCambly
• Partner with institutional research (IR)
• Engage college administration to help secure data, if needed
• Work collaboratively with Debra Bragg and the leadership team (led
by Heather Macdonald) on your assessment plan
• Start the outcomes assessment in Spring 2017 and continue to the
end of the grant

13. Outcome Measure: Geoscience Course Enrollment
Question: What is the enrollment in
geoscience courses, including courses
receiving an “educational change”, and
how does course enrollment change from
Spring 2017 to the end of the grant?
Measure: Term-by-term enrollment at
census date in geoscience courses,
including geoscience courses identified as
having received an “educational change”,
from Spring 2017 to the end of the grant.
Method
1. List geoscience courses
offered by your college
2. For each term, indicate
whether course is
offered and whether the
course has received an
“educational change”
3. For each geoscience
course offered, count
the total number of
students enrolled in
each course at census
date.

14. Outcome Measure: Broadening Participation in
Geoscience Courses
Question: What is the enrollment of
underrepresented student groups in
geoscience courses, and does
geoscience course enrollment by
underrepresented student groups
change from Winter/Spring 2017 to
the end of the grant?
Measure: Term-by-term geoscience
course enrollment by student sub-
group (e.g., gender, race, ethnicity).
Method
For each course and by term
offered, count students
affiliated within sub-groups by
gender, race, ethnicity, and
other demographic variables
(according to IPEDS
definitions) at census date.

15. Outcomes Measure: Success in Geoscience Courses
Question: How many students and
what percentage of students in
geoscience courses succeed, and does
course success change from
Winter/Spring 2017 to the end of the
grant?
Measure: Term-by-term geoscience
course completion with a grade of “C”
or higher.
Method
1. For each course and by
term, count the number
of students who
completed and passed
the course with a grade
of “C” or higher.
2. Compute percentage of
students deemed
successful (#1) by the
total enrollment in same
geoscience course.

16. Outcome Measure: Geoscience Course Success by Student
Sub-Group
Question: By sub-group, how
many students succeed in
geoscience courses, and does the
percentage of course enrollee
success change from
Winter/Spring 2017 to the end of
the grant?
Measure: Term-by-term
geoscience course completion
with a grade of “C” or higher by
student sub-group.
Method
1. For course and by term,
count the number of
students within sub-groups
who completed and passed
the geoscience course with
a grade of “C” or higher.
2. Divide #1 by the total count
of students within sub-
groups who enrolled in the
same geoscience course.

17. Outcomes Measure: Pathways Progression
Question: How many students take
more than one geoscience course,
how many students major in
geoscience, and do these numbers
change from Winter/Spring 2017 to
the end of the grant?
Method:
• Student enrollment in more than
one geoscience course.
• Students with declared major or
concentration in geoscience.
1. By course and by term,
count the number of
students who enroll at
census date who have
enrolled in more than one
geoscience course.
2. By course and by term,
count the number of
students who enroll at
census data who are
declared geoscience
majors.
Method

18. Outcomes Measure: Pathways Progression by
Student Sub-Group
Question: How many students by
sub-group who take more than one
geoscience course and major in
geoscience? How do these numbers
change from Winter/Spring 2017 to
the end of the grant?
Method:
• Student sub-group enrollment in
more than one geoscience course.
• Student sub-group with declared
major in geoscience.
1. By course and by term,
count the number of
students by sub-group who
enroll at census date who
have enrolled in more than
one geoscience course.
2. By course and by term,
count the number of
students by sub-group who
enroll at census data who
are declared geoscience
majors.
Method

19. Outcomes Measures
• Access and Participation
– Term-by-term enrollment at census date in geoscience courses
– Term-by-term geoscience course enrollment by student sub-group
• Course Success
– Term-by-term geoscience course completion with a grade of “C” or higher
– Term-by-term geoscience course completion with a grade of “C” or higher
by student sub-group
• Pathway Progression
– Student enrollment in more than one geoscience course
– Student sub-group enrollment in more than one geoscience course
– Students with declared major or concentration in geoscience
– Student sub-group with declared major in geoscience

20. Timeline for Outcomes Assessment
• November 2016 – this webinar
• Mid-December 2016 – Outcomes Assessment Guide and Worksheets
shared with Change Agents
• January 2017 – telephone conference calls to review outcomes assessment
plan with Change Agents
• Educational practices inventories
– Individual: December 2016 - January 2017
– Department: Early winter, 2017
• Winter/Spring 2017 – Change Agents launch outcomes assessment process
• June 2017 – Change Agent Annual Workshop in Tacoma, WA

21. Outcomes Assessment Resources
Primary Sources:
– Change Agent Action Plans
– SAGE 2YC Proposal and Logic Model
Other Sources:
– Gates Foundation/Institute for Higher Education & Policy’s
scan of 20+ improvement initiatives
– Alliance for Quality Career Pathways Framework
– The Voluntary Framework for Accountability
– Pathways to Results Equity & Outcomes Templates

22. Questions?
Contacts:
• Outcomes Assessment: Debra Bragg, Bragg.Associates.Inc@gmail.com
• Inventories: Pamela Eddy, peddy@wm.edu
• Overall project: Heather Macdonald, rhmacd@wm.edu