This session describes a portfolio assessment process used at Ozarka College to evaluate student writing. Students complete six essays over the course of a semester with the portfolio essay being required to pass. Faculty then randomly select and score 15-20% of portfolios to assess achievement of general education outcomes. This process provides data to faculty for critical discussion on student learning and informs curriculum improvements. Challenges include calibration of rubric scoring and participation of adjunct faculty. The assessment process has been highlighted by accreditors and involves students, faculty, and administration.
2. Session Description
This session will describe how two simple,
straightforward yearly writing assessments at one
community college with sites at multiple locations
can
-capture data at multiple levels,
-create opportunity among
-address most General Education Outcomes and
+students,
+full-time English faculty,
+adjunct English faculty and
+other department faculty
for the critical conversations where deep
assessment takes place.
6. Semester Begins:
Students Write
• Teach outcomes
• Students typically
write five essays.
• Essay six is the
portfolio
assignment.
• The portfolio must
be completed to
pass the course.
8. Compile Portfolio:
Random Selection
• Students assemble
coursework for
portfolio.
• Portfolio essays quote
coursework to prove
outcome
accomplishment.
• Faculty randomly
selects 15-20% of all
portfolios for
assessment.
10. Faculty Meets:
Score Portfolios
• Calibrate scoring with
one essay.
• Every essay is
scored twice.
• Third score if the
scores differ by a
letter grade.
• Closest two scores
are kept.
12. What We Know about Data and
Assessment
We should assess what we value.
We will value what we assess.
Assessment should influence change.
Change should improve student learning.
Will that improved learning be reflected in
the data?
13.
14. Student Grade by
Objective
72% Comp. I Jan 2012
70%
68%
66%
64%
62%
60%
58%
56%
54%
1 2 3 4 5 6
15. Example of Low Performance and
Modification to Improve Student
Learning
Fall 2010: 54% of students achieved the
outcome “Address diversity issues.”
Modified the outcome to read “Address
diverse audiences.”
Fall 2011: 73% of students achieved this
outcome.
16. Challenges to Overcome
2008
Received only 13
scorable portfolios.
No variation of
grades in sampling.
Unfamiliarity with
rubric.
Discussion affecting
scoring.
17. Challenges to Overcome
2009
Rubric - 5 point
scale, Grades - 10 point
scale.
Rubric did not match
objectives.
Some objectives not
measurable.
Too much focus on letter.
APA or MLA? Both?
Portfolio overlap in Comp. II.
18. Challenges to Overcome
2010
Students did not
understand diversity
objective.
Completion rates low.
Correlation between portfolio
and course grades.
Acceptable number of grammar errors?
Letter format was awkward.
19. Challenges to Overcome
2011
Low scores on the
critical reading
outcome.
Need to further
streamline the rubric.
20. Rubrics: Issues
Have experimented with multiple
rubrics: descriptive, non-
descriptive, rating scale, etc.
Effective calibration takes more
time than the committee feels
comfortable requesting from
participants.
The right rubric might address
this issue.
21. More Challenges
Serving needs of ALL other
departments or just Arts and
Sciences?
Serving needs of ALL adjuncts?
(impossible to schedule a “perfect”
time)
Serving needs of concurrent courses?
(none have participated yet)
22. Improvements /Changes to the
Courses
Moved from objectives to
outcomes
Unified outcomes
Unified portfolio directions
Instructors more focused
more on explaining/teaching
the outcomes
23. Improvements /Changes to
the Courses
Changed textbooks
Began teaching APA in Comp
II
Revised rubrics
Changed the diversity
outcome
Moved to a capstone essay
in Comp II
Reworded critical reading
outcome
24. Addressing
Administration’s Needs
Provides some assurance that
different sections of the same course
meet course outcomes and rigor
requirements.
When assessment is faculty-owned, it
continues no matter what changes
come in administration.
What has this meant at Ozarka
College?
25. HLC Acknowledgement
English Assessment was
highlighted as a good
assessment for the
following accreditation
needs:
The 2008 Self-Study (no
follow-up, no focus
report)
The 2012 Distance
Education accreditation
visit
The 2012 Additional Site
accreditation visit
26. Developed Learning
Outcomes:
-Involves students in metacognitive
activities focusing on achievement of
course outcomes.
-Involves faculty in deep conversations
about outcomes vs. objectives; no
groupthink occurs.
-Involves faculty and administrators in
checking various program outcomes as
well as General Education Outcomes.
27. Developed Learning
Outcomes:
-Involves multiple
departments and
administration in
conversations about
writing assessment.
-Involves meeting the
writing needs of both the
workforce as well as
transfer institutions .
28. Questions?
Contact Us:
Joanna Fulbright:
jfulbright@ozarka.edu
Jeremy Nicholson:
jnicholson@ozarka.edu
Chris Lorch:
clorch@ozarka.edu