This session describes a portfolio assessment process used at Ozarka College to evaluate student writing. Students complete six essays over the course of a semester with the portfolio essay being required to pass. Faculty then randomly select and score 15-20% of portfolios to assess achievement of general education outcomes. This process provides data to faculty for critical discussion on student learning and informs curriculum improvements. Challenges include calibration of rubric scoring and participation of adjunct faculty. The assessment process has been highlighted by accreditors and involves students, faculty, and administration.

1. Joanna Fulbright
Jeremy Nicholson
Chris Lorch
Ozarka College,
Melbourne, Arkansas
Based on a process created by the
Des Moines Area Community College System

2. Session Description
This session will describe how two simple,
straightforward yearly writing assessments at one
community college with sites at multiple locations
can
-capture data at multiple levels,
-create opportunity among
-address most General Education Outcomes and
+students,
+full-time English faculty,
+adjunct English faculty and
+other department faculty
for the critical conversations where deep
assessment takes place.

3. The Portfolio:
A Circle of
Assessment

4. Clever Introduction

5. The Portfolio:
A Circle of
Assessment

6. Semester Begins:
Students Write
• Teach outcomes
• Students typically
write five essays.
• Essay six is the
portfolio
assignment.
• The portfolio must
be completed to
pass the course.

7. The Portfolio:
A Circle of
Assessment

8. Compile Portfolio:
Random Selection
• Students assemble
coursework for
portfolio.
• Portfolio essays quote
coursework to prove
outcome
accomplishment.
• Faculty randomly
selects 15-20% of all
portfolios for
assessment.

9. The Portfolio:
A Circle of
Assessment

10. Faculty Meets:
Score Portfolios
• Calibrate scoring with
one essay.
• Every essay is
scored twice.
• Third score if the
scores differ by a
letter grade.
• Closest two scores
are kept.

11. The Portfolio:
A Circle of
Assessment

12. What We Know about Data and
Assessment
 We should assess what we value.
 We will value what we assess.
 Assessment should influence change.
 Change should improve student learning.
 Will that improved learning be reflected in
the data?

14. Student Grade by
Objective
72% Comp. I Jan 2012
70%
68%
66%
64%
62%
60%
58%
56%
54%
1 2 3 4 5 6

15. Example of Low Performance and
Modification to Improve Student
Learning
 Fall 2010: 54% of students achieved the
outcome “Address diversity issues.”
 Modified the outcome to read “Address
diverse audiences.”
 Fall 2011: 73% of students achieved this
outcome.

16. Challenges to Overcome
2008
 Received only 13
scorable portfolios.
 No variation of
grades in sampling.
 Unfamiliarity with
rubric.
 Discussion affecting
scoring.

17. Challenges to Overcome
2009
 Rubric - 5 point
scale, Grades - 10 point
scale.
 Rubric did not match
objectives.
 Some objectives not
measurable.
 Too much focus on letter.
 APA or MLA? Both?
 Portfolio overlap in Comp. II.

18. Challenges to Overcome
2010
 Students did not
understand diversity
objective.
 Completion rates low.
 Correlation between portfolio
and course grades.
 Acceptable number of grammar errors?
 Letter format was awkward.

19. Challenges to Overcome
2011
 Low scores on the
critical reading
outcome.
 Need to further
streamline the rubric.

20. Rubrics: Issues
 Have experimented with multiple
rubrics: descriptive, non-
descriptive, rating scale, etc.
 Effective calibration takes more
time than the committee feels
comfortable requesting from
participants.
 The right rubric might address
this issue.

21. More Challenges
 Serving needs of ALL other
departments or just Arts and
Sciences?
 Serving needs of ALL adjuncts?
 (impossible to schedule a “perfect”
time)
 Serving needs of concurrent courses?
 (none have participated yet)

22. Improvements /Changes to the
Courses
 Moved from objectives to
outcomes
 Unified outcomes
 Unified portfolio directions
 Instructors more focused
more on explaining/teaching
the outcomes

23. Improvements /Changes to
the Courses
 Changed textbooks
 Began teaching APA in Comp
II
 Revised rubrics
 Changed the diversity
outcome
 Moved to a capstone essay
in Comp II
 Reworded critical reading
outcome

24. Addressing
Administration’s Needs
 Provides some assurance that
different sections of the same course
meet course outcomes and rigor
requirements.
 When assessment is faculty-owned, it
continues no matter what changes
come in administration.
 What has this meant at Ozarka
College?

25. HLC Acknowledgement
 English Assessment was
highlighted as a good
assessment for the
following accreditation
needs:
 The 2008 Self-Study (no
follow-up, no focus
report)
 The 2012 Distance
Education accreditation
visit
 The 2012 Additional Site
accreditation visit

26. Developed Learning
Outcomes:
-Involves students in metacognitive
activities focusing on achievement of
course outcomes.
-Involves faculty in deep conversations
about outcomes vs. objectives; no
groupthink occurs.
-Involves faculty and administrators in
checking various program outcomes as
well as General Education Outcomes.

27. Developed Learning
Outcomes:
-Involves multiple
departments and
administration in
conversations about
writing assessment.
-Involves meeting the
writing needs of both the
workforce as well as
transfer institutions .

28. Questions?
Contact Us:
Joanna Fulbright:
jfulbright@ozarka.edu
Jeremy Nicholson:
jnicholson@ozarka.edu
Chris Lorch:
clorch@ozarka.edu