Your SlideShare is downloading. ×
0
TYCA Northeast
Annual Conference 2010
Capitol Improvements: The Two-Year College as an Agency of Change
Collaborative
Assessment:
Working Together Toward Institutional Change
Kelly Bender and Elizabeth Nesius
Passaic County Co...
This presentation can be found on
SlideShare
This presentation looks at methods of collaboration for
course-and program-level assessment and data-sharing to
help both ...
PCCC Writing Initiative
 Title V Grant: $2.5 million over 5 years
 Develop 20 Distinct, Writing Intensive
Courses (with ...
Writing Intensive Courses
 19 Distinct WI courses currently
developed or in development across the
curriculum
 Departmen...
What We Assess
 Students
◦ Assessed in writing, critical thinking, and information
literacy skills
◦ Assessed in the cour...
How We Assess
 Students
◦ Instructors: student grades
◦ Administrators: student performance on CWE
 Courses
◦ Course mat...
Assessing Writing
 First we had to determine the criteria for
writing assessment:
◦ Appropriateness
◦ Unity & Focus
◦ Dev...
Writing Rubric Student __________________________________ Assignment ______________________________________
Beginning Deve...
EN 205 – Introduction to
Literature Grading Rubric
Student __________________________________ Assignment _________________...
Assessing Writing
 Faculty use rubric with students for
formative writing assessment
 Extra space to add content or othe...
TOOLS: The Writing Center
The Writing Center
provides support for
students at all stages
of the writing process,
whether i...
The Writing Center
While the Writing Center made the
assessment of student writing “easier,” it
was also helpful in terms ...
Assessing Critical Thinking
 Criteria for critical thinking assessment
◦ Identifying arguments and counter-arguments
◦ As...
Assessing Critical Thinking
Critical Thinking Rubric STUDENT _________________________________________ Assignment ________...
Assess Information Literacy
 Criteria for information literacy
assessment
◦ Defines and articulates need for information
...
Assessing Information Literacy
Information Literacy Rubric Student ________________________ Reader ____________________ Sc...
Assessing the Course
Course assessment is essential in
determining:
 Both the quality and quantity of learning for
studen...
Methods of Course Assessment
 By Instructors
◦ Faculty institutes
◦ ePortfolios
◦ Course Survey
 By Administrators
◦ Cou...
Course Assessment: Faculty
Institutes
Faculty Administrators
 Institute Part 1
◦ Overview of Initiative
◦ Overview of WI ...
Faculty Institutes
Faculty Administrators
 Institute Part 2
◦ Some materials tested
in regular section of
courses
◦ Mater...
Assessing the Initiative
 The program is assessed for
accomplishment of its goals
◦ For faculty development
◦ For student...
ePortfolio Information
 Electronic portfolios offer three distinct benefits:
1) They are a creative, lively form in which...
ePortfolio Information
 The portfolio not only offers a tool for authentic
assessment but also a means for students to be...
ePortfolio Information
 Assessment findings derived from students, tutors,
and teachers indicate that electronic portfoli...
Assessment via ePortfolios
Faculty Administrators
 Used for student and
course assessment
 Students upload all formal
as...
Sample ePortfolios
 www.stevecuello.v2efolioworld.mnscu.edu
 www.kristycancel.v2efolioworld.mnscu.ed
u
 www.luzcampusan...
Course Survey
 Used by instructor for course assessment
 Provides the instructor with feedback from
his or her students
...
Student Survey & Results
1. When you registered for this course, were you
aware that it was a Writing Intensive course?
68...
Student Testimonials
 “I like to read more because of this class
and Professor Bender.”
 “I never thought I was going to...
WI Student Survey
 Used by WI Administrators for program
assessment
 Conducted across all WI courses
 Neither students ...
WI Student Survey Results
 Were you aware that you registered for a WI
course?
◦ 57% Yes
◦ 43% No
 Rating of services
◦ ...
WI Student Survey Results
 Rating of services
◦ Libguides
 Excellent or Good 50.4%
 Fair or Poor 13.5%
 Did not use 36...
WI Student Survey Results
 Rating of skills
◦ Information literacy
 Improved 63.4%
 About the same 31.3%
◦ Critical Thi...
Faculty Survey
 All WI Faculty surveyed
 Survey administered online, anonymous
 Allows faculty to report on services of...
Faculty Survey Results
 Familiar with Initiative goals for students
◦ Use the writing process: 100%
◦ Support of claims w...
Faculty Survey Results
 100% found the goals helpful for improving
student learning
 67% used rubrics for evaluation bef...
Faculty Survey Results
 100% incorporated tutoring into their WI
courses
 67% felt their students were more
comfortable ...
Focus Groups
 Conducted by Institutional Research
 Recorded discussion of strengths and
weaknesses of program
 WI Admin...
Focus Group Results
Faculty Administrators
 Felt that there were too
many technological
elements required in
the course
...
For the Future
Intro. to Literature Writing Initiative
 Work more closely with
counselors/registrar/
advisors to ensure t...
Thank you!
Kelly Bender: kbender@pccc.edu
Elizabeth Nesius: enesius@pccc.edu
Portfolio Results
 Writing
◦ Minimum pass rate or higher: 77%
◦ Below passing: 23%
 Critical Thinking
◦ Minimum pass rat...
Sample ePortfolios
Sample ePortfolios
Sample ePortfolios
Upcoming SlideShare
Loading in...5
×

Collaborative Assessment: Working Together Toward Institutional Change

600

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
600
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Collaborative Assessment: Working Together Toward Institutional Change"

  1. 1. TYCA Northeast Annual Conference 2010 Capitol Improvements: The Two-Year College as an Agency of Change
  2. 2. Collaborative Assessment: Working Together Toward Institutional Change Kelly Bender and Elizabeth Nesius Passaic County Community College
  3. 3. This presentation can be found on SlideShare
  4. 4. This presentation looks at methods of collaboration for course-and program-level assessment and data-sharing to help both faculty and program administrators. Instructors can improve the WAC elements of their courses without sacrificing content, while still furthering the program mission of helping students to write well and think critically in any discipline. Writing program administrators, on the other hand, can improve both the writing program implementation and the training of its faculty to teach WAC courses. Students will benefit by experiencing WAC courses that are clearly defined, engaging, and optimized for their learning.
  5. 5. PCCC Writing Initiative  Title V Grant: $2.5 million over 5 years  Develop 20 Distinct, Writing Intensive Courses (with multiple sections, across 3 campuses)  Develop and provide online resources for technology and writing  Support students and faculty with an on- ground writing center and online tutoring  Students required to take 2 WI courses (1 if getting AAS degree)
  6. 6. Writing Intensive Courses  19 Distinct WI courses currently developed or in development across the curriculum  Departments include: Computer Information Systems, English, Humanities, Math, and Science  Courses designed to contain writing, critical thinking, and information literacy elements
  7. 7. What We Assess  Students ◦ Assessed in writing, critical thinking, and information literacy skills ◦ Assessed in the course by instructors  Courses ◦ Course materials created collaboratively in Faculty Institutes ◦ Course materials checked and approved by WI Administrators  Program ◦ Students assessed across courses by WI administrators in the 3 areas ◦ Students and instructors self-assess by means of a survey
  8. 8. How We Assess  Students ◦ Instructors: student grades ◦ Administrators: student performance on CWE  Courses ◦ Course materials collected at the beginning of each semester ◦ Evidence of materials being used by student use of ePortfolios  Program ◦ Student writing assessed with 3 rubrics: writing, critical thinking, information literacy ◦ Student and faculty surveys
  9. 9. Assessing Writing  First we had to determine the criteria for writing assessment: ◦ Appropriateness ◦ Unity & Focus ◦ Development ◦ Organization ◦ Sentence Structure ◦ Usage ◦ Mechanical Conventions  Additional criteria for course writing are left up to individual instructors
  10. 10. Writing Rubric Student __________________________________ Assignment ______________________________________ Beginning Developing Competent Accomplished Vocabulary/Language appropriate to this course Words used incorrectly; displays fundamental errors in vocabulary usage; lacks variety. Generally correct usage but little variety; some variety but words used incorrectly. The majority correctly used with some variety in vocabulary. Varied vocabulary; correct and sophisticated use of language. Conventions mechanics, spelling, sentence structure, word forms (problems noted below) Many serious errors which persistently interfere with meaning. Some serious errors that may interfere with meaning. A few errors which do not interfere with meaning. No serious errors; few minor errors. Organization includes: appropriate length, order (logical, chronological etc.) and transitions No clear structure; organizational requirements not met. Some structure and order evident; minimally, an introduction, body & conclusion present. Generally well-organized; clear and logical flow of ideas. Fulfills the assignment. Well-organized with appropriate structure and smooth transitions. Content includes addressing the assigned topic Lacks thesis; incomplete development of evidence and/or minimal relevance to topic. May lack thesis. Some evidence underdeveloped or off-topic. Has supporting evidence and examples for thesis. Strong, convincing evidence to support thesis and argument.  Tense formation  subject-verb agreement  pronoun usage/agreement  word choice/meaning  proper modifiers  Variety of formations  correct construction  Spelling  Capitalization  punctuation COMMENTS: WI v10.1 Assessing Writing
  11. 11. EN 205 – Introduction to Literature Grading Rubric Student __________________________________ Assignment ______________________________________ Accomplished Competent Developing Beginning Vocabulary/Language appropriate to this course Varied vocabulary; correct and sophisticated use of language. The majority correctly used with some variety in vocabulary. Generally correct usage but little variety. Words used incorrectly; displays fundamental errors in vocabulary usage; lacks variety. Conventions (problems noted below) No fundamental errors; few minor errors. A few errors which do not interfere with meaning. Some fundamental errors that may interfere with meaning. Many fundamental errors which persistently interfere with meaning. Organization includes: appropriate length, order (logical, chronological etc.) and transitions Well-organized with appropriate structure and smooth transitions. Generally well- organized; follows basic assignment guidelines; clear and logical flow of ideas. Some structure and order evident, attempt at following assignment guidelines; minimally, introduction, body & conclusion. No clear structure; does not follow assignment guidelines; incoherent. Content includes addressing the assigned topic Strong, convincing evidence to support thesis and argument. Has supporting evidence and examples for thesis. May lack thesis or conclusion. Evidence underdeveloped or off-topic. Lacks thesis and/or conclusion. Minimal relevance to topic. Citation of sources Uses citations when appropriate and demonstrates mastery of formatting; goes beyond the amount of citation or sources required. Uses required amount of citations when appropriate and uses proper formatting most of the time. Uses citations infrequently and shows inconsistent formatting; understands some elements of citation but not others. Uses little or no citation and displays little knowledge of formatting; clearly cannot differentiate between most of the elements of citation. USAGE  Tense formation  subject-verb agreement  pronoun usage/agreement  word choice/meaning  proper modifiers SENTENCE CONSTRUCTION  Variety of formations  correct construction MECHANICS  Spelling  Capitalization  punctuation COMMENTS: Revised 11/1/2010
  12. 12. Assessing Writing  Faculty use rubric with students for formative writing assessment  Extra space to add content or other elements for assessment based on assignment  WI Administrators use rubric for summative assessment of writing (without additional elements)
  13. 13. TOOLS: The Writing Center The Writing Center provides support for students at all stages of the writing process, whether it is brainstorming a topic, structuring an essay, or editing and revising. It also offers support to students for their Writing Intensive courses in the forms of face-to-face and small group tutoring/instruction.
  14. 14. The Writing Center While the Writing Center made the assessment of student writing “easier,” it was also helpful in terms of assessing the course. Attendance at the Writing Center was made mandatory. This created a direct correlation between the outcome of student writing and the effectiveness of a support tool.
  15. 15. Assessing Critical Thinking  Criteria for critical thinking assessment ◦ Identifying arguments and counter-arguments ◦ Assessing validity of both sides ◦ Drawing conclusions based on those assessments  Faculty use rubric with students for formative critical thinking assessment  Administrators assess critical thinking across courses (in writing assignments that require critical thinking) using same rubric
  16. 16. Assessing Critical Thinking Critical Thinking Rubric STUDENT _________________________________________ Assignment ______________________________ Beginning Developing Competent Accomplished Evidence  Biased interpretations  Insufficient search for evidence  or inappropriate evidence choices  Misinterprets some evidence  Limited information search  limited source evaluation or attribution  Accurately interprets the majority of evidence  Sufficient search for evidence  Shows some evaluation of sources  Appropriate evidence  Accurately interprets the evidence presented Arguments  Fails to identify argument  or counter-arguments  Has argument but inadequate identification of relevant counter- arguments.  Identifies relevant argument(s)  including counter- arguments  Identifies pertinent arguments, pro and con  Rebuts counter arguments Conclusion(s) Does not reach any conclusion justified by evidence. Conclusion based on incorrect, irrelevant , or insufficient evidence Conclusion(s) are reasonably justified or reasonably explained. Draws thoroughly justified conclusions by appropriate interpretations of evidence Comments V4 June 2010
  17. 17. Assess Information Literacy  Criteria for information literacy assessment ◦ Defines and articulates need for information ◦ Information retrieval ◦ Citation of sources ◦ Evaluation of Sources ◦ Uses information effectively  Information literacy required in at least one assignment  Assessment by faculty and administrators using the same rubric
  18. 18. Assessing Information Literacy Information Literacy Rubric Student ________________________ Reader ____________________ Score _____________ Beginning Developing Competent Accomplished Defines and articulates the need for information No clear statement of purpose; does not associate information need with the assignment Shows purpose but not clearly articulated; associates information need with the assignment but does not articulate a thesis statement Purpose is clearly articulated and information need is clearly recognized Purpose is highly focused and/or nuanced; thesis statement or research question takes multiple viewpoints into account Information retrieval Retrieves little or none of the information required for the assignment; or, information retrieved clearly does not meet the information need Retrieves some of the information required for the assignment; some of the information retrieved may not meet the information need Retrieves the required amount of information for the assignment and the majority of it meets the information need Retrieves much more than the required amount to complete the assignment; information retrieved meets the information need; multiple viewpoints are considered Citation of sources Uses little or no citation and displays little knowledge of formatting; clearly cannot differentiate between most of the elements of citation Uses citations infrequently and shows inconsistent formatting; understands some elements of citation but not others Uses citations when appropriate and uses proper formatting most of the time Uses citations when appropriate and demonstrates mastery of formatting; goes beyond the amount of citation or sources required Evaluation of Sources Little or no recognition of the relevance and quality (authority, currency, objectivity) of information sources Recognizes the relevance of information sources but has difficulty evaluating their quality Recognizes both the quality and relevance of information sources Information sources are highly relevant and evaluation of sources is highly nuanced Uses information effectively in their writing assignments Uses little or no useful information from outside sources Attempts to incorporate information from outside sources; sometimes off-topic Uses information from outside sources to support the topic; shows some synthesis with own ideas Use of information from outside sources is highly refined, articulated, and synthesized with own ideas Comments: Revised 6/6/08
  19. 19. Assessing the Course Course assessment is essential in determining:  Both the quality and quantity of learning for students  An appropriate response to their personal development needs  Whether students have acquired the knowledge, skills and competencies consistent with the aims of the course  The effectiveness of the curriculum and faculty development
  20. 20. Methods of Course Assessment  By Instructors ◦ Faculty institutes ◦ ePortfolios ◦ Course Survey  By Administrators ◦ Course materials ◦ ePortfolios
  21. 21. Course Assessment: Faculty Institutes Faculty Administrators  Institute Part 1 ◦ Overview of Initiative ◦ Overview of WI course components ◦ Look at course as it stands for writing, critical thinking and information literacy elements ◦ Begin development of new materials with rubrics and templates  Institute Part 1 ◦ Collect current course materials (syllabus and assignments) ◦ Assist in development of new materials ◦ Collect and file new materials ◦ Survey faculty on success of workshop components
  22. 22. Faculty Institutes Faculty Administrators  Institute Part 2 ◦ Some materials tested in regular section of courses ◦ Materials refined based on in-course testing ◦ More materials developed ◦ Integration of media and other resources  Institute Part 2 ◦ Discuss piloting of materials from part 1 ◦ Assist in refining materials and development of new materials ◦ Assist in incorporating media, other resources and support services
  23. 23. Assessing the Initiative  The program is assessed for accomplishment of its goals ◦ For faculty development ◦ For student development  Methods of Program Assessment ◦ ePortfolios ◦ Student Survey ◦ Faculty Survey ◦ Focus Groups
  24. 24. ePortfolio Information  Electronic portfolios offer three distinct benefits: 1) They are a creative, lively form in which to show a student’s best work; 2) They allow students to gain an awareness of a larger audience (a portfolio can be published on an intranet or the Internet or saved to a CD and given to college representatives, employers, or anyone a student wants to impress); 3) They can evolve and grow over a student’s entire academic career. "From Worn-Out to Web - Based : Better Student Portfolios." Phi Delta Kappan 85.10 (June 2004): 792-794. Academic Search Premier. EBSCO. [Library name], [City], [State abbreviation]. 27 Aug. 2009 <http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=13292501&site =ehost-live>.
  25. 25. ePortfolio Information  The portfolio not only offers a tool for authentic assessment but also a means for students to be reflective practitioners, emphasizing the how and why as much as the what. Time spent in portfolio assessment is not time taken away from teaching or academics, but time refocused and redefined, with the portfolio viewed as a natural complement to learning. Lombardi, Judy. "To Portfolio or not to Portfolio: Helpful or Hyped?." College Teaching 56.1 (Winter2008 2008): 7-10. Academic Search Premier. EBSCO. [Library name], [City], [State abbreviation]. 27 Aug. 2009 <http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=31161341&site =ehost-live>.
  26. 26. ePortfolio Information  Assessment findings derived from students, tutors, and teachers indicate that electronic portfolios have had several positive effects on student learning. These portfolios vividly record writing as a process, providing students and teachers with an effective means of assessing the development of that process over a semester. In addition, they effectively display the final results of the semester, including a student’s self-assessments in the form of written reflections. Click, Ben A., and Sarah C. Magruder.. "Implementing Electronic Portfolios for Performance Assessment:A Pilot Program Involving a College Writing Center." Assessment Update 16.4 (July 2004): 1-15. Academic Search Premier. EBSCO. [Library name], [City], [State abbreviation]. 27 Aug. 2009 <http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=13998617&site =ehost-live>.
  27. 27. Assessment via ePortfolios Faculty Administrators  Used for student and course assessment  Students upload all formal assignments to portfolios  Allows instructors to look at student progress throughout the course  Students write reflections on various aspects of the course, or on the course as a whole  Used for course and program assessment  Random selection of portfolios to review across courses  10% of portfolios assessed  Look at assignments for writing, critical thinking, and information literacy elements
  28. 28. Sample ePortfolios  www.stevecuello.v2efolioworld.mnscu.edu  www.kristycancel.v2efolioworld.mnscu.ed u  www.luzcampusano.v2efolioworld.mnscu. edu
  29. 29. Course Survey  Used by instructor for course assessment  Provides the instructor with feedback from his or her students  Through narrowly focused questions, specific feedback is received on: ◦ Exact assignment ◦ Support systems in place (i.e. The Writing Center, eTutoring.org) ◦ Text (book and reading selections) ◦ Instructor ◦ Grading Rubric
  30. 30. Student Survey & Results 1. When you registered for this course, were you aware that it was a Writing Intensive course? 68% Yes 2. What is your overall opinion of the material covered in the course? 3. Would you take another WI course? 93% Yes 4. What is your experience with eTutoring.org? 5. What is your experience with the Writing Center? 6. What did you like best about this course? 7. What, if anything, would you change/add/remove from the course?
  31. 31. Student Testimonials  “I like to read more because of this class and Professor Bender.”  “I never thought I was going to have so much fun in a Literature course or on an intensive writing course. I’m glad it was a great experience.”  I definitely improved my writing skills. I think if this was just an intro to lit class it would be boring, the writing is what made it fun. I feel so much more confident in my writing.”
  32. 32. WI Student Survey  Used by WI Administrators for program assessment  Conducted across all WI courses  Neither students nor courses identified  Allows students to report on various aspects of the WI experience ◦ Workload ◦ Support services ◦ Self-assessment on improvement in writing, critical thinking and information literacy
  33. 33. WI Student Survey Results  Were you aware that you registered for a WI course? ◦ 57% Yes ◦ 43% No  Rating of services ◦ eTutoring  Excellent or Good 68.2%  Fair or Poor 20.5%  Did not use 11.4% ◦ Writing Center  Excellent or Good 62.2%  Fair or Poor 6%  Did not use 31.8%
  34. 34. WI Student Survey Results  Rating of services ◦ Libguides  Excellent or Good 50.4%  Fair or Poor 13.5%  Did not use 36.1% ◦ ePortfolios  Excellent or Good 44.7%  Fair or Poor 24.2%  Did not use 31.1%  Rating of skills ◦ Writing  Improved 71.5%  About the same 27.7%
  35. 35. WI Student Survey Results  Rating of skills ◦ Information literacy  Improved 63.4%  About the same 31.3% ◦ Critical Thinking  Improved 66.4%  About the same 29.1% ◦ Technology  Improved 50.7%  About the same 42.5%
  36. 36. Faculty Survey  All WI Faculty surveyed  Survey administered online, anonymous  Allows faculty to report on services offered in the Writing Initiative ◦ Institutes ◦ Professional development opportunities ◦ Support services  Faculty report on observations of student improvement in the three areas  Report on own improvement in technology and media use and use of support services
  37. 37. Faculty Survey Results  Familiar with Initiative goals for students ◦ Use the writing process: 100% ◦ Support of claims with specific evidence: 89% ◦ Edit writing according to the rules of standard academic English: 100% ◦ Evaluate sources for credibility and academic appropriateness: 89% ◦ Employ techniques for integrating information: 67% ◦ Exhibit the ability to think critically: 100% ◦ Cite sources using an appropriate documentation style: 89%
  38. 38. Faculty Survey Results  100% found the goals helpful for improving student learning  67% used rubrics for evaluation before teaching a WI course  87% used rubrics for evaluation in WI courses  33% made use of available tutoring services for their classes before teaching WI courses
  39. 39. Faculty Survey Results  100% incorporated tutoring into their WI courses  67% felt their students were more comfortable doing and integrating research after taking a WI course  78% felt their students were more aware of critical thinking after taking a WI course  89% felt that WI components had value for their other courses
  40. 40. Focus Groups  Conducted by Institutional Research  Recorded discussion of strengths and weaknesses of program  WI Administrators and WI Faculty met separately  IR wrote report analyzing the results for each group and making recommendations
  41. 41. Focus Group Results Faculty Administrators  Felt that there were too many technological elements required in the course  Wanted an increase in communication among WI faculty. They felt well-supported by the staff and often communicated with them, however.  Desire more help from the college community as a whole to overcome obstacles to program success  See communication, especially among faculty and program administrators, as key to program success
  42. 42. For the Future Intro. to Literature Writing Initiative  Work more closely with counselors/registrar/ advisors to ensure that students are aware of the course they are taking  Possibly incorporate more use of the Writing Center  Increase the amount of available tech support to allow for easier ePortfolio use  Better marketing of WI courses to students  Better marketing of the Writing Center to students  Better marketing of professional development opportunities to WI faculty  Allow faculty to use either paper or online portfolios for their classes
  43. 43. Thank you! Kelly Bender: kbender@pccc.edu Elizabeth Nesius: enesius@pccc.edu
  44. 44. Portfolio Results  Writing ◦ Minimum pass rate or higher: 77% ◦ Below passing: 23%  Critical Thinking ◦ Minimum pass rate or higher: 25% ◦ Below passing (or no evidence of CT): 75%  Information Literacy ◦ Minimum pass rate or higher: 50% ◦ Below passing in one or more areas: 50%
  45. 45. Sample ePortfolios
  46. 46. Sample ePortfolios
  47. 47. Sample ePortfolios
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×