The document summarizes the minutes from the eighth meeting of the Advancing a Massachusetts Culture of Assessment (AMCOA). Key details include:
- Representatives from 22 institutions attended the meeting hosted at Northern Essex Community College to focus on assessing student writing.
- Two assessment experiments were summarized, including a collaboration between Framingham State University and MassBay Community College to improve transfer student experiences, and an experiment comparing rubrics between Bristol Community College and Massasoit Community College.
- An update was provided on a draft proposal for continued Davis Educational Foundation funding for AMCOA activities in areas like system-wide learning outcomes assessment, campus assessment support, and increased faculty/staff engagement.
1. Minutes of the Eighth AMCOA Meeting, February 29, 2012
Prepared by Kerry McNally
Host Campus: Northern Essex Community College, Haverhill Campus
I. Attendance
The eighth AMCOA meeting was hosted by Northern Essex Community
College (NECC), Haverhill from 10:00 a.m.-1:00 p.m. on February 29, 2012.
Representatives from 22 institutions attended the meeting (See list in
Appendix A), and Peggy Maki, Consultant under the Davis Educational
Foundation Grant awarded to the Department of Higher Education, opened
and chaired the meeting.
Peggy thanked NECC for hosting the meeting.
II. Welcome: President Lane Glenn, Northern Essex Community College
President Glenn welcomed AMCOA Team members to this month’s working
session focused on scoring student work that demonstrates written
communication. He congratulated members on their progress and
emphasized his commitment to this project.
III. Updates and Foci of Next Two Working Sessions: Peggy Maki
Peggy asked the group to please sign up for leading a group, contributing an
institutionally developed scoring rubric that demonstrates critical thinking or
quantitative reasoning, or submitting an exit-level student sample.
2. IV. Update on the Fourth Statewide Assessment Conference: Jim Gubbins
Jim reported that he has received three or four new proposals in addition to
the planned descriptions of the assessment experiments. He also stated that
he would like campuses that haven’t been represented before at conferences
or meetings to step up and present. His planning group, consisting of Chris
Cratsley, Neal Bruss, Donna Kuizenga and Dawne Spangler, will have its first
Skype meeting next week. (Appendix B is the Call for Proposals for the April
23rd
AMCOA conference.)
Requests for more Assessment 101-type sessions were spread across the
Comments in the February 9th
Evaluations. Peggy Maki volunteered to give
sessions on beginning assessments. The question was raised as to what
Assessment 101 means. Peggy said that it would entail very basic terminology
and overarching assessment principles and practices. She suggested that she
would invite institutional reps to share their approaches in this basic session.
V. Summaries of Two Assessment Experiments: Susan Chang, Director of
Assessment, Framingham State University, and Joanne M. Preston, Dean of
Humanities, Bristol Community College
Framingham State University (FSU) and MassBay Community College are
collaborating on an assessment experiment that aims to identify ways to
improve the academic experience and success of students who transfer from
a community college to a 4-year institution by using assessment tools. While
there is an emphasis on a transfer student’s academic success, there is a
recognized need to collect qualitative data to understand how the overall
college experience of a transfer student can be improved. Some of the
transfer students reported feeling not connected to the campus, citing lack of
knowledge about resources. There are six primary objectives of the
3. experiment: (1) continue to build upon assessment processes already in
development at MassBay and FSU; (2) identify strengths and weaknesses in
the General Education curriculum at MassBay and FSU; (3) utilize
demographics for a seamless transition and identify indicators of student
success; (4) explore technology as a cross-campus assessment platform; (5)
influence and inspire changes to the curriculum; and (6) create and continue a
partnership between a two- and four-year institution.
Some of the progress they are making and some of the issues that are
emerging include:
• Conversations at the administrative level that don’t trickle down to the
faculty. That needs to be improved.
• Exploration of students’ strengths and weaknesses in written
communication, quantitative reasoning and critical thinking are.
• Identification of indicators of student success.
• Review of different kinds of assessment software applicable to
documenting 2-year and 4-year students’ achievements
The project is starting with the AAC&U VALUE rubrics, asking if they will work.
FSU has an assessment advisory group and they know where they are going.
Are these rubrics appropriate for both 2-year and 4-year schools? MassBay is
reviewing these rubrics to adopt for their first assessment cycle.
The first assessment cycle will provide data to identify strengths and
weaknesses in the General Education curriculum.
FSU is collecting qualitative data to complement the quantitative information
they normally gather. They are establishing benchmarks and numbers. For
example, in Fall 2008 16% of FSU transfer students came from MassBay
4. Community College. Of those who transferred from MassBay, 50% of them
graduated from FSU as of Fall 2011, compared to 39% of the total transfer
cohort.
Yesterday, FSU looked at software with people from MBCC. They need
information for both schools to make informed decisions by the end of this
calendar year. They want to inspire improvement, but they need the data.
The discussion of STEM rolled from there. They are working with PARCC and
collaborating on improving developmental courses in math. They are also
looking at a potential bridge program. And, they are working on improving
the academic experience for transfer students, for example, an orientation
program that is designed for them.
After reviewing the first timeline, they developed a second one to give faculty
more time to think about the process. By May 2013 they will look at the
overall findings and start the next assessment cycle.
For a full description of the FSU and MassBay experiment progress and future
timeline, please see a copy of the PowerPoint presentation in Appendix C.
The CONNECT AMCOA Assessment Experiment will engage faculty from the
CONNECT Consortium to learn about and apply the Liberal Education and
America’s Promise (LEAP) VALUE rubric for written communication and
support their exploration of the LEAP Essential Learning Outcomes
framework.
The Assessment Experiment will evaluate common assignments using the
Writing Rubric used at Massasoit Community College, the Writing Rubric used
at Bristol Community College and the LEAP VALUE rubric to compare and
5. contrast the three rubrics. There are three assessment teams which are each
made up of two instructors who will assess writing using the rubrics. Two of
the teams are Institutional Scoring Teams (two faculty from BCC) and (two
faculty from MCC). These team members are instructors who teach a
comparable first general education course that emphasizes composition
(English 101). The third team, the Central team, consists of one BCC faculty
member and one MCC faculty member.
The common assignment was administered to 4 writing classes at MCC and 4
writing classes at BCC. The faculty who volunteered to use the common
assignment in their course, received a small stipend after turning in the
writing. JP Nadeau, Professor at BCC, created a random sample of 25
assignments.
The two Institutional Scoring teams will use their college’s rubric: the MCC
team will use the CONNECT Rubric used at MCC and the BCC team will use the
rubric used at BCC to assess the stratified random sample of 25 papers. After
scoring with their college rubric, the two teams will use the LEAP VALUE
Rubric to score several papers not from the original sample of 25 to gain some
experience with the VALUE rubric. The two teams will then use a structured
guideline to document their experiences using the LEAP Rubric noting the
similarities and differences in the rubric when compared with their local
rubric and their perceptions of barriers and facilitators to using the LEAP
VALUE Rubric.
Afterwards, the Central Team, comprised of two composition instructors
selected, will assess the same 25 papers using the LEAP VALUE Rubric. Each
member of the Central Team will then score several additional papers using
their own institution’s rubric and will follow the same structured guideline to
document differences and similarities across rubric sets and identify any
challenges and facilitators to using the LEAP Rubric in comparison with their
own rubric.
6. The project will be completed by the end of May and the write-up of the
Experiment will be completed by the end of June. Individuals involved in this
project include:
• JP Nadeau is the Project Coordinator for the Experiment
• The BCC Team is Farah Habib and Michael Geary
• The MCC Team is Melissa Winchell and Rita Jones-Hyde
• The Central Team is Deb Anderson (BCC) and Susan Keith (MCC)
VI. Summary of New Davis Proposal: Pat Crosson, Senior Advisor for Academic Policy
Pat described the current Draft AMCOA II Proposal (Feb. 27, 2012) as
addressing three components:
• Component A, system-wide learning outcomes assessment focused on
developing a system-wide plan, including the development of models,
metrics, dashboards, and implementation plans. She noted that the
concept of a composite indicator has been taken off the table.
Component A work, therefore, will need to identify other kinds of
evidence we would include in a system-wide assessment reporting
approach that uses multiple sources of evidence about student
learning. Within this component there will be two new task forces and
those task forces will need to connect with the AMCOA team as well.
• Component B, support for campus assessment based on initiatives
from Phase 1 and the need to expand the good work of AMCOA across
our campuses through such activities recommended by co-chairs and
AMCOA team members as conferences, experiments, on-call help
7. teams, “thorny Issues” workshops, and continued use of the project
consultant on a more limited basis next year.
• Component C, expanded involvement that seeks to reach a larger
faculty and staff audience to promote engagement with learning
outcomes and assessment and encourage commitment to using results
for program improvement. The proposal seeks funds for a web-based
assessment repository and tool kit, use of AMCOA team members to
present at campuses’ assessment days, and creation of opportunities
for faculty and staff to look together at the ways that a total campus
environment and student engagements with the curriculum and co-
curriculum affect learning for all students.
Pat explained that the current draft requests support for travel and support
for some release time for roles, such as for the co-chairs, since individuals will
be taking on additional responsibilities this next year.
Pat asked for comments from the AMCOA team about the proposal and
hoped that they saw their ideas integrated into the draft. She also stated she
could receive comments from team members or institutions until this coming
Monday. Shortly thereafter she needs to send out the proposal. She asked
representatives to let their campus leaders know about their support for this
proposal as well. She stated that unless there is support from campuses,
Davis is unlikely to fund the proposal. The current copy is marked confidential
because not all presidents have received it or had a chance to respond. It can
be discussed on campus, but there is such a short time between when it is
due and when comments can be incorporated into the current draft.
Pat’s Draft AMCOA II Proposal-2/27/2012 is attached as Appendix D.
8. VII. Introduction of Faculty Participating in Today’s Working Session and Group
Leaders; Orientation to Today’s Working Session Focused on Assessing Exit-Level
Student Writing: Peggy Maki
Peggy said that developing the VALUE Rubrics has taken hundreds of experts and
years of work. They were created as a way to develop and use a common
language across the states. The VALUE rubrics are more general than many
institutionally developed rubrics. Conversely, many institutionally developed
rubrics are more holistic than the AAC&U rubrics. What we are trying to do, she
stated, is see how well institutionally developed rubrics align with these
nationally developed rubrics—even though they may be more detailed or more
holistic. Peggy also stated that creating these rubrics has become an alternative
way to assess student work – as opposed to using standardized tests as the sole
means of reporting student achievement.
Peggy introduced the group leaders and presenters:
Group 1: Chuck Prescott, presented Berkshire Community College’s rubric
Elise Martin, Middlesex Community College, served as Group Leader
Group 2: Ellen Wentland presented Northern Essex Community College’s
rubric and served as Group Leader
Group 3: Suzanne Van Wert presented Northern Essex Community College’s
rubric
Mark Patrick, Mass Maritime Academy, served as Group Leader
Group 4: Jennifer Arner Welsh presented Quinsigamond Community College’s
rubric
Bonnie Orcutt, Worcester State University, served as Group Leader
Group 5: Neal Bruss presented UMass Boston’s rubric
9. Martha Stassen, UMass Amherst, served as Group Leader
Group 6: Chris Cratsley presented Fitchburg State University’s
rubric
Paula Haines, UMass Lowell, served as Group Leader
Group 7: Tim McLaughlin presented Bunker Hill Community College’s
rubric
James Gubbins, Salem State University, served as Group Leader
10. Appendix A: Institutions Represented at the AMCOA February 29th
Meeting:
Berkshire Community College
Bristol Community College
Bunker Hill Community College
Cape Cod Community College
Fitchburg State University
Framingham State University
Greenfield Community College
Massachusetts Maritime Academy
Massasoit Community College
MassBay Community College
Middlesex Community College
Mount Wachusett Community College
Northern Essex Community College
Quinsigamond Community College
Roxbury Community College
Salem State University
University of Massachusetts Amherst
University of Massachusetts Dartmouth
University of Massachusetts Lowell
University of Massachusetts President’s Office
Westfield State University
12. Appendix B: Call for Proposals for the April 23rd
AMCOA Conference
13. Appendix C: “Two- or four-year institution … doesn’t matter: Student success is
student success,” a collaborative PowerPoint presentation by
Susan Chang, Ellen Zimmerman and Yves Salomon-Fernandez
outlining the progress of the Framingham State University and
MassBay Community College assessment experiment.
(Please double-click the image below to open the presentation. Then,
click once to move from one page to the next.)
Two- or four-year institution … doesn’t matter:
Student success is student success
Advancing a Massachusetts Culture of Assessment (AMCOA) Meeting
February 28, 2012
Northern Essex Community College
14.
15. Appendix D: DRAFT AMCOA II Proposal-2/27/2012 for AMCOA Team (Double-
click image below, then, use down-arrows to move through pages.)