Minutes of the Eighth AMCOA Meeting, February 29, 2012Prepared by Kerry McNallyHost Campus: Northern Essex Community College, Haverhill CampusI. AttendanceThe eighth AMCOA meeting was hosted by Northern Essex CommunityCollege (NECC), Haverhill from 10:00 a.m.-1:00 p.m. on February 29, 2012.Representatives from 22 institutions attended the meeting (See list inAppendix A), and Peggy Maki, Consultant under the Davis EducationalFoundation Grant awarded to the Department of Higher Education, openedand chaired the meeting.Peggy thanked NECC for hosting the meeting.II. Welcome: President Lane Glenn, Northern Essex Community CollegePresident Glenn welcomed AMCOA Team members to this month’s workingsession focused on scoring student work that demonstrates writtencommunication. He congratulated members on their progress andemphasized his commitment to this project.III. Updates and Foci of Next Two Working Sessions: Peggy MakiPeggy asked the group to please sign up for leading a group, contributing aninstitutionally developed scoring rubric that demonstrates critical thinking orquantitative reasoning, or submitting an exit-level student sample.
IV. Update on the Fourth Statewide Assessment Conference: Jim GubbinsJim reported that he has received three or four new proposals in addition tothe planned descriptions of the assessment experiments. He also stated thathe would like campuses that haven’t been represented before at conferencesor meetings to step up and present. His planning group, consisting of ChrisCratsley, Neal Bruss, Donna Kuizenga and Dawne Spangler, will have its firstSkype meeting next week. (Appendix B is the Call for Proposals for the April23rdAMCOA conference.)Requests for more Assessment 101-type sessions were spread across theComments in the February 9thEvaluations. Peggy Maki volunteered to givesessions on beginning assessments. The question was raised as to whatAssessment 101 means. Peggy said that it would entail very basic terminologyand overarching assessment principles and practices. She suggested that shewould invite institutional reps to share their approaches in this basic session.V. Summaries of Two Assessment Experiments: Susan Chang, Director ofAssessment, Framingham State University, and Joanne M. Preston, Dean ofHumanities, Bristol Community CollegeFramingham State University (FSU) and MassBay Community College arecollaborating on an assessment experiment that aims to identify ways toimprove the academic experience and success of students who transfer froma community college to a 4-year institution by using assessment tools. Whilethere is an emphasis on a transfer student’s academic success, there is arecognized need to collect qualitative data to understand how the overallcollege experience of a transfer student can be improved. Some of thetransfer students reported feeling not connected to the campus, citing lack ofknowledge about resources. There are six primary objectives of the
experiment: (1) continue to build upon assessment processes already indevelopment at MassBay and FSU; (2) identify strengths and weaknesses inthe General Education curriculum at MassBay and FSU; (3) utilizedemographics for a seamless transition and identify indicators of studentsuccess; (4) explore technology as a cross-campus assessment platform; (5)influence and inspire changes to the curriculum; and (6) create and continue apartnership between a two- and four-year institution.Some of the progress they are making and some of the issues that areemerging include:• Conversations at the administrative level that don’t trickle down to thefaculty. That needs to be improved.• Exploration of students’ strengths and weaknesses in writtencommunication, quantitative reasoning and critical thinking are.• Identification of indicators of student success.• Review of different kinds of assessment software applicable todocumenting 2-year and 4-year students’ achievementsThe project is starting with the AAC&U VALUE rubrics, asking if they will work.FSU has an assessment advisory group and they know where they are going.Are these rubrics appropriate for both 2-year and 4-year schools? MassBay isreviewing these rubrics to adopt for their first assessment cycle.The first assessment cycle will provide data to identify strengths andweaknesses in the General Education curriculum.FSU is collecting qualitative data to complement the quantitative informationthey normally gather. They are establishing benchmarks and numbers. Forexample, in Fall 2008 16% of FSU transfer students came from MassBay
Community College. Of those who transferred from MassBay, 50% of themgraduated from FSU as of Fall 2011, compared to 39% of the total transfercohort.Yesterday, FSU looked at software with people from MBCC. They needinformation for both schools to make informed decisions by the end of thiscalendar year. They want to inspire improvement, but they need the data.The discussion of STEM rolled from there. They are working with PARCC andcollaborating on improving developmental courses in math. They are alsolooking at a potential bridge program. And, they are working on improvingthe academic experience for transfer students, for example, an orientationprogram that is designed for them.After reviewing the first timeline, they developed a second one to give facultymore time to think about the process. By May 2013 they will look at theoverall findings and start the next assessment cycle.For a full description of the FSU and MassBay experiment progress and futuretimeline, please see a copy of the PowerPoint presentation in Appendix C.The CONNECT AMCOA Assessment Experiment will engage faculty from theCONNECT Consortium to learn about and apply the Liberal Education andAmerica’s Promise (LEAP) VALUE rubric for written communication andsupport their exploration of the LEAP Essential Learning Outcomesframework.The Assessment Experiment will evaluate common assignments using theWriting Rubric used at Massasoit Community College, the Writing Rubric usedat Bristol Community College and the LEAP VALUE rubric to compare and
contrast the three rubrics. There are three assessment teams which are eachmade up of two instructors who will assess writing using the rubrics. Two ofthe teams are Institutional Scoring Teams (two faculty from BCC) and (twofaculty from MCC). These team members are instructors who teach acomparable first general education course that emphasizes composition(English 101). The third team, the Central team, consists of one BCC facultymember and one MCC faculty member.The common assignment was administered to 4 writing classes at MCC and 4writing classes at BCC. The faculty who volunteered to use the commonassignment in their course, received a small stipend after turning in thewriting. JP Nadeau, Professor at BCC, created a random sample of 25assignments.The two Institutional Scoring teams will use their college’s rubric: the MCCteam will use the CONNECT Rubric used at MCC and the BCC team will use therubric used at BCC to assess the stratified random sample of 25 papers. Afterscoring with their college rubric, the two teams will use the LEAP VALUERubric to score several papers not from the original sample of 25 to gain someexperience with the VALUE rubric. The two teams will then use a structuredguideline to document their experiences using the LEAP Rubric noting thesimilarities and differences in the rubric when compared with their localrubric and their perceptions of barriers and facilitators to using the LEAPVALUE Rubric.Afterwards, the Central Team, comprised of two composition instructorsselected, will assess the same 25 papers using the LEAP VALUE Rubric. Eachmember of the Central Team will then score several additional papers usingtheir own institution’s rubric and will follow the same structured guideline todocument differences and similarities across rubric sets and identify anychallenges and facilitators to using the LEAP Rubric in comparison with theirown rubric.
The project will be completed by the end of May and the write-up of theExperiment will be completed by the end of June. Individuals involved in thisproject include:• JP Nadeau is the Project Coordinator for the Experiment• The BCC Team is Farah Habib and Michael Geary• The MCC Team is Melissa Winchell and Rita Jones-Hyde• The Central Team is Deb Anderson (BCC) and Susan Keith (MCC)VI. Summary of New Davis Proposal: Pat Crosson, Senior Advisor for Academic PolicyPat described the current Draft AMCOA II Proposal (Feb. 27, 2012) asaddressing three components:• Component A, system-wide learning outcomes assessment focused ondeveloping a system-wide plan, including the development of models,metrics, dashboards, and implementation plans. She noted that theconcept of a composite indicator has been taken off the table.Component A work, therefore, will need to identify other kinds ofevidence we would include in a system-wide assessment reportingapproach that uses multiple sources of evidence about studentlearning. Within this component there will be two new task forces andthose task forces will need to connect with the AMCOA team as well.• Component B, support for campus assessment based on initiativesfrom Phase 1 and the need to expand the good work of AMCOA acrossour campuses through such activities recommended by co-chairs andAMCOA team members as conferences, experiments, on-call help
teams, “thorny Issues” workshops, and continued use of the projectconsultant on a more limited basis next year.• Component C, expanded involvement that seeks to reach a largerfaculty and staff audience to promote engagement with learningoutcomes and assessment and encourage commitment to using resultsfor program improvement. The proposal seeks funds for a web-basedassessment repository and tool kit, use of AMCOA team members topresent at campuses’ assessment days, and creation of opportunitiesfor faculty and staff to look together at the ways that a total campusenvironment and student engagements with the curriculum and co-curriculum affect learning for all students.Pat explained that the current draft requests support for travel and supportfor some release time for roles, such as for the co-chairs, since individuals willbe taking on additional responsibilities this next year.Pat asked for comments from the AMCOA team about the proposal andhoped that they saw their ideas integrated into the draft. She also stated shecould receive comments from team members or institutions until this comingMonday. Shortly thereafter she needs to send out the proposal. She askedrepresentatives to let their campus leaders know about their support for thisproposal as well. She stated that unless there is support from campuses,Davis is unlikely to fund the proposal. The current copy is marked confidentialbecause not all presidents have received it or had a chance to respond. It canbe discussed on campus, but there is such a short time between when it isdue and when comments can be incorporated into the current draft.Pat’s Draft AMCOA II Proposal-2/27/2012 is attached as Appendix D.
VII. Introduction of Faculty Participating in Today’s Working Session and GroupLeaders; Orientation to Today’s Working Session Focused on Assessing Exit-LevelStudent Writing: Peggy MakiPeggy said that developing the VALUE Rubrics has taken hundreds of experts andyears of work. They were created as a way to develop and use a commonlanguage across the states. The VALUE rubrics are more general than manyinstitutionally developed rubrics. Conversely, many institutionally developedrubrics are more holistic than the AAC&U rubrics. What we are trying to do, shestated, is see how well institutionally developed rubrics align with thesenationally developed rubrics—even though they may be more detailed or moreholistic. Peggy also stated that creating these rubrics has become an alternativeway to assess student work – as opposed to using standardized tests as the solemeans of reporting student achievement.Peggy introduced the group leaders and presenters:Group 1: Chuck Prescott, presented Berkshire Community College’s rubricElise Martin, Middlesex Community College, served as Group LeaderGroup 2: Ellen Wentland presented Northern Essex Community College’srubric and served as Group LeaderGroup 3: Suzanne Van Wert presented Northern Essex Community College’srubricMark Patrick, Mass Maritime Academy, served as Group LeaderGroup 4: Jennifer Arner Welsh presented Quinsigamond Community College’srubricBonnie Orcutt, Worcester State University, served as Group LeaderGroup 5: Neal Bruss presented UMass Boston’s rubric
Martha Stassen, UMass Amherst, served as Group LeaderGroup 6: Chris Cratsley presented Fitchburg State University’srubricPaula Haines, UMass Lowell, served as Group LeaderGroup 7: Tim McLaughlin presented Bunker Hill Community College’srubricJames Gubbins, Salem State University, served as Group Leader
Appendix A: Institutions Represented at the AMCOA February 29thMeeting:Berkshire Community CollegeBristol Community CollegeBunker Hill Community CollegeCape Cod Community CollegeFitchburg State UniversityFramingham State UniversityGreenfield Community CollegeMassachusetts Maritime AcademyMassasoit Community CollegeMassBay Community CollegeMiddlesex Community CollegeMount Wachusett Community CollegeNorthern Essex Community CollegeQuinsigamond Community CollegeRoxbury Community CollegeSalem State UniversityUniversity of Massachusetts AmherstUniversity of Massachusetts DartmouthUniversity of Massachusetts LowellUniversity of Massachusetts President’s OfficeWestfield State University
Appendix B: Call for Proposals for the April 23rdAMCOA Conference
Appendix C: “Two- or four-year institution … doesn’t matter: Student success isstudent success,” a collaborative PowerPoint presentation bySusan Chang, Ellen Zimmerman and Yves Salomon-Fernandezoutlining the progress of the Framingham State University andMassBay Community College assessment experiment.(Please double-click the image below to open the presentation. Then,click once to move from one page to the next.)Two- or four-year institution … doesn’t matter:Student success is student successAdvancing a Massachusetts Culture of Assessment (AMCOA) MeetingFebruary 28, 2012Northern Essex Community College
Appendix D: DRAFT AMCOA II Proposal-2/27/2012 for AMCOA Team (Double-click image below, then, use down-arrows to move through pages.)