Assessment+summary+from+workshop+march26 2012 bonnie


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Assessment+summary+from+workshop+march26 2012 bonnie

  1. 1. 1Summary for Group 1 and Group 4: Scored Sample C for CTWORKSHEET 1SAMPLE CCT VALUE RUBRICSUMMARY OF SCORING & REASONS FOR SCORESGeneral Comment about VALUE Rubric:there is lack of fit with respect to sample C and the CT VALUE rubricbelief that the student handled the assignment relatively well but scored a bit lower on theVALUE Rubric than the group thought the student should have scoredImportant: how well the student performs seems linked to how well the assignmentpositions students with respect to demonstrating the criteria in the VALUE rubricCritical thinking is learned across courses and in multiple places;Do assignments actually ask/require/lead students to demonstrate critical thinkingskills?Does course level work make a difference in evaluation?Need to add a zero and need to add a N/A columnA. EXPLANATION OF ISSUES:General Comments: NoneScore:Milestone 2: 9 peopleMilestone 3: 3 peopleReasons Noted: No thesis; no problem; some terms described, but not well; definition there butnot comprehensive;Handled well a very challenging topic; conclusion could have been theintroduction; paragraph 1 was confusing, threw the reader off; ambiguities bystatements that were not to be explicitly arguedB. EVIDENCE:General Comments: NoneScore:Milestone 3 (or low 4): 4 peopleMilestone 2: 5 peopleIn between Milestone 2 and Benchmark 1 (1.5 score): 1 personBenchmark 1: 1 personReasons Noted: summary; not in the analysis; no interpretation or integration of the evidenceSelection of evidence was okay but it was not tied together; each selected
  2. 2. 2evidence was independent of ideas; not contemplation of why or contextC. INFLUENCE AND CONTEXT OF ASSUMPTIONS:General Comments:Not clear what this means; difficult to define context in assignment as student approaches itThere was a lot of variability in our group in scoring and in our discussion; our scoring rangedfrom 0  3; unclearScore:Milestone 3: 2 peopleMilestone 2: 2 peopleBenchmark 1: 4 peopleZero: 2 peopleUnclear: 2 peopleReasons Noted:Circular arguments; did not question assumptions; did not explore assumptions; did notunderstand the context of assumptions; did not examine or question assumptions in context ofothersD. STUDENT’S POSITION (Perspective, thesis/hypothesis)General Comments:The student’s position was not there; the group did not score this because they were not surehow to with respect to the assignment. No scores given by group 4Problems in determining the difference between (3) and (4) in the rubric(Group 1 comment: I (bonnie) am not sure if this refers to (3) or (4) with respect to scoring orwith respect to the criteria)Score:Benchmark 1: 5 peopleZero: 1 personReasons Noted:Listing rather than taking a positionE. CONCLUSIONS AND RELATED OUTCOMES (implications and consequences)General Comments:NoneScore:Milestone 2: 3 peopleBenchmark 1: 9 peopleZero: 1 personReasons Noted: Very short with no elaboration or synthesis
  3. 3. 3WORKSHEET 2SAMPLE CBHCC CT RUBRICSUMMARY OF SCORING & REASONS FOR SCORESGeneral Comments about BHCC CT RUBRIC:specificity of rubric: less room for interpreting rubric;more discipline-specific rubric; major specific;doesn’t easily apply to this student artifact;rubric is less helpfulProblems with matching the assignment and the rubricAssignment issue: problem with using this rubric to assess this assignmentSeemed to be a higher general scoring (when looking at applicable criteria) using theBHCC rubric than the LEAP VALUE RubricAssignment is problematic; not much Critical Thinking RequestedBunker Hill rubric seemed to be built upon BloomsA. IDENTIFIES ISSUESGeneral Comments: noneScore:Level 1: 1 personLevel 1.5: 3 peopleLevel 2: 4 peopleLevel 3: 5 peopleReasons Noted:Need assistance in understanding; shows minimal understanding of the assignment; says nothingabout the assignment; Mix criteria in descriptionClearly addressed the assignment; lacked clarity but had more than a superficial understanding ofthe assignment and issuesB. UNDERSTANDS CONNECTIONS:General Comments: noneThe tools did not line up for usScore:Level 1: 3 peopleLevel 1.5: 2 peopleLevel 2: 6 peopleReasons Noted:No thesis; little connection of points; connects some points – it is not just a summary; the writerhas an issue but no thesis;Many in group 4 (4 people) ranked it a level 2 but thought it might be a 2.5 or low 3
  4. 4. 4Three concepts presented but no there was no thesis statement; no connections; no synthesis; nodiscussion of each concept/characteristic of post-modern ideasC. PROVIDES SUPPORTGeneral Comments:What is evidence: linked to discipline?Group 4 eliminated the bottom bulletScore:Level 1: 1 personLevel 2: 4 peopleLevel 2.5: 1 personLevel 3: 6 people (all group 4 people)Reasons Noted:No opposing evidence presented; evidence as examples only; opposing evidence not relevant toassignment; okay supportVery …… on sub points;Not all arguments/critical thinking require an opposing viewpoint(s); this should be noted on theVALUE rubric as well.D. CONSIDERS COMPLEXITYGeneral Comments:The nature of the assignment  no opposing perspectives used; ―conclusions‖ should be in nextcategoryScore:Level 1: 3 peopleLevel 2: 3 peopleN/A as defined: group 4 did not think this criterion as defined appliedReasons group 4 gave:Limited complexity asked for;student was performing conceptual analysis;the student opens the door to introducing complexity/deepening the argument but the studentdoes not actually do so, doesn’t follow through or walk through the door;student did not unpack the context;here we thought it important to note the difference between context versus opposition;Also, we thought it important to distinguish between the complexity in the object of study versusthe complexity in the student’s analysis; we discussed that the complexity in the object of studycontributed to demonstrating the complexity in student’s thinkingDefinition of complexity is problematic: complexity in student’s thinking versus the complexityin the topic the student is attempting to understand
  5. 5. 5E. DRAWS INFERENCES:General Comments:Score 2 and Score 3 are too similar to discriminate betweenScore:Level 1: 1 personLevel 2: 8 peopleLevel 3: 2 people (2.5 to 3)Reasons Noted:Superficial Conclusion; No meta-cognitive activityConclusions were based upon evidence but were very superficialF. UNDERSTANDS ASSUMPTIONS:General Comments:Inconsistency in the rubric descriptors; rubric is confusingGroup 4 did not score; the group thought that this was not applicable; N/A the group was notsure how to assess; student’s assumptions? Assumption of authors of the readings the studentwas referring to; student’s state assumptions versus challenge assumptions of othersThere appears to be two separate processes here: stating/recognizing assumptions; challengingassumptionsScore:Level 1: 1 personLevel 2: 1 personNot Applicable: 10 peopleReasons Noted:G. IDENTIFIES IMPLICATIONS:General Comments:Group 4 noted a lot of variability in scoring here: scores ranged from N/A to level 2 to level 3 tolevel 4Score:Level 1: 6 peopleLevel 2: 2 peopleLevel 3: 3 peopleLevel 4: 1 personReasons Noted:
  6. 6. 6No thesisWORKSHEET 3: COMPARING RUBRICSGroup 1 completed Worksheet 3Specificity of criteria makes the Bunker Hill rubric harder to apply;BHCC asks about adherence to the assignment – VALUE Rubric does notBHCC rubric seems to be discipline-specific in its criteria; seems like a research paper rubricGroup 4 completed Summary Report (not Worksheet 3)Criteria that aligned:Generally speaking, the BHCC rubric yielded higher ratings (at least in our discussion andgeneral impression)Broad categories align, but the specifics varyRhetorical feature versus CT feature of BHCC rubricBHCC rubric was more suited to the Humanities versus the Social Sciences or Sciences; theBHCC rubric was less institutionally applicable than the VALUE rubricThe BHCC rubric specifically asks if the product/artifact addresses the assignment where theVALUE rubric does notGENERAL COMMENT:We should also consider rating the applicability of the assignment with respect to therubric to be used in assessing the assignment
  7. 7. 7GROUP 6 SCORED UMASS BOSTON SAMPLEWORKSHEET 1SAMPLE: UMASS BOSTON STUDENT SAMPLECT VALUE RUBRICSUMMARY OF SCORING & REASONS FOR SCORESA. EXPLANATION OF ISSUES:Score:Milestone 3: 3 peopleMilestone 4(3.5 -4): 3 peopleReasons Noted:Demonstrated meaning of ―comprehensive: and ―all‖; strong pieceB. EVIDENCE:Score:Milestone 3: 7 peopleReasons Noted:Use of superlatives; nature of timing and how to fulfill the assignmentC. INFLUENCE AND CONTEXT OF ASSUMPTIONS:Score:Capstone 4: 3 peopleMilestone 3.5: 2 peopleMilestone 2: 2 peopleReasons Noted:Comparison of authors, put in historical time; interpretation as optimisticD. STUDENT’S POSITION (Perspective, thesis/hypothesis)Score:Capstone 4: 3 peopleMilestone 3.5: 1 personMilestone 3: 3 peopleReasons Noted:Confidence in perspective; throws in side issue – militarism; because of timing wasn’t able tofully discuss limitationsE. CONCLUSIONS AND RELATED OUTCOMES (implications and consequences)Score:Capstone 4: 1 person
  8. 8. 8Milestone 3.5: 1 personMilestone 3: 5 peopleReasons Noted: difficult to know if accurate use of sourcesGROUP 6 SCORED UMASS BOSTON SAMPLEWORKSHEET 2UMASS BOSTON SAMPLEBHCC CT RUBRICSUMMARY OF SCORING & REASONS FOR SCORESA. IDENTIFIES ISSUESScore:Level 1: 4 peopleReasons Noted:All issues in the assignment or all issues in the topicB. UNDERSTANDS CONNECTIONS:Score:Level 1: 5 peopleLevel 2: 3 peopleReasons Noted:Depends on how many times the essay is read for the smaller threads to fitC. PROVIDES SUPPORTScore:Level 1: 6 peopleLevel 2: 2 peopleReasons Noted:Taking timing into consideration; easier to learn to 4 because of the descriptors in the rubric;more accurate descriptors to decide 3 or 4A few times wanted more informationD. CONSIDERS COMPLEXITYScore:Level 2: 8 peopleReasons Noted:Difference between 2 and 3 is too greatE. DRAWS INFERENCES:General Comments:
  9. 9. 9Rubric is constrictiveScore:Level 1: 8 peopleF. UNDERSTANDS ASSUMPTIONS:Score:Level 1: 4 peopleLevel 2: 4 peopleReasons Noted:Nothing seen as misrepresentations; assumptions clearly identified and built into the conclusionSuperlatives were too extremeG. IDENTIFIES IMPLICATIONS:Score:Level 1: 7 peopleHigh Level 2/Low Level 1: 1 personReasons Noted:Use of ―most‖ significant wording in rubricWORKSHEET 3: COMPARING RUBRICSGroup 6 completed this worksheet, not the summary sheetWays in which criteria are similar:BH level 4 is closer to a 3 on the VALUEBH 1 is the zero categoryCategories are similarWays in which criteria differ:Clearly addresses assignment – on BHCC ; not on VALUEEasier to make differences in 3 and 4 by descriptors on the BHCC rubricBHCC distinctions in all categories easier to makeVALUE rubric is broaderBH language is more useful, VALUE Rubric language is ―too nice‖
  10. 10. 10Summary of Groups 2 and 3WORKSHEET 1SAMPLE CCT VALUE RUBRICSUMMARY OF SCORING & REASONS FOR SCORESGeneral Comments about VALUE rubric:Assignment is a big part of the problem. Need professional development for faculty onhow to elicit critical thinking since some assignments do not ask for CTNeed N/AA. EXPLANATION OF ISSUESScore:Benchmark 1 personMilestone 2: 9 peopleMilestone 3: 4 peopleReasons Noted: Issues undefined—gives list of examples—no real analysisB. EVIDENCE:General Comments: NoneScore:Benchmark 1: 5 peopleMilestone 2: 7 peopleMilestone 3: 2 peopleReasons Noted: Only asked to reference 1 source; has some interpretationStudent did not address either topic but rather bits of eachC. INFLUENCE AND CONTEXT OF ASSUMPTIONS:Score:Benchmark 1: 5 peopleMilestone 2: 6 people2.5: 1 personZero: 1 personReasons Noted: Had relevant quotations that aligned with assertions—didn’t questionassumptionsNo questioning of assumptions
  11. 11. 11D. STUDENT’S POSITION (Perspective, thesis/hypothesis)Score:Benchmark 1: 9 peopleMilestone 2: 3 peopleMilestone 3: 2 peopleReasons Noted:Didn’t find a coherent starting off point, so can’t get where he needs to be.Assignment is poorly designedThis is basically a description of postmodernism with examplesE. CONCLUSIONS AND RELATED OUTCOMES (implications and consequences)Score:Level 1: 7 peopleMilestone 2: 8 peopleZero: 3 peopleReasons Noted: Clear conclusion but no opposing viewpoints—simplistic conclusionAlso assignment issueIf we were looking at a sample of papers that are representative of this sample,then students are not demonstrating critical thinking skillsQuestion arose from group regarding appropriateness of assignment itselffor critical thinking—i.e., lack of outside evidenceDescribe knowledge necessary to make evaluations based on somecharacteristicsWe don’t know if sample reflects thinking on behalf of the student or aregurgitation of what was said in class“Zero” rating vs. “unable to rate” because it’s not applicable, etc.WORKSHEET 2SAMPLE CBHCC CT RUBRICSUMMARY OF SCORING & REASONS FOR SCORESA. IDENTIFIES ISSUESGeneral Comments: none
  12. 12. 12Score:Level 1: 3 peopleLevel 2: 9 peopleReasons Noted:Didn’t show understanding of assignmentB. UNDERSTANDS CONNECTIONS:General Comments: noneThe tools did not line up for usScore:Level 1: 6 peopleLevel 2: 6 peopleReasons Noted:No real thesis; however, did connect key pointsC. PROVIDES SUPPORTGeneral Comments:Score:Level 1: 4 peopleLevel 2: 1 personLevel 2.5: 1 personLevel 3: 2 peopleN/A: 2 peopleReasons Noted:N/A. This is not an assignment permitting opposing viewpointsD. CONSIDERS COMPLEXITYScore:Level 1: 5 peopleLevel 2: 7 peopleReasons Noted:No multiple perspectives—student does not understand notion of complexity and doesunderstand postmodernismNo focus on how examples do or don’t illustrate
  13. 13. 13E. DRAWS INFERENCES:Score:Level 1: 1 personLevel 2: 7 peopleLevel 3: 3 peopleLevel 4: 1 personReasons Noted:Conclusions are consistently drawn even if somewhat simplisticF. UNDERSTANDS ASSUMPTIONS:Score:Level 1: 1 personLevel 2: 3 peopleLevel 3: 7 peopleReasons Noted:Good coherence; set groundwork for conclusionsLogical and consistent argumentG. IDENTIFIES IMPLICATIONS:Score:Level 1: 2 peopleLevel 2: 5 peopleLevel 3: 5 peopleReasons Noted:We felt the assignment didn’t ask for thisDo we need more than the assignment—maybe a checklist form of sorts or narrativethat “instructor” has identified appropriateness of assignment for particularassessmentIf we were doing this assessment ―for real,‖ we would disqualify the assignment becauseit is not appropriate for critical thinkingOr if critical thinking is the outcome, should we even consider the original assignment?Not knowing what was covered in lecture does not allow for some sense of exit thinkingor repeating lectureBullets within ratings for each characteristic ask if both or if just one achieved—whatdoes that mean for the rating assigned to it?
  14. 14. 14Worksheet 3 (from groups 2 and 3)Identify Ways in Which Criteria are Similar (AAC&U and BHCC rubric)o Both look at identification of issueso Both include student perspective/positiono Both ask about evidence/supporto Elements of AAC&U are included in Bunker Hill rubrico Different scales in eacho Disciplines should be considered if appropriate vs. Joe/Jane vs. exit-levelconsiderations need to be takenIdentify Ways in Which Criteria Differo AAC&U’s does not allow for evaluation of whether or not artifact meetsassignment requirementsSummary Worksheet (from Group 3)Identify criteria that alignThe institutional rubric had more distinctions. However, when reviewing the AAC&Urubric criteria, there are distinctions. Albeit slightly more hiddenIdentify criteria that do not align and reasons why there is lack of alignmentAssignment directions completely included on institutional rubricDiscipline influence is strong—or can be especially strong—particularly with higherlevels on AAC&U rubricList recommended ways to achieve greater alignment between the two sets of rubrics oridentify ways in which institutional rubrics capture the criteria in the AAC &U rubricThe institutional rubric did not capture the criteria in the AAC&U rubric—although theseparation of bullet points within the cells created confusion in our groupThe type of assignment seemed key to the use of rubricsWhile some of us were less comfortable with the institutional rubric, others were morecomfortable with the outlined format