Download to read offline
![Collective Sense of Purpose Assessment...requires us to work together , and to do unfamiliar things like setting common goals and standards, devising methods of assessment, interpreting the results, and using them to improve and coordinate our teaching. [It thus] possesses all the appeal and efficiency of committee work, in particular the kind visited upon us by administrators. Robert Holyer, Change , Sept./Oct., 1998](https://image.slidesharecdn.com/siskiyousshowjan07-120103161001-phpapp01/75/Outcomes-Assessment-Overview-7-2048.jpg)
More Related Content
Similar to Outcomes Assessment Overview
Outcomes Assessment Overview
- 1. Assessment as aWay of Understanding & Improving Student Learning William S. Moore, Ph.D. Policy Associate, Assessment, Teaching & Learning WA State Board for Community & Technical Colleges bmoore@sbctc.ctc.edu 360-704-4346 College of the Siskyous January 12, 2007
- 2. As you canclearly see on slide 397… Courtesy of “Dilbert” & Scott Adams Oh, no—not another case of PowerPoint poisoning!!!
- 3. Overview of WorkshopUnderstanding the Context Beginning with the End in Mind Facilitating Student Learning Making Judgments about Progress Putting the Pieces Together Exploring other Questions Focusing on Assignments as Assessments
- 4.
- 5. Accreditation: Caught betweenCompeting Worldviews? Compliance: “prove” Self-reflection; “improve” “ Them” (external) Measures & answers “ Us” (internal) Evidence & better-informed judgments ACCREDITATION WHY? WHO? WHAT? ACCOUNTABILITY ASSESSMENT
- 6. The Promise ofAssessment Create a new notion of educational excellence for higher education Reframe our understanding of accountability Revive/strengthen a sense of campus community by focusing on significant, collective purposes
- 7. Collective Sense ofPurpose Assessment...requires us to work together , and to do unfamiliar things like setting common goals and standards, devising methods of assessment, interpreting the results, and using them to improve and coordinate our teaching. [It thus] possesses all the appeal and efficiency of committee work, in particular the kind visited upon us by administrators. Robert Holyer, Change , Sept./Oct., 1998
- 8. Beginning with the End in Mind
- 9. Underlying Perspectives Drive Key Questions Curriculum What do we want students to know & be able to do?? Knowledge
- 10. Knowledge as a Set of Tools “ Situated” or grounded in specific contexts in which it is used (and learned) Expertise as body of knowledge organized around “big ideas,” not isolated facts
- 11. Making Judgments about Progress
- 12. Underlying Perspectives Drive Key Questions Assessment What do we want students to know & be able to do?? Knowledge Curriculum Evidence How do we judge competence on key outcomes?
- 13. A Sampling ofAssessment Approaches Research papers, essay tests Self-evaluations Interviews Performance tasks (e.g., cases, problems, etc.) Multiple-choice tests Projects, field work Standardized tests, surveys Peer evaluations Portfolio collections of work External assessor ratings Focus groups
- 14. Core Principles ofAssessment Assess the things that really matter, not just the things easily assessed Emphasize the quality and quantity of conversations about assessment evidence Use a variety of approaches and multiple indicators
- 15.
- 16. Underlying Perspectives Drive Key Questions How do we judge competence on key outcomes? What do we want students to know & be able to do?? Knowledge Assessment Curriculum Pedagogy Grading Learning How do we promote learning most effectively?
- 17. College Learning??? Basically,you learn two kinds of things in college: Things you will need to know in later life (2 hours)… Things you will NOT need to know in later life (1198 hours). These are the things you learn in classes whose names end in ‘-ology’, ‘-osophy’, ‘-istry’, ‘-ics’, and so on. The idea is, you memorize these things, then write them down in little exam books, then forget them. If you fail to forget them you become a professor and have to stay in college the rest of your life. Dave Barry, 1981
- 18. Learning as Deep Understanding Collaborative work around authentic, “situated” activities “ Learning to be” vs. “learning about” Teacher’s roles: modeling, scaffolding, fading, coaching
- 19. Putting the PiecesTogether
- 20. Course Department/Program InstitutionMultiple Levels of Assessment
- 21. General Education Assessment Decisions: Definition : what matters? Focus : individual students or programs? Level of emphasis : what’s a program? Nature of evidence : direct or indirect? Approach : external or embedded?
- 22. Student Learning Institutional Outcomes Effectiveness Reflecting regularly on the strengths & weaknesses of the institution Asking fundamental questions about learning and the conditions for learning Observing & judging performance based on explicit criteria Providing feedback based on those judgments
- 23. Scary Issues AroundAssessment Assuming “accountability” leads to real improvement Getting lost in administrivia Worshipping numbers Obsessing over “objectivity” & “being scientific”
- 24. Every complex questionhas a simple answer… And it’s wrong. H.L. Mencken
- 25. A Sampling ofAssessment Resources/Gatherings The Research and Planning Group for California Community Colleges http://www.rpgroup.org/ Academic Quality Improvement Program http://www.aqip.org/ Western Assessment Conference http://business.fullerton.edu/events/AssessmentConf/ International Assessment & Retention Conference http://www.naspa.org/assessment/index.cfm Partial listing of other assessment conferences http://www.assessmentconferences.com/ WA Assessment, Teaching, & Learning site http://www.sbctc.ctc.edu/college/e_assessment.aspx (annual conference, regular e-newsletter)
- 26.
- 27. Focusing on Assignments as Assessments
- 28. Clarifying ‘Good Work’:Making Implicit & Private Judgments Explicit & Public Review concrete examples of real work Develop shared notions of quality Define what constitutes adequate evidence of quality Discuss learning experiences that help foster the desired performances
- 29. Good assessment tasksare interchangeable with good instructional tasks. Lorrie Shepard “ The role of assessment in a learning culture,” 2000
- 30. The best assessmenttools are the minds of teachers and learners
Editor's Notes
- #2 in a societal and political context where accountability seems to be ascendant, and accreditation falls somewhere in between
- #3 Power corrupts, and PowerPoint corrupts absolutely
- #5 Whole group br ainstorm At its simplest, “The process of gathering information to make decisions”
- #6 Emhasis in standards is on 2 distinct approaches--institutional effectiveness & student learning outcomes—but more critical distinction derives from underlying worldviews, reflecting more than just particular programs or strategies people use— what’s competing are fundamental, underlying worldviews that shape and inform the ways people approach this work—sometimes at a conscious level, but often not. For an excellent in-depth elaboration of some of the historical roots and present-day consequences of these worldviews, see Lorrie Shepard’s article in the October, 2000 Educational Researcher (vol. 29, #7, p. 4-14) entitled ‘The role of assessment in a learning culture.” Need rigorous, useful peer review process that links or balances accountability & assessment perspectives Challenge is involving thoughtful peers, producing work meaningful for campus and translatable to external audiences
- #7 EXCELLENCE: Performance, not prestige Results, not resources/reputation Incorporate that notion of excellence into peer review processes, I.e., accreditation ACCOUNTABILITY Professional accountability, not bureaucratic accountability—a kind of accountability to ourselves, our peers, our students that involves continual self-reflection and learning—the kind most of us are already doing CAMPUS COMMUNITY Overcome traditional isolation of academic work and higher education culture Important to struggle to define collectively—which is not at all easy for us to find the time for, let alone to do—the goals/ends we have in common for our students, what we think the institution should stand for and be about
- #9 (individually, then small groups) Think about the intro course for your discipline or program. What core learning goals do you have for students who take and complete that course?
- #10 Traditional question: what do we teach? What do we cover? It’s important to understand that the conversations/debates we have about these areas are informed and shaped by our operating assumptions/personal theories about the more fundamental issues of knowledge , learning , assessment We need to examine these often tacit assumptions and re-frame the questions—
- #11 Knowledge=“Competence with respect to valued enterprises”-- Etienne Wenger, Communities of Practice Understanding it requires using it Using it involves both changing the user’s view of the world & adopting the belief system of the culture in which it is used
- #12 Discussion—see handout
- #13 Traditional question: How do we grade? It’s important to understand that the conversations/debates we have about these areas are informed and shaped by our operating assumptions/personal theories about the more fundamental issues of knowledge , learning , evidence Evidence involves: Degrees of certainty/uncertainty Subjective interpretations, even about “objective” data Meaning-making and judgment processes for teachers, students, and assessors
- #15 Assessment: the process of gathering evidence to make judgments toward the goal of improvement 1) Initial reaction to calls for assessment is often to do what’s easiest, cheapest, quickest way to get “them” (accreditors, legislators, administrators, ______) off “our” backs—this approach rarely serves the “assessment as learning” framework very well 2) at least as much as the precision of the evidence you gather What you do with and how you use the evidence or data you gather is far more important than the “rigor” of the evidence—rigor is fine, but someone will always want to argue about the evidence anyway, so make sure you spend time and energy figuring out how to have constructive arguments rather than rigorous reports that get ignored 3) to assess performance over time Multiple perspectives tend to produce higher-quality judgments about evidence and what to do about it—you rarely have complete information, but multiple data sources provide a clearer picture [2-minute table group QUESTION INTERLUDE]
- #17 Traditional question: How do we teach it? It’s important to understand that the conversations/debates we have about these areas are informed and shaped by our operating assumptions/personal theories about the more fundamental issues of knowledge , learning , assessment More often than not these are tacit assumptions; if we can make them visible we can get clearer about our understandings and their relationship to current scholarship on teaching and learning. LEARNING: …Being able to repeat facts and plug numbers into formulae to get the right answers is handy, even essential. But it is not what education is fundamentally about… Learning should be about changing the ways in which learners understand, or experience, or conceptualize the world around them … (Paul Ramsden, 1988, 1992)
- #18 We laugh at this, in part because it’s a laugh of recognition, isn’t it? And underneath the exaggeration and humor there’s more truth to this perspective than we care to admit most of the time—not to mention the fact that it reflects fairly common perceptions of our business.
- #19 Notion of “cognitive apprenticeship”-- Enculturating student into specific “communities of practice ” Meaningful problems, tasks situated in specific domains Include at least some of the “messy” attributes of real-world problems Preferably linked to student interests Learning as qualitative changes in how people interpret core concepts, reason about key content areas
- #21 COURSE: Grading/evaluation processes regarding core concepts/themes, skills and ways of reasoning Classroom assessment efforts (Cross/Angelo) DEPT./PROGRAM Faculty consensus around shared goals & essential student learning outcomes/abilities Adequate (not minimum) performance standards INSTITUTION IE approaches General education abilities Student tracking processes/indicators
- #22 Pierce: common rubrics Green River: community rubrics Big Bend, Spokane, Spokane Falls, Yakima Valley, … Capstone courses (LCC) Artifact collection (Johnson County CC)
- #23 Student learning outcomes focus Institutional effectiveness focus Both are driven by critical underlying processes reflected by Alverno definition of assessment-as-learning
- #24 Cf Peter Ewell’s plenary speech to AAHE Assessment Forum, June 2002 (http://www.aahe.org/assessment/2002/ ) No clear evidence that efforts to “make people accountable” to some outside group actually improves quality Authentic assessment critical at all levels—classroom, program and institution Nancy Cochran: “We have been overzealous in our application of a useful but limited technique [statistical analysis in social sciences]…” “ An Essay on the Inappropriateness of Program Evaluation,” 1980 Serious potential unintended consequences w/ quantified assessments, including the fact numbers have an air of authenticity about them even if they are inaccurate or false, and the way quantified standards can become ends in themselves and thus distort behavior in unproductive ways Peter Johnston: need to question the assumption “that it is possible to obtain an objective, valid unbiased empirical description of human learning activity and that it will serve educational stakeholders to do so …” “ Constructive Evaluation and the Improvement of Teaching and Learning,” 1989 We can’t treat the world as it were simply sitting there available to our assessment of it—our actions and our language affect the environment in the process of assessment, and if we want to understand our world, let alone change it, we need to attend carefully to those effects The point is that no matter how we go about educational evaluation, it involves interpretation, and we simply need to be conscious of that and figure out where possible to articulate and improve the judgment processes involved.
- #27 Encouraging and supporting this kind of work among your peers, in your department, across the campus, …??? Role of accreditation in terms of helping and/or hindering this work?
- #29 Collaborative process with colleagues 2) in student performances or products 3) what’s good enough? What meets the standards you’ve set for your students--and do they know what they are??? 4) Focus on brainstorming ways to help students achieve the learning goals you have for them (scholarship of teaching, and of assessment)