arguing for the need to be true to the notion of "authentic" assessment in pursuing institutional assessment efforts in higher education, despite (or perhaps because of) the strong and growing pressure around a narrow vision of accountability for higher education

1. Complex Questions & ‘Kitchen Stories’: Pursuing Authentic Institutional Assessment in an Age of Accountability William S. Moore, Ph.D. Coordinator, Assessment, Teaching & Learning WA State Board for Community & Technical Colleges bmoore@sbctc.ctc.edu 360-704-4346 National Student Affairs Assessment & Retention Conference June 2005 Atlanta, GA

2. Adapted from “Dilbert” & Scott Adams As you can clearly see on slide 397… Oh, no—not another case of PowerPoint poisoning!!!

4. Age of Accountability

6. Accounting or Accountability? Assessment as learning can be reconciled with assessment for accountability only if we fundamentally rethink accountability…[as] a deeply human enterprise that depends on open sharing of information and continuing conversations… Lorna Earl & Paul LeMahieu, 1997

8. Role of Underlying Beliefs in Accountability The question of how we exhibit educational accountability hinges on beliefs about what constitutes knowledge and how it can be demonstrated. Paul Theobald & Ed Mills, 1995

9. Assessment grows out of public discussions of the dimensions of excellence , exchanging informed clinical judgments and uncovering overlaps of or tensions between opinions. Its purpose is not accountability, but learning. Dennie Palmer Wolf & Nancy Pistone, 1991

10. It’s Learning That Matters

12. “ The major aim of the freshman year should be to win the student to the intellectual enterprise.” Nevitt Sanford, The Modern American College , 1962

13. Learning as Transforming Understanding … Being able to repeat facts and plug numbers into formulae to get the right answers is handy, even essential. But it is not what education is fundamentally about… Learning should be about changing the ways in which learners understand, or experience, or conceptualize the world around them … Paul Ramsden

14. Hope & Loss: Real Learning Takes Courage … It may be a great joy to discover a new and more complex way of thinking and seeing, but what do we do about the old simple world? What do we do about the hopes that we had invested and experienced in those simpler terms? When we leave those terms behind, are we to leave hope, too? Bill Perry, 1978 “ Sharing in the cost of growth”

15. Exploring Assumptions and Complex Questions

16. “ Assessment is…”

18. Assessment as Learning Observing & judging performance based on explicit criteria Providing feedback based on those judgments Reflecting regularly on the strengths & weaknesses of the institution Asking fundamental questions about learning and the conditions for learning

19. Assessment as Meaning-Making Ultimately, however, the real value of science may lie in uncertainty… Scientific knowledge is a body of statements of varying degrees of certainty… We scientists take it for granted that it is perfectly consistent to be unsure, that it is possible to live and not to know. But I don’t know whether everyone realizes that this is true… Richard Feynman, 1970 from “The Value of Science”

20. Assessment as Dialogue What can we truly know about our students—and about each other—without some dialogue and human connection?

25. Every complex question has a simple answer… … and it’s wrong. H.L. Mencken

26. Authentic Institutional Assessment

29. Collective Sense of Purpose Assessment...requires us to work together, and to do unfamiliar things like setting common goals and standards, devising methods of assessment, interpreting the results, and using them to improve and coordinate our teaching. [It thus] possesses all the appeal and efficiency of committee work, in particular the kind visited upon us by administrators. Robert Holyer, Change , Sept./Oct., 1998

31. Shared Notions of Quality … It is through developing a shared language about student work and shared set of values and criteria that assessment can affect the larger educational system with the power reformers desire and in the direction they advocate... Karen Sheingold & John Frederiksen, “ Linking Assessment with Reform,” ETS, 1994

33. Public Trust Much psychometric evaluation is ostensibly in the service of public trust, but there are more effective ways of establishing public trust than through the use of psychometrics . Peter Johnston, 1989

35. In Conclusion… I know I have not succeeded in answering all of your questions. Indeed, it feels as though I have not completely answered ANY of your questions. The answers I have given often serve simply to raise a new set of questions. In some ways it seems clear that you are as confused as ever. However, you’ll be happy to know that I believe firmly that you are now confused on a much higher level and about much more important things than before! Adapted from John McMahon, Seattle Central Community College

36. Key Issues/ Lingering Questions?