A presentation given at the Association of American Colleges and Universities Annual Meeting, January 20, 2010 in Washington, DC.

1. Deliberative Assessment with E-Portfolios Darren Cambridge Association of American Colleges and Universities Annual Meeting January 20, 2010, Washington, DC

2. Tensions in Portfolio Use

4. Rubrics

6. Through study in the sciences and mathematics, social sciences, humanities, histories, languages, and the arts

7. Intellectual and Practical Skills

8. Inquiry and analysis

9. Critical and creative thinking

10. Written and oral communication

11. Quantitative literacy

12. Information literacy

13. Teamwork and problem solving

14. Personal and Social Responsibility

15. Civic knowledge and engagement—local and global

16. Intercultural knowledge and competence

17. Ethical reasoning and action

18. Foundations and skills for lifelong learning

19. Integrative Learning

21. Reflection as an End of Its Own Dewey: Rigorous analytical thinking Schön et. al.: Key to professional practice Keegan: Essential to challenges of adult life Boud: United cognitive and affective Mezirowet. al.: Understanding and challenging domination

24. Clear learning outcomes Opportunities for students to achieve them Assessment of that achievement Use of the results for improvement

25. Useful Cost-effective Reasonably accurate and truthful Multiple Direct Planned, organized, systematized and sustained Kinds of direct evidence Portfolios of student work Student reflections on their values, attitudes, and beliefs, if developing those are intended outcomes of the course or program

27. Virginia Tech

28. Sometimes they align …

29. Deliberative Assessment Assessment as a means for participation in collective decision making Deliberative democracy Decision making Legitimation

30. Principles of Deliberation Publicity Deliberative system which informs and holds accountable Inclusiveness All impacted by decisions can participate Reasonableness Economy of moral objections Respect for reasonable disagreement Provisionality Openess to changing positions and decisions

31. Scholarship as Deliberative System Publicity Deliberative system which informs and holds accountable Inclusiveness All impacted by decisions can participate Reasonableness Economy of moral objections Respect for reasonable disagreement Provisionality Openess to changing positions and decisions

32. A New Role for Competencies Standardized: Matching performance to a pre-defined set of outcomes Deliberative: Capture standards all stakeholders value as enacted in practice and examining alignment of both student and programmatic performance

33. Competencies in Organizational Learning Standardized: Articulating expectations to students Deliberative: Means for mutually accountable connection between individual and organizational learning Boundary objects: “Boundary objects are objects that are both plastic enough to adapt to local needs and constraints of the several parties employing them, yet robust enough to maintain a common identity across sites” (Leigh Star 1989)

34. Deliberative Assessment Standardized: Objectivist/utilitarian Expressive: Subjectivist/intuitionist (Gray 2002) Deliberative assessment Learning complex and situated Judgment based in embodied expertise Students as “authoritative informants about their own learning” (Yancey 1998) Institutional values and outcomes the result of deliberation based these sources of expertise

35. Deliberative Assessment Student are privileged informants about their own learning Evidence of learning needs to come from multiple contexts, and the relationships between them need to be articulated Assessment should be a deliberative process that makes programs more responsive to all stakeholders

37. Through study in the sciences and mathematics, social sciences, humanities, histories, languages, and the arts

38. Intellectual and Practical Skills

39. Inquiry and analysis

40. Critical and creative thinking

41. Written and oral communication

42. Quantitative literacy

43. Information literacy

44. Teamwork and problem solving

45. Personal and Social Responsibility

46. Civic knowledge and engagement—local and global

47. Intercultural knowledge and competence

48. Ethical reasoning and action

49. Foundations and skills for lifelong learning

50. Integrative Learning

52. Ineffable  Essentially Contested Ineffable outcomes: Things we all think are important but don’t think we can measure E.g., ethics, leadership, social responsibility Essentially contested concept (Gallie, 1956) More optimal development because of contestation

53. Eportfolios for Contested Outcomes Makes multiple understands of outcomes visible Requires reasoning to be articulated Grounds understanding in evidence and experience Puts multiple positions into conversation

54. Eportfolios for Contested Outcomes Measurable learning outcome: Ability to articulate a reasoned stance based on evidence Makes multiple understandings of outcomes visible Requires reasoning to be articulated Grounds understanding in evidence and experience Puts multiple positions into conversation

55. NCC Competencies Communication Critical Thinking Strategic Problem Solving Valuing Group Interaction Global Understanding Effective Citizenship Aesthetic Awareness Information Technology

56. NCC Graduation Portfolio No predefined standard for what constitutes satisfactory performance in each competency Students exposed to (and assessed with) many models and standards through coursework and experiential learning Students redefine each competency, beginning with “official definition” Synthesizing multiple perspectives Integrating evidence from own experience Taking ownership and planning for the future

61. Three curricula Kathleen Yancey, Teaching Literature as Reflective Practice

62. Academics as Test of Self We intended for curricular content to be an central source of evidence and ideas and strategies, but it didn’t show up this way Class work functioned as A demonstration of character virtues An experience A goal putting aspiration towards those virtues in action

63. Complicating Evidence Link between evidence and reflection distinguishes eportfolios and other digital means for supporting reflective learning Managing information about knowledge, skills, abilities and experiences “Evidence” is the documents included in a portfolio on which the author reflects Use of evidence in practice is more complex than the eportfolio literature often acknowledges

64. Evidence in Reflection Research at Alverno College suggests that, as students become more skilled at reflection, they Draw on analysis of their own experiences rather than appealing to external authorities Reference a wider range of activities and artifacts Research deals only with the content of the reflections, not the evidence itself

65. Types of Evidence Types of evidence in science portfolios (Collins, 1992): Artifacts Attestations Reproductions Mixes analytically distinct dimensions, such as characteristics of evidence and purpose

66. Project Background Portfolio contexts: Integrative approach to learning with specific attention to classroom-based, experiential, and co-curricular learning NCC and portfolio-based assessment Intentional collaboration with University Life Small data sets over two cohorts (spring ’07; spring ’08); additional cohort beginning in fall ’08 Member of cohort 3 of the Inter/National Coalition for Electronic Portfolio Research (I/NCEPR)

67. I/NCEPR Institutional research teams examining the impact of electronic portfolio practice on learning 50 institutions in five cohorts Third cohort focuses on student affairs -academic affairs collaboration US, Canada, England, Scotland, Netherlands Book to be published by Stylus in November More information on website: ncepr.org.

68. An Emergent Typology of Use of Evidence in ePortfolios Characteristics of item used as evidence Agency Media Purpose of incorporating evidence Rhetorical Function Object Characteristics of associated learning activities Sponsorship Participation

69. Matches and Mismatches Reflective description of evidence Content of evidence Local – site of specific evidence use Global – the whole portfolio Match and mismatches yield more sophisticated understanding and resources for supporting portfolio authors

70. An Example: Richard Zepp’s ePortfolio

73. Electronic Portfolios 2.0: Emergent Findings and Shared Questions Collection of 24 chapters detailing research from cohorts I, II, and III of the Inter/National Coalition for Electronic Portfolio Research Published by Stylus, March 2009