Recommended
Recommended
More Related Content
Similar to Linking up Innovations in Assessment: Eportfolios, Open Badges, and Learning Analytics
Similar to Linking up Innovations in Assessment: Eportfolios, Open Badges, and Learning Analytics (20)
More from dcambrid
More from dcambrid (20)
Recently uploaded
Recently uploaded (20)
Linking up Innovations in Assessment: Eportfolios, Open Badges, and Learning Analytics
- 1. LINKING UP INNOVATIONS IN ASSESSMENT: EPORTFOLIOS, OPEN BADGES, AND LEARNING ANALYTICS DARREN CAMBRIDGE @ DCAMBRID AMERICAN INSTITUTES FOR RESEARCH EPIC 2013, LONDON, 10 JULY 2013
- 4. SCALE OF REPRESENTATION Macro Eportfolios Mezo Digital Badges Micro Learning Analytics
- 5. COMMON GOALS Enable decisions more grounded in evidence of learning and identity (with the assumption that they are inextricable) Make that evidence richer and more diverse (with the assumption decisions will be more valid)
- 7. FOUR PROPOSITIONS 1. Reflection on pieces of evidence within an eportfolio and on the eportfolio as a whole provides information for assessment that isn't available by other means. 2. For meaningful assessment, interaction of pieces of evidence within an eportfolio is more important than single pieces of evidence. 3. The material practice of eportfolio composition generates distinctive knowledge about learning. 4. Eportfolios enable meaningful comparison of student learning without standardization.
- 8. Proposition Principle Reflection Learners are key interpreters of evidence of learning and performance Interaction Focus on relationships rather than individual nodes Materiality Pay attention to embodied consequences Comparison Promote standards for dialogic interpretation of evidence rather than for evidence itself
- 9. LEARNERS AS INTERPRETERS: CURATING BADGE BACKPACKS
- 10. LEARNERS AS INTERPRETERS: PEER AWARDED BADGES
- 12. LEARNERS AS INTERPRETERS: PERSONAL ANALYTICS
- 13. RELATIONSHIPS NOT NODES: EMERGENT PROFESSIONAL PROFILES Pre-defined links to standards or activities Stipulated Individual patterns based on observed success Emergent Group patterns based on observed success Collective
- 14. RELATIONSHIPS NOT NODES: SOCIAL LEARNING NETWORK ANALYSIS
- 15. RELATIONSHIPS NOT NODES: EPISTEMIC NETWORK ANALYSIS STEM professions defined by SKIVE Skills, Knowledge, Values, Identity, Epistemology
- 16. EMBODIED CONSEQUENCES Digital badges: Focus on audiences as was well as issuers and earners Learning analytics: Research use by actual decision makers and its effects on learners
- 18. Our goal is to develop an understanding of validity that begins with the questions that are being asked; that can develop, analyze, and integrate multiple types of evidence at different levels of scale; that is dynamic in the sense that questions, available evidence, and interpretations can evolve dialectically as inquirers learn from their inquiry; that allows attention to the antecedents and anticipated and actual consequents … and that situates the assessment in the broader context in which it is used. A validity theory can also be construed as the representation of an epistemology—a philosophical stance on the nature and justification of knowledge claims—which entails a philosophy of science. Moss, et.al., “Validity in Educational Assessment”
- 19. STANDARDS FOR INTERPRETIVE DIALOG Hermeneutics and deliberative democracy theory as alternative epistemologies for validity theories Set standards for how we hold discussions about evidence to make decisions rather than about what evidence to consider Badges and learning analytics as heuristics, mediators of meaning making conversations Need tools and techniques for interpretive discussion
- 20. PRINCIPLES OF DELIBERATION Publicity • Deliberative system which informs and holds accountable Inclusiveness • All impacted by decisions can participate Reasonableness • Economy of moral objections • Respect for reasonable disagreement Provisionality • Openess to changing positions and decisions
- 21. BETTER TOGETHER Badges and learning analytics can enrich eportfolios through new types and structures of evidence algorithmically identified patterns Eportfolios offer authenticity for legitimacy and commitment integration for context and meaningfulness direct contribution to learning through compostion
- 22. CONTACT Darren Cambridge American Institutes for Research @dcambrid dcambridge@air.org
Editor's Notes
- Summary of 2003 story identity pathwaysbyrestructuring a set of decisions
- Learning analytics = large volumes of heterogenous data relevant to understanding and predicting learning in form that decision makers in the field can interpret for themselves in as close to real time as possible
- Decisions = not just, whom should I hire? But what do I care about?
- Eportfolio movement needs to constructively engage!! – won’t shape what happens just by rejecting out of hand, need to apply what we know and believe to say how, not if Active debate about role of identity/learning agency Digital learning now as example from corporatized/automated version of “personalized learning” valuing more diverse evidence, seeing identity as importantnot yet a commitment to agency of learners, but learning from e-portfolios practices could helpUsing people’s identity vs. helping people act with theirs Vs. CCSSO innovation network Student voice and choice key to personalizationPersonalization vs. individualization/differentiation (Bray and McCulsky)
- Will explore how the first two principles apply to the three means of representation and analysis and show how benefit from application of the fourth
- Nothing stopping you from giving yourself a badge, but power is in well-definedattestation of another curating to audience as form of interpretation
- Community badges Peer awarded by those who have earned the right Also for mentors
- Access to data first step
- Individual activity/standard: predefined – Piecemeal linking of badges to CEUs or specific standards (like a certificate) or levels within a system (we’ll be doing this for CEM, so don’t mean to trivialize) or predefined set of competencies for a profession vs. Individual profile: based on patterns in actual earning -teachers who end up being successful in this context tend to have this set of badges (with certain commonalities, from certain issuers and endorsers), and here’s how you compare to them and what it says about what learning experiences you might choose nextvs. collaborative profile: not just what they look like for individual teachers, but what they look like for an effective PLC, school, or district --This is what the open badges ecosystem is about! --Notional so far, because adoption isn’t widespread enough yet, but could be not far away for software development
- Learningof the community, health of the social learning space, not just average achievement of individual learners (e.g., activity that leads to more sustained and broadbased engagement) Concepts and their relationships, not just key words (e.g., “use” key to favorable and unfavorable but not “fun”) Way to direct community managers’ attention to decide what meaning, if any, there is rather than determining See papers on ConnectedEducators.org
- Professions defined by “epistemic frames”: SKIVE – skills, knowledge, identity, values, epistemology Coding chat transcripts from simulation for markers of each coaching suggestions
- Add picture here
- Absurdity of standarddescriptive statistics – use statistician jokeVs. Shafer – “a relational approach to psychometrics” PatriciaCross distinction between SoTL and traditional educational research - we care about specific people in this specific context and all of them (and about the character of the whole)
- Badges as mediators of meaning making conversations vs. determinants LA output as heuristic for contextualized meaning making through dialog rather than automated judgmentDevelop tools for interpretative discussion, not just earning/awarding/displaying or capture/visualization/prediction Hermeneutics and deliberative democracy theory as alternative epistemologies for validity theories
- Apply equally to e-portfolios and badges and analytics
- authentic representation – interpretation of evidence in relationship to articulation self in relationship to shared horizons of intelligibility (right term – use Guignon quote?)Provide legitimacy, create commitment • integration across levels of evidence and spheres of life (Keegan adult challenge) Accounts for contextBoth richer by virtue of new types and structures of evidence badges and LA offer and potentially meaningful patterns algorithmic analysis can identify Capitalizes on Badges and LA to directly aid learning through composition