Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Workshop powerpoint

465 views

Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Workshop powerpoint

  1. 1. ASSESSING TRANSFER: USING REFLECTION TO EVALUATE TRANSFER OF KNOWLEDGE AT CRITICAL TRANSITIONS IN WRITING PROGRAMSB O B B R O A D , I L L I N O I S S T A T E U N I V E R S I T Y H E I D I K E N A G A , W A Y N E S T A T E U N I V E R S I T Y D A N A D R I S C O L L , O A K L A N D U N I V E R S I T Y J O S E P H P A S Z E K , W A Y N E S T A T E U N I V E R S I T YW E N D Y D U P R E Y , W A Y N E S T A T E U N I V E R S I T Y L I A N E R O B E R T S O N , W I L L I A M P A T E R S O NU N I V E R S I T YG W E N G O R Z E L S K Y , W A Y N E S T A T E U N I V E R S I T Y D A V I D S L O M P , U N I V E R S I T YO F L E T H B R I D G EJ A R E D G R O G A N , W A Y N E S T A T E U N I V E R S I T Y K A R A T A C Z A K , U N I V E R S I T Y O F D E N V E RA D R I E N N E J A N K E N S , W A Y N E S T A T E U N I V E R S I T Y T H O M A S T R I M B L E , W A Y N ES T A T E U N I V E R S I T Y
  2. 2. WORKSHOP OVERVIEW• Defining & assessing transfer• Key principles for measuring transfer• Case studies on assessing transfer• Working Groups • Critical transitions • Developing goals • Developing assessment plansTo access all workshop materials online:http://assessingtransfer.pbworks.com
  3. 3. BASIC DEFINITIONS• Smit (2007) argues that the ability to transfer knowledge is what the term “learning” actually means (p. 130).• The National Research Council (1999) argues that “The ultimate goal of learning is to have access to information for a wide set of purposes—that the learning will in some way transfer to other circumstances” (p. 61).• Other transfer terms: boundary crossing, knowledge building
  4. 4. TRADITIONAL COGNITIVE AND TASK- BASED TRANSFER• Definition: “The application of knowledge learned in one situation to a new situation.”• Research method: “improved performance on tasks” (primarily through experimental design)• Research questions: “Was transfer obtained? What conditions facilitate transfer?” (Loboto 2003)
  5. 5. Jimmy: Each situation is viewed as Jimmy unique and nothing is carried to the next situation. Rhetorical Product and Analysis Analysis of Genre Whitepaper Presidential Analysis in in Business Candidates FYC ClassBriel: Transfers knowledge Brielbetween tasks and buildsher knowledge of analysis.
  6. 6. CONTEXT-BASED TRANSFER (ACTIVITY THEORY) • Actor-Oriented Transfer focuses more on the context of learning and works within the realm of activity theoryDefinition: “The personal construction of relations of similarityacross activities (i.e., seeing situations as the same).”Research method: “Researchers look for the influence of prioractivity on current activity and how actors construe situations assimilar.”Research questions: “What relations of similarity are created?How are they supported by the environment?” (Loboto, 2003)
  7. 7. Briel American Civic Engagement Activity System FYC PoliticalActivity System Voting Activity System BUS300 Activity System University Activity System Jimmy
  8. 8. CONTEXTUAL-DISPOSITIONAL (HYBRID) APPROACHES• Wells and Driscoll (under review) argue that both task- based and contextual approaches are useful, but provide an incomplete picture of transfer.• Rather, they argue it is the intersection of the task, context, and the individual learner’s dispositions. These include: • motivation, self efficacy, help-seeking, willingness to engage in mindful abstraction, developing a metacognitive mindset, beliefs, attitudes etc.• This approach examines the relationship between learner, the task, and the context and is particularly useful to assessment.• Bio-ecological assessment can also fit within a hybrid approach (as described next)
  9. 9. BIO-ECOLOGICAL THEORY OF TRANSFERBronfenbrenner and Morris (2006); Slomp (2012) Dispositions Proximal Processes Resources • Processes through which Individual Transaction learning occurs (within local Demand environment) Characteristics
  10. 10. ASSESSING TRANSFERCHALLENGES AND CONSIDERATIONS
  11. 11. 3 CHALLENGES1. Choosing a theory of transfer that captures a full picture of the factors that support or inhibit transfer.1. Defining/adopting a socially-situated construct model of writing ability.1. Overcoming technocentric limitations in current writing assessment practices
  12. 12. CHALLENGE 1:DEFINING A ROBUST THEORY OF TRANSFER
  13. 13. THEORIES OF TRANSFER• Clearly define how you are conceptualizing transfer: • The theory of transfer you adopt will determine your research/assessment focus.• Carefully consider which theory of transfer you base your assessment on. • Traditional conceptions of transfer have underrepresented what people know and are able to do (Schwartz, Bransford & Sears, 2005).
  14. 14. TASK-BASED CONCEPTION OF TRANSFER Task 2:Task 1: Complete Are students able to draw worksheetComplete on knowledge of comma requiring studentworksheet on rules developed in task 1 to properly insertrules for using to complete task 2? commas on acommas. page of unpunctuated sentences.
  15. 15. CONTEXTUAL THEORIES OF TRANSFERActivity Systems: (Wardle, 2007) Context 2: WritingContext 1:FYC Across the UniversityOrganization Do students perceive that what they learned in FYC has helped them with Students did notskills, process later writing assignments across the generalize fromknowledge, cr university? FYC because theitical How does the curriculum structurereading, subje activity system did influence generalization? not encouragect knowledge them to do so.
  16. 16. CONTEXTUAL THEORIES OF TRANSFER • Transformation and repurposing (Roozen, 2010)Context 1: Context 2:Religious Academicengagements engagements How does prior knowledgePrayer inform current practices? Notejournaling, ver How is prior knowledge taking, outlining, ase- repurposed when applied in nd organizingcopying, serm new contexts? academicon outlining arguments.
  17. 17. BIO-ECOLOGICAL THEORY OF TRANSFER Slomp and Sargent (2009, forthcoming) Context 2: Writing Across theContext 1: FYC University In what ways do characters of the Barriers to transferWriting process developing individual, proximalknowledge, m included: processes, and the ecologicaletacognitive systems in which students learn curriculumknowledge, di shape their capacity to structures, priorscourse generalize, repurpose and/or experience, persocommunity transfer knowledge about writing? nalityknowledge, issues, challenging home environments.
  18. 18. CHALLENGE 2:DEFINING A CONSTRUCT MODEL
  19. 19. CONSTRUCT VALIDITY• The construct writing ability is defined through the lenses of developmental theories (Camp 2012): Syntactic Maturity = Complexity of syntactic constructions Stage models of = Cognitive maturation development Socially situated theories of = Discourse community writing knowledge & metacognitive knowledge
  20. 20. CHALLENGE 3:OVERCOMING TECHNOCENTRIC LIMITATIONS
  21. 21. METHODOLOGICAL CONSIDERATIONS• Writing assessment has traditionally been limited by its technocentric orientation (Huot 2002): • Emphasis on achieving high degrees of reliability • Constrained by reliance on current technologies of assessment
  22. 22. METHODOLOGICAL CONSIDERATIONS• Assessment-as-research (Huot, 2002): • Focus on defining information needs • Choose assessment methodologies that help you achieve those information needs. • Shift away from technocentric views of reliability toward a rhetorical orientation (Parkes, 2007). • Emphasis on validity
  23. 23. METHODOLOGICAL CONSIDERATIONS
  24. 24. Table 1 Overview of Ecological Assessment Design in Wardle and Roozen (2012) Assessment Focus MethodECOLOGICAL FYC Assessment Understand student performance and Student writing portfolios. Pre and post surveys ofASSESSMENT experience in FYC. FYC courses.DESIGN Ethnographic Assessment Develop a broad picture of students as literate Longitudinal case studies. learners.Wardle and Roozen Identify institutional(2012) structures that support or limit transfer. Writing Center Understand how tutoring Statistical analysis of user Tutorial Assessment supports transfer. data. Tutoring in the Understand how tutoring Class portfolios. Disciplines program is supporting Document analysis. Assessment student development. Observations of tutoring sessions. Interviews with students and teachers. Assessment of Understand how programs Development and Writing in Various are supporting student assessment of writing Majors development. related outcomes in the majors. General Education Assess student Eportfolios developed in Program Assessment development across their FYC and carried on undergraduate program. through undergraduate program.
  25. 25. BIO-ECOLOGICALASSESSMENTDESIGN
  26. 26. METACOGNITION, TRANSFER, AND A NEW RESEARCH PARADIGM
  27. 27. TWO TYPES OF TRANSFER High Road Low Road• Results from mindful abstraction • Results from extended• Can occur quickly, without long- practice term practices • Spontaneous, automatic, wit• Example: applying the “count to h little need for reflective ten” rule learned in childhood to thinking inhibit tantrums during adulthood to prevent impulse buying • Example: driving a car to• Promotes greater driving a truck understanding, reflective • Increased speed and evaluation, and conscious efficiency adaptation of previously learned • Decreased long-term concepts and skills memory and analytic - Salomon and Perkins (1989) reflection • Potential for negative transfer
  28. 28. MINDFUL ABSTRACTIONAbstraction Mindful Abstraction• Identifying key qualities, • Using metacognitive attributes, or patterns thinking to decontextualize• Decontextualizing information to construct information and re- principles, patterns, strategi representing it as a set of es, or procedures principles or schemasMindfulness -- Salomon and Perkins (1989)• Thinking guided by metacognition and conscious reflection on target task, context, known strategies, and potential adaptations
  29. 29. PRINCIPLES FOR PROMOTING TRANSFER THROUGH METACOGNITION• Requiring students to actively monitor their learning• Providing feedback on students’ use of new knowledge• Showing contrasting cases to highlight key features• Foregrounding the transfer potential of new knowledge• Teaching new knowledge in multiple contexts• Moving from specific to general levels• Helping students abstract principles• Balancing specific examples with general principles - --National Research Council (1999)
  30. 30. PROMOTING TRANSFER THROUGH REFLECTIONReflection – both a theory and a practice:• From the work of Schon – on reflective practitioners• From the work of Yancey – students as agents in their own learning process
  31. 31. PROMOTING TRANSFER THROUGH REFLECTIONReflection’s Connection to Transfer: Our Starting Point• Significant research on each separately• Absence of research that explicitly explores reflection’s connection to transfer• Beaufort’s knowledge domains - reflection discussed as important for metacognition but not explicitly pursued• Schon and Yancey - reflection both theory/practice
  32. 32. PROMOTING TRANSFER THROUGH REFLECTIONReflection’s Relationship to Transfer:• Kara’s Research: questions whether or not reflection is one of the vehicles by which students transfer knowledge and practices of writing to other academic writing situations.• Liane’s Research: questions which content transfers effectively and how reflection as a reiterative practice fosters the transfer of that content.
  33. 33. PROMOTING TRANSFER THROUGH REFLECTIONDefinition of Reflection:• We define as: systematic, explicit, intentional• Both an intellectual act and a physical act • aligns with Perkins and Salomon’s claim that “conditions of a classroom affect transfer” • aligns with the notion of mindful abstraction – active self- monitoring arouses mindfulness• Students’ ability to monitor their own thinking processes is what leads to mindful abstraction – alertness to the activity in which they are engaged
  34. 34. PROMOTING TRANSFER THROUGH REFLECTIONPractice of Reflection:• We use reflection as: reiterative practice • by which students learn to define and apply their own theory of writing as a way to foster transfer of knowledge and practices from one academic writing situation to another.• Theory of Writing – main reflective practice for our FYC course • Students create a framework of writing knowledge • Students begin to develop metacognitive ability
  35. 35. PROMOTING TRANSFER THROUGH REFLECTIONOur Theory of Writing Component:• Systematic, explicit, intentional • Explicitly encourages transfer • Asks students to be mindful about what they are learning • Reiterative assignment feature - ten writing assignments related to theory of writing throughout the semester• Combines learning about writing theory, through a set of key terms and through reading reflective theory, with the practice of systematic, explicit, intentional reflection
  36. 36. PROMOTING TRANSFER THROUGH REFLECTION1. Begins As 3. Ends As• Practice more than • Metacognitive thinking; theory Abstraction• Not mindful • Development of a theory of• No abstraction, direct writing knowledge and application practice2. Progresses Toward• Increased active self- 4. Continues As monitoring • Application of knowledge• Key terms understood as and practice in new writing writing concepts contexts• Mindfulness develops
  37. 37. PROMOTING TRANSFER THROUGH REFLECTIONSample Reflective Activities:• Early Guided Reflection• Reiterative Reflection
  38. 38. MOTIVATION AND DISPOSITIONS Factors Teaching Approaches• Task difficulty level • Devising challenging• Perceived relevance but do-able• Performance vs. assignments learning orientation • Designing tasks that -- National Research Council (1999) demonstrate relevance • Providing social support for risk-taking
  39. 39. DEVELOPING THE NEW RESEARCH PARADIGM: INTEGRATING INDIVIDUAL, SOCIAL, AND DEVELOPMENTAL ASPECTSPreparing for Future Learning PFL Enables Writing Studies (PFL) Researchers to: • Evaluate students’• Focusing on Interpretive metacognition in relation to Knowledge writing studies conceptual• Evaluating Interpretive and procedural knowledge Knowledge • Evaluate how students use this knowledge to learn• Incorporating opportunities about unfamiliar genres and for learning into assessment rhetorical situations• Focusing on both “transfer • Triangulate data from in” and “transfer out” assessments of individual -- Schwartz, Bransford, and Sears (2005) factors like dispositions and motivations and of social factors like curriculum
  40. 40. DEVELOPING THE NEW RESEARCH PARADIGM: INTEGRATING INDIVIDUAL, SOCIAL, AND DEVELOPMENTAL ASPECTS• Pre- and post-semester surveys evaluating students’ motivation and dispositions related to writing instruction• Text-based interviews that ask students to describe their drafting choices, particularly the conceptual and procedural knowledge they used• Textual analyses comparing discourse features of students’ reflective writing with those of their academic writing• Textual analyses of students’ reflective writing on their uses of conceptual and procedural knowledge about writing.
  41. 41. • Tasks used and knowledge required • Learners’ priorASSESSINGTRANSFER experiences andDevelop a dispositionshybrid, locally focusedmodel that takes intoaccount all of the • Classroom and curricular contextsfactors affectingtransfer.
  42. 42. REFERENCESDriscoll, D. and Wells, J. (under review). Toward a Dispositional Model of Writing Transfer: The Impact of the Individual Learner.Loboto, J. (2003). How Design Experiments Can Inform a Rethinking of Transfer and Vice Versa. Educational Researcher, 32(1), 17-20.National, R. C. (1999). How people learn: brain, mind, experience, and school. . Washington D.C.: National Academy Press.Royer, J. M., Mestre, J. P., & Dufresne, R. J. (2005). Introduction: Framing the transfer problem. . Greenwich, CT: Information Age Publishing.Salomon. D. N., and Perkins, G. S. a (1989). Rocky Roads to Transfer: Rethinking the Mechanisms of a Neglected Phenomenon. Educational Psychologist, 24(2), 113-142.Smit, D. (2007). The End of Composition studies. Carbondale, IL: Southern Illinois University Press.Tuomi-Grohn, T., & Engestrom, Yrjo (2003). Between school and work : new perspectives on transfer and boundary-crossing (1st ed.). . Boston, MA: Pergamon.

×