Dynamic assessment of academic writing: macro-Theme and hyper-Theme


Published on

  • @wrossg Dear Ross, you are right that DA is not simply about assessment but it is about integrating assessment with teaching and learning. Learning is at the heart of DA. I would like to clarify that scaffolding is an aspect of DA. It comes into play when ZPD (Zone of Proximal Development) is considered in relation to a learner - the level of support that is needed in order to move the learner to the next level. For details, you may want to see my article published in Assessing Writing. A pre-print copy is available from my University's research repository: http://oro.open.ac.uk/31269/
    Are you sure you want to  Yes  No
    Your message goes here
  • I am very interested in learning more about DA. A project I am involved in with partners in Ethiopia is looking at ways to improve the teaching of academic writing, and a number of action researchers over there are looking at methods of feedback. If I am not mistaken, DA is really about teaching rather than assessment, and has much in common with 'scaffolding'. Could you comment on this?
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Ask the question: IS anyone familiar with the term dynamic assessment? Any one researching writing assessment?
  • Give a brief summary of the talk – go through bullet points.
    Part of a larger study.
  • As an EAP tutor, Motivated by the fact that students need various types of support in higher education for. their sustainable writing development.
    Since DA targets individual learner’s needs, it may help with resolving challenges in academic writing. But it was essential to investigate how it does that.
    This paper focuses on macro-Themes and hyper-Themes in student assignments
  • Focus on ‘promise’ of a writer – the potential of a learner. Interested in the future, not in the past!
  • Some Assumptions of Vygotskian SCT: (1) learning always takes place in a social and historical context; (2) learning is a collaborative enterprise; (3) higher mental activity is always mediated through psychological tools such as human artifacts, language and signs; (4) learning leads to development; (5) theoretical learning is more powerful than empirical learning to mediate knowledge.
  • Go through points
  • Basic tenets of SFL: (1) language is a resource for making meaning in a particular social context; (2) language is used to serve multiple functions of human activity; (3) language structure varies for each function; (4) basic unit analysis for research is text, not sentence; etc.
    These variables are represented by the language used in a text
  • In academic writing SFL has been shown to be useful in the development of assessment criteria (e.g., Rose, Rose, Farrington, & Page, 2008) and as a diagnostic tool (e.g., Bonanno & Jones, 2007). In this respect, it appears that SFL provides useful analytical tools for analysing students’ texts and deciding whether their texts demonstrate (or otherwise) the textual features expected in a certain discipline.
    Whether used for diagnostic purposes or measuring achievement, the SFL-based genre approach is instrumental for defining assessment criteria for student writing and examining assessment texts to see the textual features of a genre in a discipline. There does not appear to be a similarly strong focus on systematic procedures in formative or summative assessment at higher education level.
  • Again, focus on tutor-mediation
  • Dynamic assessment of academic writing: macro-Theme and hyper-Theme

    1. 1. Dynamic assessment of academic writing: macro- Themes and hyper- Themes Prithvi Shrestha OpenELT, Department of Languages The Open University, UK
    2. 2. What I want to talk about Why this study? Theoretical frameworks: DA and Systemic Functional Linguistics (SFL) Research methodology Data collection Analysis and findings Conclusion Questions and suggestions
    3. 3. Why this study?  Lack of DA research in academic writing assessment (see Poehner 2008 for a review) and call for more targeted and ‘dialogic’ tutor support (i.e., feedback) to students with their assignments (e.g., York 2003; Lillis 2006)  An academic writing tutor’s search for sustainable writing development support This study explored:  Do Dynamic Assessment (DA) procedures enhance students’ writing development?
    4. 4. Defining academic writing and academic writing assessment Academic writing: academic texts expected to be written by students in a particular discipline in higher education (HE), which entails having the epistemological, linguistic and socio-cultural knowledge required in order to become a part of that disciplinary community. Academic writing assessment: the range of procedures used to “describe the promise and limitations of a writer” working in a particular rhetorical, linguistic and socio-cultural context (emphasis added, Huot 2003, p. 107)
    5. 5. Theoretical frameworks (1): DA Dynamic Assessment (DA) driven by Vygotskian (1978) sociocultural theory of learning Definition of DA: an “approach to understanding individual differences and their implications for instruction … [that] embeds intervention within the assessment procedure” (Lidz and Gindis, 2003 p.99). Goal of DA: identify learners’ Zone of Proximal Development (ZPD) and actual ability and help them to move to the next level of zpd; focus on process and not on product; concerned with future rather than the past Two key concepts in DA: (1) zone of proximal development (ZPD) and (2) mediation
    6. 6. Theoretical frameworks (1): DA (contd.) ZPD: “the distance between the actual development level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance or in collaboration with more capable peers” (Vygotsky 1978, p. 86). Mediation: making sense of the world around us via human artefacts, symbolic tools and other humans (Kozulin, 2003).
    7. 7. Dynamic Assessment (DA) in the literature General education (see review in Haywood and Lidz 2007) Language education (e.g., Poehner 2005; Anton 2009; Oskoz 2005) – all in the context of foreign/ second language in a face-to-face situation
    8. 8. Theoretical frameworks (2): genre theory based on Systemic Functional Linguistics Genre theory based on Systemic Functional Linguistics (SFL) as developed by Halliday and his colleagues (e.g., Halliday and Martin 1993) Definition of genre: ‘staged, goal-oriented social processes’ (Martin 1997) Texts/ genres have different functions in different social contexts (e.g., narration, argumentation). Such functions/ purposes are realised through register variables - field (subject matter), tenor (reader-writer relationship) and mode (medium of communication).
    9. 9. SFL genre theory and aca- demic writing assessment Heavy focus on textual analysis (i.e., textual features) Diagnostic tool (e.g., Bonnano & Jones 2007) Criteria for assessment (e.g., Rose, Rose, Farrington, & Page, 2008) To date no strong focus on SFL-oriented systematic procedures in formative or summative assessment in higher education
    10. 10. Research context Academic writing course in open and distance learning Majority of students native speakers of English (ca. 400 students) Tutor-student interaction via online forum and emails Researcher tutoring one group (ca. 20 students) – two students volunteered to participate
    11. 11. Data collection methods Dynamic assessment ‘sessions’ over six months (i.e., tutor-student interaction, various drafts of students’ assessment texts) – the assignment tasks required students to apply a business study framework to a business situation/ problem (i.e., case study analysis) Tutor mediation in these DA sessions followed the basic principles proposed by Haywood and Lidz (2007, p. 42). Focus of the present presentation: macro-Themes and hyper-Themes of DA1 and DA2 two drafts each
    12. 12. Data analysis An analysis of macro-Themes and hyper-Themes in students’ assessment texts following SFL genre theory (e.g., Martin & Rose 2007) macro-Theme: refers to the introductory paragraph in an essay or analysis which predicts what the text will be about (Martin 1993). Hyper-Theme: is defined as ‘a clause (or combination of clauses) predicting a pattern of clause Themes constituting a text’s method of development’ (Martin 1993, p. 245).
    13. 13. Findings: macro-Themes (1) Michelle’s (pseudonym) macro-Themes DA1 draft 1: lengthy background information; lacks sufficient orientation for the reader DA1 final draft: same except a few minor changes DA2 first and final drafts: concise and focused; clearly indicates how the analysis will develop following the STEP framework (classification criteria)
    14. 14. Findings: macro-Themes (2) Natasha’s (pseudonym) macro-Themes DA1 draft 1: some orientation but no clear classification criteria (i.e., STEP factors) DA1 final draft: clearly states what the analysis will do; clear classification criteria DA2 first and final drafts: similar to DA1 final; clearly sets out how the analysis will be developed; classification criteria explicit; final version with more business concepts and sentences better connected
    15. 15. Findings: hyper-Themes (1) Michelle’s hyper-Themes (examples) DA1 draft 1: A major development by Heineken introduced a better tasting non alcoholic beer onto the market. (P4) DA1 final: Technological factors affect Heineken's marketing strategy in one main way. (P4)  DA2 draft 1:Although safety has been one of the main focuses for the safer syringe market, there have been some Technological factors which have influenced the market. (P3) DA2 final: Technological factors influencing the Safer Syringe market include the production of Safer Syringes in large quantities and the type of products available. (P3)
    16. 16. Findings: hyper-Themes (2) Natasha’s hyper-Themes (examples) DA1 draft 1: Technological factors: It was important to create a tasty product for the regular beer consumers. (P3) DA1 final: Several technological factors influenced Buckler 's development. (P3) DA2 draft 1: Technology plays a central role in adoption of the safer syringes. (P3) DA2 final: Technology also plays a central role in the adoption of safer syringes.
    17. 17. Findings: learner perception “… New method is more relaxed whilst the traditional one is more stressful, more personal… The method was very encouraging – prompted to think about the problem. Gave an opportunity to try again with some prompts. If required explicit help was given. Very ‘dynamic’ approach…” (Michelle). “… The new method of assessment helped a lot because there were more specific details that guided in the process of writing. When I see more details, I remember next time while writing… I would like to have a similar experience in other courses… I wish I could do the project longer… Can we start the research again?...” (Natasha).
    18. 18. Conclusion  The findings suggest that dynamic assessment procedures do enhance learners’ academic writing development, albeit at varying levels.  It was also found that both the students showed progress regarding their ability to manage macro-Themes and hyper-Themes in Taxonomic Explanations.  Overall, it appears that DA procedures do help students improve their academic writing if implemented appropriately by considering individual students’ potential to develop. Although these findings cannot be generalised given the specific context in which the DA procedures were implemented, this study has pedagogical implications for writing assessment in a specific discipline in higher education.
    19. 19. Questions and suggestions? Any questions? Any suggestions?
    20. 20. References Antón, M. (2009). Dynamic Assessment of Advanced Second Language Learners. Foreign Language Annals, 42(3), 576-598. Bonanno, H., & Jones, J. (2007). Measuring the Academic Skills of University Students. Sydney: Learning Centre, the University of Sydney. Halliday, M. A. K. & Martin, J. R. (eds.) (1993) Writing science: Literacy and discursive Power, London, The Falmer Press. Haywood, H. C., & Lidz, C. S. (2007). Dynamic assessment in practice : clinical and educational applications. Cambridge: Cambridge University Press. Huot, B. (2002). (Re)Articulating Writing Assessment: Assessment for Teaching and Learning. Logan, Utah: Utah State University Press. Kozulin, A. (2003) “Psychological tool and mediated learning”, in Kozulin, A., Gindis, B., Ageyev, V.S. and Miller, S.M. (eds.) Vygotsky’s educational theory in cultural context, Cambridge, Cambridge University Press. Lidz, C.S. and Gindis, B. (2003) “Dynamic assessment of the evolving cognitive functions in children” in Kozulin, A., Gindis, B., Ageyev, V.S. and Miller, S.M. (eds.) Vygotsky’s educational theory in cultural context, Cambridge, Cambridge University Press. Lillis, T. M. (2006). Moving towards an Academic Literacies Pedagogy: Dialogues of Participation. In L. Ganobcsik-Williams (Ed.), Teaching Academic Writing in UK Higher Education. Basingstoke: Palgrave Macmillan.
    21. 21. References (contd.) Martin, J. R. (1993) “Life as a Noun: Arresting the Universe in Science and Humanities”, in Halliday, M. A. K. & Martin, J. R. (eds.) Writing science: Literacy and discursive Power, London, The Falmer Press. Martin, J. R., & Rose, D. (2007). Working with Discourse: Meaning beyond the Clause. London and New York: Continuum. Martin, J. R. (1997). Analysing genre: functional parameters. In J. R. Martin & F. Christie (Eds.), Genre and Institutions: Social Processes in the Workplace and School (Vol. first). London and Washington: Cassell. Oskoz, A. (2005). Students' dynamic assessment via onlline chat. CALICO Journal, 22(3), 513-536. Poehner, M. E. (2005). Dynamic assessment of oral proficiency among advanced L2 learners of French. Unpublished PhD Dissertation/Thesis,, Pennsylvania State University, University Park. Poehner, M. E. (2008). Dynamic assessment : a Vygotskian approach to understanding and promoting L2 development. [New York?]: Springer. Rose, D., Rose, M., Farrington, S., & Page, S. (2008). Scaffolding academic literacy with indigenous health sciences students: An evaluative study. Journal of English for Academic Purposes, 7(3), 165-179. Vygotsky, L.S. (1978) Mind in Society: The Development of Higher Psychological Processes, Cambridge, MA, Harvard University Press. Yorke, M. (2003). Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice. Higher Education, 45(4), 477-501.