Your SlideShare is downloading. ×
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Designing and testing visual representations of draft essays for Higher Education Students
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Designing and testing visual representations of draft essays for Higher Education Students

212

Published on

This presentation reports the findings of an empirical investigation which set out to test a set of rainbow exercises. The rainbow diagrams are pictorial representations of formal graphs that are …

This presentation reports the findings of an empirical investigation which set out to test a set of rainbow exercises. The rainbow diagrams are pictorial representations of formal graphs that are derived automatically from student essays. They were designed to allow students to discover how key concepts in a well written essay are connected together. The students would then be able to compare a rainbow diagram of their own essay with a good essay and make changes to it before submission to their tutor. A trial was undertaken with academics, teaching and learning staff, doctoral students at The Open University of Catalonia and the Open University UK, before implementation into the web application known as OpenEssayist.

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
212
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • TODO: finding the right LOGO for Primma
  • Transcript

    • 1. Designing and Testing Visual Representations of Draft Essays for Higher Education Students Denise Whitelock  , Debora Field  , Stephen Pulman , John Richardson  ,  Institute of Educational Technology – The Open University  Department of Computer Science – University of Oxford
    • 2. Learning Analytics and Student Feedback • Assessment drives learning (Rowntree, 1987) • Communicating assessment in a meaningful way • What, when and How? DMW LAK Workshop March 2014
    • 3. SAFeSEA Professor Denise Whitelock Professor John Richardson Professor Stephen Pulman An automated tool supporting online writing and assessment of essays providing accurate targeted feedback SAFeSEA: Supportive Automated Feedback for Short Essay Answers http://www.open.ac.uk/researchprojects/safesea/ DMW LAK Workshop March 2014
    • 4. DMW LAK Workshop March 2014 Grand Challenge representing analysis that can be readily understood
    • 5. Feedback to prompt Self Reflection • Analysis must prompt “Advice for Action” • Self reflective discourse with computer feedback • Visual representation of feedback can open a discourse between tutor and student • Prompt peer to peer discourse DMW LAK Workshop March 2014
    • 6. Don’t forget the Human Reaction DMW LAK Workshop March 2014
    • 7. DMW LAK Workshop March 2014 www.storiesabout.com www.storiesabout.com/cr eativepdp c.mckillop@rgu.ac.uk
    • 8. DMW LAK Workshop March 2014
    • 9. Computer Feedback • Non- judgemental/objective • Is not the tutor who marks the final essay • Feedback can be accessed many times • Can lead to contact with tutor with more questions DMW LAK Workshop March 2014
    • 10. Talk Back • Checking understanding by ‘talk back’ • Summaries in OpenEssayist • Key words = key ideas • http://www.open.ac.u k/researchprojects/s afesea/ DMW LAK Workshop March 2014
    • 11. OpenEssayist: What it tells you • The system’s focus is to present summaries of students’ own work in different ways, to encourage them to reflect constructively on what they have written. • In other words Open Essayist tells them from its analysis what are the most important or key points in their essay. They can then think about whether that was what they intended to emphasise in their essay. If not then they can make the appropriate changes. • A very important aspect of the OpenEssayist system is that it will not tell students what to write, or how to rewrite sections of their essay, or even what is correct or incorrect in their essay. DMW LAK Workshop March 2014
    • 12. OpenEssayist: A tool for reflection • Different views with distinct roles presented • The purpose of this feedback is to encourage you • to reflect on the draft text you submitted, • to help you consider how your essay is organised, • how the key terms are being used across the essay • how they combine to form a cohesive discussion. DMW LAK Workshop March 2014
    • 13. OpenEssayist: How it gives feedback • Three aspects of the students’ essays are analysed by the system: • the structure of the essay (which paragraphs constitute the introduction, the conclusion, the discussion sections, etc.), • the key words and key phrases of their essay (which are the most important words and phrases, the ones that are most representative of the essay's overall meaning) • the key sentences of their essay (which are whole sentences that are most representative of the essay's overall meaning). DMW LAK Workshop March 2014
    • 14. ESSAY GRAPHICS ANALYSIS
    • 15. Key word spring DMW LAK Workshop March 2014
    • 16. Chord diagram DMW LAK Workshop March 2014
    • 17. Key sentence adjacency matrix DMW LAK Workshop March 2014
    • 18. Middle Space • Develop new analytic methods • Computational • Representational • Statistical • Visualisation DMW LAK Workshop March 2014
    • 19. Visualisations as a Thinking Tool • Visualising text (Bertin, 1981; Johnson et al, 1993) • Free text visualisation still problematic • Do users need training with visualisations? • Convention vs. Instruction? DMW LAK Workshop March 2014
    • 20. Short text for illustration of Rainbow Diagrams DMW LAK Workshop March 2014
    • 21. Sentence graph of short text DMW LAK Workshop March 2014
    • 22. OU essay awarded high grade DMW LAK Workshop March 2014
    • 23. OU essay awarded low grade DMW LAK Workshop March 2014
    • 24. Stanford University Boothe Prize essay DMW LAK Workshop March 2014
    • 25. Pretend essay: 50 identical sentences DMW LAK Workshop March 2014
    • 26. Pretend essay: 10 identical paragraphs DMW LAK Workshop March 2014
    • 27. Participants for Empirical Study Occupation University Professor of Teaching & Learning OUC Senior Lecturer Computer Science OUC Administrator x 2 OUC Professor OUC Ph.D student x 6 OUC Professor of Computer Science OUC Vice Rector OUC Senior Lecturer x 2 OU UK Learning & Teaching Officer x 3 OU UK Project Manager OU UK OU student x 2 OU UK Professor OU UK Senior Administrator OU UK Course Administrator OU UK DMW LAK Workshop March 2014
    • 28. Findings DMW LAK Workshop March 2014 “One of the clues was talking about the way the color nodes, that the red ones are at the end. If they are a good connection with a good explanation for each paragraph. They should be connected together but the darker ones should be in the middle. But you see the color groups together so for me it automatically pulls my eye to this page because all the colors are closer together and more in the middle. So that would be the student course assignment essay with the highest mark.” “You don’t show me anybody’s text. You are not revealing anyone else’s essay. So students cannot plagiarize. But you are saying ‘Look hang on, this is the way this essay connects together.’ That’s what telling a good story is about this linking.”
    • 29. Creating teaching and learning dialogues: towards guided learning supported by technology • Learning to judge • Providing reassurance • Providing a variety of signposted routes to achieve learning goals DMW LAK Workshop March 2014
    • 30. Challenges • Visualisation • Social network analysis • Communication and collaboration, • Discourse analytics • Interpretation training • What are the necessary and sufficient conditions for successful implementation of learning analytics? DMW LAK Workshop March 2014
    • 31. References • Van Labeke, N., Whitelock, D., Field, D., Pulman, S. & Richardson, J. (2013) ‘OpenEssayist: Extractive Summarisation & Formative Assessment of Free-Text Essays’. Workshop on Discourse-Centric Learning Analytics, 3rd Conference on Learning Analytics and Knowledge (LAK 2013), Leuven, Belgium • Whitelock, D. (2011) Activating Assessment for Learning: are we on the way with Web 2.0? In M.J.W. Lee & C. McLoughlin (Eds.) Web 2.0-Based-E-Learning: Applying Social Informatics for Tertiary Teaching. IGI Global. pp. 319–342. • Field, D., Pulman, S., Van Labeke, N., Whitelock, D. and Richardson, J.T.E. 2013. Did I really mean that? Applying automatic summarisation techniques to formative feedback. Proceedings of the 9th International Conference Recent Advances in Natural Language Processing (Hissar, Bulgaria, Sep. 2013). • Van Labeke, N., Whitelock, D., Field, D., Pulman, S. and Richardson, J.T.E. 2013. What is my essay really saying? Using extractive summarization to motivate reflection and redrafting. Proceedings of the AIED Workshop on Formative Feedback in Interactive Learning Environments (Memphis, TN, Jul. 2013). • Alden, B., Van Labeke, N., Field, D., Pulman, S., Richardson, J.T.E. and Whitelock, D. 2013. Using student experience to inform the design of an automated feedback system for essay answers. Proceedings of the 2013 International Computer Assisted Assessment Conference (Southampton, UK, Jul. 2013). • Field, D., Richardson, J.T.E., Pulman, S., Van Labeke, N. and Whitelock, D. 2013. Reflections on characteristics of university students’ essays through experimentation with domain-independent natural language processing techniques. Proceedings of the 2013 International Computer Assisted Assessment Conference (Southampton, UK, Jul. 2013). • Van Labeke, N., Whitelock, D., Field, D., Pulman, S. and Richardson, J.T.E. 2013. OpenEssayist: extractive summarisation and formative assessment of free-text essays. Proceedings of the 1st International Workshop on Discourse-Centric Learning Analytics (Leuven, Belgium, Apr. 2013). DMW LAK Workshop March 2014

    ×