• Save
An Architecture for the Automated Detection of Textual Indicators of Reflection
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

An Architecture for the Automated Detection of Textual Indicators of Reflection

  • 1,128 views
Uploaded on

Presented at the 1st European Workshop on Awareness and Reflection in Learning Networks. In conjunction with the EC-TEL 2011 conferece, Palermo, Italy. ...

Presented at the 1st European Workshop on Awareness and Reflection in Learning Networks. In conjunction with the EC-TEL 2011 conferece, Palermo, Italy.
Proceedings online at: http://ceur-ws.org/Vol-790/

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,128
On Slideshare
395
From Embeds
733
Number of Embeds
5

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 733

http://kmi.open.ac.uk 718
http://people.kmi.open.ac.uk 8
http://dev.kmi.open.ac.uk 4
http://translate.googleusercontent.com 2
https://www.google.de 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Wong used the model of Boud et al. 1985: Attending to feelings, association, integration, validation, appropriation, outcome of reflection Sumsion an adapted model of Boud: highly reflective, moderately reflective, not reflective The high agreement of the second model of Wong (non-reflector, reflector and critical reflector) could be explained with the intense training of coders with the first model.
  • Which one does not belong here?
  • Together with a set of interrogative questions like: which, what, whose, who, whom, where, whence, how

Transcript

  • 1. An Architecture for the Automated Detection of Textual Indicators of Reflection Thomas Daniel Ullmann, Fridolin Wild, Peter Scott KMi - The Open University 1 st Workshop on Awareness and Reflection in Learning Networks 21 st of September 2011
  • 2. Overview
    • A scenario
    • Architecture
    • Two core problems
    • 3 elements of reflections and their analysis engine
    • Examples
    • Conclusion
  • 3. Finding deep reflections on the Web is like… Pearl hunting Finding a needle in the haystack
  • 4. An example haystack – Research Network
    • TOTAL urls: 4906
    • status 1 (db_unfetched): 1
    • status 2 (db_fetched): 4780
    • status 3 (db_gone): 1
    • status 4 (db_redir_temp): 121
    • status 5 (db_redir_perm): 3
    • 12 th of September 2011
    • How to find reflections in 4780 pages?
    • Network is continuously growing
    • Actuality is important
  • 5. Example of a written reflection
    • I decided to do this course in core reflection, because I wanted to improve my way of supervising. The ideal situation I envisaged involves a method with which both parties— student teacher and supervisor—feel comfortable. In the past, I experienced on several occasions that when the supervisor gains access to the beliefs and the identity of the student, the conversation takes on a new content. To make this possible, the student must enter into a relationship of trust with the teacher educator. Such a relationship does not automatically develop with every student. I did not want to fall into the same trap as some supervisors I have observed, who take on the role of a therapist and delve into the individual’s past with dubious results. I have always rejected this manner of working on the part of supervisors, because of the danger that the person being supervised will ultimately become resistant. My view of the task of a supervisor involves a strict separation between the student’s private domain and his professional development.
    Korthagen, F., Vasalos, A.: Levels in reflection: core reflection as a means to enhance professional growth. Teachers and Teaching: Theory and Practice. 11, 47-71 (2005).
  • 6. The Architecture
  • 7. Two core problems
    • It is already for humans difficult to analyze texts regarding dimensions of reflection.
    • Reflection has many facets.
  • 8. Assessment of Reflection
    • 6 dimensions: 3 out of 5 coders achieved reliability between 50% and 70%. “We have observed in the literature review that there is more discussion of the concept of reflection, or the extent of its use, than empirical research reports on the assessment of reflection in learning” [1].
    • 3 dimensions: 50% inter-coder reliability. “ At present, there are substantial difficulties involved in attempting to identify and assess reflection ” [2].
    • With intense training of the coders good inter-coder agreement can be achieved: 88% (3 dimensions).
    1. Wong, F.K., Kember, D., Chung, L.Y.F., Yan, L.: Assessing the level of student reflection from reflective journals. Journal of Advanced Nursing. 22, 1, 48-57 (1995). 2. Sumsion, J., Fleet, A.: Reflection: can we assess it? Should we assess it? Assessment & Evaluation in Higher Education. 21, 2, 121 (1996).
  • 9. The many facets of reflections
    • On theoretical level
      • Core reflection theories: Dewey
      • Strong connected theories: Critical thinking
      • Reflection as a component: Pintrich
    • Elements of reflection
      • Critical evaluation, critical analysis, meta-cognition, awareness of feelings, development of new perspectives, …
  • 10. From unstructured to structured data
  • 11. Reflective Keywords
    • Some words are semantically more similar than others.
    • Example:
      • To reflect about something.
      • To muse about something.
      • To praise something.
  • 12. Using WordNet
    • Underlying technology
    • Annotator based on WordNet 3.0
    • Words are organized in SynSets
    • Semantic relations between SynSets
    • Using the Java WordNet Library
    • Process
    • Four seed words (reflection, reflect, reflective, reflectively)
    • Sense filtering (manual)
    • Automatic retrieval of all related synsets
      • Synonyms, hypernyms, hyponyms, coordinate terms, verb groups
  • 13. Result: 194 synsets containing 416 words
    • Examples
    • Positives: well thought out, chew over, think over, meditate, ponder, excogitate, contemplate, muse, reflect, mull, mull over, muser, muller, ponderer, convergent thinking, line of thought, reflective, convergent thinker, reason, deliberative, think, retrospection, reflectively, brainworker, philosophizer, philosophiser, divergent thinker, puzzle over, out-of-the-box thinking, …
    • False positive: rocket scientist, source, seed, Parkinson's law, wisecrack, dally …
    • Context dependent: solution, answer , speculate, value statement, look at,…
  • 14. Thought-provoking questions King, A.: Inquiry as a tool in critical thinking. Changing college classrooms: New teaching and learning strategies for an increasingly complex world. 13–38 (1994).
  • 15. Premise and Conclusion Halpern, D.F.: Thought and Knowledge. An Introduction to Critical Thinking. L. Erlbaum Associates (2003).
  • 16. Example Annotations I Moon, J.A.: A handbook of reflective and experiential learning. Routledge (2004).
  • 17. Example Annotations II Wikipedia: Awareness
  • 18. Example Annotations III
    • <result>
    • I decided to do this course in core
    • <Thinking begin=&quot;36&quot; end=&quot;46&quot; ReflectiveWord=&quot;Reflective Word”>reflect</Thinking>
    • ion, because I wanted to improve my way of supervising. The
    • <Thinking begin=&quot;104&quot; end=&quot;108&quot; ReflectiveWord=&quot;Reflective Word&quot;>deal</Thinking>
    • situation I envisaged involves a method with which both parties— student teacher and supervisor—feel comfortable. In the past, I experienced on several occasions that when the supervisor gains access to the beliefs and the identity of the student, the conversation
    • <Thinking begin=&quot;374&quot; end=&quot;378&quot; ReflectiveWord=&quot;Reflective Word&quot;>take</Thinking>
    • s on a new
    • <Thinking begin=&quot;389&quot; end=&quot;396&quot; ReflectiveWord=&quot;Reflective Word&quot;>content</Thinking>
    • . To make this possible, the student must enter into a relationship of trust with the teacher educator. Such a relationship does not automatically develop with every student. I did not want to fall into the same trap as some supervisors I have
    • <Thinking begin=&quot;640&quot; end=&quot;647&quot; ReflectiveWord=&quot;Reflective Word”>observe</Thinking>
    • d, who
    • <Thinking begin=&quot;654&quot; end=&quot;658&quot; ReflectiveWord=&quot;Reflective Word&quot;>take</Thinking>
    • on the role of a therapist and delve into the individual’s past with dubious
    • <Thinking begin=&quot;738&quot; end=&quot;744&quot; ReflectiveWord=&quot;Reflective Word&quot;>result</Thinking>
    • s. I have always rejected this manner of working on the part of supervisors, because of the danger that the person being supervised will ultimately become resistant.
    • </result>
    Korthagen, F., Vasalos, A.: Levels in reflection: core reflection as a means to enhance professional growth. Teachers and Teaching: Theory and Practice. 11, 47-71 (2005).
  • 19. Conclusion
    • Indicators != reflection
    • Three analysis engines as a starting point
    • Aggregation of individual detectors
  • 20. Thomas Ullmann [email_address] http://twitter.com/ThomasUllmann
  • 21. Credits
    • Needle in a haystack: http://www.flickr.com/photos/t_buchtele/3422507814/
    • Pearl hunter: http://en.wikipedia.org/wiki/File:Ama2.jpg