Relationship Between Text Editing Process and Evaluation
•
0 likes•195 views
This document discusses analyzing the relationship between the text editing process and evaluation of written texts. It proposes a new metric called the Editing Operation Indicator (EOI) to measure writing activity. An experiment was conducted where students wrote texts that were evaluated and revised. The relationship between evaluation scores and EOIs was analyzed. For initial texts there was low correlation between scores and EOIs, but for revised texts there was partial correlation, especially with improvement scores. This indicates EOIs may help detect text improvement. Further analysis with more data could help guide writing instruction in real-time.
Recommended
Recommended
More Related Content
Similar to Relationship Between Text Editing Process and Evaluation
Similar to Relationship Between Text Editing Process and Evaluation (20)
Recently uploaded
Recently uploaded (20)
Relationship Between Text Editing Process and Evaluation
- 1. Future University Hakodate Analysis of Relationship Between Text Editing Process and Evaluation of Written Text in Logical Writing Michiko Oba1 Taku Yamaguchi2 Shigeko Takahashi3 Tatsuo Kobayashi4 1 Future University Hakodate (FUN) 2 Independent Researcher 3 Heartiness Co., Ltd. 4 Scholex Co., Ltd.
- 2. Agenda 1. Background 2. Previous Research 3. Issues and Approach 4. Experiment 5. Conclusion 2Michiko Oba Future University Hakodate
- 3. Background Michiko Oba Future University Hakodate 3 1.
- 4. Background n At universities, logical writing ability is required. n The text editing operations record is useful for the analysis of the intellectual process of writing. pDeveloped a text editing system which records text editing operations. n Aim to apply this analysis to writing classes at schools. n We report examine the results of an analysis of the relationship between the text editing process on our system and the evaluations of the written text. Michiko Oba Future University Hakodate 4
- 5. Previous Research Michiko Oba Future University Hakodate 5 2.
- 6. Measurement and analysis of writing activity Michiko Oba Future University Hakodate 6 Topic Writer Writing Analytics
- 7. The time-series change of editing target Michiko Oba Future University Hakodate 7 Topic Writer Writing Analytics
- 8. Co-occurrence Matrix of the Editing Operation Public work sheet for presentation evaluation Example of Co-‐occurrence Matrix of the Editing Operation 8Michiko Oba Future University Hakodate Topic Writer Writing Analytics
- 9. Issues and Approach Michiko Oba Future University Hakodate 9 3.
- 10. Issues 1. In order to foster logical writing abilities and programming skills, it is not enough to analyze the relationship between the respective results. → Analyze the relationship between thought process and creation process and good or bad result 2. Analyze their correlation statistically that activity we need some idea: editing activities are discontinuous event series , and evaluation score for output text is scalar data Michiko Oba Future University Hakodate 10
- 11. Approach for Issue1 nIssue 1：Analyze the relationship between the writing activity and evaluation scores of the output text pUse an editing tool which records editing operations in writing classes pScore the output text from several points of view whether it is logical and easy-to-understand. Michiko Oba Future University Hakodate 11
- 12. Approach for Issue2 nA scalar index derived from co- occurrence matrix of text editing operations pThe index should have a property that when the next editing target is near then that index value is low, and when it is far from the previous target then that index value is high. pEOI：Editing Operation Indicators Michiko Oba Future University Hakodate 12
- 13. Editing Operation Indicator : EOI Michiko Oba Future University Hakodate 13
- 14. Experiment Michiko Oba Future University Hakodate 14 4.
- 15. Subjects and Theme of writing nSubjects p38 students in business writing class in 3rd- and 4th grade in O University. nTheme of the writing p ”glossary” of ”smartphone”. p Text must be written within 200 characters. The supposed reader of the text is a person in 60’s . He/She has never used smartphones but feature phones (Garakae). Michiko Oba Future University Hakodate 15
- 16. Worksheet for the experiment 16Michiko Oba Future University Hakodate
- 17. Rubric. Originally written in Japanese 17Michiko Oba Future University Hakodate
- 18. Time Student Teacher Data Operation Log 1 Operation Log 2 Score 2Score 1 Evaluation 1 Evaluation 2 Editing operation 1 Editing operation 2 Initial text Improvement = Points2 -‐ Points1 Co-‐occurrence matrix 1 Co-‐occurrence matrix 2 EOI 1 EOI 2 Analysis Relationship 1 Relationship 2 Relationship 3EOI: Editing Operation Indicator Experiment Process and EOI Michiko Oba Future University Hakodate 18 Peer review comments Revised text
- 19. Editing Operation Indicator : EOI Michiko Oba Future University Hakodate 19
- 20. Time Student Teacher Data Operation Log 1 Operation Log 2 Score 2Score 1 Evaluation 1 Evaluation 2 Editing operation 1 Editing operation 2 Initial text Improvement = Points2 -‐ Points1 Co-‐occurrence matrix 1 Co-‐occurrence matrix 2 EOI 1 EOI 2 Analysis Relationship 1 Relationship 2 Relationship 3 EOI: Editing Operation Indicator Result of Relationship between Evaluation Scores and EOIs 20 Peer review comments Revised text Low correlation Low correlation Partly Correlated
- 21. Relationship between the degree of improvement and the EOI2 (Table 7) Michiko Oba Future University Hakodate 21 Correlated Definition
- 22. Conclusion Michiko Oba Future University Hakodate 22 5.
- 23. Conclusion n Proposal p We introduced a new scalar index EOI (Editing Operation Indicator) for measuring writing activities. p We proposed a method to analyze the relationship between writing activity and text evaluation score. n Experiment and Result p We applied the proposed method to actual writing exercises and analyzed the relationship between the evaluation scores of texts and EOI, for the initial text and revised texts based on the reviews. p The relation with writing activity appears due to the degree of improvement score. These make it possible to detect text improving situation observing EOI value and change. p We would like to make use of the result for timely guidance for writing. n Issue and Future Plans p Investigate with more experimental data p Consider and analyze another EOI p Real-time analysis method using EOI, visualization, way to alert, and so on 23Michiko Oba Future University Hakodate
- 24. Future University Hakodate Thank you. . Michiko Oba Future University Hakodate, Professor michiko@fun.ac.jp