PhD Mini Viva Talk

2,171 views

Published on

This is the presentation of my mini viva talk given to examiners who assess my PhD's 1st year following the probationary report. It is a summary of my research aims, what I have been doing since the beginning of my 1st year and my plans for the following years of the PhD

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,171
On SlideShare
0
From Embeds
0
Number of Embeds
580
Actions
Shares
0
Downloads
29
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

PhD Mini Viva Talk

  1. 1. Mini-Viva Presentation Duygu Simsek duygu.simsek@open.ac.uk Supervisors: Prof. Simon Buckingham Shum, Dr. Anna De Liddo & Dr. Rebecca Ferguson Examiners: Prof. Steve Swithenby & Prof. Denise Whitelock The Open Science Laboratory 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  2. 2. Research Aim • To investigate • Whether or not computational techniques can automatically identify attributes of good scholarly writing • What is the potential of these techniques for student essay analysis? • How we can best feedback the results of such analysis in a way that learners can value to improve the quality of their writing. • How educators can use these results for automatic or semi- automatic assessment of their students writing. 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  3. 3. Good Scholarly Writing? Quality of Writing? • Signalled by the use of metadiscourse markers in the text. • Metadiscourse: • Linguistic cues in the text • Expresses a viewpoint, the problem, claim, argument, the evidence and the implications • Engages the readers, and signals the writer's stance. Italicised words are example metadiscourse markers 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  4. 4. Xerox Incremental Parser (XIP) • Automatic processing of scientific documents • Recognition of the rhetorically significant sentences • 8 categories of Rhetorical Moves • Background Knowledge • Summarising • Tendency • Novelty • Significance • Surprise • Open Question • Contrasting Ideas 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  5. 5. Xerox Incremental Parser (XIP) 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  6. 6. Original Contribution of PhD • Carrying XIP into the education field • For professional scientific articles written by experienced researchers • But now for analysis of student essays • Hypothesis: An outcome of the XIP processed scientific documents can demonstrate the quality of the author’s written discourse; and therefore can be used to scaffold and assess scholarly writing. • First in depth opportunity to • Assess a state of the art language technology • Integrate its services into software tools for academic writing 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  7. 7. Research Questions • Main Research Question • How can we support students’ scholarly writing skills to improve the quality of their writing through automated metadiscourse analysis? • Sub-Question 1 • How reliable and sufficient are the automated discourse analysis tools for finding good attributes of scholarly writing within student essays? • Sub-Question 2 • To what extent is there a relation between the existences of various kinds of argumentative discourse moves in student essays with final grades? • Sub-Question 3 • To what extent automated metadiscourse analysis of discipline- independent student essays can be used to provide formative feedback? 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  8. 8. Academic Writing Where this research sits? 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  9. 9. Computational Linguistics Academic Writing Where this research sits? 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  10. 10. Learning Analytics Computational Linguistics Academic Writing Where this research sits? 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  11. 11. Learning Analytics Computational Linguistics Academic Writing Where this research sits? Scientific Writing 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  12. 12. Learning Analytics Computational Linguistics Academic Writing Where this research sits? Scientific Writing Rhetorical Parsers 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  13. 13. Learning Analytics Computational Linguistics Academic Writing Where this research sits? Scientific Writing Rhetorical Parsers Discourse Centric Learning Analytics 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  14. 14. Learning Analytics Computational Linguistics Academic Writing Where this research sits? Scientific Writing Rhetorical Parsers Discourse Centric Learning Analytics 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  15. 15. Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  16. 16. Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  17. 17. Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  18. 18. Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  19. 19. Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  20. 20. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  21. 21. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  22. 22. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  23. 23. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  24. 24. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  25. 25. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation • Academic Writing: Writers signal argumentative moves by using well-established patterns • Debate on whether these patterns are discipline independent or not
  26. 26. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation • Academic Writing: Writers signal argumentative moves by using well-established patterns • Debate on whether these patterns are discipline independent or not
  27. 27. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation • Academic Writing: Writers signal argumentative moves by using well-established patterns • Debate on whether these patterns are discipline independent or not
  28. 28. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation • Academic Writing: Writers signal argumentative moves by using well-established patterns • Debate on whether these patterns are discipline independent or not
  29. 29. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation • Academic Writing: Writers signal argumentative moves by using well-established patterns • Debate on whether these patterns are discipline independent or not
  30. 30. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation • Academic Writing: Writers signal argumentative moves by using well-established patterns • Debate on whether these patterns are discipline independent or not • Computational Linguistics: Possible automated analysis of scientific & technical writing but barely deployed in educational context! • Need: XIP output is not educator/learner friendly.
  31. 31. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation • Academic Writing: Writers signal argumentative moves by using well-established patterns • Debate on whether these patterns are discipline independent or not • Computational Linguistics: Possible automated analysis of scientific & technical writing but barely deployed in educational context! • Need: XIP output is not educator/learner friendly.
  32. 32. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation • Academic Writing: Writers signal argumentative moves by using well-established patterns • Debate on whether these patterns are discipline independent or not • Computational Linguistics: Possible automated analysis of scientific & technical writing but barely deployed in educational context! • Need: XIP output is not educator/learner friendly.
  33. 33. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation • Academic Writing: Writers signal argumentative moves by using well-established patterns • Debate on whether these patterns are discipline independent or not • Computational Linguistics: Possible automated analysis of scientific & technical writing but barely deployed in educational context! • Need: XIP output is not educator/learner friendly.
  34. 34. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation • Academic Writing: Writers signal argumentative moves by using well-established patterns • Debate on whether these patterns are discipline independent or not • Computational Linguistics: Possible automated analysis of scientific & technical writing but barely deployed in educational context! • Need: XIP output is not educator/learner friendly.
  35. 35. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation • Academic Writing: Writers signal argumentative moves by using well-established patterns • Debate on whether these patterns are discipline independent or not • Computational Linguistics: Possible automated analysis of scientific & technical writing but barely deployed in educational context! • Need: XIP output is not educator/learner friendly.
  36. 36. • Learning Analytics: Promising potential of automated, timely & formative feedback. • Unresolved Question: “What does analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?” Literature Review Journey 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation • Academic Writing: Writers signal argumentative moves by using well-established patterns • Debate on whether these patterns are discipline independent or not • Computational Linguistics: Possible automated analysis of scientific & technical writing but barely deployed in educational context! • Need: XIP output is not educator/learner friendly. • Run XIP on essays from different disciplines • Validate XIP in educational context • If we can show there is a value for learners & educators then it has a potential for formative assessment of writing.
  37. 37. Pilot Work: XIP Dashboard • Aim: Visualise XIP output • Question in mind: Can visual XIP analysis of literature help users to assess current state of the art in terms of trends, patterns, gaps, and connections? 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  38. 38. Pilot Work: XIP Dashboard • Aim: Visualise XIP output • Question in mind: Can visual XIP analysis of literature help users to assess current state of the art in terms of trends, patterns, gaps, and connections? 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation Data Set Proceedings of the Learning Analytics and Knowledge (LAK) conferences and a journal special issue, and the Educational Data Mining (EDM) conferences and journal.
  39. 39. Pilot Work: XIP Dashboard • Aim: Visualise XIP output • Question in mind: Can visual XIP analysis of literature help users to assess current state of the art in terms of trends, patterns, gaps, and connections? 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation Data Set Proceedings of the Learning Analytics and Knowledge (LAK) conferences and a journal special issue, and the Educational Data Mining (EDM) conferences and journal.
  40. 40. Pilot Work: XIP Dashboard • Aim: Visualise XIP output • Question in mind: Can visual XIP analysis of literature help users to assess current state of the art in terms of trends, patterns, gaps, and connections? 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation XIP Analysis All of the papers (66 LAK and 239 EDM papers 305 in total) were analysed using XIP, extracting 7847 sentences and 40163 concepts. Data Set Proceedings of the Learning Analytics and Knowledge (LAK) conferences and a journal special issue, and the Educational Data Mining (EDM) conferences and journal.
  41. 41. Pilot Work: XIP Dashboard • Aim: Visualise XIP output • Question in mind: Can visual XIP analysis of literature help users to assess current state of the art in terms of trends, patterns, gaps, and connections? 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation XIP Analysis All of the papers (66 LAK and 239 EDM papers 305 in total) were analysed using XIP, extracting 7847 sentences and 40163 concepts. Data Set Proceedings of the Learning Analytics and Knowledge (LAK) conferences and a journal special issue, and the Educational Data Mining (EDM) conferences and journal.
  42. 42. Pilot Work: XIP Dashboard • Aim: Visualise XIP output • Question in mind: Can visual XIP analysis of literature help users to assess current state of the art in terms of trends, patterns, gaps, and connections? XIP Output: Not learner friendly 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation XIP Analysis All of the papers (66 LAK and 239 EDM papers 305 in total) were analysed using XIP, extracting 7847 sentences and 40163 concepts. Data Set Proceedings of the Learning Analytics and Knowledge (LAK) conferences and a journal special issue, and the Educational Data Mining (EDM) conferences and journal.
  43. 43. Pilot Work: XIP Dashboard • XIP Dashboard is a set of visual analytics modules built on XIP output. 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  44. 44. Pilot Work: XIP Dashboard • XIP Dashboard is a set of visual analytics modules built on XIP output. Paper Prototype & User Evaluation 6 user sessions were conducted with 1st year PhD students from OU 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  45. 45. Pilot Work: XIP Dashboard • XIP Dashboard is a set of visual analytics modules built on XIP output. Paper Prototype & User Evaluation 6 user sessions were conducted with 1st year PhD students from OU 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  46. 46. Pilot Work: XIP Dashboard • XIP Dashboard is a set of visual analytics modules built on XIP output. Paper Prototype & User Evaluation 6 user sessions were conducted with 1st year PhD students from OU XIP Dashboard 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  47. 47. Dissemination of Work • Various poster presentations & talks. 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  48. 48. Dissemination of Work • Various poster presentations & talks. 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  49. 49. Dissemination of Work • Various poster presentations & talks. 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  50. 50. Dissemination of Work • Various poster presentations & talks. 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  51. 51. Dissemination of Work • Various poster presentations & talks. 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  52. 52. Dissemination of Work • Various poster presentations & talks. 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  53. 53. Dissemination of Work • Various poster presentations & talks. 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  54. 54. Dissemination of Work • Various poster presentations & talks. 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  55. 55. What are the Next Plans? • Design refinements to the XIP Dashboard • User evaluations • XIP as an API, Web Service • Integrate to software tools, XIP Dashboard • Test XIP’s power on student essays 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  56. 56. Data Collection & Analysis What? When? How? Why? Student Essays of OU’s S288 S288-12B (2012 Course) • Analysis of student essays through XIP • Comparison of XIP findings with final grades • To see whether XIP can identify important parts of student essays • To see whether or not we can correlate XIP results with final grades Student Essays of OU’s S288 S288-13B (2013 Course) Same as two above • Use of Google Docs for collaboratively written report where we back up the revision history & analyse through XIP Same as two above • To see whether XIP can reveal interesting predictive patterns about the quality of the end document and the final grade. Student Essays of OU’s S288 S288-14B (2014 Course) Same as above • Develop software with XIP Visual Analytics Dashboard integrated • Get users’ reactions Same as above • Analyses student essays & provide real-time analytics of students’ essays as a feedback to students. Student Essays (soft domains) N/A Same as above Same as above • Test the discipline independency of XIP 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation
  57. 57. Data Collection & Analysis What? When? How? Why? Student Essays of OU’s S288 S288-12B (2012 Course) • Analysis of student essays through XIP • Comparison of XIP findings with final grades • To see whether XIP can identify important parts of student essays • To see whether or not we can correlate XIP results with final grades Student Essays of OU’s S288 S288-13B (2013 Course) Same as two above • Use of Google Docs for collaboratively written report where we back up the revision history & analyse through XIP Same as two above • To see whether XIP can reveal interesting predictive patterns about the quality of the end document and the final grade. Student Essays of OU’s S288 S288-14B (2014 Course) Same as above • Develop software with XIP Visual Analytics Dashboard integrated • Get users’ reactions Same as above • Analyses student essays & provide real-time analytics of students’ essays as a feedback to students. Student Essays (soft domains) N/A Same as above Same as above • Test the discipline independency of XIP 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation RQ1: How reliable and sufficient are the automated discourse analysis tools for finding good attributes of scholarly writing within student essays?
  58. 58. Data Collection & Analysis What? When? How? Why? Student Essays of OU’s S288 S288-12B (2012 Course) • Analysis of student essays through XIP • Comparison of XIP findings with final grades • To see whether XIP can identify important parts of student essays • To see whether or not we can correlate XIP results with final grades Student Essays of OU’s S288 S288-13B (2013 Course) Same as two above • Use of Google Docs for collaboratively written report where we back up the revision history & analyse through XIP Same as two above • To see whether XIP can reveal interesting predictive patterns about the quality of the end document and the final grade. Student Essays of OU’s S288 S288-14B (2014 Course) Same as above • Develop software with XIP Visual Analytics Dashboard integrated • Get users’ reactions Same as above • Analyses student essays & provide real-time analytics of students’ essays as a feedback to students. Student Essays (soft domains) N/A Same as above Same as above • Test the discipline independency of XIP 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation RQ1: How reliable and sufficient are the automated discourse analysis tools for finding good attributes of scholarly writing within student essays? RQ2: To what extent is there a relation between the existences of various kinds of argumentative discourse moves in student essays with final grades?
  59. 59. Data Collection & Analysis What? When? How? Why? Student Essays of OU’s S288 S288-12B (2012 Course) • Analysis of student essays through XIP • Comparison of XIP findings with final grades • To see whether XIP can identify important parts of student essays • To see whether or not we can correlate XIP results with final grades Student Essays of OU’s S288 S288-13B (2013 Course) Same as two above • Use of Google Docs for collaboratively written report where we back up the revision history & analyse through XIP Same as two above • To see whether XIP can reveal interesting predictive patterns about the quality of the end document and the final grade. Student Essays of OU’s S288 S288-14B (2014 Course) Same as above • Develop software with XIP Visual Analytics Dashboard integrated • Get users’ reactions Same as above • Analyses student essays & provide real-time analytics of students’ essays as a feedback to students. Student Essays (soft domains) N/A Same as above Same as above • Test the discipline independency of XIP 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation RQ1: How reliable and sufficient are the automated discourse analysis tools for finding good attributes of scholarly writing within student essays? RQ2: To what extent is there a relation between the existences of various kinds of argumentative discourse moves in student essays with final grades? RQ3: To what extent automated metadiscourse analysis of discipline- independent student essays can be used to provide formative feedback?
  60. 60. Data Collection & Analysis What? When? How? Why? Student Essays of OU’s S288 S288-12B (2012 Course) • Analysis of student essays through XIP • Comparison of XIP findings with final grades • To see whether XIP can identify important parts of student essays • To see whether or not we can correlate XIP results with final grades Student Essays of OU’s S288 S288-13B (2013 Course) Same as two above • Use of Google Docs for collaboratively written report where we back up the revision history & analyse through XIP Same as two above • To see whether XIP can reveal interesting predictive patterns about the quality of the end document and the final grade. Student Essays of OU’s S288 S288-14B (2014 Course) Same as above • Develop software with XIP Visual Analytics Dashboard integrated • Get users’ reactions Same as above • Analyses student essays & provide real-time analytics of students’ essays as a feedback to students. Student Essays (soft domains) N/A Same as above Same as above • Test the discipline independency of XIP 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation RQ1: How reliable and sufficient are the automated discourse analysis tools for finding good attributes of scholarly writing within student essays? RQ2: To what extent is there a relation between the existences of various kinds of argumentative discourse moves in student essays with final grades? RQ3: To what extent automated metadiscourse analysis of discipline- independent student essays can be used to provide formative feedback? How can we support students’ scholarly writing skills to improve the quality of their writing through automated metadiscourse analysis?
  61. 61. Validation of XIP XIP Quality Grades Science Social Sciences Art History Marking Rubrics Representations Educators Tutors Students 10/09/2013,Tuesday,TheOpen University MiniVivaPresentation

×