The document outlines the review process for research papers. It discusses the grading scheme, key dates, and allocation of paper reviews to students. Each student must complete 5 reviews by January 14th, 2009. The document provides guidance on how to conduct effective peer reviews, including reading the paper thoroughly and providing comments on content, presentation, and conclusions. It emphasizes that the peer review process is important for academic research and improves students' own papers.

1. Review Process

2. Outline Research paper grading scheme Review Process and Allocation Why to Review How to Review Using Easy Chair

3. Grading Scheme The Research worth 20 points of your final course grade. It is divided as follows: 3 points for proposal 10 for my evaluation 3 for average peers evaluation 2 for doing peer reviewing 2 for final conference report (i.e. for those who will not present in the conference).

4. Key dates W4 - Friday 7/11/2008 hand in proposals W10 - Friday 19/12/2008 hand in draft paper W13 - Wed. 14/1/2009 end of peer reviews of papers W15 – Wed. 21/1/2009 hand in final versions of papers W16 – Tuesday 27/1/2009 attend conference it will be from 8 till 4 Proposals Draft Paper Review Final Paper Conference

5. Review Allocation Reviewers have been allocated E-mail on Friday Afternoon (say now if you haven’t!) Each of you has 5 reviews to write Deadline for reviews is 12pm Wed. 14/1/2009 We will then send you the reviews to summarize Together this peer reviewing process is worth 2 points of the course grade

6. Why Review The Peer Review Process is: A way of regulating the published literature A mechanism to generate constructive criticism A check against plagiarism A key social and professional activity for academic research

7. How to Review Read your assigned papers Get an idea of the overall message Get the Key points straight in your mind Review your papers Much longer activity 30-60 minutes per paper Go through the paper making notes Then turn your notes into a final review (around 250-500 words)

8. The Shape of a Review Summary A paragraph summarising the paper for the committee Review Tone: First paragraph sets the tone (did you think it was good or not – why?) Content: Comments on the work itself Presentation: Comments on the style, language and structure Conclusion: Be constructive!

9. Conclusion Presentation Content This is a great paper that identifies some genuine shortcomings with existing structural systems and models. The structural concepts the authors present is an important one, and its apparent lack of support in current structural models helps explain the different approaches taken by researchers who have different emphasis and goals to one another. In Section 4.2. the authors describe some initial reactions to their environment. While their findings are interesting, I would have liked to have seen more detailed evidence to back up their claims. For example, in terms of expressiveness what are some examples of the test structures? It seems odd to leave such an important sentence as "there are general graphs that cannot be expressed with our structures." without further clarification. The paper itself is very well written with ideas expressed formally and explained clearly. I do have some minor comments about the paper itself, which are detailed below. - Section 2.1, paragraph 5. I thought that interrupting the flow of this paragraph with numerous 'e.g.'s' was a little clumsy in terms of English. - Figure 2. If I understand the figure correctly the “SLP" module is not shown in the start state, this made it difficult to understand the first operation. This paper represents an important step in thinking about structures and the various ways in which they have been represented. While the paper could do with some more concrete examples in its discussion section, I still recommend that it be accepted. Tone

10. Conclusion Presentation Content I'm afraid that this paper is deeply flawed, and I cant possibly recommend it for acceptance. The authors have a lively writing style, but unfortunately this leads to numerous errors in punctuation, grammar and also style; which makes their claims seem overblown, under-researched and somewhat arrogant. The paper is also badly structured, with no proper introduction or conclusion. The paper is successful in providing a literature overview of the e-learning and adaptive hypermedia fields, and is pleasingly comprehensive. However, there is no thread drawn through the survey and no attempt to make it relevant to the point of the paper. When the final architecture is proposed it is badly explained, For example, how does the system use the user’s profile to make adaptive decisions, and what is the mechanism that it uses for personalizing content? The author has chosen an unfortunate name for their proposed architecture, perhaps they are genuinely unaware of the connotations of the word 'ultimate', but it is both inappropriate and almost certainly inaccurate. This is a very confused paper, that doesn't know whether it is reviewing the field, making observations on the nature of adaptive e-learning systems or proposing a new architecture. As well as all the problems I have already described it is double the standard page limit. I would encourage the author to think carefully about which parts of the literature are most relevant to their work, and to explore how they can make incremental improvements at the points where there are perceived shortcomings. Any paper which then presents their solution must carefully describe the problem that they are addressing and also give details on the parts of their system that deals with it. Tone

11. Evaluating the Content Relevancy Is the topic relevant to the conference? Structure Appropriate for topic? Topic covered in depth? Argument Accurate presentation of evidence? Logically developed argument? Sources Adequate acknowledgement of sources? Correct citation of sources? Copied? - search using Google

12. Evaluating the Presentation Style Fluent Succinct writing? Succinct (adj). Characterized by clear, precise expression in few words Presentation Legible and well set out work Reasonable length (2-6 pages) Title Mechanics Sentences that are grammatically correct Correct spelling and punctuation throughout Effective use of figures and tables Correct and consistent use of units

13. Content Relevancy Is the topic relevant to the conference/journal? Structure Appropriate for topic? Topic covered in depth? Argument Accurate presentation of evidence? Logically developed argument? Sources Adequate acknowledgement of sources? Correct citation of sources? Presentation Style Fluent Succinct writing? Succinct (adj). Characterized by clear, precise expression in few words Presentation Legible and well set out work Reasonable length (6 pages) Mechanics Sentences grammatical Correct spelling and punctuation throughout Effective use of figures and tables Correct and consistent use of units

14. Your Reviews are IMPORTANT! You have written a paper and become knowledgeable in 1 area Write your reviews and become knowledgeable in 5 more areas! Treat the review process as a core exercise! Turn it into a study activity Follow up some of the references Think through the arguments yourself Looking at other people’s papers will help you improve your own

15. Summary Reviews Reviews are aimed at the paper’s authors Summary reviews are aimed at the Program Committee Summarise the reviews for the Program Committee Reflect all the opinions equally Try and form a consensus (what is the general opinion?) You can add your own opinion if you feel it is necessary (but its not your job to review the paper) 250-500 words Give one recommendation (best paper, best poster/demo) Still some feedback to the author What are essential modifications to the paper?

16. Using Easy Chair (1-5)

17. Using Easy Chair (2-5)

18. Using Easy Chair (3-5)

19. Using Easy Chair (4-5)

20. Using Easy Chair (5-5)

21. Key dates W4 - Friday 7/11/2008 hand in proposals W10 - Friday 19/12/2008 hand in draft paper W13 - Wed. 14/1/2009 end of peer reviews of papers W15 – Wed. 21/1/2009 hand in final versions of papers W16 – Tuesday 27/1/2009 attend conference it will be from 8 till 4 Proposals Draft Paper Review Final Paper Conference Slides are partially adopted from Dr. Dave Millard