Reviewing it Right!


Published on

Published in: Education, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Background to this presentation Friendship with Gail. My visit to ICR with her in 1997, her training in Systematic Review Methodology. Our conversations about applying this to IS. When I got back put it on hold. Ideas have been percolating for a while and finally crystallised about a month ago when I finally decided to move on from the mire of my PhD and read the recent ITAG report on the Knowledge Economy. When I get excited about the idea, I have a, possibly rather, optimistic vision that it could provide a way for us, as a department, to make an international name for ourselves in the IS research field…but I know this is a big vision!
  • Reviewing it Right!

    1. 1. NACCQ 2007 Research Workshop Reviewing it Right! Dr Clare Atkins School of Business & Computer Technology Nelson Marlborough Institute of Technology [email_address]
    2. 2. Background <ul><li>NACCQ conference 2004 </li></ul><ul><ul><li>Panel session – Demystifying the reviewing process </li></ul></ul><ul><ul><li>Thanks to all panel members </li></ul></ul><ul><li>NACCQ Research Workshop 2004 </li></ul><ul><ul><li>Reviewing Conference and Journal Papers </li></ul></ul><ul><ul><li>Thanks to Tony Clear (AUT) and Carmel McNaught (CUHK) </li></ul></ul>Editor – New Zealand Journal of Applied Computing and IT Editor – NMIT Occasional Paper Series Editorial Board – Journal of Database Management Reviewer for various international conferences.
    3. 3. Overview <ul><li>What is reviewing? </li></ul><ul><li>How to do a useful review </li></ul><ul><li>How to get accepted as a reviewer </li></ul>
    4. 4. What is reviewing? <ul><li>Everyone has an opinion on what reviewing should be…or what makes a good reviewer….! So let’s start by determining the playing field! </li></ul><ul><li>How many of you have had work reviewed? </li></ul><ul><li>How many of you have had work ‘blind’ reviewed? </li></ul><ul><li>How many of you have reviewed for </li></ul><ul><ul><li>a journal? </li></ul></ul><ul><ul><li>a conference? </li></ul></ul><ul><ul><li>an international journal or conference? </li></ul></ul><ul><ul><li>a book? </li></ul></ul><ul><li>( Activity ) </li></ul>
    5. 5. What is reviewing? <ul><li>Purpose </li></ul><ul><ul><li>to recommend whether to provide a public platform for the ‘submitter(s)’ to put forward their work . (to stop people getting egg on their faces!) </li></ul></ul><ul><li>Issues for the Editor </li></ul><ul><ul><li>maintain (or enhance) the credibility and standing of the public platform (conference/journal etc….) </li></ul></ul><ul><ul><li>protect credibility of the presenter </li></ul></ul><ul><ul><li>will it attract interest from others (will they subscribe to the journal, come to the conference, link to the website) </li></ul></ul><ul><li>So reviewers are </li></ul><ul><ul><li>the people who assist the editor to make the decision </li></ul></ul><ul><li>The Editorial Board are </li></ul><ul><ul><li>those people who confirm the editor’s decision (not reviewers) </li></ul></ul>
    6. 6. What is reviewing? <ul><li>Why is a reviewed paper considered more ‘prestigious’ than a non-reviewed paper? </li></ul><ul><ul><li>Objective evaluation </li></ul></ul><ul><ul><li>Judged to be of value by your peers </li></ul></ul><ul><li>What does an editor want from a reviewer? </li></ul><ul><ul><li>A clear recommendation! </li></ul></ul><ul><ul><li>Reasoned arguments! </li></ul></ul><ul><ul><li>Something useful to say to the author – particularly if reject </li></ul></ul><ul><li>What does an author want from a reviewer? </li></ul><ul><ul><li>Praise!! </li></ul></ul><ul><ul><li>Constructive criticism </li></ul></ul><ul><ul><li>New ideas for content or structure </li></ul></ul><ul><ul><li>To be taken seriously. </li></ul></ul><ul><ul><li>Reasons for recommendation…… </li></ul></ul>
    7. 7. What is reviewing? <ul><li>What does it take to be a ‘good’ reviewer? </li></ul><ul><ul><li>Ability to remain as objective as possible </li></ul></ul><ul><ul><li>Reasonable content knowledge </li></ul></ul><ul><ul><li>Good understanding of structure </li></ul></ul><ul><ul><li>Good understanding of what the journal or conference requires </li></ul></ul><ul><ul><li>Good understanding of the audience </li></ul></ul><ul><ul><li>Good understanding of the preferred style </li></ul></ul><ul><ul><li>Ability to be kind but critical at the same time. </li></ul></ul><ul><ul><li>Patience! </li></ul></ul><ul><ul><li>Ability to know when you don’t know enough…. </li></ul></ul><ul><ul><li>Ability to summarise your comments and thoughts </li></ul></ul>
    8. 8. How to write a useful review <ul><li>What does it take to write a useful review? </li></ul><ul><li>There are a number of things that you might need before you start to write a review…. </li></ul><ul><li>Working with a partner, brainstorm some of the things you might need or do before beginning to write your review…… </li></ul><ul><li>Now </li></ul><ul><li>Write them as a list and prioritise them…… </li></ul>
    9. 9. How to write a useful review <ul><li>What does it take write a useful review? </li></ul><ul><ul><li>Things I might need before I start…. </li></ul></ul><ul><ul><li>A decent cup of coffee! </li></ul></ul><ul><ul><li>Previous copy of the journal or call for papers of conference or call for chapters for book </li></ul></ul><ul><ul><li>Copy of paper (watch out for lost appendices!) </li></ul></ul><ul><ul><li>List of questions/criteria from editor or </li></ul></ul><ul><ul><li>Checklist of things you think are important </li></ul></ul><ul><ul><li>Sometimes it can be useful to have some reference material e.g. </li></ul></ul><ul><ul><ul><li>Greenhalgh, T. (1997) How to read a paper . BMJ Publishing Group, London </li></ul></ul></ul><ul><ul><li>At least an hour of uninterrupted time…… </li></ul></ul>
    10. 10. How to write a useful review <ul><li>Questions to ask yourself </li></ul><ul><ul><li>What is this paper about? </li></ul></ul><ul><ul><li>Why was this paper written? </li></ul></ul><ul><ul><li>What type of research was done? </li></ul></ul><ul><ul><li>Was the type of research appropriate to the question/hypothesis? </li></ul></ul><ul><ul><li>Is the research method well described? </li></ul></ul><ul><ul><li>Is the topic (or findings) interesting, original, innovative? </li></ul></ul><ul><ul><li>Is the paper well structured? </li></ul></ul><ul><ul><li>Does the paper do what it promises at the beginning? </li></ul></ul><ul><ul><ul><ul><ul><li>( Aside ) </li></ul></ul></ul></ul></ul>
    11. 11. What do reviewers get asked? <ul><li>Usually editor will provide questions for the specific reviewing process – if not ask for them! </li></ul><ul><li>Each journal/conference will have its own set. </li></ul><ul><li>You will almost always be asked whether a paper should be </li></ul><ul><ul><ul><ul><li>accepted as is, </li></ul></ul></ul></ul><ul><ul><ul><ul><li>accepted with major revisions, </li></ul></ul></ul></ul><ul><ul><ul><ul><li>accepted with minor revisions, </li></ul></ul></ul></ul><ul><ul><ul><ul><li>rejected </li></ul></ul></ul></ul><ul><li>You have to make that call and be able to justify it </li></ul>
    12. 12. What do reviewers get asked? <ul><ul><li>Does this paper represent a completed piece of work? </li></ul></ul><ul><ul><li>Is the research methodology sound ? </li></ul></ul><ul><ul><li>Is previous relevant research acknowledged/utilised? </li></ul></ul><ul><ul><li>Are the findings interesting/original? </li></ul></ul><ul><ul><li>Are the findings clearly based on an analysis of the results? </li></ul></ul><ul><ul><li>Is the structure and the language of the paper appropriate? </li></ul></ul><ul><ul><li>Is it appropriate to the conference/journal </li></ul></ul><ul><ul><li>Is it of interest/value to the community </li></ul></ul><ul><ul><li>Are there modifications to be made before publication? </li></ul></ul><ul><ul><li>Any comments to feedback to the author? </li></ul></ul><ul><ul><li>Any comments to feedback to the editor? </li></ul></ul>
    13. 13. How to start …… <ul><ul><li>Read abstract, introduction and conclusion </li></ul></ul><ul><ul><ul><li>Get a feel for the paper, the general area, the type of research, the writing style. Remember many readers will read no more than that. </li></ul></ul></ul><ul><ul><li>Scrutinise the references </li></ul></ul><ul><ul><ul><li>Do the references look reasonable, i.e. up-to-date, good coverage, any obvious holes, on-line? </li></ul></ul></ul><ul><ul><li>Read from beginning to end </li></ul></ul><ul><ul><ul><li>Don’t be critical just read! Try to get into author’s head! </li></ul></ul></ul><ul><ul><ul><li>Try to enjoy </li></ul></ul></ul><ul><ul><li>Have another cup of coffee! </li></ul></ul><ul><ul><ul><li>Seriously! Take some short time out to reflect on what you have read. </li></ul></ul></ul>
    14. 14. Now you are underway …… <ul><ul><li>Read from beginning to end – again </li></ul></ul><ul><ul><ul><li>Look for things you missed first time through </li></ul></ul></ul><ul><ul><ul><li>Why did you miss them? How do they affect your overall judgement? </li></ul></ul></ul><ul><ul><li>Write a short summary of what the paper is about </li></ul></ul><ul><ul><ul><li>Although this is for your eyes only at this stage it will help you be clear about the topic that is covered </li></ul></ul></ul><ul><ul><li>Write a short summary of what the author did </li></ul></ul><ul><ul><ul><li>Again this is only for you initially but it will help you focus on what the author did. </li></ul></ul></ul><ul><ul><li>Check for consistency between the two summaries </li></ul></ul><ul><ul><ul><li>This will tell you a lot about the quality of the paper </li></ul></ul></ul>
    15. 15. Now you are refining your thoughts….. <ul><li>By this stage you have probably formed your opinion! </li></ul><ul><li>but </li></ul><ul><li>It is now useful to check the “5 Rs”** </li></ul><ul><ul><li>Relevance, </li></ul></ul><ul><ul><ul><li>To audience, to community, to journal/conference </li></ul></ul></ul><ul><ul><li>Rigour, </li></ul></ul><ul><ul><ul><li>Does the research have methodological rigour </li></ul></ul></ul><ul><ul><li>Readability, </li></ul></ul><ul><ul><ul><li>Use of jargon, good English, good structure </li></ul></ul></ul><ul><ul><li>Requirements, </li></ul></ul><ul><ul><ul><li>Does it meet with the editorial requirements – Length etc.. </li></ul></ul></ul><ul><ul><li>‘ Riginality </li></ul></ul><ul><ul><ul><li>Does it have something new to say? </li></ul></ul></ul><ul><li>**Thanks to participants of research workshop in Hamilton for these! </li></ul>
    16. 16. Now write them up ….. <ul><li>Write summary of what paper is about for editor </li></ul><ul><ul><li>Use your original summaries to prepare a short paragraph </li></ul></ul><ul><li>Answer any questions that editor gave you </li></ul><ul><ul><li>You’ve done this already – just write it up </li></ul></ul><ul><li>Write comments to editor </li></ul><ul><ul><li>You can be pretty free with what you say here </li></ul></ul><ul><ul><li>Highlight areas where you are not sure or that are outside your area of expertise </li></ul></ul><ul><ul><li>Declare any bias here! </li></ul></ul><ul><ul><li>Don’t try to be clever just be clear. </li></ul></ul>
    17. 17. Now write them up ….. <ul><li>Write comments to author </li></ul><ul><ul><li>This is your chance to help the author improve their paper </li></ul></ul><ul><ul><li>Be kind but firm! </li></ul></ul><ul><ul><li>Don’t let self-doubt creep in </li></ul></ul><ul><ul><li>Always try to begin with something positive </li></ul></ul><ul><ul><li>Always try to indicate that you have understood the major points </li></ul></ul><ul><ul><li>Point out the revisions that are required (if any) </li></ul></ul><ul><ul><li>If you have recommended a reject explain why and provide hope! </li></ul></ul><ul><li>Make final judgement on approve/reject </li></ul>
    18. 18. How to get accepted as a reviewer ….. <ul><li>Often all you need to do is ask! </li></ul><ul><ul><li>Pick a journal or conference that you would like to publish in </li></ul></ul><ul><ul><ul><li>Have a good look through past papers </li></ul></ul></ul><ul><ul><ul><li>Email editor and offer your services </li></ul></ul></ul><ul><ul><li>Make contact with someone in your discipline area who already reviews and offer….. </li></ul></ul><ul><ul><li>Find out if there are email lists relating to your area, subscribe and keep an eye out for calls for reviewers </li></ul></ul><ul><ul><li>Watch out for conferences and again approach the programme chair or proceedings editor and offer….. </li></ul></ul><ul><ul><li>Tell me! </li></ul></ul><ul><ul><li>Just occasionally you may be asked to do a trial review… </li></ul></ul>
    19. 19. Final thoughts <ul><ul><li>Reviewing is only scary the first time you do it! After that it’s just stressful! </li></ul></ul><ul><ul><li>Reviewing is a really good way of improving your own ‘hit rate’ for publication </li></ul></ul><ul><ul><li>Reviewing makes you much more open to suggestions from your own reviewers! </li></ul></ul><ul><ul><li>Reviewing helps you to build networks of people researching in similar areas </li></ul></ul><ul><ul><li>Reviewing keeps you in touch with interesting work in your areas of interest </li></ul></ul><ul><ul><li>Reviewing builds your self-confidence and knowledge </li></ul></ul><ul><li>GO! DO! </li></ul>
    20. 20. What is reviewing? <ul><li>Working with the person next to you come up with some answers to these questions…. </li></ul><ul><ul><li>What is the purpose of reviewing? </li></ul></ul><ul><ul><li>Why is a reviewed paper considered more ‘prestigious’ than a non-reviewed paper? </li></ul></ul><ul><ul><li>What does an editor want from a reviewer? </li></ul></ul><ul><ul><li>What does an author want from a reviewer? </li></ul></ul><ul><ul><li>What does it take to be a ‘good’ reviewer? </li></ul></ul><ul><ul><li>In 5 words what has been your experience of </li></ul></ul><ul><ul><ul><ul><ul><li>being reviewed? </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>being a reviewer? </li></ul></ul></ul></ul></ul><ul><ul><li>Back </li></ul></ul>
    21. 21. Structure of a Research Article <ul><li>Beginning – What I’m going to talk about </li></ul><ul><li>Middle - Talking about it </li></ul><ul><li>End – What I talked about! </li></ul><ul><li>Yes anyone can do it!! </li></ul><ul><li> </li></ul>
    22. 22. <ul><li>I do not know what it means to be an American because I am from Mars. </li></ul><ul><li>On Mars we don't have any houses or teachers. All you do on Mars is ride around in Space Bubbles like in Radioactive Man. You never have to go home on Mars because your Space Bubble is like your house. It's got a TV and a big refrigerator which makes whatever food you feel like eating. The other thing about Space Bubbles which isn't in the comic books is that you can do tricks like flipping it upside down or on its side while you're flying at a gazillion miles an hour. Also, there is no school on Mars. </li></ul><ul><li>That is what it means to be an American . Back </li></ul><ul><li> </li></ul>