NACCQ 2007 Research Workshop Reviewing it Right! Dr Clare Atkins School of Business & Computer Technology Nelson Marlborough Institute of Technology [email_address]

Background NACCQ conference 2004 Panel session – Demystifying the reviewing process Thanks to all panel members NACCQ Research Workshop 2004 Reviewing Conference and Journal Papers Thanks to Tony Clear (AUT) and Carmel McNaught (CUHK) Editor – New Zealand Journal of Applied Computing and IT Editor – NMIT Occasional Paper Series Editorial Board – Journal of Database Management Reviewer for various international conferences.

NACCQ conference 2004

Panel session – Demystifying the reviewing process

Thanks to all panel members

NACCQ Research Workshop 2004

Reviewing Conference and Journal Papers

Thanks to Tony Clear (AUT) and Carmel McNaught (CUHK)

Overview What is reviewing? How to do a useful review How to get accepted as a reviewer

What is reviewing?

How to do a useful review

How to get accepted as a reviewer

What is reviewing? Everyone has an opinion on what reviewing should be…or what makes a good reviewer….! So let’s start by determining the playing field! How many of you have had work reviewed? How many of you have had work ‘blind’ reviewed? How many of you have reviewed for a journal? a conference? an international journal or conference? a book? ( Activity )

Everyone has an opinion on what reviewing should be…or what makes a good reviewer….! So let’s start by determining the playing field!

How many of you have had work reviewed?

How many of you have had work ‘blind’ reviewed?

How many of you have reviewed for

a journal?

a conference?

an international journal or conference?

a book?

( Activity )

What is reviewing? Purpose to recommend whether to provide a public platform for the ‘submitter(s)’ to put forward their work . (to stop people getting egg on their faces!) Issues for the Editor maintain (or enhance) the credibility and standing of the public platform (conference/journal etc….) protect credibility of the presenter will it attract interest from others (will they subscribe to the journal, come to the conference, link to the website) So reviewers are the people who assist the editor to make the decision The Editorial Board are those people who confirm the editor’s decision (not reviewers)

Purpose

to recommend whether to provide a public platform for the ‘submitter(s)’ to put forward their work . (to stop people getting egg on their faces!)

Issues for the Editor

maintain (or enhance) the credibility and standing of the public platform (conference/journal etc….)

protect credibility of the presenter

will it attract interest from others (will they subscribe to the journal, come to the conference, link to the website)

So reviewers are

the people who assist the editor to make the decision

The Editorial Board are

those people who confirm the editor’s decision (not reviewers)

What is reviewing? Why is a reviewed paper considered more ‘prestigious’ than a non-reviewed paper? Objective evaluation Judged to be of value by your peers What does an editor want from a reviewer? A clear recommendation! Reasoned arguments! Something useful to say to the author – particularly if reject What does an author want from a reviewer? Praise!! Constructive criticism New ideas for content or structure To be taken seriously. Reasons for recommendation……

Why is a reviewed paper considered more ‘prestigious’ than a non-reviewed paper?

Objective evaluation

Judged to be of value by your peers

What does an editor want from a reviewer?

A clear recommendation!

Reasoned arguments!

Something useful to say to the author – particularly if reject

What does an author want from a reviewer?

Praise!!

Constructive criticism

New ideas for content or structure

To be taken seriously.

Reasons for recommendation……

What is reviewing? What does it take to be a ‘good’ reviewer? Ability to remain as objective as possible Reasonable content knowledge Good understanding of structure Good understanding of what the journal or conference requires Good understanding of the audience Good understanding of the preferred style Ability to be kind but critical at the same time. Patience! Ability to know when you don’t know enough…. Ability to summarise your comments and thoughts

What does it take to be a ‘good’ reviewer?

Ability to remain as objective as possible

Reasonable content knowledge

Good understanding of structure

Good understanding of what the journal or conference requires

Good understanding of the audience

Good understanding of the preferred style

Ability to be kind but critical at the same time.

Patience!

Ability to know when you don’t know enough….

Ability to summarise your comments and thoughts

How to write a useful review What does it take to write a useful review? There are a number of things that you might need before you start to write a review…. Working with a partner, brainstorm some of the things you might need or do before beginning to write your review…… Now Write them as a list and prioritise them……

What does it take to write a useful review?

There are a number of things that you might need before you start to write a review….

Working with a partner, brainstorm some of the things you might need or do before beginning to write your review……

Now

Write them as a list and prioritise them……

How to write a useful review What does it take write a useful review? Things I might need before I start…. A decent cup of coffee! Previous copy of the journal or call for papers of conference or call for chapters for book Copy of paper (watch out for lost appendices!) List of questions/criteria from editor or Checklist of things you think are important Sometimes it can be useful to have some reference material e.g. Greenhalgh, T. (1997) How to read a paper . BMJ Publishing Group, London At least an hour of uninterrupted time……

What does it take write a useful review?

Things I might need before I start….

A decent cup of coffee!

Previous copy of the journal or call for papers of conference or call for chapters for book

Copy of paper (watch out for lost appendices!)

List of questions/criteria from editor or

Checklist of things you think are important

Sometimes it can be useful to have some reference material e.g.

Greenhalgh, T. (1997) How to read a paper . BMJ Publishing Group, London

At least an hour of uninterrupted time……

How to write a useful review Questions to ask yourself What is this paper about? Why was this paper written? What type of research was done? Was the type of research appropriate to the question/hypothesis? Is the research method well described? Is the topic (or findings) interesting, original, innovative? Is the paper well structured? Does the paper do what it promises at the beginning? ( Aside )

Questions to ask yourself

What is this paper about?

Why was this paper written?

What type of research was done?

Was the type of research appropriate to the question/hypothesis?

Is the research method well described?

Is the topic (or findings) interesting, original, innovative?

Is the paper well structured?

Does the paper do what it promises at the beginning?

( Aside )

What do reviewers get asked? Usually editor will provide questions for the specific reviewing process – if not ask for them! Each journal/conference will have its own set. You will almost always be asked whether a paper should be accepted as is, accepted with major revisions, accepted with minor revisions, rejected You have to make that call and be able to justify it

Usually editor will provide questions for the specific reviewing process – if not ask for them!

Each journal/conference will have its own set.

You will almost always be asked whether a paper should be

accepted as is,

accepted with major revisions,

accepted with minor revisions,

rejected

You have to make that call and be able to justify it

What do reviewers get asked? Does this paper represent a completed piece of work? Is the research methodology sound ? Is previous relevant research acknowledged/utilised? Are the findings interesting/original? Are the findings clearly based on an analysis of the results? Is the structure and the language of the paper appropriate? Is it appropriate to the conference/journal Is it of interest/value to the community Are there modifications to be made before publication? Any comments to feedback to the author? Any comments to feedback to the editor?

Does this paper represent a completed piece of work?

Is the research methodology sound ?

Is previous relevant research acknowledged/utilised?

Are the findings interesting/original?

Are the findings clearly based on an analysis of the results?

Is the structure and the language of the paper appropriate?

Is it appropriate to the conference/journal

Is it of interest/value to the community

Are there modifications to be made before publication?

Any comments to feedback to the author?

Any comments to feedback to the editor?

How to start …… Read abstract, introduction and conclusion Get a feel for the paper, the general area, the type of research, the writing style. Remember many readers will read no more than that. Scrutinise the references Do the references look reasonable, i.e. up-to-date, good coverage, any obvious holes, on-line? Read from beginning to end Don’t be critical just read! Try to get into author’s head! Try to enjoy Have another cup of coffee! Seriously! Take some short time out to reflect on what you have read.

Read abstract, introduction and conclusion

Get a feel for the paper, the general area, the type of research, the writing style. Remember many readers will read no more than that.

Scrutinise the references

Do the references look reasonable, i.e. up-to-date, good coverage, any obvious holes, on-line?

Read from beginning to end

Don’t be critical just read! Try to get into author’s head!

Try to enjoy

Have another cup of coffee!

Seriously! Take some short time out to reflect on what you have read.

Now you are underway …… Read from beginning to end – again Look for things you missed first time through Why did you miss them? How do they affect your overall judgement? Write a short summary of what the paper is about Although this is for your eyes only at this stage it will help you be clear about the topic that is covered Write a short summary of what the author did Again this is only for you initially but it will help you focus on what the author did. Check for consistency between the two summaries This will tell you a lot about the quality of the paper

Read from beginning to end – again

Look for things you missed first time through

Why did you miss them? How do they affect your overall judgement?

Write a short summary of what the paper is about

Although this is for your eyes only at this stage it will help you be clear about the topic that is covered

Write a short summary of what the author did

Again this is only for you initially but it will help you focus on what the author did.

Check for consistency between the two summaries

This will tell you a lot about the quality of the paper

Now you are refining your thoughts….. By this stage you have probably formed your opinion! but It is now useful to check the “5 Rs”** Relevance, To audience, to community, to journal/conference Rigour, Does the research have methodological rigour Readability, Use of jargon, good English, good structure Requirements, Does it meet with the editorial requirements – Length etc.. ‘ Riginality Does it have something new to say? **Thanks to participants of research workshop in Hamilton for these!

By this stage you have probably formed your opinion!

but

It is now useful to check the “5 Rs”**

Relevance,

To audience, to community, to journal/conference

Rigour,

Does the research have methodological rigour

Readability,

Use of jargon, good English, good structure

Requirements,

Does it meet with the editorial requirements – Length etc..

‘ Riginality

Does it have something new to say?

**Thanks to participants of research workshop in Hamilton for these!

Now write them up ….. Write summary of what paper is about for editor Use your original summaries to prepare a short paragraph Answer any questions that editor gave you You’ve done this already – just write it up Write comments to editor You can be pretty free with what you say here Highlight areas where you are not sure or that are outside your area of expertise Declare any bias here! Don’t try to be clever just be clear.

Write summary of what paper is about for editor

Use your original summaries to prepare a short paragraph

Answer any questions that editor gave you

You’ve done this already – just write it up

Write comments to editor

You can be pretty free with what you say here

Highlight areas where you are not sure or that are outside your area of expertise

Declare any bias here!

Don’t try to be clever just be clear.

Now write them up ….. Write comments to author This is your chance to help the author improve their paper Be kind but firm! Don’t let self-doubt creep in Always try to begin with something positive Always try to indicate that you have understood the major points Point out the revisions that are required (if any) If you have recommended a reject explain why and provide hope! Make final judgement on approve/reject

Write comments to author

This is your chance to help the author improve their paper

Be kind but firm!

Don’t let self-doubt creep in

Always try to begin with something positive

Always try to indicate that you have understood the major points

Point out the revisions that are required (if any)

If you have recommended a reject explain why and provide hope!

Make final judgement on approve/reject

How to get accepted as a reviewer ….. Often all you need to do is ask! Pick a journal or conference that you would like to publish in Have a good look through past papers Email editor and offer your services Make contact with someone in your discipline area who already reviews and offer….. Find out if there are email lists relating to your area, subscribe and keep an eye out for calls for reviewers Watch out for conferences and again approach the programme chair or proceedings editor and offer….. Tell me! Just occasionally you may be asked to do a trial review…

Often all you need to do is ask!

Pick a journal or conference that you would like to publish in

Have a good look through past papers

Email editor and offer your services

Make contact with someone in your discipline area who already reviews and offer…..

Find out if there are email lists relating to your area, subscribe and keep an eye out for calls for reviewers

Watch out for conferences and again approach the programme chair or proceedings editor and offer…..

Tell me!

Just occasionally you may be asked to do a trial review…

Final thoughts Reviewing is only scary the first time you do it! After that it’s just stressful! Reviewing is a really good way of improving your own ‘hit rate’ for publication Reviewing makes you much more open to suggestions from your own reviewers! Reviewing helps you to build networks of people researching in similar areas Reviewing keeps you in touch with interesting work in your areas of interest Reviewing builds your self-confidence and knowledge GO! DO!

Reviewing is only scary the first time you do it! After that it’s just stressful!

Reviewing is a really good way of improving your own ‘hit rate’ for publication

Reviewing makes you much more open to suggestions from your own reviewers!

Reviewing helps you to build networks of people researching in similar areas

Reviewing keeps you in touch with interesting work in your areas of interest

Reviewing builds your self-confidence and knowledge

GO! DO!

What is reviewing? Working with the person next to you come up with some answers to these questions…. What is the purpose of reviewing? Why is a reviewed paper considered more ‘prestigious’ than a non-reviewed paper? What does an editor want from a reviewer? What does an author want from a reviewer? What does it take to be a ‘good’ reviewer? In 5 words what has been your experience of being reviewed? being a reviewer? Back

Working with the person next to you come up with some answers to these questions….

What is the purpose of reviewing?

Why is a reviewed paper considered more ‘prestigious’ than a non-reviewed paper?

What does an editor want from a reviewer?

What does an author want from a reviewer?

What does it take to be a ‘good’ reviewer?

In 5 words what has been your experience of

being reviewed?

being a reviewer?

Back

Structure of a Research Article Beginning – What I’m going to talk about Middle - Talking about it End – What I talked about! Yes anyone can do it!! http://www.thesimpsons.com/

Beginning – What I’m going to talk about

Middle - Talking about it

End – What I talked about!

Yes anyone can do it!!

http://www.thesimpsons.com/

I do not know what it means to be an American because I am from Mars. On Mars we don't have any houses or teachers. All you do on Mars is ride around in Space Bubbles like in Radioactive Man. You never have to go home on Mars because your Space Bubble is like your house. It's got a TV and a big refrigerator which makes whatever food you feel like eating. The other thing about Space Bubbles which isn't in the comic books is that you can do tricks like flipping it upside down or on its side while you're flying at a gazillion miles an hour. Also, there is no school on Mars. That is what it means to be an American . Back http://animatedtv.about.com/gi/dynamic/offsite.htm?site=http%3A%2F%2Fhometown.aol.com%2Fgene168%2Fessay.html

I do not know what it means to be an American because I am from Mars.

On Mars we don't have any houses or teachers. All you do on Mars is ride around in Space Bubbles like in Radioactive Man. You never have to go home on Mars because your Space Bubble is like your house. It's got a TV and a big refrigerator which makes whatever food you feel like eating. The other thing about Space Bubbles which isn't in the comic books is that you can do tricks like flipping it upside down or on its side while you're flying at a gazillion miles an hour. Also, there is no school on Mars.

That is what it means to be an American . Back

http://animatedtv.about.com/gi/dynamic/offsite.htm?site=http%3A%2F%2Fhometown.aol.com%2Fgene168%2Fessay.html