PhD training workshop material on critical reading skills for Edinburgh Napier University research students.

1. Critical reading skills
Dr Hazel Hall
Professor of Social Informatics
School of Computing
http://hazelhall.org
http://www.slideshare.net/hazelhall

2. Purpose of the session
Let’s start with a question
 Why are you here today?

3. Reasons to be here




To learn how to evaluate and analyse literature
To learn how to read strategically
To learn what to do with the output of PhD reading activity
To learn how to save time




To network with other PhD students?
To have a break from the everyday routine of a PhD student?
To be entertained by Hazel?
To have a wee snooze?

4. The use of the term “critical”
“Critical” in this context means
 “to analyse and evaluate” – for the particular purposes of your PhD
work
It does not mean
 “to identify material that is critical to my PhD”
 “to make unkind remarks”

5. Agenda
Main themes to be covered
 Evaluation of material prior to studying it in detail
 Critical reading
 Reading output and the literature reviewing process

6. EVALUATING MATERIAL PRIOR
TO STUDYING IT IN DETAIL

7. Exercise: rating publications
List the criteria that you currently use to rate
 Journal and conference publications
 Individual journal articles and conference papers
 Books
 Commercial online databases for literature searching
You may list the criteria that apply to all formats, and any that are formatspecific.
If you have anything else to say about other formats of “literature”, e.g.
web pages and blogs, please also note this for discussion.

8. Evaluating journal/conference
publications
Academic
 Publisher: who publishes this title - a society; a body (university,
museum) a mainstream publisher?
 Who is on the editorial board?
 Evaluate membership as you would authors (see later)
 Review policy: is it a peer-reviewed journal?
 Longevity: how long has the title been in existence?
 Scope: are there indications of the publication’s’ scope in its title –
what do the terms “international” and “review” mean?
 Indexing: which databases index this title?
 Implication – you need to evaluate commercial databases

9. Evaluating journal/conference
publications
Professional/trade journals and conferences?
 Are any professional/trade journals of importance to your work?

10. Evaluating individual articles/papers
Type of article/paper
 Is this a report of empirical research/literature review/domain
classic?
 Which version is this?
 Who funded the study reported in the text?
Relevance of the material
 What are the main points of the paper, and is it relevant to your
work?
 In which domain is the work situated?
 Is this material still current? What about the references?
Presentation of the work
 Is it accessible, does it inspire confidence (authority)?

11. Evaluating books
Scope
 For PhD work, the scope of a book is very important. This is
because in many subject domains research literature is not
normally published in book format
 Exceptions – the research monograph in arts and humanities, and
some social sciences
Publisher
 Who publishes this title - a society; a body (university, museum) a
mainstream publisher?
Longevity
 Has this work been republished, i.e. in an edition other than the
first?

12. Evaluating commercial online databases
Provenance
 Who is the supplier?
Scope
 What is the database’s coverage, e.g. in terms of geography,
language?
 What is the level of access – basic bibliographic details or full-text?
Access
 Is there “free” access from the University?

13. Evaluating authors
Stature of authors
 Many different types of author: journalists, teaching
academics, research-active academics, fellow PhD students, fulltime researchers
 Have you heard of them?
 Does your supervisor know them?
 Where do they work?
Academic impact of authors’ work
 What is their publication track record?
 Is their work of a consistently high standard?
 How often is their work cited?

14. Academic researchers
may publish across a
range of publications for
different reasons
Tools that give metrics
help distinguish the
“best” work and “best”
researchers

15. Not everything
published makes it into
databases such as Web
of Knowledge

16. Citation indexes
help assess the
impact of work
following publication

17. Some pieces of work
are more highly cited
than others!

18. Google Scholar can
be used in a similar
way to citations
databases
Here the rankings
broadly reflect those
of Web of Science
However, Google
Scholar is less
discriminating in
terms of coverage

19. H-index for authors as
measure of productivity
and impact

20. Individual journal titles also
provide ranking information
Useful for understanding the
importance of a piece of work
within the domain of study

21. Altmetrics
 Alternative metrics (Altmetrics) becoming increasingly important as
indicators of quality
 See Altmetrics: achieving and measuring success in communicating
research in the digital age
 http://hazelhall.org/2013/07/14/altmetrics-achieving-and-measuringsuccess-in-communicating-research-in-the-digital-age

22. CRITICAL READING

23. Exercise: critical reading
Consider the order in which you read the main elements of the following
material
 An Ian Rankin Inspector Rebus novel
 A bus timetable
 The Argos catalogue
 An academic paper
 A recipe
 A court report
 Your old school friend’s Christmas newsletter

24. Critical reading - order
The order in which you do something
has an impact on its
• effectiveness
• efficiency

25. Which order?










Abstract
Acknowledgements
Conclusions
Discussion
Introduction
Methods
References/bibliography
Results
Section headings
Title

26. Which order?










Abstract – short cut to relevance
Acknowledgements – may tell you more about the author
Conclusions – short cut to relevance
Discussion
Introduction – short cut to relevance
Methods
References/bibliography – short cut to identifying domain
Results
Section headings
Title

27. Critical reading - attention
Some aspects of the material require more attention than others
Like making a film, you may need a range of
pictures:
• “long shots”
• “medium shots”
• “close-ups”
This depends on the purpose of your reading
the paper, e.g.
• if you are interested in outcomes of an
empirical study  focus on methods;
• For literature review article  evaluate
the scope of the material covered.

28. Critical reading – what you read
Decisions on what to read
 Output from literature searches
 Recommendations from others
 The librarian
 Your supervisor
 “Recommendations” from citation pearling, RSS feeds, blogs that you
follow
 Decisions become easier over time as you become familiar with
the literature of your domain

29. Critical reading – handling
Time is short – minimise double-handling of material
 Read and take notes
 Highlight material in the text, or notes in the margin
 Take a break, then extract what is genuinely useful
 Hand-written linear notes, word-processed notes, or mind maps – but
do not copy verbatim
 Only go back to the original if absolutely necessary
 Learn when enough is enough
 Keep a dictionary at hand to help clarify terms, enlarge your
vocabulary, learn the discourse of your subject area

30. Critical reading – the focus
When reading academic work you are evaluating the level of argument
presented
Look out for
 Claims/conclusions
 Reasons/interpretations of data that lead to the above
 Evidence on which above is built
 Any qualifications for the claims/conclusions

31. Critical reading – the focus
Illustration
 Claims/conclusions
 You really should read this novel
 Reasons/interpretations of data that lead to the above
 The author is fantastic
 Evidence on which above is built
 She won all these awards
 Any qualifications for the claims/conclusions
 They are all for crime writing

32. Checking the level of argument








Do the arguments that the authors put forward make sense?
Are the views of the authors consistent with the evidence provided?
Is it possible to distinguish fact and opinion?
Are there any omissions in this work?
Is there ambiguity?
Is there bias?
How current is the material?
How well referenced is the work?

33. READING OUTPUT AND THE
LITERATURE REVIEWING PROCESS

34. Your review and the focus of your study
You need to read critically in order to identify
 Deficiencies in the treatment of your subject area, e.g. important
issues possibly misunderstood
 Gaps in existing knowledge
 The context for your work
Thus you are able to validate the sense of undertaking your own study in
a critique, rather than straightforward report
NB the literature review in the final version of your thesis will be a version
of the one written at the start of your work – be prepared for revisions!

35. Literature review
Do parallel
literatures exist
for this topic?
In which
subject areas
has the topic
been studied?
What are the key
concepts in this area?
How is this topic
approached by
others?
Who are
these
“others”?
Which
discussions?
Which aspects of this
work are of most
relevance to my
study?
Which subthemes?
Which writers?
What are the
main
perspectives
on this topic in
previous
research?
Coherent synthesis
of past and present
research in the
domain of study
What have been the
main research
questions?
What are the main
conclusions on
previous research in
this area?
Where are the gaps in
literature?
Where is existing
knowledge “thin”?
What are the key
areas of debate in this
area?
Which work is
subject to
challenge?
Which existing
work could be
extended?

36. Literature review
Do parallel
literatures exist
for this topic?
In which
subject areas
has the topic
been studied?
What are the key
concepts in this area?
How is this topic
approached by
others?
Who are
these
“others”?
Which
discussions?
Which aspects of this
work are of most
relevance to my
study?
Which subthemes?
Which writers?
What are the
main
perspectives
on this topic in
previous
research?
Coherent synthesis
of past and present
research in the
domain of study
What have been the
main research
questions?
What are the main
conclusions on
previous research in
this area?
Where are the gaps in
literature?
Which existing
work could be
extended?
Where is existing
knowledge “thin”?
What are the key
Exercise to completeWhich work is own time
in your
areas of debate in this
subject to
area?an A3 copy of challenge?
 Make
the map
 Using post-its, impose on the structure
your assessment of the literature of the
domain associated with your study