Unit 2: Using the Web Effectively &
Evaluating Research Resources
Unit 1: Research Resource
Unit 2: Using the Web Effectively
& Evaluating Research Resource Results
Unit 3:Tracking Down
Unit 4:Managing Your
Unit 5: Ethics in Using
Unit 6: Publishing /
Disseminating Your Research
Unit 7: Using Archives
for Research in Ireland
Unit 8: Using Special
Collections for Research
Assessment: 5 credits
You are required to attend Units 1-6 and to
submit a Short Report (1500 words) which will
be marked on a pass/fail basis, to be
completed by Wednesday 17th December.
Note: There will no percentage mark
Unit 2: Aim and Objectives
• To provide an overview of web resources
for research and networking.
• To examine the importance of critical
thinking skills and to consider the use of an
analytical framework within which to
evaluate the information you find.
Unit 2: Learning Outcomes
On completion of this module the learner should be able to:
•Apply best-practice techniques for identifying, locating
and utilising key web resources in your research area.
•Identify best-practices in effective networking and the
effective use of web technologies.
•Apply criteria for critical appraisal.
•Use an analytical approach to evaluating information.
•Contextualise information in your own research area.
Part 1: Using the Web More Effectively
1. Google & other search engines, portals/gateways, repositories
2. Web Technologies for Information Exchange
- RSS, social bookmarking, blogs, social networking, research
Part 2: Evaluating Information Sources
1. Importance of evaluation
2. Critical Analysis/Appraisal
3. Analytical framework
4. Citation analysis: an introduction
Searching the Web More Effectively: Overview
Web: Indexed Web 2.18 billion pages??
Google: uses index/database of web pages compiled from sites found
by it’s spider programs. Full text of sites sent to Google index.
‘Pagerank’ retrieves based on word occurrence, proximity, location
on page, links to the page, traffic etc. about 100-200 ‘ingredients’.
For the most part a keyword search. See here for more
Anyone can publish: quality control?
Problem: finding relevant scholarly material
Quantity and quality of information
1. Use ‘advanced search’ on Google and other search engines
2. Use Google Scholar, Google Books & ‘more’
3. Try other search engines – not just Google
4. Don’t always use search engines, go straight to good portals
and sites suitable for your topic
5. Use the ‘social web/media’ to locate, bookmark and share
6. Always evaluate your findings for quality
1. Use Advanced Search Techniques
Won’t guarantee quality , but may help control quantity
•Consider: word order, word choice, ‘stop words’,
•Use Google advanced search, operators, domain search, phrase
search etc. see: Inside search and Googleguide
•Remember that Google personalises your search. See ‘Verbatim’
See: Google Basic Search Video and
Google Advanced Search Video on Library website here
2. Use Google Scholar/Books & ‘more’
• Works better on-campus, or from home through the
Databases A-Z (log in to your Library account)
• If no full-text, check the journal portal and the Library
catalogue as usual
• See advanced search and email alerts on Google Scholar
• See also: Microsoft Academic Search as a possible
alternative to Google Scholar
• Look for scholarly web material on:
Religion in the lives of Irish emigrants
3. Use Other Search Engines
• Don’t use same search engine for everything - try others
(examples below). See here also:
• Examples: Exalead, Ask Jeeves, Bing ,Blekko, JURN, Scirus,
Microsoft Academic Search, ScientificWebPlus, iSEEK, BASE,
4. Don’t Always Use Search Engines
Save time by going straight to sites likely to be the most useful –
use gateways, catalogues and repositories.
Examples: Europeana, Euscreen , Internet Archive, H-Net,
DHO Discovery, HSIS, Irish Resources in the Humanities, Digital
Repository of Ireland, Academicinfo.net, Voice of the Shuttle , Library
of Congress (U.S.): digital collections, EDSITEment,
Archives Hub (U.K.), Irish Archives Resource, MHRA,
Social Science Research Network, Infomine, projectbamboo.org , etc.
•Look at subject guides
• Search catalogues from elsewhere:
COPAC, British Library, Library of Congress, The European Library,
Worldcat, Digital Public Library of America, World Digital Library,
Cornucopia Also: Hathi Trust Digital Library, Project Gutenberg,
OAPEN, Librophile, DOAB, ETHOS, DART-Europe E-theses Portal
• Open Repositories: CORA , RIAN, OAIster, Driver, Institutional
repository search U.K. [Unit 6]
• Images: Find images for your coursework & research (guide)
Take a look at the following sites:
• Internet Archive (archiv.org)
• The European Library
See also: 100 time-saving search-engines for serious scholars
5: Use the Social Web/Social Media
• Social web: usually free, easy to use, shared interests, interactive
• Includes: Social bookmarking, Blogs, Wikis, RSS feeds, Social
networking sites, Twitter, etc.
• Why use it as a researcher?
1. Find (& organise) information that you may not find through
2. Network with others who share an interest in your research
Who are ‘Researchers’?
Masters, doctoral, contract researchers, early career researchers,
established academic staff, senior researchers, experts
New Review of Information Networking (2007) 13(2): 81-99
• Multidisciplinary research: dispersed across disciplines,
geographical and other boundaries (e.g. UCC interdisciplinary
• Public & Private research: public-private partnership, knowledge
transfer, technological transfer, business-led collaborative research,
accountability. See: UCC Research Support Services
• Global nature of research: what’s going on elsewhere?
Partnership between universities: Asia, South America
• Importance of sharing information and building effective networks
Formal and Informal: at every stage of the scholarly
• Informal: colleagues, acquaintances, friends, family,
social events etc.
• Formal: professional networking groups, academic
community, project teams, committees, training
programmes, conferences, presentations etc.
• Why network? For information gathering
- your research area
- your college/university/research community
- your career and professional development
Also: disseminating your ideas/research
• Your network supports you:
- Group projects
- Staff/team conflicts
- Peer issues
- ‘Managing’ your boss/supervisor/director of studies
• Sustaining your network:
- Ongoing, strategic, practiced consistently
- Information sharing must be mutually rewarding
• Sharing Information: formally and informally
• Social Bookmarking: store bookmarks remotely, some
allow you to share with others. Examples:
General: Evernote - Delicious – Diigo - Instapaper
Academic: CiteUlike - Mendeley - Papers - Zotero
• RSS Feeds: ‘Really Simple Syndication’ . Information
comes to your reader/aggregator of choice (you don’t have
to go looking for it).
E.G. Netvibes, Feedly, The Old Reader, NewsBlur ,
Bloglines, Feedreader, Genieo, etc.
Feeds from websites, blogs, databases, journals etc.
Note: Feed reader apps for mobile devices
• Blogs: online journal, facilitates discussion
- Create: Blogger, Livejournal, Wordpress, Typepad, tumblr
- Find: Blogsearchengine
See: 38 reasons to blog about your research
• Twitter: micro-blogging; 140 characters; quick updates.
‘Twitter and me: Using Twitter as a PhD Researcher’
• Wikis: create/edit web content; many contributors,
collaborative e.g. Wikipedia
- Create: Wikispaces (basic is free), Mediawiki
- Find: wiki.com, wiki spot
• Social/Academic Networking & Profiles: create a profile,
join a network e.g. Facebook, Google+, LinkedIn,
Academia.edu, ResearchGate, Methodspace ,
Academic Room, MLA Commons
See also: Google Scholar Citations, ORCID, ResearcherID
• Other resources:
- Dropbox , Google Drive, Box.com, OneDrive: upload, share
- Slideshare: share presentations
- Google groups: have discussions and upload documents
Take a look at:
• Academia research networking site
• ResearchGate research networking site
http://blog.impactstory.org/impact-challenge-day-1-academia-• Feeling Better Connected’: Academics’ Use of Social Media
Part 2: Overview: Evaluating Information Sources
• Critically important for 4th level research. Quality sources:
fundamental to developing a review of the literature that can be
used as a foundation on which new research can be built.
• A critical approach helps determine relevance and value to your
• Traditionally: journals, monographs, conference proceedings,
primary sources. Now new information formats e.g. blogs, wikis,
discussion lists, open access journals, open repositories, preprint
• How can you decide if material is of sufficient quality, suitable for
inclusion in your literature review?
Overview: Evaluating Information Sources
Generally we think of:
• Scholarly resources: aimed at those within the field; disseminate
research within that discipline. Scholarly methods, make claims that
are valid and trustworthy.
• Popular resources: aimed at a wider public/mass audience;
entertain, inform, promote viewpoints, sell products & services
So at basic level, is it:
• Factual, methodical, ‘scientific’, based on clearly referenced sources
• subjective, journalistic, personal accounts/impressions, opinion?
But: even material accepted for publication not always reliable:
E.G. 1. Research of Andrew Wakefield (‘MMR controversy’)
o The Lancet 02/28/98, Vol. 351 Issue 9103, p637
o The Lancet 06/03/04, Volume 363 Issue 9411, p750
o The Lancet 06/02/10, Volume 375 Issue9713, p445
o The crash and burn of an autism guru
o Booster shots
o Why did the Lancet take so long?
E.G. 2. Dutch Social Psychologist Diederik Stapel
Articles in: Science, European Journal of Social Psychology, Journal of
Personality and Social Psychology, Journal of Experimental Social
•Diederik Stapel Under Investigation by Dutch Prosecutors
•Dutch University Sacks Social Psychologist Over Faked Data
•Tilburg University Press release
•Dutch News article
•Science journal ‘expression of concern’ and retraction
•Dutch Psychologist Diederik Stapel Relinquishes His Ph.D
•Stapel Investigation Website
• ‘Plastic Fantastic’
• ‘Elsevier published 6 fake journals’
• ‘Merchants of Doubt’ 174.95 ORES
• Joachim Boldt
• Office of Research Integrity
• Focussed approach: process of reflective, reasonable, rational
thinking to gather, interpret and evaluate information
• Adopting an analytical and reflective mind-set at every stage
of the research process will help you to
- find the best sources
- evaluate the information you find there
- make decisions about your research methodology
- discuss new findings
- weigh-up evidence and form conclusions
- make recommendations and contribute effectively to your
i.e. your literature review becomes a solid foundation on which
you can build new research, ideas, theories etc.
An Analytical Framework
An analytical approach to assessing quality/value of
- Currency of information
Apply this checklist to all information sources that you
Why you wish to include a source; appropriate in the
context of your research needs? Screen content and
look for general clues before deciding whether to digest
– Books: look at title, keywords, contents, index
– Journal: look at abstract, keywords, descriptors
– Websites: look at title bar, document title, links to and
from the site, author
• Level: detailed/general/simple?
• Emphasis: not always obvious from keywords/abstract
• Geographical: what countries/regions are included?
• Context of your own research: a unique insight into an
aspect of your own research? Are ideas confirmed/refuted
by the research?
N.B. save time, reduce information overload
• Author: someone whose opinion or testimony is accepted;
affiliated, qualified expert, academic credentials OR
journalist/other author/commentator. Has their research been
frequently cited? Is there a ‘H index’? In what publications? Peer-reviewed?
Where indexed? Impact factor? (see Citation Analysis)
For books: who is the publisher? Established academic
publisher within your field, university press? etc.
• Organisation: is it commercial, non-profit, government,
research/educational? Vested interest? Contactable?
Web Sites: (see also the ‘Internet Detective’)
• Look at the ‘about’ or ‘who are we’ section of the site
• How well established is it and when was the site last updated?
• Evidence of sponsorship?
• Is it clear who wrote the content and is that person contactable?
• URL will often give a clue to type of organisation/country of origin:
- Commercial company? (.com or .co.uk or .biz)
- A non-profit organisation? (.org)
- A government body? (.gov)
- A research/educational organisation? (.org or .ac.uk or .edu)
e.g. www.moriartytribunal.com v www.moriarty-tribunal.ie
3. Method of Production & Methodology
• Type of Publication (Rem: anyone can publish on the web)
- Peer-review procedure and instructions for authors
process by which an academic journal passes a paper
submitted for publication to independent experts for comments
on its suitability and worth; refereeing.
e.g. British Journal of Sociology Information for Authors
- Editorial board members check journal website
e.g. British Journal of Sociology , Mind
See: Google Scholar & Junk Science
See also: Ulrich’s Periodicals Directory
Method of Production & Methodology (contd.)
• Research Methodology:
- Validity: tests what it aims to test? claims made that are
trustworthy? Applicable to the case or circumstances?
- Reliability: Extent to which a measurement made repeatedly in
identical circumstances will yield concordant results, consistent.
Methods appropriate in terms of sample size, use of control groups,
quantitative or qualitative survey design etc. to ensure
What is the objective?
To further understanding? To contribute to the field OR news,
entertainment, opinion; editorial agenda/viewpoint
Sponsors? bias/vested interests? Hidden bias, whether
deliberate or not (GM foods, organic foods, climate change).
Backed up with evidence? Clearly referenced in a bibliography?
Has the evidence been interpreted in a balanced/unbiased manner?
How is the information presented?
- Colour & font, general appearance
- Language and writing style serious/sober OR glossy
- Use of diagrams and images
- Structure and layout; logical?
- Quality of reproduction
- Advertising: minimal/substantial; target
How up to date is the information/research?
• The importance of currency depends on the context within which
you plan to refer to that research in your literature review.
• Factors to consider include:
- Is it clear when the information was produced?
- Does the date of the information meet my requirements?
- Is it obsolete or superseded?
Scholarly Communication Process
• “Scholarly communication is the system through which
research and other scholarly writings are created,
evaluated for quality, disseminated to the scholarly
community, and preserved for future use.” Source: ACRL
(Association of College and Research Libraries)
• A Process: From funding to the eventual dissemination
of research results through formal/traditional and less
• Traditional Approach: from informal to formal with
communication of research occurring at every stage:
Informal: meetings, discussions, seminars, emails,
blogs, social networking sites etc.
Report on-going research at conference (conference
Publication in an academic journal, book AND/OR
completion of thesis
Indexed in research databases, catalogues, repositories
• Recent changes in scholarly communication: new opportunities
and challenges e.g. self-archiving, repositories, open access
publishing, digital humanities, social web
• ‘Peer-reviewed’/‘refereed’ – academic/scholarly
‘The process by which an academic journal passes a paper
submitted for publication to independent experts for comments
on it’s suitability and worth; refereeing’
• Accepted/rejected: contribution to the field/new ideas, bias/conflicts
of interest, suitability for journal
• Types: Double-blind, Single-bind, Open
• Future: Open online peer-review? e.g. blogs, Twitter
[See: ‘Peer-review: a guide for researchers’]
(i.e. Not peer-reviewed):
1. Popular Magazines: (written by journalists/commentators)
- Substantive news e.g. ‘The Economist’, ‘National
Geographic’, ‘New Scientist’
- Other Magazines
2. Trade Journals (specific industry, enable practitioners share
market and product information within an industry) e.g.
‘Macworld’, ‘Restaurant Business’, ‘Chemical Week’. ‘Beverage
World’, ‘Computerworld’ etc.
Citation Analysis (Bibliometrics)
• Number of times a paper or researcher is cited by other
scholars in the field; assumes influential researchers/authors
and important works cited more often. Citations can be used to
develop metrics such as h-index, impact factor etc.
• Where to find these metrics:
1. Web of Science (Thomson Reuters) 9,000+ peer-reviewed
2. Scopus (Elsevier) 16,000+ peer-reviewed journals; more than
4,000 international publishers; 1996 on
3. Scimago Country & Journal Rank Database journals and
country-specific scientific indicators developed from data contained
4. Google Scholar provides links to ‘cited by’ information. ‘Publish
or Perish’ software can generate metrics based on this.
Example: Number of times cited
O’Callaghan, C. and Linehan, D. (2007). "Identity, politics and conflict in
dockland development in Cork, Ireland: European Capital of Culture 2005."
Cities 24(4): 311-323.
1. Go to Web of Knowledge
2. Pick ‘Web of Science’
3. Enter the Title to locate article
4. Note Number of ‘Times Cited’
5. Go to ‘Scopus’
6. Enter the Title to locate article
7. Note ‘Cited by’
5. Go to Google Scholar
6. Type article title in box
7. Note ‘Cited by -’
N.B Number of cites can only provide an indication of
Article Title: The second generation problematic: Rethinking democracy
and civil-military relations
Author(s): Cottey, A., Edmunds, T., Forster, A.
Source: ARMED FORCES & SOCIETY 29(1) Pages: 31- 56
How many times has this paper been cited in:
• Web of Science?
• Google Scholar?
• Journal Impact Factor: average number of times
articles from the journal published in past two years
have been cited in JCR year
• ‘Journal Citation Reports’ (Journal Performance
Metrics) includes impact factors. Part of Thomson
Reuter’s Web of Science
• Scimago Country & Journal Rank Database: (free)
journals and country scientific indicators developed
from information in ‘Scopus’
Examples: Impact Factor
1. What is the impact factor of the Journal ‘Cities’?
1. Go to Web of Knowledge
2. Click on ‘Select a database’
3. Pick ‘Journal Citation Reports’
4. Choose ‘JCR Social Science Edition’ and ‘Search for Specific Journal’
5. Click on ‘submit’
6. Type ‘Cities’ in box and search for ‘Full Journal Title’
Impact factor 2013 = 1.836
2. What are the highest impact journals for History?
1. Go to Web of Knowledge
2. Click on ‘Select a database’
3. Pick ‘Journal Citation Reports’
4. Choose ‘JCR Social Science Edition’ and ‘View a Group of Journals by
6. Click on ‘Submit’
7. Scroll down to ‘History’’
8. Click on ‘Submit’
9. Sort by Impact factor
Scopus and Scimago
Scimago Country & Journal Rank Database: (free) journals and
country scientific indicators developed from information in
‘Scopus’ - better for humanities and social sciences
• N.B. SJR: ‘Scimago Journal Rank’ reflects prestige of source
where such citations come from. Covers a 3 year citation
• SNIP: ‘Source normalized impact per paper’: corrects for
differences in the frequency of citation across research fields-citations
given a weighting based on the total number of
citations in that field. N.B. Scopus only
• What is the 2013 impact factor for the
journal ‘Applied Linguistics’?
• What is the SJR (from Scimago website)?
• What Sociology journal had the highest
impact factor for 2013?
• Look for the same in Scimago
A scientist has index h if h of his/her Np papers have at least h
citations each, and the other (Np-h) papers have no more than h
•highest number of papers a scientist has that have each received at
least that number of citations e.g. someone with h-index 50 has written
50 papers that have each had at least 50 citations.
Where to find it:
– Web of Science
– Google Scholar
– Publish or Perish
• InCites: uses Web of Science data to generate metrics for
institutions and units within those institutions -
• Essential Science Indicators: Institution rankings and
top papers derived from Wed of Science Data ‘NATL UNIV
IRELAND UNIV COLL CORK’
• Webometrics: quantifying the impact of web sites
• Altmetrics: attempt to track the impact of scholarly works
in the social web
• National systems to classify journals, publishers
How many times cited?
Stapel, DA. And Koomen, W (2001).
“I, we, and the effects of others on me: How self-construal
level moderates social comparison effects.”
Journal of Personality and Social Psychology 80(5): 766-
Note: Citation data is then used to create h-index for authors,
impact factor of the journal, metrics for the university etc. etc.
Useful tools, but not a measure of true quality e.g.:
• current popularity of topic & availability of article/journal: increases
citations / higher IF
• IF based on average over all articles: underestimates citations of
top cited articles, exaggerates number of citations of the average
• comparison of impact factors between different fields is invalid
e.g. not as relevant for literature (books citing other books)
• coverage differs in each resource (Web of Science, Scopus,
Google Scholar); none is complete and reliable e.g. for IF only ISI
database journals used; undercounts number of citations from other
journals e.g. ‘less-developed’ countries, other languages
Bibliometrics- limitations (contd.)
• Why have the papers been cited? e.g. Review articles, Wakefield
• Well over 50% of papers are never cited
• What about books, book chapters, conference papers, digital
• Takes time for papers to collect citations, especially in some fields
• H-index is a poor metric for young researchers
• H-index underestimates the importance of seminal articles
• Pressure on academics to publish may lead to a drop in quality?
• Google metrics are not yet reliable enough in isolation
Impact Factor- limitations (contd.)
• ‘Like nuclear energy, the impact factor is a mixed
• ‘The use of journal impacts in evaluating individuals
has its inherent dangers. In an ideal world, evaluators
would read each article and make personal
Garfield, E. (2006) ‘The History and Meaning of the
Impact Factor’. JAMA, 295(1): pp 90-93 [Garfield
invented scholarly citation in the 1960’s]
H-index - limitations (contd.)
‘Obviously a single number can never give more than a
rough approximation to an individual’s multifaceted
profile, and many other factors should be considered
in combination in evaluating an individual.’
Hirsch, J.E. (2005) ‘An index to quantify an
individual's scientific research output’.
arXiv:physics/0508025 v5 29 Sep 2006
(Hirsch proposed the H-index in 2005)
• Can’t always rely on citation data etc.
• Each researcher must make their own judgment about the
‘quality’ and suitability of any article or information source.
• Depends a lot on the context i.e. how the research is to be
used or referred to in your literature review. You may need
to draw on material from newspapers, conferences, web
material etc. with no peer review available.
• Refer to the framework/checklist outlined above and check
with your supervisor when in doubt.
• Evaluation is an art; there no perfect indicator of quality.
You need to look for clues, and ultimately judge on the
basis of usefulness for your research question