Your SlideShare is downloading. ×
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. PG6009 Unit 2: Using the Web Effectively & Evaluating Research Resources
  • 2. Module Structure Unit 1: Research Resource Discovery Unit 2: Using the Web Effectively & Evaluating Research Resource Results Unit 3:Tracking Down Results & Keeping Up-to-date Unit 4:Managing Your Information Unit 5: Ethics in Using Research Information Unit 6: Publishing / Disseminating Your Research Unit 7: Using Archives for Research in Ireland Unit 8: Using Special Collections for Research in Ireland Compulsory
  • 3. Assessment: 5 credits You are required to attend Units 1-6 and to submit a Short Report (1500 words) which will be marked on a pass/fail basis, to be completed by Wednesday 17th December. Note: There will no percentage mark awarded.
  • 4. Unit 2: Aim and Objectives • To provide an overview of web resources for research and networking. • To examine the importance of critical thinking skills and to consider the use of an analytical framework within which to evaluate the information you find.
  • 5. Unit 2: Learning Outcomes On completion of this module the learner should be able to: •Apply best-practice techniques for identifying, locating and utilising key web resources in your research area. •Identify best-practices in effective networking and the effective use of web technologies. •Apply criteria for critical appraisal. •Use an analytical approach to evaluating information. •Contextualise information in your own research area.
  • 6. Topics Part 1: Using the Web More Effectively 1. Google & other search engines, portals/gateways, repositories 2. Web Technologies for Information Exchange - RSS, social bookmarking, blogs, social networking, research profiles etc. Part 2: Evaluating Information Sources 1. Importance of evaluation 2. Critical Analysis/Appraisal 3. Analytical framework 4. Citation analysis: an introduction
  • 7. Searching the Web More Effectively: Overview Web: Indexed Web 2.18 billion pages?? Google: uses index/database of web pages compiled from sites found by it’s spider programs. Full text of sites sent to Google index. ‘Pagerank’ retrieves based on word occurrence, proximity, location on page, links to the page, traffic etc. about 100-200 ‘ingredients’. For the most part a keyword search. See here for more Anyone can publish: quality control? Problem: finding relevant scholarly material Quantity and quality of information
  • 8. Tips 1. Use ‘advanced search’ on Google and other search engines 2. Use Google Scholar, Google Books & ‘more’ 3. Try other search engines – not just Google 4. Don’t always use search engines, go straight to good portals and sites suitable for your topic 5. Use the ‘social web/media’ to locate, bookmark and share information 6. Always evaluate your findings for quality
  • 9. 1. Use Advanced Search Techniques Won’t guarantee quality , but may help control quantity •Consider: word order, word choice, ‘stop words’, •Use Google advanced search, operators, domain search, phrase search etc. see: Inside search and Googleguide •Remember that Google personalises your search. See ‘Verbatim’ tool. See: Google Basic Search Video and Google Advanced Search Video on Library website here
  • 10. 2. Use Google Scholar/Books & ‘more’ • Works better on-campus, or from home through the Databases A-Z (log in to your Library account) • If no full-text, check the journal portal and the Library catalogue as usual • See advanced search and email alerts on Google Scholar • See also: Microsoft Academic Search as a possible alternative to Google Scholar
  • 11. Practice • Look for scholarly web material on: Religion in the lives of Irish emigrants
  • 12. 3. Use Other Search Engines • Don’t use same search engine for everything - try others (examples below). See here also: • Examples: Exalead, Ask Jeeves, Bing ,Blekko, JURN, Scirus, Microsoft Academic Search, ScientificWebPlus, iSEEK, BASE, Yippy
  • 13. 4. Don’t Always Use Search Engines Save time by going straight to sites likely to be the most useful – use gateways, catalogues and repositories. Examples: Europeana, Euscreen , Internet Archive, H-Net, DHO Discovery, HSIS, Irish Resources in the Humanities, Digital Repository of Ireland,, Voice of the Shuttle , Library of Congress (U.S.): digital collections, EDSITEment, Archives Hub (U.K.), Irish Archives Resource, MHRA, Social Science Research Network, Infomine, , etc. etc. •Look at subject guides
  • 14. 4. (contd.) • Search catalogues from elsewhere: COPAC, British Library, Library of Congress, The European Library, Worldcat, Digital Public Library of America, World Digital Library, Cornucopia Also: Hathi Trust Digital Library, Project Gutenberg, OAPEN, Librophile, DOAB, ETHOS, DART-Europe E-theses Portal • Open Repositories: CORA , RIAN, OAIster, Driver, Institutional repository search U.K. [Unit 6] • Images: Find images for your coursework & research (guide)
  • 15. Practice Take a look at the following sites: • Worldcat • Europeana • Internet Archive ( • H-Net • The European Library • RIAN • OAIster See also: 100 time-saving search-engines for serious scholars
  • 16. 5: Use the Social Web/Social Media • Social web: usually free, easy to use, shared interests, interactive • Includes: Social bookmarking, Blogs, Wikis, RSS feeds, Social networking sites, Twitter, etc. • Why use it as a researcher? 1. Find (& organise) information that you may not find through regular sources 2. Network with others who share an interest in your research area
  • 17. Research Environment Who are ‘Researchers’? Masters, doctoral, contract researchers, early career researchers, established academic staff, senior researchers, experts New Review of Information Networking (2007) 13(2): 81-99 Important trends: • Multidisciplinary research: dispersed across disciplines, geographical and other boundaries (e.g. UCC interdisciplinary institutes) • Public & Private research: public-private partnership, knowledge transfer, technological transfer, business-led collaborative research, accountability. See: UCC Research Support Services • Global nature of research: what’s going on elsewhere? Partnership between universities: Asia, South America • Importance of sharing information and building effective networks
  • 18. Networking (1.) Formal and Informal: at every stage of the scholarly communication process • Informal: colleagues, acquaintances, friends, family, social events etc. • Formal: professional networking groups, academic community, project teams, committees, training programmes, conferences, presentations etc. • Why network? For information gathering - your research area - your college/university/research community - your career and professional development Also: disseminating your ideas/research
  • 19. Networking (2.) • Your network supports you: - Group projects - Staff/team conflicts - Peer issues - ‘Managing’ your boss/supervisor/director of studies • Sustaining your network: - Ongoing, strategic, practiced consistently - Information sharing must be mutually rewarding • Sharing Information: formally and informally
  • 20. • Social Bookmarking: store bookmarks remotely, some allow you to share with others. Examples: General: Evernote - Delicious – Diigo - Instapaper Academic: CiteUlike - Mendeley - Papers - Zotero • RSS Feeds: ‘Really Simple Syndication’ . Information comes to your reader/aggregator of choice (you don’t have to go looking for it). E.G. Netvibes, Feedly, The Old Reader, NewsBlur , Bloglines, Feedreader, Genieo, etc. Feeds from websites, blogs, databases, journals etc. Note: Feed reader apps for mobile devices
  • 21. • Blogs: online journal, facilitates discussion - Create: Blogger, Livejournal, Wordpress, Typepad, tumblr - Find: Blogsearchengine See: 38 reasons to blog about your research • Twitter: micro-blogging; 140 characters; quick updates. ‘Twitter and me: Using Twitter as a PhD Researcher’ • Wikis: create/edit web content; many contributors, collaborative e.g. Wikipedia - Create: Wikispaces (basic is free), Mediawiki - Find:, wiki spot
  • 22. • Social/Academic Networking & Profiles: create a profile, join a network e.g. Facebook, Google+, LinkedIn,, ResearchGate, Methodspace , Academic Room, MLA Commons See also: Google Scholar Citations, ORCID, ResearcherID • Other resources: - Dropbox , Google Drive,, OneDrive: upload, share docs - Slideshare: share presentations - Google groups: have discussions and upload documents - Storify etc.
  • 23. Practice Take a look at: • Academia research networking site • ResearchGate research networking site •• Feeling Better Connected’: Academics’ Use of Social Media
  • 24. Part 2: Overview: Evaluating Information Sources • Critically important for 4th level research. Quality sources: fundamental to developing a review of the literature that can be used as a foundation on which new research can be built. • A critical approach helps determine relevance and value to your field. • Traditionally: journals, monographs, conference proceedings, primary sources. Now new information formats e.g. blogs, wikis, discussion lists, open access journals, open repositories, preprint repositories etc. • How can you decide if material is of sufficient quality, suitable for inclusion in your literature review?
  • 25. Overview: Evaluating Information Sources Generally we think of: • Scholarly resources: aimed at those within the field; disseminate research within that discipline. Scholarly methods, make claims that are valid and trustworthy. • Popular resources: aimed at a wider public/mass audience; entertain, inform, promote viewpoints, sell products & services (vested interests). So at basic level, is it: • Factual, methodical, ‘scientific’, based on clearly referenced sources and documentation? OR • subjective, journalistic, personal accounts/impressions, opinion?
  • 26. But: even material accepted for publication not always reliable: E.G. 1. Research of Andrew Wakefield (‘MMR controversy’) o The Lancet 02/28/98, Vol. 351 Issue 9103, p637 o The Lancet 06/03/04, Volume 363 Issue 9411, p750 o The Lancet 06/02/10, Volume 375 Issue9713, p445 See also: o The crash and burn of an autism guru o Booster shots o Why did the Lancet take so long?
  • 27. E.G. 2. Dutch Social Psychologist Diederik Stapel Articles in: Science, European Journal of Social Psychology, Journal of Personality and Social Psychology, Journal of Experimental Social Psychology etc. •Diederik Stapel Under Investigation by Dutch Prosecutors •Dutch University Sacks Social Psychologist Over Faked Data •Tilburg University Press release •Dutch News article •Science journal ‘expression of concern’ and retraction •Dutch Psychologist Diederik Stapel Relinquishes His Ph.D •Stapel Investigation Website
  • 28. More examples: • ‘Plastic Fantastic’ • ‘Elsevier published 6 fake journals’ • ‘Merchants of Doubt’ 174.95 ORES • Woo-Suk-Hwang • Joachim Boldt • Office of Research Integrity
  • 29. Critical Analysis/Appraisal • Focussed approach: process of reflective, reasonable, rational thinking to gather, interpret and evaluate information • Adopting an analytical and reflective mind-set at every stage of the research process will help you to - find the best sources - evaluate the information you find there - make decisions about your research methodology - discuss new findings - weigh-up evidence and form conclusions - make recommendations and contribute effectively to your field i.e. your literature review becomes a solid foundation on which you can build new research, ideas, theories etc.
  • 30. An Analytical Framework An analytical approach to assessing quality/value of information: - Relevance - Authority - Method - Objectivity - Presentation - Currency of information Apply this checklist to all information sources that you use
  • 31. 1. Relevance Why you wish to include a source; appropriate in the context of your research needs? Screen content and look for general clues before deciding whether to digest or reject: – Books: look at title, keywords, contents, index – Journal: look at abstract, keywords, descriptors – Websites: look at title bar, document title, links to and from the site, author
  • 32. Relevance (contd.) Focus on: • Level: detailed/general/simple? • Emphasis: not always obvious from keywords/abstract • Geographical: what countries/regions are included? • Context of your own research: a unique insight into an aspect of your own research? Are ideas confirmed/refuted by the research? N.B. save time, reduce information overload
  • 33. 2. Authority • Author: someone whose opinion or testimony is accepted; affiliated, qualified expert, academic credentials OR journalist/other author/commentator. Has their research been frequently cited? Is there a ‘H index’? In what publications? Peer-reviewed? Where indexed? Impact factor? (see Citation Analysis) For books: who is the publisher? Established academic publisher within your field, university press? etc. • Organisation: is it commercial, non-profit, government, research/educational? Vested interest? Contactable?
  • 34. Authority (contd.) Web Sites: (see also the ‘Internet Detective’) • Look at the ‘about’ or ‘who are we’ section of the site • How well established is it and when was the site last updated? • Evidence of sponsorship? • Is it clear who wrote the content and is that person contactable? • URL will often give a clue to type of organisation/country of origin: - Commercial company? (.com or or .biz) - A non-profit organisation? (.org) - A government body? (.gov) - A research/educational organisation? (.org or or .edu) e.g. v
  • 35. 3. Method of Production & Methodology • Type of Publication (Rem: anyone can publish on the web) - Peer-review procedure and instructions for authors Peer review/Refereed/Juried: process by which an academic journal passes a paper submitted for publication to independent experts for comments on its suitability and worth; refereeing. e.g. British Journal of Sociology Information for Authors e.g. Addiction - Editorial board members check journal website e.g. British Journal of Sociology , Mind See: Google Scholar & Junk Science See also: Ulrich’s Periodicals Directory
  • 36. Method of Production & Methodology (contd.) • Research Methodology: - Validity: tests what it aims to test? claims made that are trustworthy? Applicable to the case or circumstances? - Reliability: Extent to which a measurement made repeatedly in identical circumstances will yield concordant results, consistent. Methods appropriate in terms of sample size, use of control groups, quantitative or qualitative survey design etc. to ensure reproducibility?
  • 37. 4. Objectivity: What is the objective? To further understanding? To contribute to the field OR news, entertainment, opinion; editorial agenda/viewpoint Sponsors? bias/vested interests? Hidden bias, whether deliberate or not (GM foods, organic foods, climate change). Backed up with evidence? Clearly referenced in a bibliography? Has the evidence been interpreted in a balanced/unbiased manner?
  • 38. 5. Presentation How is the information presented? - Colour & font, general appearance - Language and writing style serious/sober OR glossy appearance - Use of diagrams and images - Structure and layout; logical? - Quality of reproduction - Advertising: minimal/substantial; target audience
  • 39. 6. Currency How up to date is the information/research? • The importance of currency depends on the context within which you plan to refer to that research in your literature review. • Factors to consider include: - Is it clear when the information was produced? - Does the date of the information meet my requirements? - Is it obsolete or superseded?
  • 40. Scholarly Communication Process • “Scholarly communication is the system through which research and other scholarly writings are created, evaluated for quality, disseminated to the scholarly community, and preserved for future use.” Source: ACRL (Association of College and Research Libraries) • A Process: From funding to the eventual dissemination of research results through formal/traditional and less formal mechanisms
  • 41. Scholarly Communication • Traditional Approach: from informal to formal with communication of research occurring at every stage: Informal: meetings, discussions, seminars, emails, blogs, social networking sites etc. Report on-going research at conference (conference proceedings) Publication in an academic journal, book AND/OR completion of thesis Indexed in research databases, catalogues, repositories • Recent changes in scholarly communication: new opportunities and challenges e.g. self-archiving, repositories, open access publishing, digital humanities, social web
  • 42. Peer-review • ‘Peer-reviewed’/‘refereed’ – academic/scholarly ‘The process by which an academic journal passes a paper submitted for publication to independent experts for comments on it’s suitability and worth; refereeing’ • Accepted/rejected: contribution to the field/new ideas, bias/conflicts of interest, suitability for journal • Types: Double-blind, Single-bind, Open • Future: Open online peer-review? e.g. blogs, Twitter [See: ‘Peer-review: a guide for researchers’]
  • 43. Other Periodicals (i.e. Not peer-reviewed): 1. Popular Magazines: (written by journalists/commentators) - Newspapers - Substantive news e.g. ‘The Economist’, ‘National Geographic’, ‘New Scientist’ - Other Magazines 2. Trade Journals (specific industry, enable practitioners share market and product information within an industry) e.g. ‘Macworld’, ‘Restaurant Business’, ‘Chemical Week’. ‘Beverage World’, ‘Computerworld’ etc.
  • 44. Citation Analysis (Bibliometrics) • Number of times a paper or researcher is cited by other scholars in the field; assumes influential researchers/authors and important works cited more often. Citations can be used to develop metrics such as h-index, impact factor etc. • Where to find these metrics: 1. Web of Science (Thomson Reuters) 9,000+ peer-reviewed journals 2. Scopus (Elsevier) 16,000+ peer-reviewed journals; more than 4,000 international publishers; 1996 on 3. Scimago Country & Journal Rank Database journals and country-specific scientific indicators developed from data contained in ‘Scopus.’ 4. Google Scholar provides links to ‘cited by’ information. ‘Publish or Perish’ software can generate metrics based on this.
  • 45. Example: Number of times cited O’Callaghan, C. and Linehan, D. (2007). "Identity, politics and conflict in dockland development in Cork, Ireland: European Capital of Culture 2005." Cities 24(4): 311-323. 1. Go to Web of Knowledge 2. Pick ‘Web of Science’ 3. Enter the Title to locate article 4. Note Number of ‘Times Cited’ 5. Go to ‘Scopus’ 6. Enter the Title to locate article 7. Note ‘Cited by’ 5. Go to Google Scholar 6. Type article title in box 7. Note ‘Cited by -’ N.B Number of cites can only provide an indication of ‘quality’.
  • 46. Exercise Article Title: The second generation problematic: Rethinking democracy and civil-military relations Author(s): Cottey, A., Edmunds, T., Forster, A. Source: ARMED FORCES & SOCIETY 29(1) Pages: 31- 56 Published: 2002 How many times has this paper been cited in: • Web of Science? • Scopus? • Google Scholar?
  • 47. Journal Impact • Journal Impact Factor: average number of times articles from the journal published in past two years have been cited in JCR year Check: • ‘Journal Citation Reports’ (Journal Performance Metrics) includes impact factors. Part of Thomson Reuter’s Web of Science • Scimago Country & Journal Rank Database: (free) journals and country scientific indicators developed from information in ‘Scopus’
  • 48. Examples: Impact Factor 1. What is the impact factor of the Journal ‘Cities’? 1. Go to Web of Knowledge 2. Click on ‘Select a database’ 3. Pick ‘Journal Citation Reports’ 4. Choose ‘JCR Social Science Edition’ and ‘Search for Specific Journal’ 5. Click on ‘submit’ 6. Type ‘Cities’ in box and search for ‘Full Journal Title’ Impact factor 2013 = 1.836 2. What are the highest impact journals for History? 1. Go to Web of Knowledge 2. Click on ‘Select a database’ 3. Pick ‘Journal Citation Reports’ 4. Choose ‘JCR Social Science Edition’ and ‘View a Group of Journals by Subject Category’ 6. Click on ‘Submit’ 7. Scroll down to ‘History’’ 8. Click on ‘Submit’ 9. Sort by Impact factor
  • 49. Journal impact Scopus and Scimago Scimago Country & Journal Rank Database: (free) journals and country scientific indicators developed from information in ‘Scopus’ - better for humanities and social sciences • N.B. SJR: ‘Scimago Journal Rank’ reflects prestige of source where such citations come from. Covers a 3 year citation window. • SNIP: ‘Source normalized impact per paper’: corrects for differences in the frequency of citation across research fields-citations given a weighting based on the total number of citations in that field. N.B. Scopus only
  • 50. Exercise • What is the 2013 impact factor for the journal ‘Applied Linguistics’? • What is the SJR (from Scimago website)? • What Sociology journal had the highest impact factor for 2013? • Look for the same in Scimago
  • 51. H-Index A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np-h) papers have no more than h citations each •highest number of papers a scientist has that have each received at least that number of citations e.g. someone with h-index 50 has written 50 papers that have each had at least 50 citations. Where to find it: – Web of Science – Scopus – Google Scholar – Publish or Perish
  • 52. Other Metrics • InCites: uses Web of Science data to generate metrics for institutions and units within those institutions - benchmarking • Essential Science Indicators: Institution rankings and top papers derived from Wed of Science Data ‘NATL UNIV IRELAND UNIV COLL CORK’ • Webometrics: quantifying the impact of web sites • Altmetrics: attempt to track the impact of scholarly works in the social web • National systems to classify journals, publishers
  • 53. How many times cited? Stapel, DA. And Koomen, W (2001). “I, we, and the effects of others on me: How self-construal level moderates social comparison effects.” Journal of Personality and Social Psychology 80(5): 766- 781. Note: Citation data is then used to create h-index for authors, impact factor of the journal, metrics for the university etc. etc.
  • 54. Bibliometrics- limitations Useful tools, but not a measure of true quality e.g.: • current popularity of topic & availability of article/journal: increases citations / higher IF • IF based on average over all articles: underestimates citations of top cited articles, exaggerates number of citations of the average article • comparison of impact factors between different fields is invalid e.g. not as relevant for literature (books citing other books) • coverage differs in each resource (Web of Science, Scopus, Google Scholar); none is complete and reliable e.g. for IF only ISI database journals used; undercounts number of citations from other journals e.g. ‘less-developed’ countries, other languages
  • 55. Bibliometrics- limitations (contd.) • Why have the papers been cited? e.g. Review articles, Wakefield • Well over 50% of papers are never cited • What about books, book chapters, conference papers, digital humanities etc.? • Takes time for papers to collect citations, especially in some fields • H-index is a poor metric for young researchers • H-index underestimates the importance of seminal articles • Pressure on academics to publish may lead to a drop in quality? • Google metrics are not yet reliable enough in isolation
  • 56. Impact Factor- limitations (contd.) • ‘Like nuclear energy, the impact factor is a mixed blessing.’ • ‘The use of journal impacts in evaluating individuals has its inherent dangers. In an ideal world, evaluators would read each article and make personal judgments.’ Garfield, E. (2006) ‘The History and Meaning of the Impact Factor’. JAMA, 295(1): pp 90-93 [Garfield invented scholarly citation in the 1960’s]
  • 57. H-index - limitations (contd.) ‘Obviously a single number can never give more than a rough approximation to an individual’s multifaceted profile, and many other factors should be considered in combination in evaluating an individual.’ Hirsch, J.E. (2005) ‘An index to quantify an individual's scientific research output’. arXiv:physics/0508025 v5 29 Sep 2006 (Hirsch proposed the H-index in 2005)
  • 58. Final word: • Can’t always rely on citation data etc. • Each researcher must make their own judgment about the ‘quality’ and suitability of any article or information source. • Depends a lot on the context i.e. how the research is to be used or referred to in your literature review. You may need to draw on material from newspapers, conferences, web material etc. with no peer review available. • Refer to the framework/checklist outlined above and check with your supervisor when in doubt. • Evaluation is an art; there no perfect indicator of quality. You need to look for clues, and ultimately judge on the basis of usefulness for your research question
  • 59. Slides: Further reading: