0
15/06/14 www.rba.co.uk 1
Karen Blakeman
karen.blakeman@rba.co.uk, www.rba.co.uk
Slides available at http://www.rba.co.uk/a...
All change!
Search engines
– new algorithms, ranking and display, quality of results
– EU ruling on “right to be forgotten...
Impact of Social Sciences – The right to read is the right
to mine: Text and data mining copyright exceptions introduced i...
Search engines
Google in particular is undergoing major change
Don’t forget alternative search engines (they may still be
...
Five things you need to know about Google search
1. Google personalises your search
Personalises search based on
– locatio...
Private browsing - quickest way “un-personalise”search
Chrome - New Incognito window Ctrl+Shift+N
FireFox Ctrl+Shift+P
Int...
Five things you need to know about Google search
2. Google automatically looks for variations on your search terms
and som...
Google Verbatim
Google now showing missing search terms?
Not always shown – possibly still a live experiment?
15/06/14 www.rba.co.uk 9
Five things you need to know about Google search
3. Google web search does not search everything it has in its
database
– ...
Five things you need to know about Google search
4. Google changes its algorithms several hundred times a
year
How Google ...
Five things you need to know about Google search
5. We are all Google’s lab rats
Just Testing: Google Users May See Up To ...
What I see on my screen will not be what
you see on your screen, will not be what
your colleagues see on theirs, will not ...
Hummingbird
Not just an update but a completely new algorithm
Tries to make “sense” of your query and put it into context,...
So called “right to be forgotten” ruling
15/06/14 www.rba.co.uk 15
Edition of Monday, January
19, 1998, page 23 -
Newspape...
Information is NOT removed from the web
Subject can apply to have links in search results that point to
specific informati...
Could have a serious impact on business and
legal research
How far will it go?
People already trying to get links to their...
Four things we’ve learned from the EU Google judgment | ICO
Blog
http://iconewsblog.wordpress.com/2014/05/20/four-things-w...
Last Week Tonight with John Oliver (HBO)
Right To Be Forgotten - YouTube
https://www.youtube.com/watch?v=r-ERajkMXw0&featu...
How to get around it?
Until we see how it works in practice, difficult to give advice
Google says it will indicate on the ...
Search commands (Google, Bing)
Think file format
– PDF for research documents, government reports, industry papers,
compan...
Search commands (Google, Bing)
Filetype may not always work
– The filetype: command may not always work even when a site
o...
Search commands – site: (most search engines)
Site search
For searching large websites, or groups of sites by type for
exa...
Searching in multiple languages
A significant amount of information is in the local language
Google has removed the extrem...
15/06/14 www.rba.co.uk 25
Evaluating resources
Date of publication, 'last updated'
Check text for clues of publication date
Stated date for a web pa...
Who owns a website?
http://www.domaintools.com/ good starting point for
identifying who owns a domain name but....
– may b...
Domaintools check – good news
15/06/14 www.rba.co.uk 28
Domaintools check – what you do not want to see
15/06/14 www.rba.co.uk 29
Quality of statistics
Read the definitions
– Definitions and scope may have changed over the years and
may be different fo...
Some web sites......
15/06/14 www.rba.co.uk 31
Gov.uk
15/06/14 www.rba.co.uk 32
UK Government Web Archive | The National Archives
http://www.nationalarchives.gov.uk/webarchive/
Browse by category or cho...
UK Government Web Archive | The National Archives
http://www.nationalarchives.gov.uk/webarchive/
15/06/14 www.rba.co.uk 34
Wayback Machine http://www.archive.org/
15/06/14 www.rba.co.uk 35
http://www.legislation.gov.uk/
Launched in 2010
Not all legislation has been updated
Pending updates are flagged
15/06/14 ...
UK Parliament http://www.parliament.uk/
15/06/14 www.rba.co.uk 37
Monitoring progress of legislation
http://services.parliament.uk/bills/2013-14/pensions.html
15/06/14 www.rba.co.uk 38
data.parliament.uk http://www.data.parliament.uk/
15/06/14 www.rba.co.uk 39
The Gazette | Official Public Record
https://www.thegazette.co.uk/
15/06/14 www.rba.co.uk 40
FLARE - Foreign Law Research Group Website
http://ials.sas.ac.uk/flare/flare.htm
15/06/14 www.rba.co.uk 41
FLARE - Union List of Official Gazettes: Europe
http://ials.sas.ac.uk/flare/flare_fog_unionlist_europe.htm
15/06/14 www.rb...
They work for you http://www.theyworkforyou.com/
15/06/14 www.rba.co.uk 43
WhatDoTheyKnow - Freedom of Information (FOI)
requests https://www.whatdotheyknow.com/
15/06/14 www.rba.co.uk 44
Official company information
Companies House http://www.companieshouse.gov.uk/
Lists of official company registers
– Offic...
Company Check
http://companycheck.co.uk/
UK & Ireland companies and director search
Some info free, some free with login, ...
Company Check
15/06/14 www.rba.co.uk 47
Company Check
15/06/14
www.rba.co.uk 48
Company Check Dashboard
15/06/14 www.rba.co.uk 49
DUEDIL http://www.duedil.com/
15/06/14 www.rba.co.uk 50
OpenCorporates https://opencorporates.com/
15/06/14 www.rba.co.uk 51
OpenCorporates http://opencorporates.com/
15/06/14 www.rba.co.uk 52
OpenCorporates https://opencorporates.com/
15/06/14 www.rba.co.uk 53
OpenCorporates and Google Refine
15/06/14 www.rba.co.uk 54
How to use OpenCorporates to match
companies in Google Refine o...
OpenCharities http://opencharities.org/
15/06/14 www.rba.co.uk 55
OpenCharities http://opencharities.org/
15/06/14 www.rba.co.uk 56
Mapping Corporate Networks With OpenCorporates
http://www.slideshare.net/psychemedia/mapping-corporate-netw
15/06/14 www.r...
ICIJ Offshore Leaks Database
http://offshoreleaks.icij.org/
15/06/14 www.rba.co.uk 58
ICIJ Releases Offshore Leaks Databas...
Open Access Research Literature
15/06/14 www.rba.co.uk 59
Open Access vs open access vs open access
Open Access
Publicly funded research made available free of charge to the
user
O...
Mandated Open Access
US
All research publications resulting from work funded by the US
National Institutes of Health are e...
UK Gold versus Green OA
Gold OA
– researchers publish their articles in journals that offer open access
publishing (can be...
Jeffrey Beall
List of Predatory Publishers 2014 | Scholarly Open Access
http://scholarlyoa.com/2014/01/02/list-of-predator...
Fragmentation of open access
Where are the open access publications?
– Individual OA articles within existing subscription...
ResearchGate
http://www.researchgate.net/
Increasingly used to request copies of articles from authors
15/06/14 www.rba.co...
Google Scholar http://scholar.google.com/
15/06/14 www.rba.co.uk 66
Google Scholar
Does not cover all key journals in all subjects – no source list
Top publications for subjects and language...
Google Scholar
15/06/14 www.rba.co.uk 68
Google Scholar
Includes open access material, pre-prints, institutional
repositories (but not necessarily author self arch...
Google Scholar
15/06/14 www.rba.co.uk 70
Does NOT use the publishers’ metadata
Sometimes gets the author wrong
Beware the ...
Institutional repositories and open access
BASE - Bielefeld Academic Search Engine http://www.base-search.net/
CORE (COnne...
Specialist search tools for research information
A selection can be found at
http://www.rba.co.uk/search/links.shtml#resea...
Specialist search tools for research information
Europe PubMed Central http://europepmc.org/
Mendeley http://www.mendeley....
Elsevier’s take down notices
Elsevier clamps down on academics posting their own papers
online (Wired UK)
http://www.wired...
Grey literature
Literature that has been “peer reviewed” or assessed/approved in
some way by colleagues or subject experts...
OpenGrey http://www.opengrey.eu/
15/06/14 www.rba.co.uk 76
Article may be OA but public access outside of the
institution may be not allowed or difficult
15/06/14 www.rba.co.uk 77
W...
BBC News - Public libraries get online access to
research journals
http://www.bbc.co.uk/news/education-25981183
15/06/14 w...
Public Library Initiative by PLS and ProQuest | Access to
Research http://www.accesstoresearch.org.uk/
List of participati...
Free patent information
Patent Searching 101: A Patent Search Tutorial
IPWatchdog.com Patents & Patent Law
http://www.ipwa...
Google Patents
https://www.google.com/patents
Coverage:
– US
– Canada
– European Patent Office (EPO)
– Germany
– China
– W...
Google Patents
15/06/14 www.rba.co.uk 82
Google Patents advanced search
15/06/14 www.rba.co.uk 83
https://www.google.co.uk/advanced_patent_search
Statistics
More open data but....
Raw data – no pretty layouts or visualisations, you have to do
that
Data may need weedin...
Official statistics
OFFSTATS http://www.offstats.auckland.ac.nz/
UK National Statistics Publication Hub http://www.statist...
http://www.offstats.auckland.ac.nz/
15/06/14 www.rba.co.uk 86
UK National Statistics & ONS
http://www.statistics.gov.uk/, http://www.ons.gov.uk/
15/06/14 www.rba.co.uk 87
Publication Hub (http://www.statistics.gov.uk/) is an “index” to
what is available and links through to other sites
ONS (h...
Publication Hub search (statistics.gov.uk)
15/06/14 www.rba.co.uk 89
http://statswales.wales.gov.uk/
15/06/14 www.rba.co.uk 90
data.gov.uk http://data.gov.uk/
15/06/14 www.rba.co.uk 91
data.gov.uk http://data.gov.uk/
Not all of the data on this site is open data – may be restrictions
on use
Download links ...
Eurostat http://ec.europa.eu/eurostat
15/06/14 www.rba.co.uk 93
European Union - Open Data Portal http://open-
data.europa.eu/open-data/
15/06/14 www.rba.co.uk 94
European Union - Open Data Portal http://open-
data.europa.eu/open-data/
15/06/14 www.rba.co.uk 95
Google Public Data Explorer
http://www.google.com/publicdata/
One of Google's best kept secrets!
Public data sets made ava...
Google Public Data Explorer Minimum Wage
15/06/14 www.rba.co.uk 97
Google Public Data Explorer Minimum Wage
15/06/14 www.rba.co.uk 98
Eurostat - Minimum Wage
15/06/14 www.rba.co.uk 99
15/06/14 www.rba.co.uk 100
Google Public Data Explorer – DPT immunizations
15/06/14 www.rba.co.uk 101
World Bank DPT
15/06/14 www.rba.co.uk 102
Datamarket http://datamarket.com/
Open portal to datasets worldwide and market research
Creates visualisations of the data...
Statista http://www.statista.com/
“The Statistics Portal for Market Data, Market Research and Market
Studies”
– 60,000 top...
Statista
15/06/14 www.rba.co.uk 105
Zanran http://www.zanran.com/
Searches graphs, charts, tables, PDFs, spreadsheets
Can limit by location of server, date, a...
Zanran
http://www.zanran.com/
Zanran – great for data in tables, charts and graphs
http://www.rba.co.uk/wordpress/2013/01/...
Guardian Data Store http://www.guardian.co.uk/data
15/06/14 www.rba.co.uk 108
Data and analysis on topics that are in the ...
Migrants crossing the Mediterranean: key numbers
http://www.theguardian.com/news/datablog/2014/jun/10/migrants-crossing-th...
Public Data Group
https://www.gov.uk/government/groups/public-data-group
15/06/14 www.rba.co.uk 110
Land Registry http://www.landregistry.gov.uk/
15/06/14 www.rba.co.uk 111
Land Registry price paid data
15/06/14 www.rba.co.uk 112
http://www.rightmove.co.uk/
15/06/14 www.rba.co.uk 113
6 months ago
the sold price
listed was
£185,000 and
for 2012.
Land Registry summary of the postcode
15/06/14 www.rba.co.uk 114
Title document
15/06/14 www.rba.co.uk 115
Missing data
15/06/14 www.rba.co.uk 116
Error report filed with the Land Registry three weeks ago
Still waiting for a resp...
Price paid data report builder
15/06/14 www.rba.co.uk 117
House price indices
How the Land Registry undercooks house prices by £90k [Daily Mail
alert!] http://www.thisismoney.co.uk...
Crime statistics http://www.police.uk/
15/06/14 www.rba.co.uk 119
Crime statistics http://www.police.uk/
15/06/14 www.rba.co.uk 120
Crime statistics http://www.police.uk/
15/06/14 www.rba.co.uk 121
Electricity micro-generation
15/06/14 www.rba.co.uk 122
Variable Pitch http://www.variablepitch.co.uk/
Electricity micro-generation
15/06/14 www.rba.co.uk 123
Variable Pitch http://www.variablepitch.co.uk/stations/1310/
Electricity micro-generation
15/06/14 www.rba.co.uk 124
Variable Pitch http://www.variablepitch.co.uk/stations/2580/
Virgi...
FoI request generation data for Virginia Station
15/06/14 www.rba.co.uk 125
https://www.whatdotheyknow.com/request/request...
Chart and image gallery: 30+ free tools for data visualization and
analysis - Computerworld
http://www.computerworld.com/s...
Google Fusion Tables
15/06/14 www.rba.co.uk 127
And finally....
Per capita consumption of mozzarella cheese (US) correlates with Civil
engineering doctorates awarded (US)...
Upcoming SlideShare
Loading in...5
×

The Brave New World of Free, Open Data and Open Access

1,224

Published on

Pre-conference seminar given at BIALL 2014, 11th June 2014, Harrogate

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,224
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
9
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "The Brave New World of Free, Open Data and Open Access"

  1. 1. 15/06/14 www.rba.co.uk 1 Karen Blakeman karen.blakeman@rba.co.uk, www.rba.co.uk Slides available at http://www.rba.co.uk/as/ This work is licensed under a Creative Commons Attribution License The Brave New World of Free, Open Data and Open Access Pre-conference workshop, BIALL 2014, Harrogate Wednesday, 11th June 2014 #BIALL2014
  2. 2. All change! Search engines – new algorithms, ranking and display, quality of results – EU ruling on “right to be forgotten”, how much is being censored/removed? Free government and legal resources – how reliable are they? – data and information being moved to Gov.uk (is it?) or web archives (really?) Open Access vs open access vs “open access” – how accessible is it? – authority, version – predatory publishers Open data – how “open” is it really – findability, ease of use, quality Changes to copyright – data/text mining 15/06/14 www.rba.co.uk 2
  3. 3. Impact of Social Sciences – The right to read is the right to mine: Text and data mining copyright exceptions introduced in the UK. http://blogs.lse.ac.uk/impactofsocialsciences/2014/06/04/the-right-to-read-is-t 15/06/14 www.rba.co.uk 3
  4. 4. Search engines Google in particular is undergoing major change Don’t forget alternative search engines (they may still be subject to recent EU ruling on search results) Need to understand how search tools work, how the different country versions work and how they present results Google and Bing (to a lesser degree) personalise results How to assess quality of results 15/06/14 www.rba.co.uk 4
  5. 5. Five things you need to know about Google search 1. Google personalises your search Personalises search based on – location – device that you are using – past search history – past browsing activity – activity in other areas of Google e.g. YouTube, blogs, images – content from contacts in your personal networks may be given priority (possibly) 15/06/14 www.rba.co.uk 5
  6. 6. Private browsing - quickest way “un-personalise”search Chrome - New Incognito window Ctrl+Shift+N FireFox Ctrl+Shift+P Internet Explorer Ctrl+Shift+P Opera Ctrl+Shift+N Will not remove country personalisation
  7. 7. Five things you need to know about Google search 2. Google automatically looks for variations on your search terms and sometimes drops terms from your search – Google may or may not tell you that it has ignored some of your terms – “..” around terms, phrases, names, titles of documents does not always work – To force an exact match and inclusion of a term in a search prefix it with ‘intext:’ public transport intext:algal biofuels – Use Verbatim for an exact match search
  8. 8. Google Verbatim
  9. 9. Google now showing missing search terms? Not always shown – possibly still a live experiment? 15/06/14 www.rba.co.uk 9
  10. 10. Five things you need to know about Google search 3. Google web search does not search everything it has in its database – two indexes: main, default index and the supplemental index – supplemental index may contain less popular, unusual, specialist material – supplemental index comes into play when Google thinks your search has returned too few results – Verbatim and some advanced search commands seems to trigger a search in the supplemental index
  11. 11. Five things you need to know about Google search 4. Google changes its algorithms several hundred times a year How Google makes improvements to its search algorithm - YouTube http://youtu.be/J5RZOU6vK4Q
  12. 12. Five things you need to know about Google search 5. We are all Google’s lab rats Just Testing: Google Users May See Up To A Dozen Experiments http://searchengineland.com/just-testing-google-searchers-may-se Mostly minor effects on search but sometimes totally bizarre results
  13. 13. What I see on my screen will not be what you see on your screen, will not be what your colleagues see on theirs, will not be what your users see. 15/06/14 www.rba.co.uk 13
  14. 14. Hummingbird Not just an update but a completely new algorithm Tries to make “sense” of your query and put it into context, natural language queries Not just search history but also your location, device being used Now difficult to predict how Google will handle your search and how results will be displayed Layout of results and menu options depend on type of search 15/06/14 www.rba.co.uk 14
  15. 15. So called “right to be forgotten” ruling 15/06/14 www.rba.co.uk 15 Edition of Monday, January 19, 1998, page 23 - Newspaper - Lavanguardia.es http://hemeroteca.lavanguardia.c
  16. 16. Information is NOT removed from the web Subject can apply to have links in search results that point to specific information removed from the results Only applies to searches conducted in the EU + Norway, Switzerland, Iceland and Lichtenstein Not automatic – subject has to apply and request will be assessed to see if the information is “inadequate, irrelevant or no longer relevant, or excessive in relation to the purposes for which they were processed.” Request form available at https://support.google.com/legal/contact/lr_eudpa?product=websearch 15/06/14 www.rba.co.uk 16
  17. 17. Could have a serious impact on business and legal research How far will it go? People already trying to get links to their company directorships, bankruptcy, IVAs etc removed Land Registry records Crime reports in the newspapers Unfavourable reviews of services 15/06/14 www.rba.co.uk 17
  18. 18. Four things we’ve learned from the EU Google judgment | ICO Blog http://iconewsblog.wordpress.com/2014/05/20/four-things-weve-learned-from-the-eu-googl iconewsblog.files.wordpress.com/2014/05/key-points-of-cjeu-case.pdf http://iconewsblog.files.wordpress.com/2014/05/key-points-of-cjeu-case.pdf BBC News - More Google 'forget' requests emerge after EU ruling http://www.bbc.co.uk/news/technology-27439194 EU Law Analysis: The CJEU's Google Spain judgment: failing to balance privacy and freedom of expression http://eulawanalysis.blogspot.no/2014/05/the-cjeus-google-spain-judgment-failing.html Google Offers Webform To Comply With Europe’s ‘Right To Be Forgotten’ Ruling | TechCrunch http://techcrunch.com/2014/05/30/right-to-be-forgotten-webform/ The Myths & Realities Of How Of The EU’s New “Right To Be Forgotten” In Google Works http://searchengineland.com/eu-right-forgotten-191604 15/06/14 www.rba.co.uk 18
  19. 19. Last Week Tonight with John Oliver (HBO) Right To Be Forgotten - YouTube https://www.youtube.com/watch?v=r-ERajkMXw0&feature=youtu.be 15/06/14 www.rba.co.uk 19
  20. 20. How to get around it? Until we see how it works in practice, difficult to give advice Google says it will indicate on the results page if information has been excluded Will we be able to use Google.com or Google.ca, for example, or will Google recognise we are in the EU from our IP address and still block results? Use alternative search engine with no business footprint in Europe? New search engines starting up without a European presence? Anonymous proxy servers? 15/06/14 www.rba.co.uk 20
  21. 21. Search commands (Google, Bing) Think file format – PDF for research documents, government reports, industry papers, company reports – ppt or pptx for presentations, tracking down an expert on a topic – xls or xlsx for spreadsheets containing data Use the advanced search screen or the filetype: command Land registry tax evasion data mining filetype:pdf tax evasion data mining filetype:ppt tax evasion data mining filetype:pptx tax evasion UK filetype:xls tax evasion UK filetype:xlsx 15/06/14 www.rba.co.uk 21
  22. 22. Search commands (Google, Bing) Filetype may not always work – The filetype: command may not always work even when a site offers data, for example, in spreadsheet format – Sometimes this is because the data is held in a database and the files are created from a subset of the data when you request the file. – Instead of filetype:xlsx or filetype:xls simply include the word Excel, xls or csv in your search. 15/06/14 www.rba.co.uk 22
  23. 23. Search commands – site: (most search engines) Site search For searching large websites, or groups of sites by type for example government, NHS, academic Can exclude sites using -site: agricultural occupational asthma UK site:nhs.uk agricultural occupational asthma UK site:ac.uk agricultural occupational asthma UK site:gov.uk agricultural occupational asthma UK site:gov.uk –site:hse.gov.uk 15/06/14 www.rba.co.uk 23
  24. 24. Searching in multiple languages A significant amount of information is in the local language Google has removed the extremely useful “Translated foreign pages” search option  This is how it can be done now 1.Use Google Translate (http://translate.google.com/) to translate your search into the required language. 2.Copy the translated search and paste it into Google search. 3.Google Chrome will offer to translate page If using another browser click on the ‘Translate this page’ link next to a result to view a translation of just that page. 15/06/14 www.rba.co.uk 24
  25. 25. 15/06/14 www.rba.co.uk 25
  26. 26. Evaluating resources Date of publication, 'last updated' Check text for clues of publication date Stated date for a web page or document may be automatically generated when it is put onto the web site After a web site redesign pages are re-uploaded and are assigned a new time-stamp Some pages are generated "on the fly" so will always have today's date Type of web site for example: – gov.uk, ac.uk, .gov, .edu Who is really behind the site? – use a domain name register such as http://whois.domaintools.com 15/06/14 www.rba.co.uk 26
  27. 27. Who owns a website? http://www.domaintools.com/ good starting point for identifying who owns a domain name but.... – may be hiding behind an agent – may be using a service such as Privacy Protect ( http://www.privacyprotect.org) – if in the UK and an individual with a personal page then contact details other than name are not publicly available (Data Protection) Run a Google or Bing search on the ‘registrant’ or agent and see if you can find out anything more about them 15/06/14 www.rba.co.uk 27
  28. 28. Domaintools check – good news 15/06/14 www.rba.co.uk 28
  29. 29. Domaintools check – what you do not want to see 15/06/14 www.rba.co.uk 29
  30. 30. Quality of statistics Read the definitions – Definitions and scope may have changed over the years and may be different for each country Political manipulation e.g. unemployment statistics Some data may not exist for some countries e.g. minimum wage Sudden jumps or flatlining in graphs should alert you to oddities in the data (see later) Official data not immune from errors May need an industry expert to help interpret and analyse sector specific data e.g. energy reserves vs resources, “liquids”/oil/petroleum 15/06/14 www.rba.co.uk 30
  31. 31. Some web sites...... 15/06/14 www.rba.co.uk 31
  32. 32. Gov.uk 15/06/14 www.rba.co.uk 32
  33. 33. UK Government Web Archive | The National Archives http://www.nationalarchives.gov.uk/webarchive/ Browse by category or choose your organisation from an A-Z list Choose the date of the archived version of the website you want to view 15/06/14 www.rba.co.uk 33
  34. 34. UK Government Web Archive | The National Archives http://www.nationalarchives.gov.uk/webarchive/ 15/06/14 www.rba.co.uk 34
  35. 35. Wayback Machine http://www.archive.org/ 15/06/14 www.rba.co.uk 35
  36. 36. http://www.legislation.gov.uk/ Launched in 2010 Not all legislation has been updated Pending updates are flagged 15/06/14 www.rba.co.uk 36
  37. 37. UK Parliament http://www.parliament.uk/ 15/06/14 www.rba.co.uk 37
  38. 38. Monitoring progress of legislation http://services.parliament.uk/bills/2013-14/pensions.html 15/06/14 www.rba.co.uk 38
  39. 39. data.parliament.uk http://www.data.parliament.uk/ 15/06/14 www.rba.co.uk 39
  40. 40. The Gazette | Official Public Record https://www.thegazette.co.uk/ 15/06/14 www.rba.co.uk 40
  41. 41. FLARE - Foreign Law Research Group Website http://ials.sas.ac.uk/flare/flare.htm 15/06/14 www.rba.co.uk 41
  42. 42. FLARE - Union List of Official Gazettes: Europe http://ials.sas.ac.uk/flare/flare_fog_unionlist_europe.htm 15/06/14 www.rba.co.uk 42
  43. 43. They work for you http://www.theyworkforyou.com/ 15/06/14 www.rba.co.uk 43
  44. 44. WhatDoTheyKnow - Freedom of Information (FOI) requests https://www.whatdotheyknow.com/ 15/06/14 www.rba.co.uk 44
  45. 45. Official company information Companies House http://www.companieshouse.gov.uk/ Lists of official company registers – Official Company Registers http://www.rba.co.uk/sources/registers.htm – Company registration around the world http://www.commercial-register.sg.ch/home/worldwide.html – Companies House Links http://www.companieshouse.gov.uk/links/introduction.shtml#reg US – listed companies – SEC Edgar IDEA – Interactive Date Electronic Applications http://www.sec.gov/edgar.shtml Canada - listed companies – SEDAR http://www.sedar.com/ 15/06/14 www.rba.co.uk 45
  46. 46. Company Check http://companycheck.co.uk/ UK & Ireland companies and director search Some info free, some free with login, some free with “pro” account, some pay as you go Provides 5 years of figures and graphs for Cash at Bank, Net Worth, Total Current Liabilities and Total Current Assets Download up to 5 yrs of accounts Companies house documents (£2 or 99p depending on subscription) Credit risk, charges, CCJs, Monitor company for financial changes and when new accounts are filed Lists directors of a company - see what other directorships they have 15/06/14 www.rba.co.uk 46
  47. 47. Company Check 15/06/14 www.rba.co.uk 47
  48. 48. Company Check 15/06/14 www.rba.co.uk 48
  49. 49. Company Check Dashboard 15/06/14 www.rba.co.uk 49
  50. 50. DUEDIL http://www.duedil.com/ 15/06/14 www.rba.co.uk 50
  51. 51. OpenCorporates https://opencorporates.com/ 15/06/14 www.rba.co.uk 51
  52. 52. OpenCorporates http://opencorporates.com/ 15/06/14 www.rba.co.uk 52
  53. 53. OpenCorporates https://opencorporates.com/ 15/06/14 www.rba.co.uk 53
  54. 54. OpenCorporates and Google Refine 15/06/14 www.rba.co.uk 54 How to use OpenCorporates to match companies in Google Refine on Vimeo http://vimeo.com/17924204
  55. 55. OpenCharities http://opencharities.org/ 15/06/14 www.rba.co.uk 55
  56. 56. OpenCharities http://opencharities.org/ 15/06/14 www.rba.co.uk 56
  57. 57. Mapping Corporate Networks With OpenCorporates http://www.slideshare.net/psychemedia/mapping-corporate-netw 15/06/14 www.rba.co.uk 57
  58. 58. ICIJ Offshore Leaks Database http://offshoreleaks.icij.org/ 15/06/14 www.rba.co.uk 58 ICIJ Releases Offshore Leaks Database Revealing Names Behind Secret Companies, Trusts | International Consortium of Investigative Journalists http://www.icij.org/offshore/icij-releases-offshore-leaks-database-revealing-names-behin https://www.youtube.com/watch?
  59. 59. Open Access Research Literature 15/06/14 www.rba.co.uk 59
  60. 60. Open Access vs open access vs open access Open Access Publicly funded research made available free of charge to the user Open access journals Publications that are free to the reader, may or may not be a charge for publishing in the journal “Open access” predatory publishers Publications of dubious quality that have been set up purely as a means of generating money 15/06/14 www.rba.co.uk 60
  61. 61. Mandated Open Access US All research publications resulting from work funded by the US National Institutes of Health are expected to be deposited in PubMed Central (http://www.ncbi.nlm.nih.gov/pmc/) – some material embargoed for up to 12 or 24 months ( http://www.ncbi.nlm.nih.gov/pmc/journals/) – Europe PubMed Central (http://europepmc.org/) part of PMC network of international repositories UK 1st of April 2013 - researchers at UK Research Institutions are expected to publish as open access any peer reviewed research‐ papers and conference proceedings that acknowledge Research Council UK funding 15/06/14 www.rba.co.uk 61
  62. 62. UK Gold versus Green OA Gold OA – researchers publish their articles in journals that offer open access publishing (can be established “conventional” publishers) – articles can be made available free of charge to readers immediately – author or institution/department pays article processing fee – CC-BY Green OA – researchers deposit copies of articles in an institutional or subject- based repository, subject to copyright/license permissions – repository makes copies available to the public either immediately or embargoed (more common) – period of embargo varies (for example http://cdn.elsevier.com/assets/pdf_file/0018/121293/external-embargo-list. ) – CC-BY-NC 15/06/14 www.rba.co.uk 62
  63. 63. Jeffrey Beall List of Predatory Publishers 2014 | Scholarly Open Access http://scholarlyoa.com/2014/01/02/list-of-predatory-publishers-2014/ 15/06/14 www.rba.co.uk 63
  64. 64. Fragmentation of open access Where are the open access publications? – Individual OA articles within existing subscription journals – Separate OA journals, publishers website – Author’s website – Institutional repositories – Aggregators e.g. Scopus, Web of Science, Google Scholar – Mendeley, ResearchGate? 15/06/14 www.rba.co.uk 64
  65. 65. ResearchGate http://www.researchgate.net/ Increasingly used to request copies of articles from authors 15/06/14 www.rba.co.uk 65
  66. 66. Google Scholar http://scholar.google.com/ 15/06/14 www.rba.co.uk 66
  67. 67. Google Scholar Does not cover all key journals in all subjects – no source list Top publications for subjects and languages under Metrics link on home page or http://scholar.google.co.uk/citations?view_op=top_venues&hl=en Scholar indexes the full text but you may have to pay to view the whole article Groups different versions of an article together 15/06/14 www.rba.co.uk 67
  68. 68. Google Scholar 15/06/14 www.rba.co.uk 68
  69. 69. Google Scholar Includes open access material, pre-prints, institutional repositories (but not necessarily author self archived papers on personal websites) Includes material that is NOT peer reviewed but is structured and looks like an academic article (title in large font, authors, affiliations, abstract, keywords, citations) Pre-prints and IR copies may differ from final published version – charts and images may be redacted because of copyright restrictions 15/06/14 www.rba.co.uk 69
  70. 70. Google Scholar 15/06/14 www.rba.co.uk 70 Does NOT use the publishers’ metadata Sometimes gets the author wrong Beware the advanced search screen and commands – Date and author search looks in the area of the document where those elements are usually found – Page numbers, part of an address, data item may be mistaken for publication year
  71. 71. Institutional repositories and open access BASE - Bielefeld Academic Search Engine http://www.base-search.net/ CORE (COnnecting Repositories) http://core.kmi.open.ac.uk/search DART-Europe E-theses Portal http://www.dart-europe.eu/basic-search.php DOAJ: Directory of Open Access Journals http://www.doaj.org/doaj Institutional Repository Search (IRS) http://irs.mimas.ac.uk/ Open DOAR http://opendoar.org/ RIAN - Pathways to Irish Research http://rian.ie ROAR - Registry of Open Access Repositories http://roar.eprints.org/ OpenAIRE http://www.openaire.eu/ 15/06/14 www.rba.co.uk 71
  72. 72. Specialist search tools for research information A selection can be found at http://www.rba.co.uk/search/links.shtml#research ArXiv http://arxiv.org/ BioMed Central http://www.biomedcentral.com/ Chemistry Central http://www.chemistrycentral.com/ ChemSpider http://www.chemspider.com/ Deep Web Technologies Mednar http://mednar.com/ Science.gov http://www.science.gov/ Science Research http://scienceresearch.com/ WorldWideScience http://worldwidescience.org/ 15/06/14 www.rba.co.uk 72
  73. 73. Specialist search tools for research information Europe PubMed Central http://europepmc.org/ Mendeley http://www.mendeley.com/ Open Biology http://rsob.royalsocietypublishing.org/ PhilPapers: Online Research in Philosophy http://philpapers.org/ PubMed Central http://www.ncbi.nlm.nih.gov/pmc/ TechXtra http://www.techxtra.ac.uk/ 15/06/14 www.rba.co.uk 73
  74. 74. Elsevier’s take down notices Elsevier clamps down on academics posting their own papers online (Wired UK) http://www.wired.co.uk/news/archive/2013-12/17/elsevier-versus-open "Why do we send take down notices? One key reason is to ensure that the final published version of an article is readily discoverable and citable via the journal itself in order to maximise the usage metrics and credit for our authors, and to protect the quality and integrity of the scientific record. The formal publications on our platforms also give researchers better tools and links, for example to data" 15/06/14 www.rba.co.uk 74
  75. 75. Grey literature Literature that has been “peer reviewed” or assessed/approved in some way by colleagues or subject experts but is not easy to find or access Print run may have been small, possibly never published electronically Published on the web but page or site is no longer available Research and technical papers, government reports, pre-prints, market surveys, press releases, committee working papers, conference papers and presentations Use advanced Google commands and web archives to find documents May or may not be open access GreyNet International, Grey Literature Network Service – http://www.greynet.org/ – http://www.opengrey.eu/15/06/14 www.rba.co.uk 75
  76. 76. OpenGrey http://www.opengrey.eu/ 15/06/14 www.rba.co.uk 76
  77. 77. Article may be OA but public access outside of the institution may be not allowed or difficult 15/06/14 www.rba.co.uk 77 WATER report published | SCONUL http://www.sconul.ac.uk/news/water-report-published Walk-in Access To E- Resources
  78. 78. BBC News - Public libraries get online access to research journals http://www.bbc.co.uk/news/education-25981183 15/06/14 www.rba.co.uk 78 For personal research, non-commercial use.
  79. 79. Public Library Initiative by PLS and ProQuest | Access to Research http://www.accesstoresearch.org.uk/ List of participating libraries and publishers Public Library Initiative by PLS and ProQuest | Access To Research http://freetoviewjournals.pls.org.uk/ Search tool for the journals and articles covered by the agreement. List of journals covered by the agreement Not just Open Access articles but subscription services as well Gold Open Access articles can be viewed anywhere. Other articles can only be viewed on library premises 15/06/14 www.rba.co.uk 79
  80. 80. Free patent information Patent Searching 101: A Patent Search Tutorial IPWatchdog.com Patents & Patent Law http://www.ipwatchdog.com/2012/03/17/patent-searching-101-a-paten Compares Google patent search with other services “Holes in the database” – may not be able to find patents you know exist Does go back further than some e.g. to US patent no.1 but difficult to focus search See also Patents Searching with Esp@cenet, Google Patents and USPTO - University of Bradford http://www.brad.ac.uk/library/resources/patents-searching-with-espace 15/06/14 www.rba.co.uk 80
  81. 81. Google Patents https://www.google.com/patents Coverage: – US – Canada – European Patent Office (EPO) – Germany – China – World Intellectual Property Organisation (WIPO) Patents available in original language and English (Google Translate) 15/06/14 www.rba.co.uk 81
  82. 82. Google Patents 15/06/14 www.rba.co.uk 82
  83. 83. Google Patents advanced search 15/06/14 www.rba.co.uk 83 https://www.google.co.uk/advanced_patent_search
  84. 84. Statistics More open data but.... Raw data – no pretty layouts or visualisations, you have to do that Data may need weeding and cleaning before it is usable May have to do a LOT of work on the data to get anything sensible out of it To see what you could be letting yourself in for look at Tony Hirst’s blog postings on open data at http://blog.ouseful.info/, for example Reshaping Horse Import/Export Data to Fit a Sankey Diagram http://blog.ouseful.info/2013/02/18/reshaping-horse- importexport-data-to-fit-a-sankey-diagram/ 15/06/14 www.rba.co.uk 84
  85. 85. Official statistics OFFSTATS http://www.offstats.auckland.ac.nz/ UK National Statistics Publication Hub http://www.statistics.gov.uk/ Office for National Statistics http://www.ons.gov.uk/ Welsh Government | Statistics http://wales.gov.uk/topics/statistics/ Welsh Assembly Government StatsWales http://statswales.wales.gov.uk/ data.gov.uk http://data.gov.uk/ Eurostat http://ec.europa.eu/eurostat/ European Union Open Data Portal http://open-dat.europa.eu/en/ 15/06/14 www.rba.co.uk 85
  86. 86. http://www.offstats.auckland.ac.nz/ 15/06/14 www.rba.co.uk 86
  87. 87. UK National Statistics & ONS http://www.statistics.gov.uk/, http://www.ons.gov.uk/ 15/06/14 www.rba.co.uk 87
  88. 88. Publication Hub (http://www.statistics.gov.uk/) is an “index” to what is available and links through to other sites ONS (http://www.ons.gov.uk/) only shows reports since 2008 even if there are earlier editions. Use the Publication Hub to search for the report title/series. Once you have found an edition of the title click on “Current and past editions” to see the list of editions available. Then click on the relevant report. 15/06/14 www.rba.co.uk 88
  89. 89. Publication Hub search (statistics.gov.uk) 15/06/14 www.rba.co.uk 89
  90. 90. http://statswales.wales.gov.uk/ 15/06/14 www.rba.co.uk 90
  91. 91. data.gov.uk http://data.gov.uk/ 15/06/14 www.rba.co.uk 91
  92. 92. data.gov.uk http://data.gov.uk/ Not all of the data on this site is open data – may be restrictions on use Download links sometimes take you to the wrong dataset Download links sometimes completely broken It’s all or nothing! May have to filter the datasets for the information you want and produce your own graphs and charts Variety of formats 15/06/14 www.rba.co.uk 92
  93. 93. Eurostat http://ec.europa.eu/eurostat 15/06/14 www.rba.co.uk 93
  94. 94. European Union - Open Data Portal http://open- data.europa.eu/open-data/ 15/06/14 www.rba.co.uk 94
  95. 95. European Union - Open Data Portal http://open- data.europa.eu/open-data/ 15/06/14 www.rba.co.uk 95
  96. 96. Google Public Data Explorer http://www.google.com/publicdata/ One of Google's best kept secrets! Public data sets made available by Eurostat, World Bank, IMF, CSO Ireland, OECD, ITU, some national statistics offices (but not ONS), and many more. Source and date updated given. Charts and charting options can highlight oddities and missing data Look at the charts to see if there is a sudden change in the trends. 15/06/14 www.rba.co.uk 96
  97. 97. Google Public Data Explorer Minimum Wage 15/06/14 www.rba.co.uk 97
  98. 98. Google Public Data Explorer Minimum Wage 15/06/14 www.rba.co.uk 98
  99. 99. Eurostat - Minimum Wage 15/06/14 www.rba.co.uk 99
  100. 100. 15/06/14 www.rba.co.uk 100
  101. 101. Google Public Data Explorer – DPT immunizations 15/06/14 www.rba.co.uk 101
  102. 102. World Bank DPT 15/06/14 www.rba.co.uk 102
  103. 103. Datamarket http://datamarket.com/ Open portal to datasets worldwide and market research Creates visualisations of the data 15/06/14 www.rba.co.uk 103
  104. 104. Statista http://www.statista.com/ “The Statistics Portal for Market Data, Market Research and Market Studies” – 60,000 topics from over 18,000 sources – http://www.statista.com/topics/ – some information free, registration (free) required – Chart of the day 15/06/14 www.rba.co.uk 104
  105. 105. Statista 15/06/14 www.rba.co.uk 105
  106. 106. Zanran http://www.zanran.com/ Searches graphs, charts, tables, PDFs, spreadsheets Can limit by location of server, date, and filetype Title in results list is usually the title or caption to the table and not title of the document Hover over the thumbnail to see a preview of the table or page Click on the URL button next to the result to view the original URL of the document – clicking on it may take you to “page not found” 404 Click on the title of the result to see Zanran’s own copy (free registration usually required Useful if document no longer available at it’s original location and can’t be found in any of the web archives 15/06/14 www.rba.co.uk 106
  107. 107. Zanran http://www.zanran.com/ Zanran – great for data in tables, charts and graphs http://www.rba.co.uk/wordpress/2013/01/25/zanran-great-for-data-in- tables-charts-and-graphs/ 15/06/14 www.rba.co.uk 107
  108. 108. Guardian Data Store http://www.guardian.co.uk/data 15/06/14 www.rba.co.uk 108 Data and analysis on topics that are in the news Some data sets created from information obtained via FoI Links to the original datasets are provided
  109. 109. Migrants crossing the Mediterranean: key numbers http://www.theguardian.com/news/datablog/2014/jun/10/migrants-crossing-the- mediterranean-key-numbers-libya-european#start-of-comments 15/06/14 www.rba.co.uk 109
  110. 110. Public Data Group https://www.gov.uk/government/groups/public-data-group 15/06/14 www.rba.co.uk 110
  111. 111. Land Registry http://www.landregistry.gov.uk/ 15/06/14 www.rba.co.uk 111
  112. 112. Land Registry price paid data 15/06/14 www.rba.co.uk 112
  113. 113. http://www.rightmove.co.uk/ 15/06/14 www.rba.co.uk 113 6 months ago the sold price listed was £185,000 and for 2012.
  114. 114. Land Registry summary of the postcode 15/06/14 www.rba.co.uk 114
  115. 115. Title document 15/06/14 www.rba.co.uk 115
  116. 116. Missing data 15/06/14 www.rba.co.uk 116 Error report filed with the Land Registry three weeks ago Still waiting for a response Why might a property/price paid not appear in the data? Seems not that uncommon according to discussion boards – usually data entry error (but the above example was in the public data until the last update) Absence of price – gift of property or purchase of a share Impractical to calculate price e.g. bulk purchase of properties Commercial transactions http://www.landregistry.gov.uk/market-trend-data/house-price-data-exclusions
  117. 117. Price paid data report builder 15/06/14 www.rba.co.uk 117
  118. 118. House price indices How the Land Registry undercooks house prices by £90k [Daily Mail alert!] http://www.thisismoney.co.uk/money/mortgageshome/article- 2270763/How-Land-Registry-undercooks-house-prices-90k.html Which house price index can you trust? http://www.telegraph.co.uk/finance/personalfinance/houseprices/10403 980/Which-house-price-index-can-you-trust.html House Prices - Who Should You Believe? http://www.money.co.uk/article/1006113-house-prices-who-should-you- believe.htm Why do average house prices differ? - http://moneyfacts.co.uk/news/mortgages/why-do-average-house- prices-differ/ 15/06/14 www.rb a.co.uk 118
  119. 119. Crime statistics http://www.police.uk/ 15/06/14 www.rba.co.uk 119
  120. 120. Crime statistics http://www.police.uk/ 15/06/14 www.rba.co.uk 120
  121. 121. Crime statistics http://www.police.uk/ 15/06/14 www.rba.co.uk 121
  122. 122. Electricity micro-generation 15/06/14 www.rba.co.uk 122 Variable Pitch http://www.variablepitch.co.uk/
  123. 123. Electricity micro-generation 15/06/14 www.rba.co.uk 123 Variable Pitch http://www.variablepitch.co.uk/stations/1310/
  124. 124. Electricity micro-generation 15/06/14 www.rba.co.uk 124 Variable Pitch http://www.variablepitch.co.uk/stations/2580/ Virginia Station is the hydroelectric installation that feeds Windsor Castle
  125. 125. FoI request generation data for Virginia Station 15/06/14 www.rba.co.uk 125 https://www.whatdotheyknow.com/request/request_electricity_output_of_ro
  126. 126. Chart and image gallery: 30+ free tools for data visualization and analysis - Computerworld http://www.computerworld.com/s/article/9214755/Chart_and_image_ga llery_30_free_tools_for_data_visualization_and_analysis 15/06/14 www.rba.co.uk 126
  127. 127. Google Fusion Tables 15/06/14 www.rba.co.uk 127
  128. 128. And finally.... Per capita consumption of mozzarella cheese (US) correlates with Civil engineering doctorates awarded (US) http://tylervigen.com/view_correlation?id=3890 15/06/14 www.rba.co.uk 128
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×