Advanced Internet Searching for Energy Information & Market Research

Information for Energy Group, Energy Institute, New Cavendish Street, London Thursday 13 May 2010 Advanced Internet Searching for Energy Information & Market Research [email_address] , http://twitter.com/karenblakeman http://www.slideshare.net/karenblakeman http://www.rba.co.uk/ , http://www.rba.co.uk/wordpress/ This presentation is licensed under a Creative Commons Attribution 3.0 License Photo: http://www.flickr.com/photos/rbainfo/3313765895/

Search is changing
How people look for and find information is changing
Social media and networks are now an essential part of the search mix
Facebook for market research and customer feedback e.g. the marmite bar http://www.facebook.com/MarmiteBars
Business students assignment on advertising history of Cadbury’s Crème Egg and Dairy Milk – Youtube!
Twitter and blogs used as an essential part of reputation monitoring
What’s happening at conferences - Twitter, Slideshare, Flickr, videos, podcasts
Search engines now including social media and professional networks in search results

Search on a person Blogger profile Twitter profile Facebook profile LinkedIn profile 15 May 2010 Karen Blakeman www.rba.co.uk

Search on a topic – first page results 15/05/10 www.rba.co.uk Results from Google News Latest results - rolling feed from Twitter, blogs, discussions

Search on a topic – first page results 15/05/10 www.rba.co.uk Images Wikipedia article Blog posting listing videos YouTube video Results from my “social circle”: Twitter, RSS feeds, FriendFeed, Google Wave contacts etc.

Social media
It is where many companies and organisations market themselves, provide customer support and distribute information
It is where an increasing proportion of the population grow up & they don’t differentiate between the different means of communication
It is where an increasing number of people feel comfortable and share information
It’s impossible to ‘gag’ social networks
BBC NEWS | Politics | When is a secret not a secret?
http://news.bbc.co.uk/1/hi/uk_politics/8304908.stm
For serious research, social media and professional networks are not optional extras - they are an essential part of your research strategy
15 May 2010 Karen Blakeman www.rba.co.uk

Types of search tools
Humans
is a colleague or a friend already working in, or knowledgeable about the subject area?
who have you met at meetings, conferences?
discussion lists, bloggers, LinkedIn, Facebook, trade/professional associations, Twitter etc. (social “stuff”!)
Search engines
Google, Yahoo, Bing,
So called “deep web” and specialist tools, evaluated listings, subject listings, customised search engines
Databases and peer reviewed sources
Multi search engine tools, federated search
search lots of search tools or databases at once
or type in your search once and click on each search tool in turn

How up to date are search tools?
Google, Yahoo, Bing etc.
you are searching an out of date index of the web and not the web itself
may take days to months for a site to be added to the index
hierarchy of sites for updating
Real time, social media search tools
Twitter, blogs, discussion boards, RSS feeds
what is happening NOW

What are you looking for?
Authoritative facts?
Official or corporate announcements/press releases?
News articles?
Views and opinions on a topic?
from experts
from consumers
from people who don’t know what they’re talking about!
Spur of the moment rants about failed services or products?
Rumour, speculation, gossip?
Or do you want to gather new data?
e.g. set up surveys on Facebook, Survey Monkey, Twtpoll.com

Switched on by default
Google adjusts your search results according to what you have clicked on in the past
See Karen Blakeman's Blog - Your Google results are about to get weirder
http://www.rba.co.uk/wordpress/2009/12/17/your-google-results-are-about-to-get-weirder/
Before we start - is your Google web history on? 15 May 2010 Karen Blakeman www.rba.co.uk

A search engine’s results may vary
In content and presentation
from one minute to the next
different server being used
testing out different search and ranking algorithms
Country versions
different emphasis
local content
different interface
different search features
use if researching a person, company, subject located or based in another country

General search techniques
By default, the main search tools look for all of your terms in a page
Use double quote marks around phrases or names
To exclude pages containing a term, precede the term with a minus sign (-) but use with care
Focus your search in areas of the page
- use advanced search screens or commands
- inurl: for example UK inurl:carbonemissions
looks for your terms/phrase in the URL
- intitle: or allintitle: for example UK intitle:"carbon emissions"
looks for your terms/phrase in the title of the page

General search techniques
Search sites or domains using the advanced search screen or site: command
wind turbine energy generation site:statistics.gov.uk
wind turbine energy generation site:gov.uk
Imagine what you would like to appear in your ideal document and include those terms in your strategy
- renewable energy generation UK wind solar wave hydro
Repeat your key search terms in your search strategy
tar sands EROEI
tar sands EROEI EROEI EROEI
different results
Change the order of your terms
biodiesel fuel UK buses efficiency
biodiesel fuel efficiency UK buses
different results

File format search
Use advanced search options to limit your search to file types or format:
pdf or doc for government or industry/market reports
xls for data and statistics
ppt or pdf for presentations, experts on a topic
Use Google and Yahoo Advanced Search
Bing.com sometimes picks up unique content
use command filetype: in your search strategy e.g. filetype:ppt car ownership UK

Docjax
http://www.docjax.com/
searches pdf, xls, doc, ppt in Yahoo and Google
15/05/10 www.rba.co.uk

Presentations
For :
find experts on a topic
what are companies doing
introduction to a topic or a general overview
data and market statistics
company structures, who’s who
Presentation search:
Slideshare http://www.slideshare.net/
Authorstream http://www.authorstream.com/
Slideboom http://www.slideboom.com/
Slidefinder http://www.slidefinder.net/

Slideshare.net 15/05/10 www.rba.co.uk Search: biofuels UK public transport

Slidefinder
http://www.slidefinder.net/
Identifies individual slides within a presentation

Slidefinder
Can limit search to universities
Also Advanced Search to search areas of slides

Google
Additional search features previously hidden under “Show options” now in left hand side menu
Most are self explanatory
A few that aren’t....
Updates = Twitter

Google
Wonderwheel

Google
Timeline
but Google can get it wrong
15/05/10 www.rba.co.uk ?

Unique Google search features
Similar or related pages
looks for pages that are similar in type and content
click on the Similar link next to the page in your results list
or use the related: command e.g. related:www.bwea.com

Automatically looks for variations on your terms
to stop it, precede your terms with plus signs e.g. +Norne oil field to stop Google looking for ‘Horne’
or use double quote marks around the term or phrase “Norne”
(hat tip to Even Flood for this example)
Numeric range search
can be weights, distances, years, prices, measurements of any sort
use Advanced Search screen
or the search box on the Google home page
search term(s) first value..second value unit of measurement
toblerone 1..6 kg
world oil production forecasts 2010..2015

Proximity
use the asterisk (*) to stand in for one or more terms
useful when searching for people
separates the terms by one or more words
Karen * Blakeman picks up:
Karen Blakeman
Karen Mary Blakeman
Karen Sands Blakeman
Karen Marie Blakeman
solar * panels picks up:
solar heating panels
solar electricity panels
solar thermal panel
solar voltaic panels
solar concentrator panels

Google Finance
http://www.google.co.uk/finance

Yahoo!
http://www.yahoo.co.uk/
very busy interface!
http://search.yahoo.co.uk/
simpler, Google like interface

Yahoo!
http://search.yahoo.co.uk/
Advanced Search hidden under “More”
Similar options to Google Advanced Search

Yahoo!
http://www.yahoo.co.uk/
http://search.yahoo.co.uk/ http://search.yahoo.com/
Deal with Bing
Some country versions already using Bing web search, rest will follow during the year
Now includes options in the left side menu for limiting your seearch

Yahoo Finance
http://uk.finance.yahoo.com/

Bing
http://www.bing.com/ (was live.com)
“ decision engine”
Results tend to be more consumer oriented
Seems to have the most up to date database and possibly the most extensive database of web pages
Results vary considerably depending on the country version
Supposedly incorporating real time, social media results into main search
Advanced Search screen ‘minimalist” but can use commands (see Search Engine Summary and Comparison Chart http://www.rba.co.uk/search/compare.pdf )

Compare search engines
TripleMe
http://www.tripleme.com/
compares Google, Yahoo and Bing side by side
Blindsearch
http://blindsearch.fejus.com/
compare Google, Yahoo and Bing results for web or images
vote on which are the best without seeing the branding
Zuula
http://www.zuula.com /
runs your search through a range of search tools one by one
order can be customised
Browsys Finder (was Intelways/Crossengine)
http://www.browsys.com/finder
runs your search through a plethora of search tools one by one

iSEEK
http://www.iseek.com/
Clusters results into topics, people, places, organisations, date & time
“ Education” option – more research oriented pages

Biznar
http://www.biznar.com/
Real time federated search of selected business resources, some of them “hidden web” or “deep web”
Click on Advanced Search to see a list of the sources

Intute
http://www.intute.ac.uk
Funding to be discontinued but has been given a stay of execution
Excellent starting point for quality resources in all subject areas and industries

TechXtra
http://www.techxtra.ac.uk
ICBL and the Library at Heriot-Watt University, Edinburgh
Engineering, mathematics and computing
Free information and pay per view

Google Scholar
http://scholar.google.com/
No source list
Not comprehensive and omits many key scientific publications
Both peer-reviewed and un-reviewed articles, pre-prints, institutional repositories, references to books, citations
Does not use publishers’ meta data
Author search unreliable
Search on year of publication unreliable
“ Google Scholar is brain damaged” Peter Jasco, Trends in Professional and Academic Online Information Services, presented at Inforum, 22 nd May 2007, Prague
Use with caution and be aware that there are huge gaps in coverage

Alacrawiki spotlights
http://www.alacrawiki.com/
Key resources for major industry sectors
Most pages have not been updated for 2-3 years  BUT much of the information still valid

Alacrasearch
http://www.alacra.com/alacrasearch/
Google custom search engine

LGSearch
http://lgsearch.net/
Another Google custom search engine
Searches UK public sector web sites

Directionlessgov.com 15/05/10 www.rba.co.uk

http://www.search4oil.com/
New!
Information on the oil and gas sectors

http://www.search4oil.com/
People, Feature

Statistics
Some general starting points
OFFSTATS - The University of Auckland Library http://www.offstats.auckland.ac.nz/
by country or subject or both
excellent list of energy web site stats including the BP Statistical Review of World Energy (2009)
RBA Energy Information Sources http://www.rba.co.uk/sources/energy.shtml (under revision!)
Google Public data explorer http://www.google.com/publicdata/
See review Google Public Data Explorer- fine as far as it goes http://www.rba.co.uk/wordpress/2010/03/27/google-public-data-explorer-fine-as-far-as-it-goes/
Guardian datastore and datablog http://www.guardian.co.uk/data-store and World Government Data http://www.guardian.co.uk/world-government-data
the latter is a better way of searching http://data.gov.uk/

Guardian World Government Data
data.gov.uk

Office of National Statistics
http://www.statistics.gov.uk/
Official UK statistics
Search on energy
Often quicker and easier to use Google site search

Department of Energy and Climate Change http://www.decc.gov.uk/en/content/cms/statistics/statistics.aspx 15/05/10 www.rba.co.uk

RESTATS
Renewable Energy STATisticS database for the United Kingdom
http://www.restats.org.uk/

Energy Export Databrowser http://mazamascience.com/OilExport/ 15/05/10 www.rba.co.uk

Free industry and market research
Individual publishers
tables of contents, executive summaries, free sample chapters
Government bodies
US http://www.export.gov/ , http://www.bisnis.doc.gov/
regulatory bodies e.g. in the UK - Competition Commission, Ofcom
Management consultants
Academic sector
Trade, industry, manufacturing associations
Trade and industry press

Priced industry and market research
Aggregators may not be comprehensive
many publishers embargo reports for weeks or months
many publishers withhold reports from aggregators
check the publisher’s own web site or email them to see if there is additional or more recent research available
may get a better deal by going to the publisher direct
use aggregators as an index to see who is publishing on your topic, for example
http://www.marketresearch.com/
http://www.mindbranch.com/
http://www.researchandmarkets.com/
http://www.alacrastore.com/
http://www.reportbuyer.com/ (go to Energy and Utilities)
For emerging markets http://www.securities.com/

Priced industry and market research
Many sites specialising in energy reports
just search on energy market research reports
Energy market research, analysis, databases, forecasts and energy market consulting http://www.absenergyresearch.com/
Energy Market research http://www.energy-market-research.info/
GBI Research http://www.gbiresearch.com/ (Go to the Oil and Gas section)

News
Newspapers, journals, trade press
Blogs
Twitter
Video
Podcasts
Set up alerts and RSS feeds
Google News
http://news.google.co.uk/
good regional, national, international and trade press coverage
free sources only so misses some important publications
useful way of identifying industry publications

Chipwrapper http://www.chipwrapper.co.uk
Google Custom Search engine
Searches everything available on 15 free UK News Sites
“ Time slices” for last 24 hours, past week, past month, past year

Silobreaker http://www.silobreaker.com/
covers free resources
news, blogs, video, tweets
geographical location of stories
people
networks
use as a starting point for in depth research
has an energy section

RSS for ...
Current awareness and alerts
newspapers, blogs, twitter
customised alerts on topics or as part of reputation monitoring/management
Read feeds in
an online feed reader e.g. Google Reader
desktop client e.g. FeedReader, RSS Bandit, Omea
Outlook (2007 and later)
Start pages e.g. Netvibes, My Yahoo, iGoogle,

RSS feeds (in Omea) 15/05/10 www.rba.co.uk

Netvibes.com 15/05/10 www.rba.co.uk

Netvibes.com
New dashboard feature
type in your keywords and Netvibes automatically searches across news, social media, videos etc.
looks in places you might not think of

Video & Audio
Search engines’ video/audio tabs
YouTube, Vimeo.com, iTunes U
Blinkx - http://www.blinkx.com/
Truveo – http://www.truveo.com/
country versions available
Zuula - http://www.zuula.com/
click on the video tab
Browsys Finder (formerly Intelways/Crossengine)
http://www.browsys.com/finder/
click on the video tab
Podcasts associated with news stories on e.g. BBC, The Guardian, New Scientist, Nature etc.
Odeo - http://www.odeo.com/

Blogs and blog search tools
Blogs are a very easy and quick means of publishing
New postings are picked up within minutes - sometimes seconds – of publication
RSS feeds for alerts and keeping up to date
Google left side menu
Blogs
By date and time
Latest, Past 24 hours, Past week include large proportion of forums, news and press releases, blog postings
Blog search tools
Look for the blogroll or list of links on a known, trusted blog
Google blogsearch http://blogsearch.google.com/
Blogpulse http://www.blogpulse.com/
Icerocket http://www.icerocket.com/ (also Twitter and MySpace)
Technorati http://www.technorati.com/

Blogs
Many researchers and industry experts run blogs
Some have lengthy articles, analysis and serious discussion e.g. The Oil Drum ( http://www.theoldrum.com ), Energy Balance ( http://ergobalance.blogspot.com/ )

Searching Twitter and tweets
To use Twitter’s Find People you need to be signed in to Twitter
Alternatively search for people or organisations using Tweepz ( http://www.tweepz.com/ )
Searching public tweets
http://search.twitter.com/ , http://www.twazzup.com/
Backtweets.com – enter the URL of a web site and see who has referred to it in a tweet
Phil Bradley's weblog: Twitter Search - 20 alternative search engines
http://philbradley.typepad.com/phil_bradleys_weblog/2009/03/twitter-search-20-alternative-search-engines.html

Twitter search 15 May 2010 Karen Blakeman www.rba.co.uk

LinkedIn.com
Check up on individuals, companies, groups

Design your own search engine
For
regularly searched sites
selected sites on a topic
searching sites on a reading list
Google Custom Search Engines
http://www.google.com/cse /
at least hundreds of sites, maybe thousands!
can import lists of sites
Need a Google account to set one up
Cannot search password protected sources or sites where you have to fill in a form to access the information

Create your own Google CSE 15 May 2010 Karen Blakeman www.rba.co.uk

Host it on Google
Keep it private or make it public
URLs are horribly long but you can create a TinyURL, Bit.ly URL etc. to make it easier for your users

Disappearing pages
Search engine cache copies
Google, Yahoo, Bing
UK Web Archive
http://www.webarchive.org.uk/ukwa/
Wayback machine
http://www.archive.org/
from 1996 to about 1 year ago
navigate the archived site or type in the full URL of the document if known

UK Web Archive of http://www.berr.gov.uk/energy/index.html 15/05/10 www.rba.co.uk

Wayback Machine 15 May 2010 Karen Blakeman www.rba.co.uk

Further information & keeping up to date
Information World Review (IWR) and IWR Blog
http://www.iwr.co.uk/ , http://blog.iwr.co.uk/
Phil Bradley’s Blog
http://philbradley.typepad.com/
I want to
http://philbradley.typepad.com/i_want_to/
Web 2.0 tools and applications: 1100+ Web2 apps listed, annotated
http://www.philb.com/iwantto.htm
Karen Blakeman’s Blog
http://www.rba.co.uk/wordpress/
Annual Online Information Exhibition, London, usually at the beginning of December
http://www.online-information.co.uk/
Extensive programme of free seminars, updates and master classes on the exhibition floor.

Advanced Internet Searching for Energy Information & Market Research

by Karen Blakeman on May 12, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

4 Embeds 68

Statistics

Advanced Internet Searching for Energy Information & Market Research — Presentation Transcript

http://www.rba.co.uk	53
http://www.slideshare.net	13
http://www.lmodules.com	1
http://philbradley.typepad.com	1