9. THE TROUBLE WITH DATA
• You need to find data API
• Get Access – Signup for key
• Find data endpoint
• Read docs to learn what parameters you
have
• Get data in obscure format
• Use data after converting and filtering
• More APIs you use, more is your
annoyance
10. To make data access
easy on the web,
Yahoo! created YQL
20. ACCESSING PRIVATE DATA
http://query.yahooapis.com/v1/yql
Uses OAuth 1.0 for authorization
OAuth is complicated – use one of our SDKs at
https://github.com/yahoo
21. The easiest way to start
with YQL is to use the
console
http://developer.yahoo.com/yql/console
28. QUERY EXAMPLES
Find hackday tweets:
SELECT * FROM twitter.search where q='hackday’
Search Yahoo! Answers for resolved questions about cars:
select * from answers.search where query="cars" and type="resolved”
Translate tweets into Hindi:
select * from google.translate where q in (select text from
twitter.search where q="hackday") and target='hi';
Extract important terms from top stories on Yahoo! news:
select * from search.termextract where context in (select description
from rss where url='http://rss.news.yahoo.com/rss/topstories')
29. QUERY EXAMPLES
Find distance between Bangalore and Mumbai:
select * from geo.distance where place1="bangalore" and
place2="mumbai”
Get stock quotes for Yahoo!, Apple, Google & Microsoft in one query:
select * from yahoo.finance.quotes where symbol in
("YHOO","AAPL","GOOG","MSFT")
Shorten a URL:
insert into yahoo.y.ahoo.it (url, keysize) values ('http://
www.javarants.com', 5)
Search apartments in criagslist:
select * from craigslist.search where location="bangalore" and
type="apa" and query="indiranagar”
30. QUERY EXAMPLES
Scrape news from Yahoo! Finance:
select * from html where url="http://finance.yahoo.com/q?
s=yhoo" and xpath='//div[@id="yfi_headlines"]/div[2]/ul/li/a’
Select, filter data from google spreadsheets:
select * from csv where url="https://
spreadsheets.google.com/pub?key=0ArYndzim-
lbrdF8wc3A5QWl1ZGRpdkxRZk80SU9zUXc&output=csv"
and col5 like 'Bangalore%’ ;
43. WEBMEME.IN
Fetch multiple feeds in different formats like atom, RSS and
transform them into consistent RSS format
Select * from rss where url in (‘http://feeds.feedburner.com/
pluggd’, ‘http://quatrainman.blogspot.com/atom.xml’, ‘…’)
Filter news containing “india” from multiple feeds:
select * from rss where url in ('http://feeds.feedburner.com/
TechCrunch', 'http://www.readwriteweb.com/rss.xml','http://
gigaom.com/feed/') and description like '%india%’
44. YQL is open – you
can get your data
tables in our system
45. All you need to do is
write an XML
schema and put it
on Github.