THE TROUBLE WITH DATA • You need to find data API • Get Access – Signup for key • Find data endpoint • Read docs to learn what parameters you have • Get data in obscure format • Use data after converting and filtering • More APIs you use, more is your annoyance
To make data accesseasy on the web,Yahoo! created YQL
YQL turns webservices and data onthe web intodatabases.
QUERY EXAMPLESFind hackday tweets:SELECT * FROM twitter.search where q=hackday’Search Yahoo! Answers for resolved questions about cars:select * from answers.search where query="cars" and type="resolved”Translate tweets into Hindi:select * from google.translate where q in (select text fromtwitter.search where q="hackday") and target=hi;Extract important terms from top stories on Yahoo! news:select * from search.termextract where context in (select descriptionfrom rss where url=http://rss.news.yahoo.com/rss/topstories)
QUERY EXAMPLESFind distance between Bangalore and Mumbai:select * from geo.distance where place1="bangalore" andplace2="mumbai”Get stock quotes for Yahoo!, Apple, Google & Microsoft in one query:select * from yahoo.finance.quotes where symbol in("YHOO","AAPL","GOOG","MSFT")Shorten a URL:insert into yahoo.y.ahoo.it (url, keysize) values (http://www.javarants.com, 5)Search apartments in criagslist:select * from craigslist.search where location="bangalore" andtype="apa" and query="indiranagar”
QUERY EXAMPLESScrape news from Yahoo! Finance:select * from html where url="http://finance.yahoo.com/q?s=yhoo" and xpath=//div[@id="yfi_headlines"]/div/ul/li/a’Select, filter data from google spreadsheets:select * from csv where url="https://spreadsheets.google.com/pub?key=0ArYndzim-lbrdF8wc3A5QWl1ZGRpdkxRZk80SU9zUXc&output=csv"and col5 like Bangalore%’ ;
WEBMEME.INFetch multiple feeds in different formats like atom, RSS andtransform them into consistent RSS formatSelect * from rss where url in (‘http://feeds.feedburner.com/pluggd’, ‘http://quatrainman.blogspot.com/atom.xml’, ‘…’)Filter news containing “india” from multiple feeds:select * from rss where url in (http://feeds.feedburner.com/TechCrunch, http://www.readwriteweb.com/rss.xml,http://gigaom.com/feed/) and description like %india%’
YQL is open – youcan get your datatables in our system
All you need to do iswrite an XMLschema and put iton Github.