Yql hacku iitd_2012

Hacking on steroids with YQLAnshu Prateek Search/YDN @anshprat

Hacking togethersystems in 24 hours islot of fun.

Data manipulationHacks = + Data visualization

The web has lot of data around

ProgrammableWeb.Com – 6831 APIs

Yahoo! has opened up its data

http://developer.yahoo.com/everything.html

The trouble with data •  You need to find data API •  Get Access – Signup for key •  Find data endpoint •  Read docs to learn what parameters you have •  Get data in obscure format •  Use data after converting and filtering •  More APIs you use, more is your annoyance

To make data access easy on the web,Yahoo! created YQL

YQL turns webservices and data on the web into databases.

select {what} from {where} where {conditions}

You can select, filter, sort and limit

You can even insert,update and delete from it.

Finding Videos about Bangalore •  SELECT * FROM youtube.search where query=bangalore

Selec@ng Photos of HackdaY •  SELECT * FROM ﬂickr.photos.search where text="hackday” and api_key=“b5a60b2a…”

Inser@ng data •  INSERT INTO bitly.shorten (login, apiKey, longUrl) •  •  VALUES (ME, API_KEY, hYp://yahoo.com)

upda@ng data •  UPDATE social.proﬁle.status •  SET status="Using YQL UPDATE” •  WHERE guid="NJFIDHVPVVISDX7UKED2WHU"

Retrieving my Contacts •  SELECT * FROM social.contacts WHERE guid=me

Accessing Private Data •  hYp://query.yahooapis.com/v1/yql Uses OAuth 1.0 for authorizationOAuth is complicated – use one of our SDKs athttps://github.com/yahoo

You can also mix and match several webservices using the in() command.

select * from search.termextractwhere context in (select description from rss where url=http://rss.news.yahoo.com/rss/topstories)

Almost all the top APIs on web areaccessible from YQL

1255 APIs

Some of them amazon foursquare peerindex apple geo salesforce bbc github slideshare bible google themovideb boss hackernews tumblr campfire ign twitter contentanalysis intuit vimeo craigslist kiva weather delicious klout yahoo dopplr lastfm youtube etsy netflix zillow facebook paypal

You want even more?

Alright, how about this? atom json csv microformats feed rss html xml

The easiest way to start with YQL is to use the consolehttp://developer.yahoo.com/yql/console

YQL: http://developer.yahoo.com/yql/console

How to get this data in your app?

YQL is a REST API in itself and has two endpoints

The public endpoint does not need any authentication.http://query.yahooapis.com/v1/public/yql? q={query} &format={format}

The private endpoint needs oauth authentication.http://query.yahooapis.com/v1/yql? q={query}&format={format}

Output formats are XML or JSON

Let’s See It

Query Examples select * from yahoo.ﬁnance.quotes where symbol in ("^IXIC","^DJI","YHOO ","AAPL")

Query Examples select * from weather.byloca@on where loca@on in ("bangalore, in", ”new york, us")

•  Query Examples Find hackday tweets: •  SELECT * FROM twi]er.search where q=hackday’ •  Search Yahoo! Answers for resolved ques@ons about cars: •  select * from answers.search where query="cars" and type="resolved” •  Find distance between Bangalore and Mumbai: •  select * from geo.distance where place1="bangalore" and place2="mumbai” •  Extract important terms from top stories on Yahoo! news: •  select * from search.termextract where context in (select descrip@on from rss where url=h]p://rss.news.yahoo.com/rss/topstories)

Query Examples •  Get Olympic medal list •  select * from html where url=h]p://sports.yahoo.com/olympics/ medals.html and xpath=//*[@id=mediasportsoverallmedalcount]/ div[2]/table/tbody/tr/td/a •  Shorten a URL: •  insert into yahoo.y.ahoo.it (url, keysize) values (h]p:// www.javarants.com, 5) •  Search apartments in criagslist: •  select * from craigslist.search where loca@on="bangalore" and type="apa" and query="indiranagar”

Query Examples •  Scrape news from Yahoo! Finance: •  select * from html where url="h]p://finance.yahoo.com/q? s=yhoo" and xpath=//div[@id="yfi_headlines"]/div[2]/ul/li/ a’ •  Select, filter data from google spreadsheets: •  select * from csv where url="h]ps:// spreadsheets.google.com/pub?key=0ArYndzim-‐ lbrdF8wc3A5QWl1ZGRpdkxRZk80SU9zUXc&output=csv" and col5 like Bangalore%’ ;

There are lot of thingsyou can do with YQL.

Play yourself http://developer.yahoo.com/yql/

One more thing

Don’t look for features.Solve one problem

Resources •  All Yahoo! APIs and Services •  hYp://developer.yahoo.com/everything.html •  YQL Documentahon •  hYp://developer.yahoo.com/yql •  YQL Console •  hYp://developer.yahoo.com/yql/console •  YQL Github Account (Contribute Tables)

Thanks! http://www.slideshare.net/anshprat Anshu Prateek Twitter: @anshprat Github: http://github.com/anshprat Web: http://hackalyst.info

http://www.cse.iitd.ac.in	35
http://www.cse.iitd.ernet.in	24

Yql hacku iitd_2012

by Anshu Prateek on Aug 16, 2012

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 59

Statistics

Yql hacku iitd_2012 Presentation Transcript