YQL & Yahoo! APIs - Open Hack India 2011

YQL AND YAHOO! APISSaurabh SahniYDN Developer, Hacker, EvangelistTwitter: @saurabhsahni

Hacking togethersystems in 24 hours islot of fun

Data manipulationHacks = + Data visualization

The web has lot of data around

ProgrammableWeb.Com – 3517 APIs

Yahoo! has openedup its data

http://developer.yahoo.com/everything.html

THE TROUBLE WITH DATA •  You need to find data API •  Get Access – Signup for key •  Find data endpoint •  Read docs to learn what parameters you have •  Get data in obscure format •  Use data after converting and filtering •  More APIs you use, more is your annoyance

To make data accesseasy on the web,Yahoo! created YQL

YQL turns webservices and data onthe web intodatabases.

select {what} from {where} where {conditions}

You can select, filter,sort and limit

You can even insert,update and deletefrom it.

FINDING VIDEOS ABOUT BANGALORE SELECT * FROM youtube.search where query=bangalore

SELECTING PHOTOS OF HACKDAYSELECT * FROM flickr.photos.search wheretext="hackday"

INSERTING DATAINSERT INTO bitly.shorten (login, apiKey, longUrl)VALUES (ME, API_KEY, http://yahoo.com)

UPDATING DATA UPDATE social.profile.status SET status="Using YQL UPDATE” WHERE guid="NJFIDHVPVVISDX7UKED2WHU"

RETRIEVING MY CONTACTSSELECT * FROM social.contacts WHEREguid=me

ACCESSING PRIVATE DATA http://query.yahooapis.com/v1/yqlUses OAuth 1.0 for authorizationOAuth is complicated – use one of our SDKs athttps://github.com/yahoo

The easiest way to startwith YQL is to use theconsolehttp://developer.yahoo.com/yql/console

YQL: http://developer.yahoo.com/yql/console

URL STRUCTUREhttp://query.yahooapis.com/v1/yql?[params]http://query.yahooapis.com/v1/public/yql?[params] Params •  q = [ YQL QUERY ] •  format = [ XML / JSON ] •  diagnostics = [ true / false ] •  debug = [ true / false ] •  callback = [ function name ]

LET’S SEE IT

QUERY EXAMPLESFind hackday tweets:SELECT * FROM twitter.search where q=hackday’Search Yahoo! Answers for resolved questions about cars:select * from answers.search where query="cars" and type="resolved”Translate tweets into Hindi:select * from google.translate where q in (select text fromtwitter.search where q="hackday") and target=hi;Extract important terms from top stories on Yahoo! news:select * from search.termextract where context in (select descriptionfrom rss where url=http://rss.news.yahoo.com/rss/topstories)

QUERY EXAMPLESFind distance between Bangalore and Mumbai:select * from geo.distance where place1="bangalore" andplace2="mumbai”Get stock quotes for Yahoo!, Apple, Google & Microsoft in one query:select * from yahoo.finance.quotes where symbol in("YHOO","AAPL","GOOG","MSFT")Shorten a URL:insert into yahoo.y.ahoo.it (url, keysize) values (http://www.javarants.com, 5)Search apartments in criagslist:select * from craigslist.search where location="bangalore" andtype="apa" and query="indiranagar”

QUERY EXAMPLESScrape news from Yahoo! Finance:select * from html where url="http://finance.yahoo.com/q?s=yhoo" and xpath=//div[@id="yfi_headlines"]/div[2]/ul/li/a’Select, filter data from google spreadsheets:select * from csv where url="https://spreadsheets.google.com/pub?key=0ArYndzim-lbrdF8wc3A5QWl1ZGRpdkxRZk80SU9zUXc&output=csv"and col5 like Bangalore%’ ;

Let’s findopenhackindiaphotos on flickr

MAKING REQUESTS: FLICKR URLS <photo farm="3" id="5708163920" isfamily="0" isfriend="0" ispublic="1" owner="31832337@N04" secret="0075137487" server="2496" title="San Francisco"/>

MAKING REQUESTS: FLICKR URLS Photo URL http://farm{$farm}.static.flickr.com/{$server}/ {$id}_{$secret}.jpg Photo Page URL http://www.flickr.com/photos/{$owner}/{$id} Photo Owner Profile URL http://www.flickr.com/photos/{$owner}

https://github.com/saurabhsahni/Hacks/

Finding Recent Photos from ﬂickr

Some YQL Hacks

ChromYQLip is achrome extension forpage scraping via YQLOpen Hack Bangalore 2010 Winner

http://bit.ly/chromeYQL

VIDEO CLIP

http://www.webmeme.in

WEBMEME.INFetch multiple feeds in different formats like atom, RSS andtransform them into consistent RSS formatSelect * from rss where url in (‘http://feeds.feedburner.com/pluggd’, ‘http://quatrainman.blogspot.com/atom.xml’, ‘…’)Filter news containing “india” from multiple feeds:select * from rss where url in (http://feeds.feedburner.com/TechCrunch, http://www.readwriteweb.com/rss.xml,http://gigaom.com/feed/) and description like %india%’

YQL is open – youcan get your datatables in our system

All you need to do iswrite an XMLschema and put iton Github.

http://github.com/yql/yql-tables

Here is the craigslistsearch tablehttps://github.com/yql/yql-tables/tree/master/craigslist/craigslist.search.xml

USE INSTANTLY BY UPLOADING ON YOUR SITE USE http://www.mysite.com/my_table.xml AS mytable; SELECT * FROM mytable WHERE user=saurabh

You can even writeserver side javascriptto build a webserviceor augment one.h#p://developer.yahoo.com/yql/guide/yql-‐execute-‐chapter.html

There are lot ofthings you can dowith YQL.

Play yourselfhttp://developer.yahoo.com/yql/

RESOURCESAll Yahoo! APIs and Serviceshttp://developer.yahoo.com/everything.htmlYQL Documentationhttp://developer.yahoo.com/yqlYQL Consolehttp://developer.yahoo.com/yql/consoleYQL Github Account (Contribute Tables)http://github.com/yql/yql-tables

ANY QUESTIONS?http://www.slideshare.net/saurabhsahni Saurabh Sahni Twitter: @saurabhsahni Github: http://github.com/saurabhsahni Web: http://www.saurabhsahni.com

http://twitter.com	4
http://tweetedtimes.com	2
https://twitter.com	1
http://a0.twimg.com	1

YQL & Yahoo! APIs - Open Hack India 2011

by Saurabh Sahni

Accessibility

Categories

Upload Details

Usage Rights

4 Embeds 8

Statistics

YQL & Yahoo! APIs - Open Hack India 2011 Presentation Transcript