YQL is a data querying and manipulation service from Yahoo! that allows users to access and combine data from different sources on the web through an SQL-like syntax. The document demonstrates how to query data from sources like YouTube, Flickr, and Craigslist, and also discusses how users can create their own custom data tables by uploading XML definitions or writing server-side JavaScript. Finally, the document encourages users to contribute new data tables through an open GitHub repository and notes that YQL provides access to over 1200 existing data tables covering many popular web APIs.

1. YQL & YAHOO! APIS
Jai Santhosh
Frontend Engineer, Hacker
Yahoo!

2. What We’re Going to Cover
• YQL Overview
• Making Queries and the Console Sandbox
• Demo
• Open Data Tables and Execute (Server-Side
JavaScript)
• Final Lessons

3. Data manipulation
Hacks = +
Data visualization

4. The web is a large pile of data!

5. ProgrammableWeb.com – 4480 Apis

6. Yahoo! has also
opened up its data

7. http://developer.yahoo.com/everything.html

8. Trouble with Data
• Need to find data API
• Get Access – Signup for key
• Find data endpoint
• Read docs to learn what parameters you
have
• Get data in obscure format
• Use data after converting and filtering
• More APIs you use, more is your
annoyance

9. To make data access easy on
the web,
Yahoo! created YQL

10. =
YQL turns web services and
data on the web into
databases.

11. select {what} from {where}
where {conditions}

12. You can select, filter, sort
and limit data and you can
even insert, update and
delete from it.

13. YQL: http://developer.yahoo.com/yql/console

14. Finding Videos about Bangalore
• SELECT * FROM youtube.search where
query='bangalore'

15. Selecting Photos of RHOK
• SELECT * FROM flickr.photos.search where
text="rhok"

16. Inserting data
INSERT INTO bitly.shorten (login, apiKey, longUrl)
VALUES ('ME', 'API_KEY', 'http://yahoo.com')

17. updating data
UPDATE social.profile.status
SET status="Using YQL UPDATE”
WHERE guid="NJFIDHVPVVISDX7UKED2WHU"

18. Accessing Private Data
• http://query.yahooapis.com/v1/yql
Uses OAuth 1.0 for authorization
OAuth is complicated – use one of our SDKs at
https://github.com/yahoo

19. Retrieving my profile
• SELECT * FROM social.contacts WHERE
guid=me

20. Using the YQL Console
makes it very easy to
use it.

23. url structure
• http://query.yahooapis.com/v1/yql?[params]
• http://query.yahooapis.com/v1/public/yql?[para
ms]
Params
q= [ YQL QUERY ]
format = [ XML / JSON ]
diagnostics = [ true / false ]
debug = [ true / false ]
callback = [ function name ]

24. Let’s See It

25. Query Samples
• SELECT * FROM twitter.search where q='hackathon’ limit 5
• select * from answers.search where query="cars" and type="resolved"
• select * from google.translate where q="this is a test" and target="hi";
• select * from google.translate where q in (select text from twitter.search
where q="hackday") and target='hi';
• select * from search.termextract where context in (select description from
rss where url='http://rss.news.yahoo.com/rss/topstories')

26. Query Samples
• select * from yahoo.finance.quotes where symbol in
("YHOO","AAPL","GOOG","MSFT")
• insert into yahoo.shortener (url, keysize) values
('http://www.javarants.com', 5)
• select * from craigslist.search where location="bangalore" and
type="apa" and query="indiranagar”
• select * from html where url="http://finance.yahoo.com/q?s=yhoo"
and xpath='//div[@id="yfi_headlines"]/div[2]/ul/li/a’

27. Making Requests: Flickr URLs
<photo farm="3"
id="5708163920"
isfamily="0"
isfriend="0"
ispublic="1"
owner="31832337@N04"
secret="0075137487"
server="2496"
title="San Francisco"/>

28. Making Requests: Flickr URLs
Photo URL
http://farm{$farm}.static.flickr.com/{$server}/{$id
}_{$secret}.jpg
Photo Page URL
http://www.flickr.com/photos/{$owner}/{$id}
Photo Owner Profile URL
http://www.flickr.com/photos/{$owner}

29. YQL is open – you can get your
data tables in our system

30. Craigslist data table
<table>
<meta>
<author>Yahoo! Inc.</author>
<documentationURL>http://craigslist.org/</documentationURL>
<sampleQuery>
select * from {table} where location="sfbay" and type="sss" and query=“bike”
</sampleQuery>
</meta>
<bindings>
…
</bindings>
</table>

31. Craigslist data table
<table>
<meta> … </meta>
<bindings>
<select itemPath="" produces="XML">
<urls>
<url>http://{location}.craigslist.org/search/{type}?format=rss</url>
</urls>
<inputs>
<key id="location" type="xs:string" paramType="path" required="true"/>
<key id="type" type="xs:string" paramType="path" required="true"/>
<key id="query" type="xs:string" paramType="query" required="true"/>
</inputs>
</select>
</bindings>
</table>

32. Upload XML to our Github
http://github.com/yql/yql-tables

33. Or to your own Site
• The USE clause
USE 'http://www.mysite.com/my_table.xml'
AS mytable;
SELECT * FROM mytable
WHERE user='john_doe'

34. You can even write server side
javascript to build a webservice

35. Final Lessons
• YQL is a skeleton key for all Yahoo! APIs
• Open data tables allow you to create new data feeds
out of one or many different raw sources
• There are 1200+ Tables in YQL covering all major APIs
on the web
• Use SDKs when accessing private data that requires
OAuth

36. Final Lessons: Links
• All Yahoo! APIs and Services
• http://developer.yahoo.com/everything.html
• YQL Documentation
• http://developer.yahoo.com/yql
• YQL Console
• http://developer.yahoo.com/yql/console
• YQL Github Account (Contribute Tables)
• http://github.com/yql/yql-tables

37. http://isithackday.com/hackday-toolbox/

38. Questions?
http://www.slideshare.net/jaisa
nth
Jai Santhosh
Twitter: @jaisanth
Github: http://github.com/jaisanth

39. THANK YOU!