1. HackU 2013
University of Dundee
February 2013

2. The web is full of
juicy and
long lasting data
HackU 2013 – University of Dundee

3. And there is a lot of it around
HackU 2013 – University of Dundee

4. However, our
attempts to get it
can sometimes
be clumsy
HackU 2013 – University of Dundee

5. What we need is
an easy way to
get to that data
HackU 2013 – University of Dundee

6. What about using APIs?
Like RSS, JSON, XML, CSV, SOAP, REST, …
HackU 2013 – University of Dundee

7. We could use APIs. But there are
thousands of APIs...
HackU 2013 – University of Dundee

8. and it’s painful to get your head
around them!
HackU 2013 – University of Dundee

9. What if there was one API…
One API for everything
on the web?
HackU 2013 – University of Dundee

10. One World
One Web
One API ?
Can it ever exist?
Sounds like
politics!
HackU 2013 – University of Dundee

11. Yes it does exist!
YQL:
The API
for the
web
HackU 2013 – University of Dundee

12. The Yahoo Query Language, or short
YQL is a unified interface language
to the web.
What are you waiting for?
http://developer.yahoo.com/yql
HackU 2013 – University of Dundee

13. Accessing the web and its services
now becomes easy as ABC SQL
select * from web
select {what} from {service}
where {condition}
HackU 2013 – University of Dundee

14. Let’s try a real YQL example
I love kittens.
YQL get me 5 kitten photos
select * from
flickr.photos.search where
text="kitten" limit 5
HackU 2013 – University of Dundee

15. Let’s try a real YQL example
select * from
flickr.photos.search where
text="kitten" limit 5
Output can be XML or JSON
HackU 2013 – University of Dundee

16. I want to use YQL and I want it now.
That’s the easy part…
http://query.yahooapis.com/v1/public/yql?q=
<SQL_QUERY>&format=<FORMAT>
e.g.
http://query.yahooapis.com/v1/public/yql?q=
select%20*%20from%20rss%20where%20url%3D
%22http%3A%2F%2Frss.news.yahoo.com%2Frss
%2Ftopstories%22&format=json
There’s also a private endpoint which uses oauth authentication
http://developer.yahoo.com/yql/guide/authorization-access.html
HackU 2013 – University of Dundee

17. That’s not all… You can mix and
match several web services using
the in() command
HackU 2013 – University of Dundee

18. Mix and match…
Find photos in Dundee, UK with a
Creative Commons “By” license
select * from flickr.photos.info where
photo_id in (select id from
flickr.photos.search where woe_id in (select
woeid from geo.places where text=’dundee,uk')
and license=4)
HackU 2013 – University of Dundee

19. Now use your imagination… and
build away!
http://isithackday.com/hacks/cantine/index.php?loc=covent+garden
HackU 2013 – University of Dundee

20. Yahoo! offers lots of data for free!
flickr.photos.exif
local.search
mybloglog.community.find
flickr.photos.info
music.ar1st.id
mybloglog.member
flickr.photos.interes1ngness
music.ar1st.popular
mybloglog.member.contacts
flickr.photos.recent
music.ar1st.search
mybloglog.member.newwithcontacts
flickr.photos.search
music.ar1st.similar
mybloglog.member.newwithme
flickr.photos.sizes
music.release.ar1st
mybloglog.members.find
flickr.places
music.release.id
mybloglog.stats.find
flickr.places.info
music.release.popular
upcoming.category
geo.places
music.release.search
upcoming.country
geo.places.ancestors
music.track.id
upcoming.events
geo.places.belongtos
music.track.popular
upcoming.events.bes1nplace
geo.places.children
music.track.search
upcoming.groups
geo.places.neighbors
music.video.category
upcoming.metro
geo.places.parent
music.video.id
upcoming.state
geo.places.siblings
music.video.popular
upcoming.user
geo.placetypes
music.video.search
upcoming.venue
gnip.ac1vity
music.video.similar
HackU 2013 – University of Dundee

21. … and more of it. ALL FREE!
search.images
search.images
search.images
search.news
search.news
search.news
search.siteexplorer.inlinks
search.siteexplorer.inlinks
search.siteexplorer.inlinks
search.siteexplorer.pages
search.siteexplorer.pages
search.siteexplorer.pages
search.spelling
search.spelling
search.spelling
search.suggest
search.suggest
search.suggest
search.termextract
search.termextract
search.termextract
search.web
search.web
search.web
social.connec1ons
social.connec1ons
social.connec1ons
social.contacts
social.contacts
social.contacts
social.presence
social.presence
social.presence
social.profile
social.profile
social.profile
social.updates
social.updates
social.updates
weather.forecast
weather.forecast
weather.forecast
yahoo.iden1ty
yahoo.iden1ty
yahoo.iden1ty
yap.setsmallview
yap.setsmallview
yap.setsmallview
HackU 2013 – University of Dundee

22. You want more data?
amazon
nmm
bitly
nyt
delicious
opensocial
dopplr
search
etsy
shopping
friendfeed
social
github
twi@er
Greader
update.groovy
guardian
weather
imdb
wesabe
iploca1on
whitepages
las?m
yahoo
nestoria
yelp
ne?lix
zillow
http://github.com/spullara/yql-tables/tree/master
HackU 2013 – University of Dundee

23. Why don’t you create your own data
source for YQL…
Using YQL’s Open Tables
All we need to do is create a schema that
explains the data structure.
http://developer.yahoo.com/yql/guide/yql-opentables-
chapter.html
HackU 2013 – University of Dundee

24. What else can you do with YQL?
Get data of a website
select * from html where url='http://
www.telegraph.co.uk/' and xpath='//h3/a'
Get data of several websites in one query
select * from rss where url in (<url1>,<url2>);
HackU 2013 – University of Dundee

25. Anything else? Yes...
Create a View (Filtered data set)
use 'http://yqlblog.net/samples/searchrank.xml'
as searchrank;
select * from searchrank where query='pizza' and
dispurl like '%pizzahut%'
More about YQL Execute:
http://developer.yahoo.com/yql/guide/
yql-execute-chapter.html
HackU 2013 – University of Dundee

26. YQL, where to start…
User guide:
http://developer.yahoo.com/yql
YQL Console (Test your queries…)
http://developer.yahoo.com/yql/console/
HackU 2013 – University of Dundee