YQL lunch2.0 presentation

Yahoo Query Language
select * from internet

Jonathan Trevor (jtrevor@yahoo-inc.com)

data manipulation
Application =

data manipulation
Application = +
data visualization

var map = new
YMap(document.getElementById('map'));
…
var currentGeoPoint = new
dev
YGeoPoint( _c.Lat, _c.Lon );
map.addMarker(currentGeoPoint);
…

var map = new
…
dev
…

…
if (flickcurl_prepare(fc,
quot;flickr.photos.geo.correctLocationquot;,
parameters, count)) { … }
…

var map = new
…
dev
…

…
…

http://search.yahooapis.com/
ImageSearchService/V1/imageSearch?
appid=YahooDemo&query=Corvette&results=2

var map = new
…
dev
…

…
…

http://search.yahooapis.com/

http://weather.yahooapis.com/
forecastrss?p=FRXX0076&u=c

var map = new
…
dev

?
…

…

?
…

? http://search.yahooapis.com/

?
http://weather.yahooapis.com/

Yahoo! My Yahoo! Mail Welcome, guest Sign In

Search

var map = new
MY PROJECTS APIs & WEB SERVICES RESOURCES SUPPORT

YDN Maps Suite AJAX Maps

Yahoo! Maps Web Services - AJAX API Getting READY TO GET STARTED?
… You aren't signed in Sign In Help

Started Guide Get an App ID
dev Home The Tour Sign Up Explore Search
Learn the fundamentals of the Yahoo! Maps AJAX API and start creating your own maps. This page contains:
Introduction and Prerequisites Yahoo! Maps Cheatsheets
Flickr Services RECENT BLOG ARTICLES
Yahoo! My Yahoo! Mail
Examples Welcome, guest Sign In

API Documentation Feeds Your API Keys map.addMarker(currentGeoPoint);
Display a Yahoo! Map

?
Apply for a new API Key
Search
Add controls to your Yahoo! Map Leonardo da Vinci: Hacker
… January 24, 2009
Add a logger to a Yahoo! Map
Place a marker on a Yahoo! Map
MY PROJECTS APIs & WEB SERVICES RESOURCES SUPPORT
Draw Polylines on a Yahoo! Map Whaddyathink? & the Yahoo! Application
flickr.places.getPlaceTypes
Place Smart Markers on a Yahoo! Map Platform
YDN Weather RSS Feed
Add multiple custom markers on Yahoo! Map January 22, 2009
Traffic information overlay
Yahoo! Weather RSS Feed
Yahoo! Pipes RSS output information overlay Making the new Yahoo! Currency Converter
Yahoo! My Yahoo! Mail Welcome, guest Sign In
Fetches a list of available place types for Flickr.
Yahoo! Pipes JSON output information overlay accessible
YMap
Local search results overlay The Weather RSS feed enables you
Januaryto get up-to-date weather information for your location. You can save this
21, 2009
… Search
Overlay GeoRSS XML data
feed in My Yahoo! or your favorite feed aggregator, or incorporate the RSS data into your own web site or client
Overlay custom objects at YGeoPoint location
application. The Weather RSSRetrievingdynamically-generatedfrom based on zip code or Location ID.
feed is a and displaying data feed
Authentication
Overlay custom objects at YCoordPoint location
Wikipedia with YQL PROJECTS
MY APIs & WEB SERVICES RESOURCES SUPPORT
Overlay polyline on Yahoo! Map
This page describes the format of the 18, 2009 URL and the RSS response for developers. It contains these
January request

?
Export Yahoo! Map in GeoRSS format YDN Image Search Documentation for Yahoo! Search Web Services
This method does not require authentication.
sections:
GeoRSS overlay with polyline

Image Search Documentation for Yahoo! Search
Get zoom level given radius from center READY TO GET STARTED?
RSS Request
Arguments
Custom position of controls
parameters, count)) { … } By applying for an Application ID for this service, you
Summary
Change Marker Images Dynamically hereby agree to the Terms of Use
Request Parameters
Use YMapTypeControl to position map types
Submitting Image Queries
…
api_key (Required)
Other Features of the Yahoo! Maps AJAX API RSS Response
YEvent, YGeoRSS, YMarker
Get an App ID
Rate Limit Your API application key. See here for moreTop-Level Elements
details. The Image Search Web Service allows you to search the Internet for images.
Terms of Service Channel Elements
Support & Community
Image Elements

?
Example Response Item Elements
Request URL
Download all of the cheatsheets and example code in the Yahoo
Condition Codes view all
YAHOO! GROUPS DISCUSSIONS
Maps API Reference Bundle.
http://search.yahooapis.com/ImageSearchService/V1/imageSearch
<place_types> Examples
yahoo local search code in php
For detailed descriptions of the classes and methods in the API, place_type_id=quot;22quot;>neighbourhood</place_type>
<place_type Terms of Use
Tue, 27 Jan 2009
check out v3.8 Reference Manual. <place_type place_type_id=quot;7quot;>locality</place_type>
http://search.yahooapis.com/ Request parameters
<place_type place_type_id=quot;9quot;>county</place_type>
RSS Request My Web Search API shut down
<place_type place_type_id=quot;8quot;>region</place_type>
Remaining, Supporting Classes announcement
See information on constructing REST queries
<place_type place_type_id=quot;12quot;>country</place_type>
The Weather RSS feed request follows simple HTTP GET syntax: start with a base URL and then add Wed, 21 Jan 2009
Introduction and Prerequisites
<place_type place_type_id=quot;29quot;>continent</place_type> (?). Multiple parameters are separated by an ampersand (&).
parameters and values after a question mark Parameter Value Description
</place_types>
appid=YahooDemo&query=Corvette&results=2 Re: Site Explorer API returns 999 Rate Limit
appid string (required) The application ID. See Application IDs for more information.
The Yahoo! AJAX Maps API lets developers add maps to their web sites using DHTML and JavaScript®. the Weather RSS feed is
The base URL for Maps Exceeded
are fully embeddable and scriptable using the JavaScript programming language. Yahoo! Maps AS-Flash API's query string (required) The query to search for. Use + to include terms, - to exclude terms, and

?
Wed, 21 Jan 2009
Error Codes put quotes around quot;exact phrasequot;.
built-in geocoder means that you can specify a physical address or latitude/longitude coordinates for your map's
http://weather.yahooapis.com/forecastrss
location, as you like. type all (default), any, or The kind of search to submit: Reminder: Old obsolete versions of Local
In this section, Summary shows how to make an RSS request with different parameters; Request Parameters
phrase Search (V1, V2) shutting do
In order to create and test applications 100: Yahoo! Maps AJAX API, you'll need to use a supported summarizing those parameters.
using Invalid API Key shows a table web Mon, 19 Jan 2009
all returns results with all query terms.
The API key passed was not valid or has expired. any returns results with one or more of the query terms.
browser: Firefox 2, Internet Explorer 6 or 7, Opera 9 or Safari 3. Newer versions of these browsers are also
phrase returns results containing the query terms as a phrase.
supported. You can code your JavaScript applications usingcurrently unavailable
105: Service your favorite text editor or IDE.
SUMMARY results integer: default 10, The number of results to return.
The requested service is temporarily unavailable.
max 50
An application ID is required to use the Yahoo! Maps AJAX API. You can get an application ID here.
The Weather RSS feed request follows simple HTTP GET syntax: start with a base URL and then add
111: Format quot;xxxquot; not found
start integer: default 1 by an starting result position to return (1-based). The finishing position (start
The ampersand (&). For
parameters and values after a question mark (?). Multiple parameters are separated
For best results, you should launch your web pagesrequested response format was not found. applications
The from a web server. However, it's possible to run + results - 1) cannot exceed 1000.
the Weather RSS feed there are two parameters:
directly from your hard drive by double-clickingMethod This can be donefound
112: on them. quot;xxxquot; not by lowering the security settings in format any (default), bmp, Specifies the kind of image file to search for.
the Internet Explorer browser or by setting the The requested method was not found. p for location.
UniversalBrowserRead property in Firefox. gif, jpeg, png
u for degrees units (Fahrenheit or Celsius). adult_ok
http://weather.yahooapis.com/ no value or 1 The service filters out adult content by default. Enter a 1 to allow adult
114: Invalid SOAP envelope
content.
The SOAP envelope send in the requestlocationnot be parsed. required. Use this parameter to indicate the location for the weather forecast as a
The could parameter p is
Displaying a Yahoo! Map coloration any (default), color, The service returns only the images of the coloration specified (color or
zip code or Location ID.
115: Invalid XML-RPC Method Call
bw black-and-white).
All the API classes and methods necessary to interact with Yahoo! Mapsdocument couldyour application when
The XML-RPC request are availablehttp://weather.yahooapis.com/forecastrss?p=location
to not be parsed.
site string: default no A domain to restrict your searches to (e.g. www.yahoo.com). You may
you include Yahoo! Maps AJAX API library in your web page.This example uses the demonstration application
value submit up to 30 values (site=www.yahoo.com&site=www.cnn.com).
ID quot;YD-eQRpTl0_JX2E95l_xAFs5UwZUlNQhhn7lj1Hquot;; you cannot use this Application Forand must requestweather for Yahoo! Headquarters in Sunnyvale, CA, use the zip
ID example, to get one code for Sunnyvale
API Explorer output string: xml The format for the output. If json is requested, the results will be returned
here. (94089): (default), json, php in JSON format. If php is requested, the results will be returned in
Serialized PHP format.
API Explorer : flickr.places.getPlaceTypes
<html> http://weather.yahooapis.com/forecastrss?p=94089
callback string The name of the callback function to wrap around the JSON data. The
<head>
following characters are allowed: A-Z a-z 0-9 . [] and _. If output=json has
Or use the Location ID for Sunnyvale (USCA1116):
<b><script type=quot;text/javascriptquot;
not been requested, this parameter has no effect. More information on the
src=quot;http://api.maps.yahoo.com/ajaxymap?v=3.8&appid=YD-

dev

Bindings

YQL
YQL stmt

dev

Bindings

YQL
YQL stmt
select

dev

Bindings

YQL
YQL stmt
select
desc

dev

Bindings

YQL
YQL stmt
select
desc
show

Why YQL?
Thousands of Web Services and sources that
provide valuable data
Require Developers to read documentation and form
URLs/queries.
Data is isolated
Needs ﬁltering, combining, tweaking, shaping
even after it gets to the developer.

YQL
Hosted web service
SQL-Like Language
Familiar to developers
Synonymous with Data access
Expressive enough to get the right data.
Self Describing - show, desc table
Allows you to query, ﬁlter and join data across any

YQL Statement Basics
show: lists the supported tables
desc: describes the structure of a table
select: fetches data
use: use an Open Data Table

show tables
desc social.contacts

YQL Select Statement

SELECT what
FROM collection
WHERE filter condition
IN (sub-select)
LIMIT n OFFSET n

Demo:
developer.yahoo.com/yql/

Get any data:
open data tables

Open Data Tables
Bindings

YQL

Open Data Tables

Map YQL to Endpoint

YQL

Open Data Tables

Brings the power of the table model to new APIs
Schema deﬁnes mapping between YQL and
Endpoint
Keys can either be query, path or matrix parameters
Keys can be marked as required
YQL Compiler validates existence of required keys

USE Verb
use ‘http://url-to-def.xml’ as tableName;
select * from tableName where key=‘value’...

Imports the Table deﬁnition
Makes it available to following statements as
tableName
desc tableName - describes the def
Acts like a First Class Data Source just like
other YQL Tables.

Community:
github/datatables.org

Share your work
leverage the work of others to make access easier by
sharing tables
github repository:
http://github.com/spullara/yql-tables/tree/master
datatables.org mirrors the master and builds a single
environment with all tables -> faster serving
http://developer.yahoo.com/yql/console/?env=http://
datatables.org/alltables.env

YQL Execute
extends Open Data Tables with server-side
transformation using Javascript
access APIs that require authentication
Netflix OAuth, FlickrAuth, Google AuthSub
join data across services
grab New York Times article tags and generate find
associated flickr photos
combine multiple searches into a single result: twitter,
web, news and image

YQL Execute
augment data:
city/state to APIs that just return zip code
create APIs from web page
celebrity birthdays scraped from imdb, caltrain
data transformation
convert the result from xml to Google's kml format
move (some of the) business logic of your application
to the cloud

<execute>
Javascript server-side scripting
No DOM, events etc, pure JS scripting
E4X support - XML literals, ﬁltering ...
YQL no longer performs the GET request to the table
binding URL
Javascript has additional useful global objects
y.rest, y.include, y.query...
execute tag must return the data that will be the
output of the YQL select

hello world
<?xml version=quot;1.0quot; encoding=quot;UTF-8quot;?>
<table xmlns=quot;http://query.yahooapis.com/v1/schema/table.xsdquot;>
<meta>
<sampleQuery>select * from {table} where a='cat' and b='dog';</sampleQuery>
</meta>
<bindings>
<select itemPath=quot;quot; produces=quot;XMLquot;>
<urls>
<url>http://fake.url/{a}</url>
</urls>
<inputs>
<key id='a' type='xs:string' paramType='path' required=quot;truequot; />
<key id='b' type='xs:string' paramType='variable' required=quot;truequot; />
</inputs>
<execute><![CDATA[
// Your javascript goes here. We will run it on our servers
response.object = <item>
<url>{request.url}</url>
<a>{a}</a>
<b>{b}</b>
</item>;
]]></execute>
</select>
</bindings>
</table>

Uniﬁed web+image
<?xml version=quot;1.0quot; encoding=quot;UTF-8quot;?>
<table xmlns=quot;http://query.yahooapis.com/v1/schema/table.xsdquot;>
<meta>
<author>Nagesh Susarla</author>
<documentationURL>See search.web and search.images for more details</documentationURL>
</meta>
<bindings>
<select itemPath=quot;results.resultquot; produces=quot;XMLquot;>
<urls>
<url></url>
</urls>
<inputs>
<key id=quot;queryquot; type=quot;xs:stringquot; paramType=quot;queryquot; required=quot;truequot;/>
</inputs>
<execute><![CDATA[
var qs = query;
var search = y.query('select * from search.web(50) where query=@query', {query: qs}).results;
var images = [];
default xml namespace='http://www.inktomi.com/';
for each (var result in search.result) {
images.push(y.query('select * from search.images(1) where query=@query and url=@url', {url:result.url, query:qs}));
}
var i = 0;
for each (var result in search.result) {
var image = images[i++].results.result;
if (image) {
result.image = <image>{image}</image>;
}
}
response.object = search;
]]>
</execute>
</select>
</bindings>
</table>

Search Netflix catalog
<table xmlns=quot;http://query.yahooapis.com/v1/schema/table.xsdquot; https=quot;truequot;>
<meta>...</meta>
<bindings>
<select itemPath=quot;quot; produces=quot;XMLquot; >
<urls><url env=quot;allquot;>http://api.netflix.com/catalog/titles/</url></urls>
<paging model=quot;offsetquot;>
<start id=quot;start_indexquot; default=quot;0quot; />
<pagesize id=quot;max_resultsquot; max=quot;100quot; />
<total default=quot;10quot; />
</paging>
<inputs>
<key id=quot;termquot; type=quot;xs:stringquot; paramType=quot;queryquot; required=quot;truequot; />
<key id=quot;ckquot; type=quot;xs:stringquot; paramType=quot;variablequot; required=quot;truequot; />
<key id=quot;cksquot; type=quot;xs:stringquot; paramType=quot;variablequot; required=quot;truequot; />
</inputs>
<execute><![CDATA[
// Include the OAuth libraries from oauth.net
y.include(quot;http://oauth.googlecode.com/svn/code/javascript/oauth.jsquot;);
y.include(quot;http://oauth.googlecode.com/svn/code/javascript/sha1.jsquot;);
// Collect all the parameters
var encodedurl = request.url;
var accessor = { consumerSecret: cks, tokenSecret: quot;quot;};
var message = { action: encodedurl, method: quot;GETquot;, parameters: [[quot;oauth_consumer_keyquot;,ck],[quot;oauth_versionquot;,quot;1.0quot;]]};
OAuth.setTimestampAndNonce(message);
// Sign the request
OAuth.SignatureMethod.sign(message, accessor);
try {
// get the content from service along with the OAuth header, and return the result back out
response.object = request.contentType('application/xml').header(quot;Authorizationquot;, OAuth.getAuthorizationHeader(quot;netflix.comquot;,
message.parameters)).get().response;
} catch(err) {
response.object = {'result':'failure', 'error': err};
}
]]></execute>
</select>
</bindings>
</table>

flickr.auth.frob
<?xml version=quot;1.0quot; encoding=quot;UTF-8quot; ?>
<table xmlns=quot;http://query.yahooapis.com/v1/schema/table.xsdquot; https=”true”>
<meta>
<sampleQuery> select * from {table}</sampleQuery>
</meta>
<bindings>
<select itemPath=quot;rspquot; produces=quot;XMLquot;>
<urls>
<url>http://api.flickr.com/services/rest/</url>
</urls>
<inputs>
<key id='method' type='xs:string' paramType='variable' const=quot;truequot; default=quot;flickr.auth.getFrobquot; />
<key id='api_key' type='xs:string' paramType='variable' required=quot;truequot; />
<key id='secret' type='xs:string' paramType='variable' required=quot;truequot; />
</inputs>
<execute><![CDATA[
// Include the flickr signing library
y.include(quot;http://yqlblog.net/samples/flickr.jsquot;);
// GET the flickr result using a signed url
var fs = new flickrSigner(api_key,secret);
response.object = y.rest(fs.createUrl({method:method, format:quot;quot;})).get().response();
]]></execute>
</select>
</bindings>
</table>

Conclusion

More easily build applications
fewer lines of code
data processing done away from app
consistent and familiar syntax for all data access
iterative environment for developing the query

Conclusion

build applications that run faster
service in the cloud does the work
conversion, ﬁltering, parallel requests...
with smaller network footprint
fewer (client) calls
smaller data amounts
closer to the data, fatter pipes

YQL
select * from internet

http://developer.yahoo.com/yql

Jonathan Trevor
yql-questions@yahoo-inc.com

YQL lunch2.0 presentation

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (17)

Recently uploaded

Recently uploaded (11)

YQL lunch2.0 presentation