select * from internet
Yahoo Query Language
Nagesh Susarla
Application ?
Application ?
Application ?
Application ?
Application ?
Application =
Application =
data manipulation
Application =
data manipulation
data visualization
+
data
data
datadata
data
dev
dev
dev
dev
dev
dev
dev
dev
dev
dev
dev
dev
var map = new
YMap(document.getElementById('map'));
…
var currentGeoPoint = new
YGeoPoint( _c.Lat, _c.Lon );
map.addMa...
dev
var map = new
YMap(document.getElementById('map'));
…
var currentGeoPoint = new
YGeoPoint( _c.Lat, _c.Lon );
map.addMa...
dev
var map = new
YMap(document.getElementById('map'));
…
var currentGeoPoint = new
YGeoPoint( _c.Lat, _c.Lon );
map.addMa...
dev
var map = new
YMap(document.getElementById('map'));
…
var currentGeoPoint = new
YGeoPoint( _c.Lat, _c.Lon );
map.addMa...
dev
var map = new
YMap(document.getElementById('map'));
…
var currentGeoPoint = new
YGeoPoint( _c.Lat, _c.Lon );
map.addMa...
dev
var map = new
YMap(document.getElementById('map'));
…
var currentGeoPoint = new
YGeoPoint( _c.Lat, _c.Lon );
map.addMa...
dev
var map = new
YMap(document.getElementById('map'));
…
var currentGeoPoint = new
YGeoPoint( _c.Lat, _c.Lon );
map.addMa...
dev
var map = new
YMap(document.getElementById('map'));
…
var currentGeoPoint = new
YGeoPoint( _c.Lat, _c.Lon );
map.addMa...
dev
var map = new
YMap(document.getElementById('map'));
…
var currentGeoPoint = new
YGeoPoint( _c.Lat, _c.Lon );
map.addMa...
dev
var map = new
YMap(document.getElementById('map'));
…
var currentGeoPoint = new
YGeoPoint( _c.Lat, _c.Lon );
map.addMa...
dev
var map = new
YMap(document.getElementById('map'));
…
var currentGeoPoint = new
YGeoPoint( _c.Lat, _c.Lon );
map.addMa...
dev
http://weather.yahooapis.com/
forecastrss?p=FRXX0076&u=c
…
if (flickcurl_prepare(fc,
"flickr.photos.geo.correctLocatio...
dev
YQL
dev
YQL
dev
YQL
Bindings
dev
YQL
Bindings
dev
YQLYQL stmt
Bindings
dev
YQL
select
YQL stmt
Bindings
dev
YQL
select
desc
YQL stmt
Bindings
dev
YQL
select
desc
show
YQL stmt
Bindings
Open Data Tables
YQL
Open Data Tables
YQL
Open Data Tables
YQL
Open Data Tables
YQL
Open Data Tables
YQL
Bindings
Open Data Tables
YQL
Open Data Tables
YQL
Open Data Tables
YQL
Map YQL to Endpoint
Open Data Tables
YQL
Open Data Tables
YQL
Open Data Tables
Open Data Tables
USE Verb
use ‘http://url-to-def.xml’ as tableName;
select * from tableName where key=‘value’ and
type like ‘%pizza%’
YQL Execute
Why YQL Execute
Authorization/Authentication
Join data across services
Augment/Enhance data
Create APIs from web page (Scr...
YQL Execute (Choices)
Enhance YQL Language to allow for use cases
Cons: Departs from basic principle of simplicity
Cons: L...
YQL Execute
Add a new element to table schema called <execute>
Give complete control of the request lifecycle to JS
code i...
YQL Execute
Ok... No DOM, just simple server side JavaScript
YQL Execute (JS API)
Implicit/global object request that supports method
chaining
Methods to add headers, query, path and ...
YQL Execute (JS API)
Implicit object response allows setting the response
response.object = data; // can be JSON or XML
re...
YQL Execute (JS API)
Include external JS files
Make nested async YQL calls
Import external environment files for tables.
Con...
Hello world
Hello world
web + image
web + image
web + image
Unified web+image
Search Netflix catalog
Insert/Update/Delete
Insert / Update / Delete
modify data on the internet,
create tweets, add blog entries, update DBs etc
Open Data Tables spe...
insert into bitly.shorten
(login,apiKey,longUrl) values
("nagesh","R..0e","http://cnn.com")
insert into foursquare.checkin
(vid,username,password) values
("44379","foo","...")
Env Files
A way to separate the YQL queries from the table
declarations.
Enable you to declare all your external tables
Al...
Env Files
YQL Storage
Exposes Yahoo’s Distributed Data Store (Sherpa) via
open data tables.
Select/Insert/Update/Delete data across ...
YQL Storage
insert into yql.storage.admin (name, url) values ('foo',
'http://example.com/tables.env')
Conclusion
Build applications that run faster
service in the cloud does the work
conversion, filtering, parallel requests.....
data
YQL
select * from internet
http://developer.yahoo.com/yql
Nagesh Susarla
nageshs@yahoo-inc.com
twitter: nageshs
YQL: Building Open Data Tables with YQL Execute
YQL: Building Open Data Tables with YQL Execute
YQL: Building Open Data Tables with YQL Execute
YQL: Building Open Data Tables with YQL Execute
Upcoming SlideShare
Loading in …5
×

YQL: Building Open Data Tables with YQL Execute

3,000 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,000
On SlideShare
0
From Embeds
0
Number of Embeds
259
Actions
Shares
0
Downloads
43
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

YQL: Building Open Data Tables with YQL Execute

  1. 1. select * from internet Yahoo Query Language Nagesh Susarla
  2. 2. Application ?
  3. 3. Application ?
  4. 4. Application ?
  5. 5. Application ?
  6. 6. Application ?
  7. 7. Application =
  8. 8. Application = data manipulation
  9. 9. Application = data manipulation data visualization +
  10. 10. data data
  11. 11. datadata
  12. 12. data
  13. 13. dev
  14. 14. dev
  15. 15. dev
  16. 16. dev
  17. 17. dev
  18. 18. dev
  19. 19. dev
  20. 20. dev
  21. 21. dev
  22. 22. dev
  23. 23. dev
  24. 24. dev var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); …
  25. 25. dev var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } …
  26. 26. dev var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2
  27. 27. dev var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c
  28. 28. dev var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c
  29. 29. dev var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c
  30. 30. dev var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c
  31. 31. dev var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c
  32. 32. dev var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c ? ? ? ?
  33. 33. dev var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c ? ? ? ?
  34. 34. dev var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c ? ? ? ? Yahoo! My Yahoo! Mail Welcome, guest Sign In Introduction and Prerequisites Examples Display a Yahoo! Map Add controls to your Yahoo! Map Add a logger to a Yahoo! Map Place a marker on a Yahoo! Map Draw Polylines on a Yahoo! Map Place Smart Markers on a Yahoo! Map Add multiple custom markers on Yahoo! Map Traffic information overlay Yahoo! Pipes RSS output information overlay Yahoo! Pipes JSON output information overlay Local search results overlay Overlay GeoRSS XML data Overlay custom objects at YGeoPoint location Overlay custom objects at YCoordPoint location Overlay polyline on Yahoo! Map Export Yahoo! Map in GeoRSS format GeoRSS overlay with polyline Get zoom level given radius from center Custom position of controls Change Marker Images Dynamically Use YMapTypeControl to position map types Other Features of the Yahoo! Maps AJAX API Rate Limit Terms of Service Support & Community Download all of the cheatsheets and example code in the Yahoo Maps API Reference Bundle. For detailed descriptions of the classes and methods in the API, check out v3.8 Reference Manual. Yahoo! Maps Cheatsheets YMap YEvent, YGeoRSS, YMarker Remaining, Supporting Classes Yahoo! Maps Web Services - AJAX API Getting Started Guide Learn the fundamentals of the Yahoo! Maps AJAX API and start creating your own maps. This page contains: Introduction and Prerequisites The Yahoo! AJAX Maps API lets developers add maps to their web sites using DHTML and JavaScript®. Maps are fully embeddable and scriptable using the JavaScript programming language. Yahoo! Maps AS-Flash API's built-in geocoder means that you can specify a physical address or latitude/longitude coordinates for your map's location, as you like. In order to create and test applications using Yahoo! Maps AJAX API, you'll need to use a supported web browser: Firefox 2, Internet Explorer 6 or 7, Opera 9 or Safari 3. Newer versions of these browsers are also supported. You can code your JavaScript applications using your favorite text editor or IDE. An application ID is required to use the Yahoo! Maps AJAX API. You can get an application ID here. For best results, you should launch your web pages from a web server. However, it's possible to run applications directly from your hard drive by double-clicking on them. This can be done by lowering the security settings in the Internet Explorer browser or by setting the UniversalBrowserRead property in Firefox. Displaying a Yahoo! Map All the API classes and methods necessary to interact with Yahoo! Maps are available to your application when you include Yahoo! Maps AJAX API library in your web page.This example uses the demonstration application ID "YD-eQRpTl0_JX2E95l_xAFs5UwZUlNQhhn7lj1H"; you cannot use this Application ID and must request one here. <html> <head> <b><script type="text/javascript" src="http://api.maps.yahoo.com/ajaxymap?v=3.8&appid=YD- YDN Maps Suite AJAX Maps Leonardo da Vinci: Hacker Whaddyathink? & the Yahoo! Application Platform Making the new Yahoo! Currency Converter accessible Retrieving and displaying data from Wikipedia with YQL RECENT BLOG ARTICLES January 24, 2009 January 22, 2009 January 21, 2009 January 18, 2009 READY TO GET STARTED? Get an App ID Search MY PROJECTS APIs & WEB SERVICES RESOURCES SUPPORT You aren't signed in Sign In Help Flickr Services API Documentation Feeds Your API Keys Apply for a new API Key flickr.places.getPlaceTypes Fetches a list of available place types for Flickr. Authentication This method does not require authentication. Arguments api_key (Required) Your API application key. See here for more details. Example Response <place_types> <place_type place_type_id="22">neighbourhood</place_type> <place_type place_type_id="7">locality</place_type> <place_type place_type_id="9">county</place_type> <place_type place_type_id="8">region</place_type> <place_type place_type_id="12">country</place_type> <place_type place_type_id="29">continent</place_type> </place_types> Error Codes 100: Invalid API Key The API key passed was not valid or has expired. 105: Service currently unavailable The requested service is temporarily unavailable. 111: Format "xxx" not found The requested response format was not found. 112: Method "xxx" not found The requested method was not found. 114: Invalid SOAP envelope The SOAP envelope send in the request could not be parsed. 115: Invalid XML-RPC Method Call The XML-RPC request document could not be parsed. API Explorer API Explorer : flickr.places.getPlaceTypes Home The Tour Sign Up Explore Search Yahoo! My Yahoo! Mail Welcome, guest Sign In Yahoo! Weather RSS Feed The Weather RSS feed enables you to get up-to-date weather information for your location. You can save this feed in My Yahoo! or your favorite feed aggregator, or incorporate the RSS data into your own web site or client application. The Weather RSS feed is a dynamically-generated feed based on zip code or Location ID. This page describes the format of the request URL and the RSS response for developers. It contains these sections: RSS Request Summary Request Parameters RSS Response Top-Level Elements Channel Elements Image Elements Item Elements Condition Codes Examples Terms of Use RSS Request The Weather RSS feed request follows simple HTTP GET syntax: start with a base URL and then add parameters and values after a question mark (?). Multiple parameters are separated by an ampersand (&). The base URL for the Weather RSS feed is http://weather.yahooapis.com/forecastrss In this section, Summary shows how to make an RSS request with different parameters; Request Parameters shows a table summarizing those parameters. SUMMARY The Weather RSS feed request follows simple HTTP GET syntax: start with a base URL and then add parameters and values after a question mark (?). Multiple parameters are separated by an ampersand (&). For the Weather RSS feed there are two parameters: p for location. u for degrees units (Fahrenheit or Celsius). The location parameter p is required. Use this parameter to indicate the location for the weather forecast as a zip code or Location ID. http://weather.yahooapis.com/forecastrss?p=location For example, to get weather for Yahoo! Headquarters in Sunnyvale, CA, use the zip code for Sunnyvale (94089): http://weather.yahooapis.com/forecastrss?p=94089 Or use the Location ID for Sunnyvale (USCA1116): YDN Weather RSS Feed Search MY PROJECTS APIs & WEB SERVICES RESOURCES SUPPORT Yahoo! My Yahoo! Mail Welcome, guest Sign In Image Search Documentation for Yahoo! Search Submitting Image Queries The Image Search Web Service allows you to search the Internet for images. Request URL http://search.yahooapis.com/ImageSearchService/V1/imageSearch Request parameters See information on constructing REST queries Parameter Value Description appid string (required) The application ID. See Application IDs for more information. query string (required) The query to search for. Use + to include terms, - to exclude terms, and put quotes around "exact phrase". type all (default), any, or phrase The kind of search to submit: all returns results with all query terms. any returns results with one or more of the query terms. phrase returns results containing the query terms as a phrase. results integer: default 10, max 50 The number of results to return. start integer: default 1 The starting result position to return (1-based). The finishing position (start + results - 1) cannot exceed 1000. format any (default), bmp, gif, jpeg, png Specifies the kind of image file to search for. adult_ok no value or 1 The service filters out adult content by default. Enter a 1 to allow adult content. coloration any (default), color, bw The service returns only the images of the coloration specified (color or black-and-white). site string: default no value A domain to restrict your searches to (e.g. www.yahoo.com). You may submit up to 30 values (site=www.yahoo.com&site=www.cnn.com). output string: xml (default), json, php The format for the output. If json is requested, the results will be returned in JSON format. If php is requested, the results will be returned in Serialized PHP format. callback string The name of the callback function to wrap around the JSON data. The following characters are allowed: A-Z a-z 0-9 . [] and _. If output=json has not been requested, this parameter has no effect. More information on the YDN Image Search Documentation for Yahoo! Search Web Services yahoo local search code in php My Web Search API shut down announcement Re: Site Explorer API returns 999 Rate Limit Exceeded Reminder: Old obsolete versions of Local Search (V1, V2) shutting do YAHOO! GROUPS DISCUSSIONS Tue, 27 Jan 2009 Wed, 21 Jan 2009 Wed, 21 Jan 2009 Mon, 19 Jan 2009 READY TO GET STARTED? By applying for an Application ID for this service, you hereby agree to the Terms of Use Get an App ID view all Search MY PROJECTS APIs & WEB SERVICES RESOURCES SUPPORT
  35. 35. dev http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); …
  36. 36. dev YQL
  37. 37. dev YQL
  38. 38. dev YQL Bindings
  39. 39. dev YQL Bindings
  40. 40. dev YQLYQL stmt Bindings
  41. 41. dev YQL select YQL stmt Bindings
  42. 42. dev YQL select desc YQL stmt Bindings
  43. 43. dev YQL select desc show YQL stmt Bindings
  44. 44. Open Data Tables YQL
  45. 45. Open Data Tables YQL
  46. 46. Open Data Tables YQL
  47. 47. Open Data Tables YQL
  48. 48. Open Data Tables YQL Bindings
  49. 49. Open Data Tables YQL
  50. 50. Open Data Tables YQL
  51. 51. Open Data Tables YQL Map YQL to Endpoint
  52. 52. Open Data Tables YQL
  53. 53. Open Data Tables YQL
  54. 54. Open Data Tables
  55. 55. Open Data Tables
  56. 56. USE Verb use ‘http://url-to-def.xml’ as tableName; select * from tableName where key=‘value’ and type like ‘%pizza%’
  57. 57. YQL Execute
  58. 58. Why YQL Execute Authorization/Authentication Join data across services Augment/Enhance data Create APIs from web page (ScrAPI) Data transformation Logic in the Cloud
  59. 59. YQL Execute (Choices) Enhance YQL Language to allow for use cases Cons: Departs from basic principle of simplicity Cons: Learning curve Pros: Makes us look geeky? Pick a language familiar and loved by developers Pros: Doesn’t change YQLs familiar syntax
  60. 60. YQL Execute Add a new element to table schema called <execute> Give complete control of the request lifecycle to JS code in execute Provide a JS API for common tasks E4X support for manipulating XML: XML Literals, filtering
  61. 61. YQL Execute Ok... No DOM, just simple server side JavaScript
  62. 62. YQL Execute (JS API) Implicit/global object request that supports method chaining Methods to add headers, query, path and matrix parameters. Support for HTTP methods
  63. 63. YQL Execute (JS API) Implicit object response allows setting the response response.object = data; // can be JSON or XML response.maxAge= 100; // set maxage in seconds
  64. 64. YQL Execute (JS API) Include external JS files Make nested async YQL calls Import external environment files for tables. Convert between XML <-> JSON Make external REST calls.
  65. 65. Hello world
  66. 66. Hello world
  67. 67. web + image
  68. 68. web + image
  69. 69. web + image
  70. 70. Unified web+image
  71. 71. Search Netflix catalog
  72. 72. Insert/Update/Delete
  73. 73. Insert / Update / Delete modify data on the internet, create tweets, add blog entries, update DBs etc Open Data Tables specify required/optional keys Execute block supports POST/PUT/DELETE Some tables with I/U/D Twitter Status, Bit.ly shortener Yahoo profile status, application update stream
  74. 74. insert into bitly.shorten (login,apiKey,longUrl) values ("nagesh","R..0e","http://cnn.com")
  75. 75. insert into foursquare.checkin (vid,username,password) values ("44379","foo","...")
  76. 76. Env Files A way to separate the YQL queries from the table declarations. Enable you to declare all your external tables Allow including other environment files Allow the use of the “SET” keyword http://query.yahooapis.com/v1/public/yql? env=<ENV_URL>&q=.....
  77. 77. Env Files
  78. 78. YQL Storage Exposes Yahoo’s Distributed Data Store (Sherpa) via open data tables. Select/Insert/Update/Delete data across multiple colos across the world. Data is accessible via special ‘store://’ protocol scheme. YQL layers UNIX like read, write, update privileges on keys.
  79. 79. YQL Storage insert into yql.storage.admin (name, url) values ('foo', 'http://example.com/tables.env')
  80. 80. Conclusion Build applications that run faster service in the cloud does the work conversion, filtering, parallel requests... with smaller network footprint fewer (client) calls smaller data amounts (less parsing cost)
  81. 81. data
  82. 82. YQL select * from internet http://developer.yahoo.com/yql Nagesh Susarla nageshs@yahoo-inc.com twitter: nageshs

×