Yahoo Query Language
                       SELECT * FROM INTERNET




Monday, May 17, 2010
Application ?




Monday, May 17, 2010
Application ?




Monday, May 17, 2010
Application ?




Monday, May 17, 2010
Application ?




Monday, May 17, 2010
Application ?




Monday, May 17, 2010
Application =




Monday, May 17, 2010
data manipulation
         Application =




Monday, May 17, 2010
data manipulation
         Application =           +
                         data visualization




Monday, May 17, 2010
data
         data




Monday, May 17, 2010
data




Monday, May 17, 2010
data




Monday, May 17, 2010
Monday, May 17, 2010
dev




Monday, May 17, 2010
dev




Monday, May 17, 2010
dev




Monday, May 17, 2010
dev




Monday, May 17, 2010
dev




Monday, May 17, 2010
dev




Monday, May 17, 2010
dev




Monday, May 17, 2010
dev




Monday, May 17, 2010
dev




Monday, May 17, 2010
dev




Monday, May 17, 2010
dev




Monday, May 17, 2010
var map = new YMap
                       (document.getElementById('map'));
                       …
              dev    ...
var map = new YMap
                       (document.getElementById('map'));
                       …
              dev    ...
var map = new YMap
                       (document.getElementById('map'));
                       …
              dev    ...
var map = new YMap
                       (document.getElementById('map'));
                       …
              dev    ...
var map = new YMap
                       (document.getElementById('map'));
                       …
              dev    ...
var map = new YMap
                       (document.getElementById('map'));
                       …
              dev    ...
var map = new YMap
                       (document.getElementById('map'));
                       …
              dev    ...
var map = new YMap
                       (document.getElementById('map'));
                       …
              dev    ...
var map = new YMap
                           (document.getElementById('map'));
                           …
             ...
var map = new YMap
                           (document.getElementById('map'));
                           …
             ...
Yahoo!    My Yahoo!   Mail                                                                                                ...
var map = new YMap
                       (document.getElementById('map'));
                       …
              dev    ...
dev




                       YQL




Monday, May 17, 2010
dev




                       YQL




Monday, May 17, 2010
dev



                        Bindings




                       YQL




Monday, May 17, 2010
dev



                        Bindings




                       YQL




Monday, May 17, 2010
dev



                                   Bindings




                       YQL stmt
                                  Y...
dev



                                   Bindings




                       YQL stmt
                                  Y...
dev



                                   Bindings




                       YQL stmt
                                  Y...
dev



                                   Bindings




                       YQL stmt
                                  Y...
Why YQL?
              Thousands of Web Services and sources that
              provide valuable data
              Requir...
YQL
                Hosted web service and SQL-Like Language
                       Familiar to developers
               ...
YQL Statement Basics
              show: lists the supported tables
              desc: describes the structure of a table...
YQL Select Statement

              SELECT what
              FROM collection
              WHERE filter condition
       ...
Demo:
         developer.yahoo.com/yql/



Monday, May 17, 2010
Executing YQL with PHP




Monday, May 17, 2010
Download PHP5 SDK from GitHub

  http://github.com/yahoo



Monday, May 17, 2010
$query = new YQLQuery();

         $yql = “SELECT * from search.web where
         query = ‘phpday.it’“;

         $data =...
Get any data:
         open data tables



Monday, May 17, 2010
Open Data Tables



                YQL



Monday, May 17, 2010
Open Data Tables



                YQL



Monday, May 17, 2010
Open Data Tables



                YQL



Monday, May 17, 2010
Open Data Tables



                YQL



Monday, May 17, 2010
Open Data Tables
                       Bindings




                YQL



Monday, May 17, 2010
Open Data Tables



                YQL



Monday, May 17, 2010
Open Data Tables



                YQL



Monday, May 17, 2010
Open Data Tables

                       Map YQL to Endpoint


                YQL



Monday, May 17, 2010
Open Data Tables

                       Brings the power of the table model to new APIs
                       Schema defi...
Open Data Tables



                YQL



Monday, May 17, 2010
Open Data Tables



                YQL



Monday, May 17, 2010
Open Data Tables




Monday, May 17, 2010
Open Data Tables




Monday, May 17, 2010
USE Verb
          use ‘http://url-to-def.xml’ as tableName;
          select * from tableName where key=‘value’...


    ...
Basics

                       select * from {table} where {key1} = "{value1}" and
                       {key2} = {value2...
Remote vs. Local




Monday, May 17, 2010
select
                        *
                        from
                        flickr.photos.search
                ...
select
                        *
                        from
                        flickr.photos.search
                ...
select
                        *
                        from
                        flickr.photos.search
                ...
desc {table}

Monday, May 17, 2010
Local
Monday, May 17, 2010
select * from flickr.photos.search where text =
         "kittens" and tags = "domokun" and farm = 4
Monday, May 17, 2010
Limits and Paging




Monday, May 17, 2010
select * from flickr.photos.search where text = "kittens"
         and tags = "domokun" and farm = 4 limit 10



Monday, Ma...
select * from flickr.photos.search(10) where text =
         "kittens" and tags = "domokun" and farm = 4



Monday, May 17,...
select * from flickr.photos.search(20) where text =
         "kittens" and tags = "domokun" and farm = 4



Monday, May 17,...
select * from flickr.photos.search(0) where text =
         "kittens" and tags = "domokun" and farm = 4 limit 10



Monday,...
Limit Summary

              limit 10 - give me max 10 results from the data you
              have
              {table}(...
Offset




Monday, May 17, 2010
select * from {table} (10) where foo = "bar" offset
                                    10

                              ...
select * from {table} (10) where foo = "bar" offset
                                    10




                <results>
 ...
select * from {table} (10,10) where foo = "bar"




                             <results>
                               ...
Inner Queries




Monday, May 17, 2010
select * from {table1}
                where
               {key1} in
               (select id from {table2});


Monday, ...
select * from
           flickr.photos.info
           where
           photo_id in
           (select id
           from fl...
Multi-key




Monday, May 17, 2010
select * from {table1}
          where
         ({key1}, {key2}) in
         (select foo, bar from {table2});

Monday, May...
select * from flickr.places
       where
       (lat, lon) in
       (select match1, match2 from regex
       where
       ...
Filters




Monday, May 17, 2010
select * from {table}(20) |
               sort(field=”foo”,
               descending = “true”) |
               truncate ...
Filters are always applied
         last, in order



Monday, May 17, 2010
select * from {table}(20) | sort(field=”foo”, descending = “true”) | truncate 10




                <results>             ...
Super-secret Filters


              Xpath()
              Regex()
              ... there are more I can’t remember




M...
Community:
         github/datatables.org



Monday, May 17, 2010
Share your work
              leverage the work of others to make access easier by
              sharing tables
          ...
Do it your way:
         <execute>



Monday, May 17, 2010
YQL Execute
              extends Open Data Tables with server-side code
              access APIs that require authentica...
YQL Execute

              augment data:
                       city/state to APIs that just return zip code
             ...
<execute>
              Javascript server-side scripting
              No DOM, events etc, pure JS scripting
             ...
hello world
       <?xml version="1.0" encoding="UTF-8"?>
       <table xmlns="http://query.yahooapis.com/v1/schema/table....
Unified web+image
   <?xml version="1.0" encoding="UTF-8"?>
   <table xmlns="http://query.yahooapis.com/v1/schema/table.xsd...
Search Netflix catalog
         <table xmlns="http://query.yahooapis.com/v1/schema/table.xsd" https="true">
           <met...
INSERT INTO internet




Monday, May 17, 2010
Insert / Update / Delete
              With INSERT / UPDATE / DELETE SQL verbs
              developers can modify data on...
select * from twitter.status where
          id="2108869549" and username="xxx"
          and password="yyy"

    	       ...
insert into twitter.status
          (status,username,password) values
          ("new tweet from YQL",”xxx”,”yyy”);

    ...
delete from twitter.status where
          id="2108869549" and username="xxx"
          and password="yyy"


    	      	 ...
Conclusion

                       More easily build applications
                        fewer lines of code
            ...
Conclusion
                       build applications that run faster
                        service in the cloud does the...
data




Monday, May 17, 2010
YQL & PHP
     SELECT * FROM INTERNET

             http://developer.yahoo.com/yql


                                  Dus...
Upcoming SlideShare
Loading in...5
×

YQL & PHP

12,556

Published on

Published in: Technology, Sports
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
12,556
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
74
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide

YQL & PHP

  1. 1. Yahoo Query Language SELECT * FROM INTERNET Monday, May 17, 2010
  2. 2. Application ? Monday, May 17, 2010
  3. 3. Application ? Monday, May 17, 2010
  4. 4. Application ? Monday, May 17, 2010
  5. 5. Application ? Monday, May 17, 2010
  6. 6. Application ? Monday, May 17, 2010
  7. 7. Application = Monday, May 17, 2010
  8. 8. data manipulation Application = Monday, May 17, 2010
  9. 9. data manipulation Application = + data visualization Monday, May 17, 2010
  10. 10. data data Monday, May 17, 2010
  11. 11. data Monday, May 17, 2010
  12. 12. data Monday, May 17, 2010
  13. 13. Monday, May 17, 2010
  14. 14. dev Monday, May 17, 2010
  15. 15. dev Monday, May 17, 2010
  16. 16. dev Monday, May 17, 2010
  17. 17. dev Monday, May 17, 2010
  18. 18. dev Monday, May 17, 2010
  19. 19. dev Monday, May 17, 2010
  20. 20. dev Monday, May 17, 2010
  21. 21. dev Monday, May 17, 2010
  22. 22. dev Monday, May 17, 2010
  23. 23. dev Monday, May 17, 2010
  24. 24. dev Monday, May 17, 2010
  25. 25. var map = new YMap (document.getElementById('map')); … dev var currentGeoPoint = new YGeoPoint ( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … Monday, May 17, 2010
  26. 26. var map = new YMap (document.getElementById('map')); … dev var currentGeoPoint = new YGeoPoint ( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … Monday, May 17, 2010
  27. 27. var map = new YMap (document.getElementById('map')); … dev var currentGeoPoint = new YGeoPoint ( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 Monday, May 17, 2010
  28. 28. var map = new YMap (document.getElementById('map')); … dev var currentGeoPoint = new YGeoPoint ( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c Monday, May 17, 2010
  29. 29. var map = new YMap (document.getElementById('map')); … dev var currentGeoPoint = new YGeoPoint ( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c Monday, May 17, 2010
  30. 30. var map = new YMap (document.getElementById('map')); … dev var currentGeoPoint = new YGeoPoint ( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c Monday, May 17, 2010
  31. 31. var map = new YMap (document.getElementById('map')); … dev var currentGeoPoint = new YGeoPoint ( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c Monday, May 17, 2010
  32. 32. var map = new YMap (document.getElementById('map')); … dev var currentGeoPoint = new YGeoPoint ( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c Monday, May 17, 2010
  33. 33. var map = new YMap (document.getElementById('map')); … dev var currentGeoPoint = new YGeoPoint ( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); ? … … if (flickcurl_prepare(fc, ? "flickr.photos.geo.correctLocation", parameters, count)) { … } … ? http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 ? http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c Monday, May 17, 2010
  34. 34. var map = new YMap (document.getElementById('map')); … dev var currentGeoPoint = new YGeoPoint ( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); ? … … if (flickcurl_prepare(fc, ? "flickr.photos.geo.correctLocation", parameters, count)) { … } … ? http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 ? http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c Monday, May 17, 2010
  35. 35. Yahoo! My Yahoo! Mail Welcome, guest Sign In Search MY PROJECTS APIs & WEB SERVICES var map = new YMap RESOURCES SUPPORT YDN Maps Suite AJAX Maps (document.getElementById('map')); Yahoo! Maps Web Services - AJAX API Getting … READY TO GET STARTED? You aren't signed in Sign In Help Started Guide Get an App ID var currentGeoPoint = new YGeoPoint dev Home The Tour Sign Up Explore Search Learn the fundamentals of the Yahoo! Maps AJAX API and start creating your own maps. This page contains: Introduction and Prerequisites ( _c.Lat, _c.Lon ); Yahoo! Maps Cheatsheets Examples Flickr Services Yahoo! My Yahoo! Mail RECENT BLOG ARTICLES Welcome, guest Sign In API Documentation Feeds Your API Keys map.addMarker(currentGeoPoint); Display a Yahoo! Map ? Apply for a new API Key Add controls to your Yahoo! Map Leonardo da Vinci: Hacker Search Add a logger to a Yahoo! Map Place a marker on a Yahoo! Map … MY PROJECTS January 24, 2009 APIs & WEB SERVICES RESOURCES SUPPORT Draw Polylines on a Yahoo! Map Whaddyathink? & the Yahoo! Application flickr.places.getPlaceTypes Place Smart Markers on a Yahoo! Map Add multiple custom markers on Yahoo! Map YDN Weather RSS Feed Platform January 22, 2009 Traffic information overlay Yahoo! Pipes RSS output information overlay Yahoo! Weather RSS Feed Making the new Yahoo! Currency Converter Yahoo! My Yahoo! Mail Welcome, guest Sign In Local search results overlay Fetches a list of available place types for Flickr. Yahoo! Pipes JSON output information overlay YMap accessible The Weather RSS feed enables you Januaryto get up-to-date weather information for your location. You can save this 21, 2009 Overlay GeoRSS XML data Overlay custom objects at YGeoPoint location … feed in My Yahoo! or your favorite feed aggregator, or incorporate the RSS data into your own web site or client Search application. The Weather RSSRetrievingdynamically-generatedfrom based on zip code or Location ID. feed is a and displaying data feed Authentication Overlay custom objects at YCoordPoint location Overlay polyline on Yahoo! Map if (flickcurl_prepare(fc, Wikipedia with YQL PROJECTS MY APIs & WEB SERVICES RESOURCES This page describes the format of the 18, 2009 URL and the RSS response for developers. It contains these January request SUPPORT ? Export Yahoo! Map in GeoRSS format YDN Image Search Documentation for Yahoo! Search Web Services This method does not require authentication. GeoRSS overlay with polyline "flickr.photos.geo.correctLocation", sections: Get zoom level given radius from center Custom position of controls Arguments RSS Request Image Search Documentation for Yahoo! Search READY TO GET STARTED? Change Marker Images Dynamically Use YMapTypeControl to position map types parameters, count)) { … } Summary Request Parameters By applying for an Application ID for this service, you hereby agree to the Terms of Use Submitting Image Queries Rate Limit api_key (Required) Other Features of the Yahoo! Maps AJAX API … RSS Response YEvent, YGeoRSS, YMarker Your API application key. See here for moreTop-Level Elements details. The Image Search Web Service allows you to search the Internet for images. Get an App ID Terms of Service Channel Elements Support & Community Image Elements ? Example Response Item Elements Download all of the cheatsheets and example code in the Yahoo Request URL Maps API Reference Bundle. Condition Codes YAHOO! GROUPS DISCUSSIONS view all <place_types> Examples http://search.yahooapis.com/ImageSearchService/V1/imageSearch For detailed descriptions of the classes and methods in the API, place_type_id="22">neighbourhood</place_type> <place_type yahoo local search code in php Terms of Use Tue, 27 Jan 2009 check out v3.8 Reference Manual. <place_type place_type_id="7">locality</place_type> <place_type <place_type http://search.yahooapis.com/ place_type_id="9">county</place_type> RSS Request place_type_id="8">region</place_type> Remaining, Supporting Classes Request parameters My Web Search API shut down See information on constructing REST queries announcement Introduction and <place_type Prerequisites <place_type ImageSearchService/V1/imageSearch? place_type_id="12">country</place_type> The Weather RSS feed request follows simple HTTP GET syntax: start with a base URL and then add place_type_id="29">continent</place_type> (?). Multiple parameters are separated by an ampersand (&). parameters and values after a question mark Wed, 21 Jan 2009 Parameter Value Description </place_types> appid=YahooDemo&query=Corvette&results=2 The Yahoo! AJAX Maps API lets developers add maps to their web sites using DHTML and JavaScript®. the Weather RSS feed is The base URL for Maps appid string (required) The application ID. See Application IDs for more information. Re: Site Explorer API returns 999 Rate Limit Exceeded are fully embeddable and scriptable using the JavaScript programming language. Yahoo! Maps AS-Flash API's query string (required) The query to search for. Use + to include terms, - to exclude terms, and ? Wed, 21 Jan 2009 Error Codes built-in geocoder means that you can specify a physical address or latitude/longitude coordinates for your map's http://weather.yahooapis.com/forecastrss put quotes around "exact phrase". location, as you like. type all (default), any, or The kind of search to submit: Reminder: Old obsolete versions of Local In this section, Summary shows how to make an RSS request with different parameters; Request Parameters phrase Search (V1, V2) shutting do In order to create and test applications 100: Yahoo! Maps AJAX API, you'll need to use a supported summarizing those parameters. using Invalid API Key shows a table web all returns results with all query terms. Mon, 19 Jan 2009 The API key passed was not valid or has expired. browser: Firefox 2, Internet Explorer 6 or 7, Opera 9 or Safari 3. Newer versions of these browsers are also any returns results with one or more of the query terms. supported. You can code your JavaScript applications usingcurrently unavailable 105: Service your favorite text editor or IDE. phrase returns results containing the query terms as a phrase. SUMMARY results integer: default 10, The number of results to return. The requested service is temporarily unavailable. An application ID is required to use the Yahoo! Maps AJAX API. You can get an application ID here. max 50 The Weather RSS feed request follows simple HTTP GET syntax: start with a base URL and then add 111: Format "xxx" not found start integer: default 1 parameters and values after a question mark (?). Multiple parameters are separated by an starting result position to return (1-based). The finishing position (start The ampersand (&). For For best results, you should launch your web pagesrequested response format was not found. applications The from a web server. However, it's possible to run + results - 1) cannot exceed 1000. the Weather RSS feed there are two parameters: directly from your hard drive by double-clickingMethod This can be donefound 112: on them. "xxx" not by lowering the security settings in format any (default), bmp, Specifies the kind of image file to search for. the Internet Explorer browser or by setting the The requested method was not found. p for location. UniversalBrowserRead property in Firefox. gif, jpeg, png u for degrees units (Fahrenheit or Celsius). adult_ok 114: Invalid SOAP envelope http://weather.yahooapis.com/ no value or 1 content. The service filters out adult content by default. Enter a 1 to allow adult The SOAP envelope send in the requestlocationnot be parsed. required. Use this parameter to indicate the location for the weather forecast as a The could parameter p is Displaying a Yahoo! Map zip code or Location ID. coloration any (default), color, The service returns only the images of the coloration specified (color or 115: Invalid XML-RPC Method Call forecastrss?p=FRXX0076&u=c All the API classes and methods necessary to interact with Yahoo! Maps are available to your application when The XML-RPC request document could not be parsed. http://weather.yahooapis.com/forecastrss?p=location bw black-and-white). site string: default no A domain to restrict your searches to (e.g. www.yahoo.com). You may you include Yahoo! Maps AJAX API library in your web page.This example uses the demonstration application value submit up to 30 values (site=www.yahoo.com&site=www.cnn.com). ID "YD-eQRpTl0_JX2E95l_xAFs5UwZUlNQhhn7lj1H"; you cannot use this Application Forand must requestweather for Yahoo! Headquarters in Sunnyvale, CA, use the zip ID example, to get one code for Sunnyvale here. API Explorer (94089): output string: xml The format for the output. If json is requested, the results will be returned (default), json, php in JSON format. If php is requested, the results will be returned in Serialized PHP format. . <html> API Explorer : flickr.places.getPlaceTypes http://weather.yahooapis.com/forecastrss?p=94089 . <head> callback string The name of the callback function to wrap around the JSON data. The . <b><script type="text/javascript" Or use the Location ID for Sunnyvale (USCA1116): following characters are allowed: A-Z a-z 0-9 . [] and _. If output=json has . src="http://api.maps.yahoo.com/ajaxymap?v=3.8&appid=YD- not been requested, this parameter has no effect. More information on the eQRpTl0_JX2E95l_xAFs5UwZUlNQhhn7lj1H"> http://weather.yahooapis.com/forecastrss?p=USCA1116 callback can be found in the Yahoo! Developer Network JSON Monday, May 17, 2010 . </script></b> Documentation.
  36. 36. var map = new YMap (document.getElementById('map')); … dev var currentGeoPoint = new YGeoPoint ( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c Monday, May 17, 2010
  37. 37. dev YQL Monday, May 17, 2010
  38. 38. dev YQL Monday, May 17, 2010
  39. 39. dev Bindings YQL Monday, May 17, 2010
  40. 40. dev Bindings YQL Monday, May 17, 2010
  41. 41. dev Bindings YQL stmt YQL Monday, May 17, 2010
  42. 42. dev Bindings YQL stmt YQL select Monday, May 17, 2010
  43. 43. dev Bindings YQL stmt YQL select desc Monday, May 17, 2010
  44. 44. dev Bindings YQL stmt YQL select desc show Monday, May 17, 2010
  45. 45. Why YQL? Thousands of Web Services and sources that provide valuable data Require Developers to read documentation and form URLs/queries. Data is isolated Needs filtering, combining, tweaking, shaping even after it gets to the developer. Monday, May 17, 2010
  46. 46. YQL Hosted web service and SQL-Like Language Familiar to developers Synonymous with Data access Expressive enough to get the right data. Self describing - show, desc table Allows you to query, filter and join data across any structured data on the web / web services Real time engine Monday, May 17, 2010
  47. 47. YQL Statement Basics show: lists the supported tables desc: describes the structure of a table select: fetches data insert/update/delete: modify data use: use an Open Data Table show tables desc social.contacts Monday, May 17, 2010
  48. 48. YQL Select Statement SELECT what FROM collection WHERE filter condition IN (sub-select) LIMIT n OFFSET n Monday, May 17, 2010
  49. 49. Demo: developer.yahoo.com/yql/ Monday, May 17, 2010
  50. 50. Executing YQL with PHP Monday, May 17, 2010
  51. 51. Download PHP5 SDK from GitHub http://github.com/yahoo Monday, May 17, 2010
  52. 52. $query = new YQLQuery(); $yql = “SELECT * from search.web where query = ‘phpday.it’“; $data = $query->execute($yql); Monday, May 17, 2010
  53. 53. Get any data: open data tables Monday, May 17, 2010
  54. 54. Open Data Tables YQL Monday, May 17, 2010
  55. 55. Open Data Tables YQL Monday, May 17, 2010
  56. 56. Open Data Tables YQL Monday, May 17, 2010
  57. 57. Open Data Tables YQL Monday, May 17, 2010
  58. 58. Open Data Tables Bindings YQL Monday, May 17, 2010
  59. 59. Open Data Tables YQL Monday, May 17, 2010
  60. 60. Open Data Tables YQL Monday, May 17, 2010
  61. 61. Open Data Tables Map YQL to Endpoint YQL Monday, May 17, 2010
  62. 62. Open Data Tables Brings the power of the table model to new APIs Schema defines mapping between YQL and Endpoint Keys can either be query, path or matrix parameters Keys can be marked as required YQL Compiler validates existence of required keys Monday, May 17, 2010
  63. 63. Open Data Tables YQL Monday, May 17, 2010
  64. 64. Open Data Tables YQL Monday, May 17, 2010
  65. 65. Open Data Tables Monday, May 17, 2010
  66. 66. Open Data Tables Monday, May 17, 2010
  67. 67. USE Verb use ‘http://url-to-def.xml’ as tableName; select * from tableName where key=‘value’... Imports the Table definition Makes it available to following statements as tableName desc tableName - describes the def Acts like a First Class Data Source just like other YQL Tables. Monday, May 17, 2010
  68. 68. Basics select * from {table} where {key1} = "{value1}" and {key2} = {value2} select * from flickr.photos.search where text = "kittens" and tags = "domokun" Monday, May 17, 2010
  69. 69. Remote vs. Local Monday, May 17, 2010
  70. 70. select * from flickr.photos.search where text = "kittens" and tags = "domokun" Remote Monday, May 17, 2010
  71. 71. select * from flickr.photos.search where text = "kittens" and tags = "domokun" Remote Monday, May 17, 2010
  72. 72. select * from flickr.photos.search where text = "kittens" and tags = "domokun" Remote Monday, May 17, 2010
  73. 73. desc {table} Monday, May 17, 2010
  74. 74. Local Monday, May 17, 2010
  75. 75. select * from flickr.photos.search where text = "kittens" and tags = "domokun" and farm = 4 Monday, May 17, 2010
  76. 76. Limits and Paging Monday, May 17, 2010
  77. 77. select * from flickr.photos.search where text = "kittens" and tags = "domokun" and farm = 4 limit 10 Monday, May 17, 2010
  78. 78. select * from flickr.photos.search(10) where text = "kittens" and tags = "domokun" and farm = 4 Monday, May 17, 2010
  79. 79. select * from flickr.photos.search(20) where text = "kittens" and tags = "domokun" and farm = 4 Monday, May 17, 2010
  80. 80. select * from flickr.photos.search(0) where text = "kittens" and tags = "domokun" and farm = 4 limit 10 Monday, May 17, 2010
  81. 81. Limit Summary limit 10 - give me max 10 results from the data you have {table}(10) - get ten results from server {table}(0) - get enough results from the server to fill my local limit with the local filters Monday, May 17, 2010
  82. 82. Offset Monday, May 17, 2010
  83. 83. select * from {table} (10) where foo = "bar" offset 10 vs. select * from {table} (10,10) where foo = "bar" Monday, May 17, 2010
  84. 84. select * from {table} (10) where foo = "bar" offset 10 <results> <result foo=”bar”>1</result> <results> <result foo=”qux”>2</result> <result foo=”bar”>1</result> ... ... <results> </results> <result foo=”qux”>8</result> <result foo=”bar”>9</result> </query> <result foo=”bar”>9</result> <result foo=”bar”>10</result> <result foo=”bar”>10</result> </results> </results> </query> </query> Monday, May 17, 2010
  85. 85. select * from {table} (10,10) where foo = "bar" <results> <result foo=”bar”>11</result> <results> <result foo=”qux”>12</result> <result foo=”bar”>11</result> ... ... <result foo=”qux”>18</result> <result foo=”bar”>20</result> <result foo=”qux”>19</result> </results> <result foo=”bar”>20</result> </query> </results> </query> Monday, May 17, 2010
  86. 86. Inner Queries Monday, May 17, 2010
  87. 87. select * from {table1} where {key1} in (select id from {table2}); Monday, May 17, 2010
  88. 88. select * from flickr.photos.info where photo_id in (select id from flickr.photos.search where text="kitten") Monday, May 17, 2010
  89. 89. Multi-key Monday, May 17, 2010
  90. 90. select * from {table1} where ({key1}, {key2}) in (select foo, bar from {table2}); Monday, May 17, 2010
  91. 91. select * from flickr.places where (lat, lon) in (select match1, match2 from regex where text in (select entry.point from usgs.earthquakes limit 10) and expression = "(-?d+.d+) (-?d+. d+)") and total != 0 limit 10 Monday, May 17, 2010
  92. 92. Filters Monday, May 17, 2010
  93. 93. select * from {table}(20) | sort(field=”foo”, descending = “true”) | truncate 10 Monday, May 17, 2010
  94. 94. Filters are always applied last, in order Monday, May 17, 2010
  95. 95. select * from {table}(20) | sort(field=”foo”, descending = “true”) | truncate 10 <results> <results> <results> <result foo=”bar”>1</result> <result foo=”bar”>20</result> <result foo=”bar”>20</result> <result foo=”qux”>2</result> <result foo=”qux”>19</result> <result foo=”qux”>19</result> ... ... ... <result foo=”qux”>18</result> <result foo=”qux”>3</result> <result foo=”qux”>13</result> <result foo=”bar”>19</result> <result foo=”bar”>2</result> <result foo=”bar”>12</result> <result foo=”bar”>20</result> <result foo=”bar”>1</result> <result foo=”bar”>11</result> </results> </results> </results> </query> </query> </query> Monday, May 17, 2010
  96. 96. Super-secret Filters Xpath() Regex() ... there are more I can’t remember Monday, May 17, 2010
  97. 97. Community: github/datatables.org Monday, May 17, 2010
  98. 98. Share your work leverage the work of others to make access easier by sharing tables github repository: http://github.com/yql/yql-tables/tree/master datatables.org mirrors the master and builds a single environment with all tables -> faster serving http://developer.yahoo.com/yql/console/?env=http:// datatables.org/alltables.env Monday, May 17, 2010
  99. 99. Do it your way: <execute> Monday, May 17, 2010
  100. 100. YQL Execute extends Open Data Tables with server-side code access APIs that require authentication Netflix OAuth, FlickrAuth, Google AuthSub join data across services grab New York Times article tags and generate find associated flickr photos combine multiple searches into a single result: twitter, web, news and image Monday, May 17, 2010
  101. 101. YQL Execute augment data: city/state to APIs that just return zip code create APIs from web page celebrity birthdays scraped from imdb, caltrain data transformation convert the result from xml to Google's kml format move business logic of your application to the cloud Monday, May 17, 2010
  102. 102. <execute> Javascript server-side scripting No DOM, events etc, pure JS scripting E4X support - XML literals, filtering ... YQL no longer performs the GET request to the table binding URL Javascript has additional useful global objects y.rest, y.include, y.query... execute tag must return the data that will be the output of the YQL select Monday, May 17, 2010
  103. 103. hello world <?xml version="1.0" encoding="UTF-8"?> <table xmlns="http://query.yahooapis.com/v1/schema/table.xsd"> <meta> <sampleQuery>select * from {table} where a='cat' and b='dog';</sampleQuery> </meta> <bindings> <select itemPath="" produces="XML"> <urls> <url>http://fake.url/{a}</url> </urls> <inputs> <key id='a' type='xs:string' paramType='path' required="true" /> <key id='b' type='xs:string' paramType='variable' required="true" /> </inputs> <execute><![CDATA[ // Your javascript goes here. We will run it on our servers response.object = <item> <url>{request.url}</url> <a>{a}</a> <b>{b}</b> </item>; ]]></execute> </select> </bindings> </table> Monday, May 17, 2010
  104. 104. Unified web+image <?xml version="1.0" encoding="UTF-8"?> <table xmlns="http://query.yahooapis.com/v1/schema/table.xsd"> <meta> <author>Nagesh Susarla</author> <documentationURL>See search.web and search.images for more details</documentationURL> </meta> <bindings> <select itemPath="results.result" produces="XML"> <urls> <url></url> </urls> <inputs> <key id="query" type="xs:string" paramType="query" required="true"/> </inputs> <execute><![CDATA[ var qs = query; var search = y.query('select * from search.web(50) where query=@query', {query: qs}).results; var images = []; default xml namespace='http://www.inktomi.com/'; for each (var result in search.result) { images.push(y.query('select * from search.images(1) where query=@query and url=@url', {url:result.url, query:qs})); } var i = 0; for each (var result in search.result) { var image = images[i++].results.result; if (image) { result.image = <image>{image}</image>; } } response.object = search; ]]> </execute> </select> </bindings> </table> Monday, May 17, 2010
  105. 105. Search Netflix catalog <table xmlns="http://query.yahooapis.com/v1/schema/table.xsd" https="true"> <meta>...</meta> <bindings> <select itemPath="" produces="XML" > <urls><url env="all">http://api.netflix.com/catalog/titles/</url></urls> <paging model="offset"> <start id="start_index" default="0" /> <pagesize id="max_results" max="100" /> <total default="10" /> </paging> <inputs> <key id="term" type="xs:string" paramType="query" required="true" /> <key id="ck" type="xs:string" paramType="variable" required="true" /> <key id="cks" type="xs:string" paramType="variable" required="true" /> </inputs> <execute><![CDATA[ // Include the OAuth libraries from oauth.net y.include("http://oauth.googlecode.com/svn/code/javascript/oauth.js"); y.include("http://oauth.googlecode.com/svn/code/javascript/sha1.js"); // Collect all the parameters var encodedurl = request.url; var accessor = { consumerSecret: cks, tokenSecret: ""}; var message = { action: encodedurl, method: "GET", parameters: [["oauth_consumer_key",ck],["oauth_version","1.0"]]}; OAuth.setTimestampAndNonce(message); // Sign the request OAuth.SignatureMethod.sign(message, accessor); try { // get the content from service along with the OAuth header, and return the result back out response.object = request.contentType('application/xml').header("Authorization", OAuth.getAuthorizationHeader("netflix.com", message.parameters)).get().response; } catch(err) { response.object = {'result':'failure', 'error': err}; } ]]></execute> </select> </bindings> </table> Monday, May 17, 2010
  106. 106. INSERT INTO internet Monday, May 17, 2010
  107. 107. Insert / Update / Delete With INSERT / UPDATE / DELETE SQL verbs developers can modify data on the internet, e.g. create tweets, add blog entries, update DBs etc Open Data Tables specify what fields are needed and how to bind the SQL to the remote web service Some tables with I/U/D Twitter Status, Bit.ly shortener Yahoo profile status, application update stream Amazon SimpleDB Monday, May 17, 2010
  108. 108. select * from twitter.status where id="2108869549" and username="xxx" and password="yyy" <select itemPath="" produces="XML"> <urls><url>http://twitter.com/statuses/show/{id}.xml</url></urls> <inputs> <key id="username" type="xs:string" required="false" paramType="variable"/> <key id="password" type="xs:string" required="false" paramType="variable"/> <key id="id" type="xs:integer" required="true" paramType="path"/> </inputs> <execute><![CDATA[ var r = null; if (username && password) { y.include("http://yqlblog.net/samples/base64.js"); var authheader = "Basic "+Base64.encode(username+":"+password); r = request.header("Authorization",authheader).get().response; } else { r = request.get().response; } response.object = r; ]]></execute> </select> Monday, May 17, 2010
  109. 109. insert into twitter.status (status,username,password) values ("new tweet from YQL",”xxx”,”yyy”); <insert itemPath="" produces="XML"> <urls> <url>http://twitter.com/statuses/update.xml</url> </urls> <inputs> <value id="username" type="xs:string" required="true" paramType="variable"/> <value id="password" type="xs:string" required="true" paramType="variable"/> <value id="status" type="xs:string" required="true" paramType="variable"/> </inputs> <execute><![CDATA[ y.include("http://yqlblog.net/samples/base64.js"); var authheader = "Basic "+Base64.encode(username+":"+password); var content = "status="+status; response.object = request.header("Authorization",authheader).post(content).response; ]]></execute> </insert> Monday, May 17, 2010
  110. 110. delete from twitter.status where id="2108869549" and username="xxx" and password="yyy" <delete itemPath="" produces="XML"> <urls> <url>http://twitter.com/statuses/destroy/{id}.xml</url> </urls> <inputs> <key id="username" type="xs:string" required="true" paramType="variable"/> <key id="password" type="xs:string" required="true" paramType="variable"/> <key id="id" type="xs:string" required="true" paramType="path"/> </inputs> <execute><![CDATA[ y.include("http://yqlblog.net/samples/base64.js"); var authheader = "Basic "+Base64.encode(username+":"+password); response.object = request.header("Authorization",authheader).del().response; ]]></execute> </delete> Monday, May 17, 2010
  111. 111. Conclusion More easily build applications fewer lines of code data processing done away from app consistent and familiar syntax for all data access iterative environment for developing the query Monday, May 17, 2010
  112. 112. Conclusion build applications that run faster service in the cloud does the work conversion, filtering, parallel requests... with smaller network footprint fewer (client) calls smaller data amounts closer to the data, fatter pipes Monday, May 17, 2010
  113. 113. data Monday, May 17, 2010
  114. 114. YQL & PHP SELECT * FROM INTERNET http://developer.yahoo.com/yql Dustin Whittle dustin@yahoo-inc.com | @dustinwhittle Monday, May 17, 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×