YQL

YQL A Query Language for the Web Sam Pullara (samp@yahoo-inc.com) Chief Technologist @ Yahoo Wednesday, March 11, 2009 1

Application ? Wednesday, March 11, 2009 2

Application = Wednesday, March 11, 2009 3

data manipulation Application = Wednesday, March 11, 2009 3

data manipulation Application = + data visualization Wednesday, March 11, 2009 3

data data Wednesday, March 11, 2009 4

data Wednesday, March 11, 2009 4

Wednesday, March 11, 2009 6

dev Wednesday, March 11, 2009 6

dev Wednesday, March 11, 2009 7

var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new dev YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … Wednesday, March 11, 2009 7

var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new dev YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, \"flickr.photos.geo.correctLocation\", parameters, count)) { … } … Wednesday, March 11, 2009 7

var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new dev YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, \"flickr.photos.geo.correctLocation\", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 Wednesday, March 11, 2009 7

var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new dev YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, \"flickr.photos.geo.correctLocation\", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c Wednesday, March 11, 2009 7

var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new dev YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); ? … … if (flickcurl_prepare(fc, ? \"flickr.photos.geo.correctLocation\", parameters, count)) { … } … ? http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 ? http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c Wednesday, March 11, 2009 7

var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new dev YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); ? … … if (flickcurl_prepare(fc, ? \"flickr.photos.geo.correctLocation\", parameters, count)) { … } … ? http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 ? http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c Wednesday, March 11, 2009 8

Yahoo! My Yahoo! Mail Welcome, guest Sign In Search var map = new MY PROJECTS APIs & WEB SERVICES RESOURCES SUPPORT YMap(document.getElementById('map')); YDN Maps Suite AJAX Maps Yahoo! Maps Web Services - AJAX API Getting READY TO GET STARTED? … You aren't signed in Sign In Help Started Guide Get an App ID var currentGeoPoint = new dev Home The Tour Sign Up Explore Search Learn the fundamentals of the Yahoo! Maps AJAX API and start creating your own maps. This page contains: YGeoPoint( _c.Lat, _c.Lon ); Introduction and Prerequisites Yahoo! Maps Cheatsheets Flickr Services RECENT BLOG ARTICLES Yahoo! My Yahoo! Mail Examples Welcome, guest Sign In API Documentation Feeds Your API Keys map.addMarker(currentGeoPoint); Display a Yahoo! Map ? Apply for a new API Key Search Add controls to your Yahoo! Map Leonardo da Vinci: Hacker … January 24, 2009 Add a logger to a Yahoo! Map Place a marker on a Yahoo! Map MY PROJECTS APIs & WEB SERVICES RESOURCES SUPPORT Draw Polylines on a Yahoo! Map Whaddyathink? & the Yahoo! Application flickr.places.getPlaceTypes Place Smart Markers on a Yahoo! Map Platform YDN Weather RSS Feed Add multiple custom markers on Yahoo! Map January 22, 2009 Traffic information overlay Yahoo! Weather RSS Feed Yahoo! Pipes RSS output information overlay Making the new Yahoo! Currency Converter Yahoo! My Yahoo! Mail Welcome, guest Sign In Fetches a list of available place types for Flickr. Yahoo! Pipes JSON output information overlay accessible YMap Local search results overlay The Weather RSS feed enables you Januaryto get up-to-date weather information for your location. You can save this 21, 2009 … Search Overlay GeoRSS XML data feed in My Yahoo! or your favorite feed aggregator, or incorporate the RSS data into your own web site or client Overlay custom objects at YGeoPoint location application. The Weather RSSRetrievingdynamically-generatedfrom based on zip code or Location ID. feed is a and displaying data feed Authentication Overlay custom objects at YCoordPoint location if (flickcurl_prepare(fc, Wikipedia with YQL PROJECTS MY APIs & WEB SERVICES RESOURCES SUPPORT Overlay polyline on Yahoo! Map This page describes the format of the 18, 2009 URL and the RSS response for developers. It contains these January request ? Export Yahoo! Map in GeoRSS format YDN Image Search Documentation for Yahoo! Search Web Services This method does not require authentication. \"flickr.photos.geo.correctLocation\", sections: GeoRSS overlay with polyline Image Search Documentation for Yahoo! Search Get zoom level given radius from center READY TO GET STARTED? RSS Request Arguments Custom position of controls parameters, count)) { … } By applying for an Application ID for this service, you Summary Change Marker Images Dynamically hereby agree to the Terms of Use Request Parameters Use YMapTypeControl to position map types Submitting Image Queries … api_key (Required) Other Features of the Yahoo! Maps AJAX API RSS Response YEvent, YGeoRSS, YMarker Get an App ID Rate Limit Your API application key. See here for moreTop-Level Elements details. The Image Search Web Service allows you to search the Internet for images. Terms of Service Channel Elements Support & Community Image Elements ? Example Response Item Elements Request URL Download all of the cheatsheets and example code in the Yahoo Condition Codes view all YAHOO! GROUPS DISCUSSIONS Maps API Reference Bundle. http://search.yahooapis.com/ImageSearchService/V1/imageSearch <place_types> Examples yahoo local search code in php For detailed descriptions of the classes and methods in the API, place_type_id=\"22\">neighbourhood</place_type> <place_type Terms of Use Tue, 27 Jan 2009 check out v3.8 Reference Manual. <place_type place_type_id=\"7\">locality</place_type> http://search.yahooapis.com/ Request parameters <place_type place_type_id=\"9\">county</place_type> RSS Request My Web Search API shut down <place_type place_type_id=\"8\">region</place_type> Remaining, Supporting Classes announcement See information on constructing REST queries ImageSearchService/V1/imageSearch? <place_type place_type_id=\"12\">country</place_type> The Weather RSS feed request follows simple HTTP GET syntax: start with a base URL and then add Wed, 21 Jan 2009 Introduction and Prerequisites <place_type place_type_id=\"29\">continent</place_type> (?). Multiple parameters are separated by an ampersand (&). parameters and values after a question mark Parameter Value Description </place_types> appid=YahooDemo&query=Corvette&results=2 Re: Site Explorer API returns 999 Rate Limit appid string (required) The application ID. See Application IDs for more information. The Yahoo! AJAX Maps API lets developers add maps to their web sites using DHTML and JavaScript®. the Weather RSS feed is The base URL for Maps Exceeded are fully embeddable and scriptable using the JavaScript programming language. Yahoo! Maps AS-Flash API's query string (required) The query to search for. Use + to include terms, - to exclude terms, and ? Wed, 21 Jan 2009 Error Codes put quotes around \"exact phrase\". built-in geocoder means that you can specify a physical address or latitude/longitude coordinates for your map's http://weather.yahooapis.com/forecastrss location, as you like. type all (default), any, or The kind of search to submit: Reminder: Old obsolete versions of Local In this section, Summary shows how to make an RSS request with different parameters; Request Parameters phrase Search (V1, V2) shutting do In order to create and test applications 100: Yahoo! Maps AJAX API, you'll need to use a supported summarizing those parameters. using Invalid API Key shows a table web Mon, 19 Jan 2009 all returns results with all query terms. The API key passed was not valid or has expired. any returns results with one or more of the query terms. browser: Firefox 2, Internet Explorer 6 or 7, Opera 9 or Safari 3. Newer versions of these browsers are also phrase returns results containing the query terms as a phrase. supported. You can code your JavaScript applications usingcurrently unavailable 105: Service your favorite text editor or IDE. SUMMARY results integer: default 10, The number of results to return. The requested service is temporarily unavailable. max 50 An application ID is required to use the Yahoo! Maps AJAX API. You can get an application ID here. The Weather RSS feed request follows simple HTTP GET syntax: start with a base URL and then add 111: Format \"xxx\" not found start integer: default 1 by an starting result position to return (1-based). The finishing position (start The ampersand (&). For parameters and values after a question mark (?). Multiple parameters are separated For best results, you should launch your web pagesrequested response format was not found. applications The from a web server. However, it's possible to run + results - 1) cannot exceed 1000. the Weather RSS feed there are two parameters: directly from your hard drive by double-clickingMethod This can be donefound 112: on them. \"xxx\" not by lowering the security settings in format any (default), bmp, Specifies the kind of image file to search for. the Internet Explorer browser or by setting the The requested method was not found. p for location. UniversalBrowserRead property in Firefox. gif, jpeg, png u for degrees units (Fahrenheit or Celsius). adult_ok http://weather.yahooapis.com/ no value or 1 The service filters out adult content by default. Enter a 1 to allow adult 114: Invalid SOAP envelope content. The SOAP envelope send in the requestlocationnot be parsed. required. Use this parameter to indicate the location for the weather forecast as a The could parameter p is Displaying a Yahoo! Map coloration any (default), color, The service returns only the images of the coloration specified (color or zip code or Location ID. forecastrss?p=FRXX0076&u=c 115: Invalid XML-RPC Method Call bw black-and-white). All the API classes and methods necessary to interact with Yahoo! Mapsdocument couldyour application when The XML-RPC request are availablehttp://weather.yahooapis.com/forecastrss?p=location to not be parsed. site string: default no A domain to restrict your searches to (e.g. www.yahoo.com). You may you include Yahoo! Maps AJAX API library in your web page.This example uses the demonstration application value submit up to 30 values (site=www.yahoo.com&site=www.cnn.com). ID \"YD-eQRpTl0_JX2E95l_xAFs5UwZUlNQhhn7lj1H\"; you cannot use this Application Forand must requestweather for Yahoo! Headquarters in Sunnyvale, CA, use the zip ID example, to get one code for Sunnyvale API Explorer output string: xml The format for the output. If json is requested, the results will be returned here. (94089): (default), json, php in JSON format. If php is requested, the results will be returned in Serialized PHP format. API Explorer : flickr.places.getPlaceTypes <html> http://weather.yahooapis.com/forecastrss?p=94089 callback string The name of the callback function to wrap around the JSON data. The <head> following characters are allowed: A-Z a-z 0-9 . [] and _. If output=json has Or use the Location ID for Sunnyvale (USCA1116): <b><script type=\"text/javascript\" not been requested, this parameter has no effect. More information on the src=\"http://api.maps.yahoo.com/ajaxymap?v=3.8&appid=YD- callback can be found in the Yahoo! Developer Network JSON eQRpTl0_JX2E95l_xAFs5UwZUlNQhhn7lj1H\"> http://weather.yahooapis.com/forecastrss?p=USCA1116 Documentation Wednesday, March 11, 2009 8 </script </b

YQL select * from internet Wednesday, March 11, 2009 9

var map = new YMap(document.getElementById('map')); … var currentGeoPoint = new dev YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint); … … if (flickcurl_prepare(fc, \"flickr.photos.geo.correctLocation\", parameters, count)) { … } … http://search.yahooapis.com/ ImageSearchService/V1/imageSearch? appid=YahooDemo&query=Corvette&results=2 http://weather.yahooapis.com/ forecastrss?p=FRXX0076&u=c Wednesday, March 11, 2009 10

dev YQL Wednesday, March 11, 2009 10

dev Bindings YQL Wednesday, March 11, 2009 10

dev Bindings YQL YQL stmt Wednesday, March 11, 2009 10

dev Bindings YQL YQL stmt select Wednesday, March 11, 2009 10

dev Bindings YQL YQL stmt select desc Wednesday, March 11, 2009 10

dev Bindings YQL YQL stmt select desc show Wednesday, March 11, 2009 10

Why YQL? Thousands of Web Services and sources that provide valuable data Require Developers to read documentation and form URLs/queries. Data is isolated Needs combining, tweaking, shaping even after it gets to the developer. Wednesday, March 11, 2009 11

YQL SQL-Like Language Synonymous with Data access Familiar to developers Expressive enough to get the right data. Self Describing - show, desc table Allows you to query, ﬁlter and join data across WebServices. Wednesday, March 11, 2009 12

YQL Statement Basics Verbs show: lists the supported tables desc: describes the structure of a table select: fetches data use: use an Open Data Table *new* show tables desc social.contacts Wednesday, March 11, 2009 13

YQL Select Statement Syntax SELECT what FROM collection WHERE filter condition IN (sub-select) LIMIT n OFFSET n Wednesday, March 11, 2009 14

Demo: developer.yahoo.com/yql/ Wednesday, March 11, 2009 15

YQL Statement Basics Syntax -> Select -> Collection FROM collection Yahoo! Web Services Social Dir, Upcoming, MyblogLog, Y!Local, Search(BOSS) External Data Sources XML, Atom, RSS, JSON, CSV, HTML Wednesday, March 11, 2009 16

YQL Statement Basics Syntax -> Select -> WHERE clause Table data can be ﬁltered in the WHERE clause either Remotely by the table data source Locally by the YQL engine (dot notation) Wednesday, March 11, 2009 17

YQL Statement Basics Syntax -> WHERE clause -> Remote Filter WHERE <remote_key>=‘value’ Tables require Keys in where clause Speciﬁed in the describe <query> <key name=”guid” required=”true” .../> </query> Correspond to query, path, header or matrix parameters for the Endpoint Wednesday, March 11, 2009 18

YQL Statement Basics Syntax -> WHERE clause -> Local Filter WHERE <local_filter> OP ‘value’ Filter an element or an attribute <movies> Dot notation: <movie>Star Wars</movie> movies.movie LIKE ‘%Star%’ <movie>StarTrek</movie> </movies> <, >, =, LIKE, matches, not matches, not like AND, OR binary operator combinations Wednesday, March 11, 2009 19

YQL Statement Basics Syntax -> Select -> Projection SELECT <what> Uses the dot notation to project parts of matching item(s) <results> <movie> <id>101</id> Projection: (relative to root) <name>Star Wars</name> select movie.id, movie.name </movie> from ... <movie>...</movie> </results> Wednesday, March 11, 2009 20

YQL Statement Basics Syntax -> Select -> Sub-Select IN (SELECT ...) Join across data sources field IN (select guid from ...) Like SELECT but can return only N leaves Get the proﬁle for all my connections: select * from social.profile where guid in (select guid from social.connections where owner_guid = me) Wednesday, March 11, 2009 21

YQL Statement Basics Syntax -> Select -> Sub-Select IN (SELECT ...) (field, ...fieldN) IN (SELECT f1,...fn from table) Get the Location name of the given Photo: select place.name from flickr.places where (lat, lon) in (select location.latitude, location.longitude from flickr.photos.info where photo_id='3030238604') Wednesday, March 11, 2009 22

YQL Statement Basics Syntax -> Select -> Table Size Changing the Table Size Yahoo! Tables are limited to a small number by default (10) select * from local.search where zip=‘94089’ and query=‘pizza’ Increase set size as parameters to Collection select * from local.search(100) where zip=‘94089’ and query=‘pizza’ Wednesday, March 11, 2009 23

YQL Statement Basics Syntax -> Post Query Manipulation Post Select processing using ‘pipe’ symbol. select ... | sort(field=‘item.pubDate’) select ... | unique(field=‘item.title’) select ... | tail(count=10) Functions operate on return data. Operations unrelated to table or data source. Wednesday, March 11, 2009 24

YQL Statement Basics YQL Variables (@var) Like Prepared Statements for YQL Substitutes Query parameter http://<>?q=....&city=Bangalore&state=Karnataka select * from table where city=@city and state=@state Can be used in sub-selects select * from table where city in (@city, @city2, @city3) Wednesday, March 11, 2009 25

YQL Statement Basics YQL Variables (@var) Multi Valued variables http://<>?q=....&city=Bangalore&city=Mumbai select * from table where city in (@city) Wednesday, March 11, 2009 26

Sounds Good. But what about my Web Service? Wednesday, March 11, 2009 27

Can I add my data tables? Wednesday, March 11, 2009 28

Open Data Tables YQL Wednesday, March 11, 2009 29

Open Data Tables Bindings YQL Wednesday, March 11, 2009 29

Open Data Tables Map YQL to Endpoint YQL Wednesday, March 11, 2009 30

Open Data Tables Wednesday, March 11, 2009 32

Open Data Tables Wednesday, March 11, 2009 33

Open Data Tables Schema deﬁnes mapping between YQL and Endpoint Keys can either be query, path, header or matrix parameters Keys can be marked as required YQL Compiler validates existence of required keys and uses them to select an endpoint Wednesday, March 11, 2009 34

USE Verb use ‘http://url-to-def.xml’ as tableName; select * from tableName where key=‘value’... Imports the Table deﬁnition Makes it available to following statements as tableName desc tableName - describes the def Acts like a First Class Data Source just like other YQL Tables. Wednesday, March 11, 2009 35

Environment files Text files that contain multiple table “use” statements Useful for defining your own default working sets of tables No need to “use” per call Wednesday, March 11, 2009 36

Using environment ﬁles http://query.yahooapis.com/v1/yql? q=...&env=http://... http://developer.yahoo.com/v1/yql? env=http://... Add a query parameter to the web service or console call Shows up in console table list / show tables http://developer.yahoo.com/yql/console/?env=http:// github.com/spullara/yql-tables/raw/ ef685688d649a7514ebd27722366b2918d966573/alltables.env Wednesday, March 11, 2009 37

Accessing YQL OAuth Endpoint http://query.yahooapis.com/v1/yql?q=... Public Endpoint http://query.yahooapis.com/v1/public/yql?q= YQL Console http://developer.yahoo.com/yql/console Wednesday, March 11, 2009 38

Accessing YQL YQL Via Pipes Wednesday, March 11, 2009 39

YQL Console Wednesday, March 11, 2009 40

Conclusion build applications more easily fewer lines of code consistent and familiar syntax for all data access iterative environment for developing the query Wednesday, March 11, 2009 41

Conclusion build applications that run faster service in the cloud does the work with smaller network footprint fewer (client) calls smaller data amounts closer to the data, fatter pipes Wednesday, March 11, 2009 42

One API : YQL Wednesday, March 11, 2009 44

Q&A http://developer.yahoo.com/yql Follow up: http://developer.yahoo.com/ events/etech09 Wednesday, March 11, 2009 45

YQL

0 comments

Notes on slide 1

4 Favorites

YQL - Presentation Transcript

More by user

More info about this document

Categories

Tags

YQL