Yahoo Query Language:<br />select * from internet<br />Derek Gathright<br />
var map = new YMap(document.getElementById('map'));…varcurrentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon );  map.addMarker(c...
var map = new YMap(document.getElementById('map'));…varcurrentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon );  map.addMarker(c...
var map = new YMap(document.getElementById('map'));…varcurrentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon );  map.addMarker(c...
Why YQL?<br />Thousands of Web Services and sources that provide valuable data<br />Require Developers to read documentati...
What is YQL?<br />Cloud web service and SQL-Like Language <br />Familiar to developers<br />Synonymous with Data access<br...
YQL Statement Basics<br />show: lists the supported tables<br />desc: describes the structure of a table<br />select: fetc...
Insert / Update / Delete<br />With INSERT/ UPDATE/ DELETESQL verbs you can modify data on the internet or cloud<br />creat...
Demo: developer.yahoo.com/yql/<br />
Get any data:<br />Open Data Tables<br />
Open Data Tables<br /><ul><li>http://github.com/yql/yql-tables/tree/master
datatables.orgmirrors the master and builds a single environment with all tables (faster serving)
Console with community tables
Brings the power of the table model to new APIs
Schema defines mapping between YQL and Endpoint
Keys can either be query, path or matrix parameters
Keys can be marked as required
YQL Compiler validates existence of required keys</li></li></ul><li>YQL<br />Open Data Table example<br />
Open Data Table example<br />
Open Data Table example<br />
ODT <execute> element<br />Extends Open Data Tables with server-side code<br />Javascript server-side scripting<br />No DO...
What you can do with <execute><br />access APIs that require authentication<br />Netflix OAuth, FlickrAuth, Google AuthSub...
hello world<br /><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd"><br /><meta><br />    <sampleQuery>select *...
Scraping a page<br /><select produces="XML" itemPath="deals.deal"><br />      <urls><url>http://deals.yahoo.com</url></url...
Twitter Select<br />select *from twitter.statuswhere id="2108869549"andusername="xxx"and password="yyy”<br /><select itemP...
Upcoming SlideShare
Loading in...5
×

Yahoo Query Language: Select * from Internet

4,280

Published on

Here are the slides for the presentation I gave at Yahoo for a JavaScript meetup on Apr 21st, 2010.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,280
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
19
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Yahoo Query Language: Select * from Internet

  1. 1. Yahoo Query Language:<br />select * from internet<br />Derek Gathright<br />
  2. 2.
  3. 3. var map = new YMap(document.getElementById('map'));…varcurrentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint);…<br />…<br />if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … }<br />…<br />http://search.yahooapis.com/ImageSearchService/V1/imageSearch?appid=YahooDemo&query=Corvette&results=2<br />http://weather.yahooapis.com/forecastrss?p=FRXX0076&u=c<br />
  4. 4. var map = new YMap(document.getElementById('map'));…varcurrentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint);…<br />…<br />if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … }<br />…<br />http://search.yahooapis.com/ImageSearchService/V1/imageSearch?appid=YahooDemo&query=Corvette&results=2<br />http://weather.yahooapis.com/forecastrss?p=FRXX0076&u=c<br />
  5. 5. var map = new YMap(document.getElementById('map'));…varcurrentGeoPoint = new YGeoPoint( _c.Lat, _c.Lon ); map.addMarker(currentGeoPoint);…<br />…<br />if (flickcurl_prepare(fc, "flickr.photos.geo.correctLocation", parameters, count)) { … }<br />…<br />http://search.yahooapis.com/ImageSearchService/V1/imageSearch?appid=YahooDemo&query=Corvette&results=2<br />http://weather.yahooapis.com/forecastrss?p=FRXX0076&u=c<br />YQL<br />YQL stmt<br />
  6. 6. Why YQL?<br />Thousands of Web Services and sources that provide valuable data<br />Require Developers to read documentation and form URLs/queries.<br />Data is isolated<br />Needs filtering, combining, tweaking, shaping even after it gets to the developer.<br />
  7. 7. What is YQL?<br />Cloud web service and SQL-Like Language <br />Familiar to developers<br />Synonymous with Data access<br />Expressive enough to get the right data.<br />Self describing - show, desc table<br />Allows you to query, filter, join and update data across any structured data on the web / web services<br />And Yahoo’s sherpa cloud storage<br />Real time engine<br />
  8. 8. YQL Statement Basics<br />show: lists the supported tables<br />desc: describes the structure of a table<br />select: fetches data<br />insert/update/delete: modify data<br />use: use an Open Data Table<br />set: define key-values across Open Data Tables<br />
  9. 9. Insert / Update / Delete<br />With INSERT/ UPDATE/ DELETESQL verbs you can modify data on the internet or cloud<br />create tweets, add blog entries, update DBs etc<br />store tables or pieces of code in sherpa<br />Some tables with I/U/D<br />Twitter, Bit.ly shortener<br />Yahoo profile status, application update stream<br />Amazon SimpleDB<br />
  10. 10. Demo: developer.yahoo.com/yql/<br />
  11. 11. Get any data:<br />Open Data Tables<br />
  12. 12. Open Data Tables<br /><ul><li>http://github.com/yql/yql-tables/tree/master
  13. 13. datatables.orgmirrors the master and builds a single environment with all tables (faster serving)
  14. 14. Console with community tables
  15. 15. Brings the power of the table model to new APIs
  16. 16. Schema defines mapping between YQL and Endpoint
  17. 17. Keys can either be query, path or matrix parameters
  18. 18. Keys can be marked as required
  19. 19. YQL Compiler validates existence of required keys</li></li></ul><li>YQL<br />Open Data Table example<br />
  20. 20. Open Data Table example<br />
  21. 21. Open Data Table example<br />
  22. 22. ODT <execute> element<br />Extends Open Data Tables with server-side code<br />Javascript server-side scripting<br />No DOM, events etc, pure JS scripting<br />E4X support - XML literals, filtering ...<br />YQL no longer performs a GET request to the table binding URL<br />YQL provides additional useful global objects<br />request, response, y.rest, y.include, y.query...<br />Execute tag must return the data that will be the output of the YQL select<br />Move business logic of your application to the cloud<br />
  23. 23. What you can do with <execute><br />access APIs that require authentication<br />Netflix OAuth, FlickrAuth, Google AuthSub<br />join data across services<br />grab New York Times article tags and generate find associated flickr photos<br />combine multiple searches into a single result (twitter, web, news and images)<br />augment data:<br />city/state to APIs that just return zip code<br />create APIs from web page<br />celebrity birthdays scraped from imdb, caltrain<br />data transformation<br />convert the result from xml to Google's kml format<br />
  24. 24. hello world<br /><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd"><br /><meta><br /> <sampleQuery>select * from {table} where a=’hello' and b=’world';</sampleQuery><br /></meta><br /><bindings><br /><select itemPath="" produces="XML"><br /><urls><url>http://fake.url/{a}</url></urls><br /><inputs><br /><key id='a' type='xs:string' paramType='path' required="true" /><br /><key id='b' type='xs:string' paramType='variable' required="true" /><br /></inputs><br /><execute><![CDATA[<br />// Your javascript goes here. We will run it on our servers<br />response.object = <item><br /><url>{request.url}</url><br /><a>{a}</a><br /> <b>{b}</b><br /> </item>;<br /> ]]></execute><br /> </select><br /> </bindings><br /></table><br />
  25. 25. Scraping a page<br /><select produces="XML" itemPath="deals.deal"><br /> <urls><url>http://deals.yahoo.com</url></urls><br /> <execute><![CDATA[<br />vardealhtml = y.query("select * from html where url='http://deals.yahoo.com' and xpath='//div[contains(@class, "product-display")]'").results;<br />var deals = <deals/>;<br />function compress(object) {<br /> if (!object) return "";<br /> return object.toString().replace(/s+/, " ")<br />}<br />for each (var div in dealhtml.div) {<br />var deal = <deal/>;<br />deal.name += <name>{compress(div.h3.a.text())}</name>;<br />deal.link += <link>{div.div[0].a.@href}</link>;<br />…<br />deal.description += <description>{compress(div.p[0].*)}</description>;<br />deal.logo += <logo>{compress(div.div[1].*)}</logo>;<br />deals.deal += deal;<br />}<br />response.object = deals;<br /> ]]></execute><br /> </select><br />
  26. 26. Twitter Select<br />select *from twitter.statuswhere id="2108869549"andusername="xxx"and password="yyy”<br /><select itemPath="" produces="XML"><br /> <urls><url>http://twitter.com/statuses/show/{id}.xml</url></urls><br /> <inputs><br /> <key id="username" type="xs:string" required="false" paramType="variable"/><br /> <key id="password" type="xs:string" required="false" paramType="variable"/><br /> <key id="id" type="xs:integer" required="true" paramType="path"/><br /> </inputs><br /> <execute><![CDATA[<br />varr = null;<br /> if (username && password) {<br /> y.include("http://yqlblog.net/samples/base64.js");<br />varauthheader = "Basic "+Base64.encode(username+":"+password);<br />r = request.header("Authorization",authheader).get().response;<br /> } else {<br />r = request.get().response;<br /> }<br />response.object = r;<br /> ]]></execute><br /></select><br />
  27. 27. Twitter Update<br />insert into twitter.status(status,username,password)values ("new tweet from YQL",”xxx”,”yyy”);<br /><insert itemPath="" produces="XML"><br /> <urls><br /> <url>http://twitter.com/statuses/update.xml</url><br /> </urls><br /> <inputs><br /> <value id="username" type="xs:string" required="true" paramType="variable"/><br /> <value id="password" type="xs:string" required="true" paramType="variable"/><br /> <value id="status" type="xs:string" required="true" paramType="variable"/><br /> </inputs><br /> <execute><![CDATA[<br /> y.include("http://yqlblog.net/samples/base64.js");<br />varauthheader = "Basic "+Base64.encode(username+":"+password);<br />var content = "status="+status;<br />response.object = request.header("Authorization",authheader).post(content).response;<br /> ]]></execute><br /></insert><br />
  28. 28. Twitter Delete<br />delete from twitter.statuswhere id="2108869549"and username="xxx"and password="yyy”<br /><delete itemPath="" produces="XML"><br /> <urls><br /> <url>http://twitter.com/statuses/destroy/{id}.xml</url><br /> </urls><br /> <inputs><br /> <key id="username" type="xs:string" required="true" paramType="variable"/><br /> <key id="password" type="xs:string" required="true" paramType="variable"/><br /> <key id="id" type="xs:string" required="true" paramType="path"/><br /> </inputs><br /> <execute><![CDATA[<br />y.include("http://yqlblog.net/samples/base64.js");<br />varauthheader = "Basic "+Base64.encode(username+":"+password);<br />response.object = request.header("Authorization",authheader).del().response;<br /> ]]></execute><br /></delete><br />
  29. 29. Conclusion<br />More easily build applications<br />fewer lines of code<br />data processing done away from app<br />consistent and familiar syntax for all data access<br />
  30. 30. Conclusion<br />Build applications that run faster<br /><ul><li>service in the cloud does the work
  31. 31. store tables, environments, js, text files…
  32. 32. conversion, filtering, parallel requests...</li></ul>With smaller network footprint<br />fewer (client) calls<br />smaller data amounts<br />closer to the data, fatter pipes<br />
  33. 33. Conclusion<br />Flexible<br /><ul><li>Supports XML, JSON, JSONP
  34. 34. Transform documents between formats
  35. 35. Easy to integrate into any development environment w/ HTTP</li></li></ul><li>Questions?<br />http://developer.yahoo.com/yql<br />Derek Gathright<br />drg@yahoo-inc.com<br />twitter.com/derek<br />meetup.com/socaljs<br />
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×