YQL and the API hack stackJonathan LeBlancPrincipal Software Engineer / Technology EvangelistTwitter: @jcleblanc
What We’re Going to CoverYQL OverviewMaking Queries and the Console SandboxOpen Data Tables (a.k.a. Adding New Features into YQL)Execute (Server-Side JavaScript)Final Lessons
What We’re Going to CoverYQL OverviewMaking Queries and the Console SandboxOpen Data Tables (a.k.a. Adding New Features into YQL)Execute (Server-Side JavaScript)Final Lessons
Overview: Don’t reinvent the wheel!
Overview: The Components of YQLYQL ServiceWeb APIYQL TableUSER
Overview: YQL FETCHes data
Overview: YQL Sets and deletes data
Overview: IS YQL A DATABASE?
Overview: YQL is not a database
What We’re Going to CoverYQL OverviewMaking Queries and the Console SandboxOpen Data Tables (a.k.a. Adding New Features into YQL)Execute (Server-Side JavaScript)Final Lessons
Making Requests: Fetching dataSELECT myColumn, myTitleFROM myTable(0, 50) WHERE myColumn = 'value' AND myTitle = 'title‘LIMIT 3 OFFSET 10
Making Requests: Inserting dataINSERT INTO bitly.shorten (login, apiKey, longUrl)VALUES ('ME', 'API_KEY', 'http://yahoo.com')
Making Requests: updating dataUPDATE social.profile.statusSET status="Using YQL UPDATE” WHERE guid="NJFIDHVPVVISDX7UKED2WHU"
Making Requests: deleting dataDELETE FROM twittertableWHERE tweetid="12345" AND username="twitter_username" AND password="twitter_password"
Making Requests: Sub SelectsSELECT * FROM geo.places WHERE text IN (SELECT location FROM social.profile WHERE guid = me)
Making Requests: The yql consolehttp://developer.yahoo.com/yql/console
Making Requests: Flickr URLs<photo 	farm="3" id="5708163920" isfamily="0" isfriend="0" ispublic="1" owner="31832337@N04" secret="0075137487" server="2496" title="San Francisco"/>
Making Requests: Flickr URLsPhoto URLhttp://farm{$farm}.static.flickr.com/{$server}/{$id}_{$secret}.jpgPhoto Page URLhttp://www.flickr.com/photos/{$owner}/{$id}Photo Owner Profile URLhttp://www.flickr.com/photos/{$owner}
Making Requests: url structurehttp://query.yahooapis.com/v1/yql?[params]http://query.yahooapis.com/v1/public/yql?[params]Paramsq=             	[ YQL QUERY ]
format =      	[ XML / JSON]
diagnostics =	[ true / false ]
debug =       	[ true / false ]
callback =    	[ function name ]Making Requests: Private Datahttp://query.yahooapis.com/v1/yqlUses OAuth 1.0 for authorizationOAuth is complicated – use one of our SDKs at https://github.com/yahoo
Making Requests: Let’s See It
What We’re Going to CoverYQL OverviewMaking Queries and the Console SandboxOpen Data Tables (a.k.a. Adding New Features into YQL)Execute (Server-Side JavaScript)Final Lessons
Open Data Tables: Top Level<?xml version="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">        <meta></meta>        <bindings></bindings></table>
Open Data Tables: Meta Node<?xml version="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">    <meta></meta>        <bindings></bindings></table>
Open Data Tables: Meta Node<?xml version="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">    <meta>        <author>Jonathan LeBlanc</author>                   <description>My Table</description>         <documentationURL>www.site.com        </documentationURL>               <sampleQuery>SELECT * FROM {table}         </sampleQuery>    </meta>        <bindings></bindings></table>
Open Data Tables: Bindings Node<?xml version="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">        <meta></meta>    <bindings></bindings></table>
Open Data Tables: Bindings Node<?xml version="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">        <meta></meta>    <bindings><select itemPath="top.child" produces="XML">             <urls>                 <urlenv="all">http://www.site.com/{user}</url>             </urls>             <inputs>                <key id="user" type="xs:string" paramType="path"                    required="true" />            </inputs>         </select>     </bindings></table>
Open Data Tables: Using Your TableSeveral Choices:Upload the XML file to our community Github account.Upload the XML file to your own site and use the YQL ‘USE’ clause.
Open Data Tables: Community Tableshttp://github.com/yql/yql-tables
Open Data Tables: Using The USE ClauseThe USE clauseUSE 'http://www.mysite.com/my_table.xml' AS mytable; SELECT * FROM mytableWHERE user='john_doe'
What We’re Going to CoverYQL OverviewMaking Queries and the Console SandboxOpen Data Tables (a.k.a. Adding New Features into YQL)Execute (Server-Side JavaScript)Final Lessons
Execute: Execute Node<?xml version="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">        <meta></meta>        <bindings><select itemPath="top.child" produces="XML">             <urls></urls>             <inputs></inputs>         </select>     </bindings></table>
Execute: Execute Node<?xml version="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">        <meta></meta>        <bindings><select itemPath="top.child" produces="XML">             <urls></urls>             <inputs></inputs> <execute></execute>        </select>     </bindings></table>
Execute: Server-Side JavaScript<execute>    <![CDATA[    //get user specified inputs from YQL queryvararrQueries = inputs['query'];    //create new XML nodesvarelQueries = <queries/>;    //output new node as YQL response response.object = elQueries;    ]]></execute>
What We’re Going to CoverYQL OverviewMaking Queries and the Console SandboxOpen Data Tables (a.k.a. Adding New Features into YQL)Execute (Server-Side JavaScript)Final Lessons

Hack Day EU 2011 YQL

  • 2.
    YQL and theAPI hack stackJonathan LeBlancPrincipal Software Engineer / Technology EvangelistTwitter: @jcleblanc
  • 3.
    What We’re Goingto CoverYQL OverviewMaking Queries and the Console SandboxOpen Data Tables (a.k.a. Adding New Features into YQL)Execute (Server-Side JavaScript)Final Lessons
  • 4.
    What We’re Goingto CoverYQL OverviewMaking Queries and the Console SandboxOpen Data Tables (a.k.a. Adding New Features into YQL)Execute (Server-Side JavaScript)Final Lessons
  • 5.
  • 6.
    Overview: The Componentsof YQLYQL ServiceWeb APIYQL TableUSER
  • 7.
  • 8.
    Overview: YQL Setsand deletes data
  • 9.
    Overview: IS YQLA DATABASE?
  • 10.
    Overview: YQL isnot a database
  • 11.
    What We’re Goingto CoverYQL OverviewMaking Queries and the Console SandboxOpen Data Tables (a.k.a. Adding New Features into YQL)Execute (Server-Side JavaScript)Final Lessons
  • 12.
    Making Requests: FetchingdataSELECT myColumn, myTitleFROM myTable(0, 50) WHERE myColumn = 'value' AND myTitle = 'title‘LIMIT 3 OFFSET 10
  • 13.
    Making Requests: InsertingdataINSERT INTO bitly.shorten (login, apiKey, longUrl)VALUES ('ME', 'API_KEY', 'http://yahoo.com')
  • 14.
    Making Requests: updatingdataUPDATE social.profile.statusSET status="Using YQL UPDATE” WHERE guid="NJFIDHVPVVISDX7UKED2WHU"
  • 15.
    Making Requests: deletingdataDELETE FROM twittertableWHERE tweetid="12345" AND username="twitter_username" AND password="twitter_password"
  • 16.
    Making Requests: SubSelectsSELECT * FROM geo.places WHERE text IN (SELECT location FROM social.profile WHERE guid = me)
  • 17.
    Making Requests: Theyql consolehttp://developer.yahoo.com/yql/console
  • 18.
    Making Requests: FlickrURLs<photo farm="3" id="5708163920" isfamily="0" isfriend="0" ispublic="1" owner="31832337@N04" secret="0075137487" server="2496" title="San Francisco"/>
  • 19.
    Making Requests: FlickrURLsPhoto URLhttp://farm{$farm}.static.flickr.com/{$server}/{$id}_{$secret}.jpgPhoto Page URLhttp://www.flickr.com/photos/{$owner}/{$id}Photo Owner Profile URLhttp://www.flickr.com/photos/{$owner}
  • 20.
    Making Requests: urlstructurehttp://query.yahooapis.com/v1/yql?[params]http://query.yahooapis.com/v1/public/yql?[params]Paramsq= [ YQL QUERY ]
  • 21.
    format = [ XML / JSON]
  • 22.
  • 23.
    debug = [ true / false ]
  • 24.
    callback = [ function name ]Making Requests: Private Datahttp://query.yahooapis.com/v1/yqlUses OAuth 1.0 for authorizationOAuth is complicated – use one of our SDKs at https://github.com/yahoo
  • 25.
  • 26.
    What We’re Goingto CoverYQL OverviewMaking Queries and the Console SandboxOpen Data Tables (a.k.a. Adding New Features into YQL)Execute (Server-Side JavaScript)Final Lessons
  • 27.
    Open Data Tables:Top Level<?xml version="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd"> <meta></meta> <bindings></bindings></table>
  • 28.
    Open Data Tables:Meta Node<?xml version="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd"> <meta></meta> <bindings></bindings></table>
  • 29.
    Open Data Tables:Meta Node<?xml version="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd"> <meta> <author>Jonathan LeBlanc</author> <description>My Table</description> <documentationURL>www.site.com </documentationURL> <sampleQuery>SELECT * FROM {table} </sampleQuery> </meta> <bindings></bindings></table>
  • 30.
    Open Data Tables:Bindings Node<?xml version="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd"> <meta></meta> <bindings></bindings></table>
  • 31.
    Open Data Tables:Bindings Node<?xml version="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd"> <meta></meta> <bindings><select itemPath="top.child" produces="XML"> <urls> <urlenv="all">http://www.site.com/{user}</url> </urls> <inputs> <key id="user" type="xs:string" paramType="path" required="true" /> </inputs> </select> </bindings></table>
  • 32.
    Open Data Tables:Using Your TableSeveral Choices:Upload the XML file to our community Github account.Upload the XML file to your own site and use the YQL ‘USE’ clause.
  • 33.
    Open Data Tables:Community Tableshttp://github.com/yql/yql-tables
  • 34.
    Open Data Tables:Using The USE ClauseThe USE clauseUSE 'http://www.mysite.com/my_table.xml' AS mytable; SELECT * FROM mytableWHERE user='john_doe'
  • 35.
    What We’re Goingto CoverYQL OverviewMaking Queries and the Console SandboxOpen Data Tables (a.k.a. Adding New Features into YQL)Execute (Server-Side JavaScript)Final Lessons
  • 36.
    Execute: Execute Node<?xmlversion="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd"> <meta></meta> <bindings><select itemPath="top.child" produces="XML"> <urls></urls> <inputs></inputs> </select> </bindings></table>
  • 37.
    Execute: Execute Node<?xmlversion="1.0" encoding="UTF-8"?><table xmlns="http://query.yahooapis.com/v1/schema/table.xsd"> <meta></meta> <bindings><select itemPath="top.child" produces="XML"> <urls></urls> <inputs></inputs> <execute></execute> </select> </bindings></table>
  • 38.
    Execute: Server-Side JavaScript<execute> <![CDATA[ //get user specified inputs from YQL queryvararrQueries = inputs['query']; //create new XML nodesvarelQueries = <queries/>; //output new node as YQL response response.object = elQueries; ]]></execute>
  • 39.
    What We’re Goingto CoverYQL OverviewMaking Queries and the Console SandboxOpen Data Tables (a.k.a. Adding New Features into YQL)Execute (Server-Side JavaScript)Final Lessons
  • 40.
    Final LessonsYQL isa skeleton key for all Yahoo! APIsOpen data tables allow you to create new data feeds out of one or many different raw sourcesExecute allows you to manipulate data before it is returned.Use SDKs when accessing private data that requires OAuth
  • 41.
    Final Lessons: LinksAllYahoo! APIs and Serviceshttp://developer.yahoo.com/everything.htmlYQL Documentationhttp://developer.yahoo.com/yqlYQL Consolehttp://developer.yahoo.com/yql/consoleYQL Github Account (Contribute Tables)http://github.com/yql/yql-tables
  • 42.