Open Hack Taiwan 2012


Published on

Introduction to YQL for Open Hack Taiwan 2012

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • One of the biggest hurdles at Hack events is data, and how to get it. The most important part of your application is the data. It’s the data that keeps your user’s coming back for more. Without it, all you’ve got is that snazzy UI.
  • Individual companies build up massive virtual silos of data. Some companies make their data available over Web services, while others do not.
  • But, the great thing is that the internet is really just a series of tubes and all those silos are connected by these tubes. Some tubes lead to RSS feeds, others to RESTful APIs, while some just lead to websites.
  • It is our goalas hackers and makers to find new and interesting ways to combine the data available online to create new and interesting applications. This is where YQL comes in. YQL helps you to spend less time worrying about the plumbing of your application, and more time taking that data and creating value for your users.
  • YQL stands for Yahoo! Query Language. It is an SQL-like language that allows you to query, filter, and join data across any number of different data sources. YQL is not limited to Yahoo! services, but we are nice and have already provided a number of freely accessible tables to get you started. You can also test out the tables on our easy to use web console. You can easily create your own tables which implement your own custom logic, allowing you to mash-up data to your hearts content, any way you would like to (More on this later).
  • YQL is highly compatible. It is able to ingest data from XML, JSON, and CSV sources and then return XML, JSON, or JSON-P. YQL is deployed into our colos around the world. There are multiple levels of cache that you are able to use as a developer. There is a user-facing caching proxy server, an internal colo-wide memory cache, as well as a squid cache to capture responses from any backend APIs you call. All of these are controllable from with-in the JavaScript execute environment.YQL has a 99.5% performance uptime target, and can be used for commercial purposes, though it is rate limited. However, the YQL environment provided for use today is not rate limited.
  • The syntax used in YQL is very simple, and should look very familiar if you have done any work with SQL databases.
  • The best place to start learning how to use YQL is the YQL console.
  • YQL supports 3 output types, JSON, JSON-P and XML. You can select which data type you would like here.The diagnostics mode toggle will tell the YQL server to output additional diagnostic information, including any information on any backend services it connected to, how many resource units were consumed, and any log statements from custom written server-side logic.Debug mode will enable additional logging.Creating a query alias will allow you to more easily re-run the same query at a later date without having to type it all in again.This is the URL that your external application can call to execute the the last query that was run on this console.
  • What if we wanted to do a simple aggregation of the RSS feeds from TechCrunch and AllThingsD? This is simple with the feed table.
  • We can use projection to only grab the data we need, and also use the built-in sort function to sort the entries by publication date.
  • Pinterest doesn’t currently have a public API for it’s top 50 pins. But we can use the HTML table to make one.
  • The first thing you need to do to be able to use the Flickr table is to grab a free Flickr API key. If you don’t already have one, you can apply for one at the link on the screen now.
  • You’ll notice that this table doesn’t provide us with the URL to any of the images.
  • To get the URLs, we need to call a different table. We can do this by using the photo search table as a sub-query to the photo sizes table. This will grab the URL for each of the photos in our search results.
  • To help developers with the first 3 steps in developing their own YQL tables, we’ve developed the YQL editor.
  • If you notice, this table is not using the URL block shown on the previous slide. This is because instead of calling another external web service, we are going to call a different community YQL table.
  • jsonCompat prevents JSON numbers being turned into strings, and preserves single-item arrays.
  • Open Hack Taiwan 2012

    1. 1. YQLCody SchwebkeYahoo!
    2. 2. DATA
    3. 3. (DATA) SILOS
    4. 4. THE INTERNET
    6. 6. Y(AHOO!) Q(UERY) L(ANGUAGE)  SQL-like syntax  Query, filter, and join across data sources  Not limited to Yahoo! services  Easy to use web console and table editor  Create custom tables with your own server- side logic using the JavaScript execute environment
    7. 7. YQL, CONT’D Compatible  Accepts XML, JSON, CSV  Returns XML, JSON, JSON-P High Performance  Deployed around the globe  Front and Back facing caches  99.5%+ performance uptime target Rate-limited  /v1/public/* - 2,000 requests/hour per IP  /v1/yql/* - 20,000 requests/hour per IP  Per application limit (identified by your Access Key): 100,000 requests/day
    8. 8. QUERY SYNTAX Select {what} from {service} where {condition} Supports Select, Insert, Update, Delete Supports Joining with sub-selects Includes helper functions (sort, reverse, unique)  Uses ‘|’ symbol
    9. 9. CONSOLE
    10. 10. CONSOLE OPTIONS Create aliasOutput type selection Toggle diagnostics and debug modesREST query
    11. 11. REST QUERY
    13. 13. COMMUNITY TABLES 151 community tables Everything from flickr to weather Click desc for more information on each table
    15. 15. FEED, CONT’D
    16. 16. HTML
    17. 17. FLICKRGet a Flickr API key:
    23. 23. 4 STEPS TO BUILD YOUR OWN TABLE1. Define your table’s properties2. Implement custom logic using JavaScript execute3. Save your table to our servers4. Call your new table from your application
    24. 24. PIG LATIN• A language game played with the English language• Popular with elementary school students• Simple, algorithmic conversion • pig = igpay • computer = omputercay • another = another-way
    25. 25. YQL EDITOR Click here
    26. 26. STEP 1
    27. 27. STEP 1, CONT’D Optional meta data Table inputsPath to repeating Custom JavaScript response items include
    28. 28. STEP 2 Table inputs Execute a sub-query “Lossless” JSON mode E4X – XML for JavaScript
    29. 29. STEP 3 Click me! Reference keys
    30. 30. STEP 3.5, TEST IN CONSOLE Execute key
    31. 31. STEP 4
    32. 32. END RESULT
    33. 33. RECAP – BUILDING A HACK WITH YQL Pick your data source Mix and match community tables in the console Build new a table if needed in the editor and test in the console Use YUI-YQL module to get data Or just call the REST query endpoint
    34. 34. ADDITIONAL RESOURCES Docs   Community    E4X  US/docs/E4X/Processing_XML_with_E4X
    35. 35. QUESTIONS?