SELECT * FROM Internet: Using the Yahoo! Query Language

SELECT * FROM Internet Using the Yahoo! Query Language

Survey...

Who am I? • Brian DeShong, Senior Technical Yahoo • Yahoo! (Atlanta) • Developer / sys admin since 1998 • Focus on PHP, open source • Focusing on native mobile apps lately

Agenda • What’s YQL? • How to work with YQL • Demos! • Data available from Y! and others • Making your data available via YQL

What is YQL?

How does it work? • Web service with an SQL-like query syntax • SQL is familiar to many developers • Common interface for any data on the web • Query, ﬁlter, join and update data across any structured data on the web • Output formats: • XML • JSON, JSONP, JSONPX

Working With YQL

SQL syntax SELECT {what} FROM {data table} WHERE {condition}

Weather forecast SELECT * FROM weather.forecast WHERE location = ‘29424’

Location by IP SELECT * FROM ip.location WHERE ip = ’153.9.243.13’

Twitter search SELECT from_user, text, profile_image_url FROM twitter.search WHERE q = ’#barcampchs’

More than SELECT • Modify data with INSERT, UPDATE and DELETE • Tweet, add blog posts, update databases, etc. • We’ll look at Open Data Tables shortly

Demos!

Integrating YQL

RESTful API http://query.yahooapis.com/v1/public/yql ?q=[urlencoded YQL query] &format=[xml|json|etc.]

Weather URL https://query.yahooapis.com/v1/public/yql ?q=select%20*%20 from%20weather.location%20 where %20location%3D'29424'%3B

PHP output brian@roadrunner [~/Dropbox/barcampchs/code]$ php ./twitter.php XarkGirl: @mguerard need more $$, but loving life. i'm helping w/ #barcampCHS, writing a novel, working on secret project. how the hell iz u?!? barcampchs: RT @csatcofc: Pecha Kucha on Wednesday 10/10, Barcamp on Saturday, 10/13. Next week is going to be good! #pkchs #barcampchs #csatcofc csatcofc: Pecha Kucha on Wednesday 10/10, Barcamp on Saturday, 10/13. Next week is going to be good! #pkchs #barcampchs #csatcofc barcampchs: Cool, just found out that the @lullabot's will be sending someone to join us fm Savannah #barcampchs @chsdug #drupal carnellm: If you haven't bought your ticket for #BarCampCHS yet, you can try to win a free one! http://www.facebook.com/TheModernConnection Peter_Lucash:Vote for my proposal - From Idea to Launch! RT @BarCampCHS: RT @pla1: 8 days until BarCamp Charleston Sat Nov 13 2010 #barcampchs joannepicvu: RT @barcampchs: RT @pla1: 8 days until BarCamp Charleston Sat Nov 13 2010 #barcampchs barcampchs: RT @pla1: 8 days until BarCamp Charleston Sat Nov 13 2010 #barcampchs CBSWebDesigns: #barcampchs is coming up. What sessions do you want ? We hope ours is 1. Customizing WP Twenty Ten theme is one of them. http://t.co/nP5aR99 pla1: 8 days until BarCamp Charleston Sat Nov 13 2010 #barcampchs Yaenette: RT @barcampchs: Only one week left to get your ticket to #barcampchs #chs #nchs carnellm: I'm attending #BarCampCHS! Have you gotten your ticket yet? http://www.eventbrite.com/s/1EXx vhanna26: @Yaenette It will be fun. #barcampchs tybaltknight: I missed the memo on the change of venue for #BarcampCHS planning session #volunteeringfail. vhanna26: At a #barcampchs planning meeting. Only nine more days!!!

JavaScript

Yahoo! Data

Community data

Open Data Tables • Create web service for sites that do not expose public APIs • Data table is: • XML document • Translates YQL query into HTTP request • See datatables.org

Data Table XML example

Storing to Sherpa • yql.storage and yql.storage.admin tables • Allows you to store and work with data in YQL itself • Shares tables, environments and JS with all other YQL users • Layered on Yahoo! Sherpa DHT cloud storage platform

Sherpa Limits and Reqs • Size Limit • 1000 records @ 100KB each • Retention Limit • 30 day purge • Record Format • Records must be in a text-based format; authentication for New Records • yql.storage.admin access must be authorized using two-legged OAuth

Wrap Up

Why YQL? • Same interface for all of your data • Process data away from your application • Rapidly prototype and build applications • Create APIs for sites that don’t have them • And let others easily leverage them!

http://developer.yahoo.com/yql/

Thanks! bdeshong@yahoo-inc.com http://www.yahoo.com/ brian@deshong.net http://www.deshong.net/

SELECT * FROM Internet: Using the Yahoo! Query Language

by bdeshong

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 1

Statistics