YQL is an amazing tool to use and offer APIs to the world. As you can do the lot in JavaScript it is pretty simple to get started. There is however also the option that you do things wrong and make your end users and yourself unhappy. This talk works around some of the issues you might face.

1. Caching, collating and filtering
data
Using YQL sensibly

2. What is the web?

3. Was ist das Web?
Data + Interfaces

4. Lots of yummy,
yummy data.

5. Everybody benefits
from APIs.

6. Companies get their
data into
environments they
could never reach.

7. Developers can
build products
without buying data
or writing code.

8. Let’s play with two
examples.

9. Build a system to
calculate the
distance between
two places on Earth.

10. Use a map service?

11. Raw data and info
about the places
would be better to
have.

12. Simple Plan:
1. Find the location of the two places
on Earth
2. Calculate the distance.

13. Earth Data = Yahoo
GeoPlanet

14. Yahoo GeoPlanet is
a data set that has
information about
the location of
places on Earth.
http://developer.yahoo.com/geo/geoplanet/

16. http://where.yahooapis.com/v1/places.q('warsaw')?
appid={appid}&format=json

17. = Latitude+Longitude

18. Distance?

19. http://www.movable-type.co.uk/scripts/latlong-
vincenty.html

20. Putting it all
together...

21. Putting it all
together...

25. A few annoyances
1. Multiple script generation (order?
what if one breaks?)
2. Access keys readable in source.

26. Building a system to
translate foreign
tweets.

27. Twitter is
multilingual but
doesn’t translate.

28. Google has a
translation service
though.

29. A simple plan:
1. Investigate Twitter’s search API and
Google’s translation API and if
needed, get keys.
2. Get the results from Twitter for a
certain search.
3. Loop over the results, see which ones
are not in English, and then translate
them with the Google Translation
API.

30. Really not that
much difference in
code.

31. It also suffers from
the same issues.

32. 1. Asynchronous lookups with
generated script nodes are a pain to
get right - what if one breaks?
2. Depending on how many Tweets are
not in English, you have to hammer
Google’s translation API which slows
down your overall app.

33. YUI fixes a few of
those issues.
1. Using JSONP you can have success
and failure events.
2. You can also provide timeouts
IO
JSON
JSON-P
JSON-P
YQL-Query
GET

34. Still, it would be
nice to have one
request, right?

35. Simplifying access.

36. YQL http://developer.yahoo.com/yql/console/

37. YQL http://developer.yahoo.com/yql/console/
select {what} from {where}
where {conditions}

38. Foreign Tweets?

39. select text from twitter.search
where q=”ft2010” and
iso_language_code=”pl”

40. select * from google.translate
where q in (
select text from twitter.search
where q=”ft2010” and
iso_language_code=”pl”
) and target=”en”

41. Re-using cool data
on the web?

42. http://www.guardian.co.uk/news/datablog/2010/
feb/11/winter-olympics-medals-by-country

45. select * from csv where url="http://
spreadsheets.google.com/pub?
key=tpWDkIZMZleQaREf493v1Jw&output=
csv" and
columns="Year,City,Sport,Discipline,Country
,Event, Gender,Type" and Year="1924"

46. http://winterolympicsmedals.com

47. Instead of going
crazy filtering and
sorting in JS...

48. ...use the YQL server
and then have a very
simple JS for
displaying.

49. Using web services
with YQL in JS.

50. YQL is a web
service endpoint on
its own...

51. https://query.yahooapis.com/v1/public/
yql?q={uri-encoded-query}&
format={xml|json}&
diagnostics={true|false}&
callback={function}&
env=store%3A%2F%2Fdatatables.org
%2Falltableswithkeys

53. Special case:
Scraping
http://www.flickr.com/photos/fdtate/4426760544/

54. http://www.slideshare.net/cheilmann/reasons-to-be-cheerful-fronteers-2010

56. select * from html where
url="http://www.slideshare.net/
cheilmann/reasons-to-be-cheerful-
fronteers-2010"
and
xpath="//ol/li/p[contains(.,'http')]"

57. http://y.ahoo.it/r/ENSPGm

58. http://lanyrd.com/people/codepo8/

59. HTML as JSON is
not fun.

60. JSON-P-X =
HTML as a string in
a JSON-P container!

61. https://github.com/codepo8/lanyrdbadge

62. Using YQL re-use of
web content is very
easy indeed.

63. YUI3’s YQL-Query
makes it even
better!

64. Be safe,
be good...

65. Don’t rely on
data arriving -
test for it!

66. XML to JSON?

67. XML to JSON?

68. Using JSON is easy
with libraries.

69. $.getJSON(url+'&callback=?',
function(data){
});
JSON-P and jQuery:

70. $.ajax({
url: url,
dataType: 'jsonp',
jsonp: 'callback',
jsonpCallback: 'ohyeah'
});
function ohyeah(data){
}
JSON-P and jQuery:

71. Which one to use?

72. getJSON() is
dangerous with
other people’s data.

73. http://{...}&
format=json&callback=ohyeah
$.ajax():
$.getJSON():
http://{...}&
format=json&callback=jsonp1282497813335
Random number

74. Cachebreaking is
not a good idea.

75. Local caching is a
good idea.

76. Cookies
suck,
though.

77. Would be good to
have a better
solution for that.

78. localStorage =
cookies on steroids.

79. if(('localStorage' in window) &&
window['localStorage'] !== null){
localStorage.setItem(
'cake',
'much better than cookies'
)
}

80. if(('localStorage' in window) &&
window['localStorage'] !== null){
var what = localStorage.getItem(
'cake'
)
// what -> 'much better than
cookies'
}

81. localStorage only
stores Strings - use
JSON to work
around that.

82. if(('localStorage' in window) &&
window['localStorage'] !== null){
localStorage.setItem(
'cake',
JSON.stringify(
{yummy:‘yes’,candles:5}
)
);
}

83. if(('localStorage' in window) &&
window['localStorage'] !== null){
var what = JSON.parse(
localStorage.getItem('cake')
);
// what -> Object{...}
// and not [Object object]
}

84. Let’s wrap this up in
a function.

85. yql - the query
id - storage key name
cacheage - how long to cache
callback - obvious, isn’t it?
https://github.com/codepo8/yql-localcache

86. Browsers
supporting
localStorage fetch
the data every hour.

87. Others still work,
but load the data
every time.

88. callback gets an
object with two
properties:
data - guess what?
type - cached|live|freshcache

89. Libraries offer
storage fallbacks for
legacy browers via
Flash - YUI is of
course one of them.

90. Offering your own
API.

91. To get your own API
into YQL all you
need to do is write
an XML schema and
put it on GitHub.

92. http://github.com/yql/yql-tables

93. YQL allows you to
write “executable
tables”...

94. ...which means you
can convert data
with JavaScript that
will be executed
server-side.

95. Our earlier
examples as YQL
APIs are...

96. Twitter translate
example:

97. Offering your own
API.

98. Offering your own
API.

100. SELECT * FROM
twitter.translate WHERE
language="en" and
search="warszawa" and
amount="20"

103. Distance example:

106. SELECT * FROM geo.distance
WHERE place1=”london” and
place2="warsaw"

108. http://isithackday.com/hacks/geo/distance/

109. Using your JS
tables.

110. Write your schema,
put it on the web...

111. use “http://awesomeserver.com/
distance.xml” as distance;
SELECT * FROM distance WHERE
place1=”london” and
place2="warsaw"
use USE to use it!

112. Both
problems
solved
and
released
as an API
- in JS!

113. In summary
Use YQL instead of wasting time
reading API docs for a simple task
Filter data in the service and get the
info back in formats you need.
Use the fast YQL server instead of
doing lots of requests.
Write your own JS APIs using
execute.
Use local storage and don’t break
caching.
Go and use the web.
Go easy on effects.

114. Christian Heilmann
http://wait-till-i.com
http://developer-evangelism.com
@codepo8
Cheers