Using web services
in JavaScript.
Chris&an Heilmann
Fron0rends, Warsaw, Poland, October 2010

What is the web?

Was ist das Web?
Data + Interfaces

Lots of yummy,
yummy data.

If a company has a
bit of brains these
days then they build
on and release an
API for their
services.

Company benefits:
People bring your data and
services into environments
you could never reach.

Our benefits:
solve problems and get
information without buying
data or writing code.

Solving 2 problems.

Problem #1:
Distance between
two places on Earth.

Sometimes it would
be good to know
the distance
between two places
on Earth.

For this you could
use any mapping
service and
navigate from one
to the other.

But what if you
want to have that as
plain information?

Battle Plan:
1. Find the location of the two places
on Earth
2. Calculate the distance.

Yahoo GeoPlanet is
a data set that has
information about
the location of
places on Earth.

Yahoo GeoPlanet is
a data set that has
information about
the location of
places on Earth.
http://developer.yahoo.com/geo/geoplanet/

http://where.yahooapis.com/v1/places.q('warsaw')?
appid={appid}&format=json

This gives us the
latitude and
longitude of both
places...

...but how do we
calculate the
distance?

http://www.movable-type.co.uk/scripts/latlong-
vincenty.html

Putting it all
together...

Putting it all
together...

Problem #1:
Foreign Tweets.

Twitter supports
different languages,
but has no
translation service.

Google has a
translation service
though.

Battle Plan:
1. Investigate Twitter’s search API and
Google’s translation API and if
needed, get keys.
2. Get the results from Twitter for a
certain search.
3. Loop over the results, see which
ones are not in English, and then
translate them with the Google
Translation API.

The solution is very
similar to the other
solution.

It also suffers from
the same issues.

Not fun with JS:
1. Asynchronous lookups with
generated script nodes are a pain to
get right - what if one breaks?
2. Depending on how many Tweets are
not in English, you have to hammer
Google’s translation API which slows
down your overall app.

Simplifying access.

Using several APIs
in conjunction in
JavaScript is not
fun.

It doesn’t matter if
that is internally or
externally in a
company.

Yahoo had a lot of
these problems...

...which is why they
built YQL to work
around that.

Using YQL, web
services become
databases you can
access like any
other DB.

YQL http://developer.yahoo.com/yql/console/

YQL http://developer.yahoo.com/yql/console/
select {what} from {where}
where {conditions}

Foreign Tweets?

select text from twitter.search
where q=”ft2010” and
iso_language_code=”pl”

select * from google.translate
where q in (
select text from twitter.search
where q=”ft2010” and
iso_language_code=”pl”
) and target=”en”

Keywords for
Warsaw?

select * from search.web
where query="warsaw"

select abstract from
search.web where
query="warsaw"

select abstract from
search.web(100) where
query="warsaw"

select * from
search.termextract where
context in (
select abstract from
search.web(100) where
query="warsaw"
)

select * from
search.termextract where
context in (
select abstract from
search.web(100) where
query="warsaw"
)
| sort(field="Result")
| unique(field="Result")

Photos of Warsaw?

select * from
flickr.photos.search(50) where
woe_id in (select woeid from
geo.places where
text="warsaw") and
min_taken_date = "2009"

Instead of going
crazy filtering and
sorting in JS...

...use the YQL server
and then have a very
simple JS for
displaying.

Using web services
with YQL in JS.

YQL is a web
service endpoint on
its own...

https://query.yahooapis.com/v1/public/
yql?q={uri-encoded-query}&
format={xml|json}&
diagnostics={true|false}&
callback={function}&
env=store%3A%2F%2Fdatatables.org
%2Falltableswithkeys

As it supports
JSON-P you can use
it with dynamically
created script
nodes.

Special case:
Scraping
http://www.flickr.com/photos/fdtate/4426760544/

http://www.slideshare.net/cheilmann/reasons-to-be-cheerful-fronteers-2010

select * from html where
url="http://www.slideshare.net/
cheilmann/reasons-to-be-cheerful-
fronteers-2010"
and
xpath="//ol/li/p[contains(.,'http')]"

http://y.ahoo.it/r/ENSPGm

http://lanyrd.com/people/codepo8/

HTML as JSON is
not fun.

JSON-P-X =
HTML as a string in
a JSON-P container!

Using YQL re-use of
web content is very
easy indeed.

Be safe,
be good...

Using JSON-P in
JavaScript is easy.

Don’t rely on
data arriving -
test for it!

XML to JSON?

XML to JSON?

Using JSON is easy
with libraries.

JSON-P and jQuery:
$.getJSON(url+'&callback=?',
function(data){
});

JSON-P and jQuery:
$.ajax({
url: url,
dataType: 'jsonp',
jsonp: 'callback',
jsonpCallback: 'ohyeah'
});
function ohyeah(data){
}

Which one to use?

getJSON() is
dangerous with
other people’s data.

$.ajax():
http://{...}&
format=json&callback=ohyeah
$.getJSON():
http://{...}&
format=json&callback=jsonp1282497813335
Random number

Cachebreaking is
not a good idea.

Local caching is a
good idea though.

Cookies
suck,
though.

Would be good to
have a better
solution for that.

localStorage =
cookies on steroids.

if(('localStorage' in window) &&
window['localStorage'] !== null){
localStorage.setItem(
'cake',
'much better than cookies'
)
}

if(('localStorage' in window) &&
window['localStorage'] !== null){
var what = localStorage.getItem(
'cake'
)
// what -> 'much better than
cookies'
}

localStorage only
stores Strings - use
JSON to work
around that.

if(('localStorage' in window) &&
window['localStorage'] !== null){
localStorage.setItem(
'cake',
JSON.stringify(
{yummy:‘yes’,candles:5}
)
);
}

if(('localStorage' in window) &&
window['localStorage'] !== null){
var what = JSON.parse(
localStorage.getItem('cake')
);
// what -> Object{...}
// and not [Object object]
}

Let’s wrap this up in
a function.

yql - the query
id - storage key name
cacheage - how long to cache
callback - obvious, isn’t it?

Browsers
supporting
localStorage fetch
the data every hour.

Others still work,
but load the data
every time.

OK, dann machen
wir das halt..

OK, dann machen
wir das halt..

OK, dann machen
wir das halt..

There are a few
libraries offering
fallbacks to other
browers via Flash.

You can of course
also use those.

Offering your own
API.

To get your own API
into YQL all you
need to do is write
an XML schema and
put it on GitHub.

http://github.com/yql/yql-tables

YQL allows you to
write “executable
tables”...

...which means you
can convert data
with JavaScript that
will be executed
server-side.

Twitter translate
example:

Offering your own
API.

Offering your own
API.

SELECT * FROM
twitter.translate WHERE
language="en" and
search="warsaw" and
amount="20"

Distance example:

SELECT * FROM geo.distance
WHERE place1=”london” and
place2="warsaw"

http://isithackday.com/hacks/geo/distance/

Using your JS
tables.

Write your schema,
put it on the web
and use, err... USE
to, err... use it.

use “http://app.pl/distance.xml”
as distance;
SELECT * FROM distance
WHERE place1=”london” and
place2="warsaw"

Both
problems
solved
and
released
as an API
- in JS!

In summary
Use YQL instead of wasting time
reading API docs for a simple task
Filter data in the service and get the
info back in formats you need.
Use the fast YQL server instead of
doing lots of requests.
Write your own JS APIs using
execute.
Use local storage and don’t break
caching.
Go and use the web.

Christian Heilmann
http://wait-till-i.com Cheers
http://developer-evangelism.com
@codepo8