Playing with the web
or “The geek shall inherit the earth”
Christian Heilmann | http://wait-till-i.com | http://twitter.com/codepo8
Geek Meet Stockholm, December 2008

The web is an awesome
opportunity and media.

I learnt that pretty early and
left an old media to work for
the new one.

One thing that keeps amazing
me about it is how simple the
technologies driving it are.

You don’t
need to be a
rocket
scientist to
wreak havoc
on the web.

I am not talking about
malicious intent here.

I am talking about ethical
hacking.

So today I am going to show
some tools I love to use to
play and mess with the web.

Disrupting the process to
foster and drive innovation.
Mr. Buzzword
will see you now!

Let’s start with the first one –
a true Swedish product!

Their product?

cURL

cURL allows you to get raw
text data from any server.

You can create the full range
of HTTP requests from a script
or the shell.

Including POST requests,
simulating cookies and all the
other goodies :)

Which gives you an amazing
amount of power

Say you want to build an API
that doesn’t exist.

At
Mashed08 I
got bored.
http://www.flickr.com/photos/37996583811@N01/2600265124/

Someone came to me and
asked if I knew of a currency
conversion API.

And I didn’t as there is none.
(everybody pays good money for that data)

So I went to Yahoo Finance.

http://finance.yahoo.com/currency/convert?
amt=1&from=USD&to=JPY&submit=Convert

Simple, predictable URL :)

I found the location of the
result by viewing the source
of the page.

function convert($from,$to){
$url= 'http://finance.yahoo.com/currency/convert?
amt=1&from='.$from.'&to='.$to.'&submit=Convert';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$feed = curl_exec($ch);
curl_close($ch);
preg_match_all(\"/tabledata1\\\">([^<]+)<\\/td>/\",
$feed,$cells);
return $cells[1][1];
}
echo convert(’USD’,'GBP’);
http://www.wait-till-i.com/2008/06/21/currency-
conversion-api/

Turning this into an API was
easy:

header('Content-type:text/javascript');
$from = $_GET['from'];
$to = $_GET['to'];
$callback = $_GET['callback'];
if(preg_match(\"/[A-Z|a-z]{3}/\",$to) &&
preg_match(\"/[A-Z|a-z]{3}/\",$from)){
$to = strToUpper($to);
$from = strToUpper($from);
$url= ‘http://finance.yahoo.com/currency/
convert?’ .
‘amt=1&from=’.$from.’&to=’.
$to.’&submit=Convert’;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,
1);
$feed = curl_exec($ch);
curl_close($ch);

preg_match_all(\"/tabledata1\\\">([^<]+)<\\/
td>/\",$feed,$cells);
if(is_numeric($cells[1][1])){
$out = ‘{\"from\":\"’.$from.’\",\"to\":\"’.
$to.’\",\"factor\":\"’.$cells[1][1].’\"}’;
} else {
$out = ‘{\"error\":\"Could not convert
currencies, are you sure about the
names?\"}’;
}
} else {
$out = ‘{\"error\":\"Invalid Currency format,
must be three letters\"}’;
}
if(isset($callback)){
if(preg_match(\"/[a-z|A-Z|_|-|\\$|0-9|\\.]/\",
$callback)){
$out = $callback.’(’.$out.’)';

} else {
$out = ‘{\"error\":\"Invalid callback
method name\"}’;
}
}
echo $out;

Using the API is as easy...

<script type=\"text/javascript\">
function converted(obj){
if(obj.error){
alert(obj.error);
} else {
alert('one ' + obj.from + ' is ' +
obj.factor + ' ' + obj.to);
}
}
</script>
<script type=\"text/javascript\" src=\"convert.php?
from=gbp&to=usd&callback=converted\">
</script>

View Source is a good start.

However, much more win is
Firebug.

It has never been easier to
find things to get with cURL.

Say twitter information...

How about showing this as a
cool chart?

We all like to show off what
we do on the web.

Charts can be tricky...

Good thing that there’s
Google Charts.
(yeah and YUI charts)

http://www.wait-till-i.com/2008/11/23/show-the-world-your-
twitter-type-using-php-and-google-charts/

<?php
$user = $_GET['user'];
$isjs = \"/^[a-z|A-Z|_|-|\\$|0-9|\\.]+$/\";
if(preg_match($isjs,$user)){
$info = array();
$cont = get('http://twitter.com/'.$user);
preg_match_all('/<span
id=\"following_count\" class=\"stats_count
numeric\">([^>]+)<\\/span>/msi',$cont,
$follow);
$info['follower'] = convert($follow[1]
[0]);
preg_match_all('/<span id=\"follower_count\"
class=\"stats_count numeric\">([^>]+)<\\/span>/
msi',$cont,$follower);
$info['followed'] = convert($follower[1]
[0]);

preg_match_all('/<span id=\"update_count\"
class=\"stats_count numeric\">([^>]+)<\\/span>/
msi',$cont,$updates);
$info['updater'] = convert($updates[1]
[0]);
$max = max($info);
$convert = 100 / $max ;
foreach($info as $k=>$f){
if($f === $max){
$type = $k;
}
$disp[$k] = $f * $convert;
}
if($type === 'updater'){
$t = ' is an ';
}
if($type === 'follower'){
$t = ' is a ';

}
if($type === 'followed'){
$t = ' is being ';
}
$title = $user . $t . $type;
$out = array();
foreach($info as $k=>$i){
$out[] = $k.'+('.$i.')';
}
$labels = join($out,'|');
$values = join($disp,',');
header('location:http://
chart.apis.google.com/chart?
cht=p3&chco=336699&'.
'chtt='.urlencode($title).'&chd=t:'.$values.
'&chs=350x100&chl='.$labels);
}

function get($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,
1);
$feed = curl_exec($ch);
curl_close($ch);
return $feed;
}
function convert($x){
$x = str_replace(',','',$x);
$x = (int)$x;
return $x;
}
?>

What if you need to mix and
match and filter data?

http://www.wait-till-i.com/2008/09/28/useful-
tweets-with-pipe/

https://pipes.yahoo.com/

https://pipes.yahoo.com/

And *dum de dum de*...

http://developer.yahoo.com/yql/console/

/*
Useful tweets badge by Christian Heilmann
*/
var tweets = function(){
var x = document.getElementById('mytweet');
if(x){
var twitterUserId = x.className.replace('user-','');
var s = document.createElement('script');
s.type = 'text/javascript';
s.src = 'http://pipes.yahoo.com/pipes/pipe.run?' +
'_id=f7229d01b79e508d543fb84e8a0abb0d&_render=json' +
'&id=' + twitterUserId + '&_callback=tweets.tweet';
document.getElementsByTagName('head')[0].appendChild(s);
};
function tweet(data){
if(data && data.value && data.value.items){
if(typeof data.value.items.length !== 'undefined'){
var ul = document.createElement('ul');
var all = data.value.items.length;
var end = all > 5 ? 5 : all;
for(var i=0;i < end;i++){

var now = data.value.items[i];
var li = document.createElement('li');
var a = document.createElement('a');
a.href = now.link;
a.appendChild(
document.createTextNode(now.title)
);
li.appendChild(a);
ul.appendChild(li);
}
x.appendChild(ul);
}
}
};
return{
tweet:tweet
}
}();

One of my favourite toys:

Using GreaseMonkey you can
change any web page out
there with DOM and
JavaScript.

You can for example
prototype an enhancement
and show it to people with a
single link.

By playing with the web you
can do jobs that until now
cost a lot of money.

Say you want to help your
clients find good keywords to
promote their product online.

You can do some research,
surf all the competitors’ sites
and note down the
descriptions, keywords and
titles.

Or you can be a man and use
cURL to write a script to do
that.

Or you can be a clever man
and keep your eyes open and
check if there is an API for
that.

http://developer.yahoo.com/search/boss/

http://boss.yahooapis.com/ysearch/web/v1/donkeys?
format=xml&appid=...

http://boss.yahooapis.com/ysearch/web/v1/donkeys?
format=xml&view=keyterms&appid=...

All you need to do is getting
the top 20, analyzing the
keyword frequency and
create a top 20.

http://developer.yahoo.net/blog/archives/2008/11/
boss_keywords.html

Then you take YUI CSS grids,
and spend 30 minutes playing
with colours and fonts.

And you have a
product:
http://keywordfinder.org

This is all cool,
but does it
bring us
anywhere?

Yes, if you get excited about
the web and its opportunities
you can move things.

It takes determination!

And the outcome can be
rewarding beyond anything
you ever imagined.

http://www.youtube.com/watch?v=CwsDKaalgq8&
http://www.youtube.com/watch?v=QiuT0y0KR6I

So by all means, put all your
wonderful products on the
web.

Especially when they cater a
very specific need.

The prime number shitting
bear...
http://alpha61.com/primenumbershittingbear/

And never stop to fiddle,
tweak and poke at the things
people offer you on the web.

So thank you for having me
here and listening.

And I hope you will have an
awesome Christmas!
Christian Heilmann
http://scriptingenabled.org | http://wait-till-i.com
twitter/flickr: codepo8