Playing with the web
 or “The geek shall inherit the earth”




 Christian Heilmann | http://wait-till-i.com | http://twit...
The web is an awesome
opportunity and media.
I learnt that pretty early and
left an old media to work for
         the new one.
One thing that keeps amazing
me about it is how simple the
 technologies driving it are.
You don’t
need to be a
rocket
scientist to
wreak havoc
on the web.
I am not talking about
malicious intent here.
I am talking about ethical
         hacking.
So today I am going to show
 some tools I love to use to
play and mess with the web.
Disrupting the process to
foster and drive innovation.



                      Mr. Buzzword
                     will see...
Let’s start with the first one –
  a true Swedish product!
Their product?
cURL
cURL allows you to get raw
text data from any server.
You can create the full range
of HTTP requests from a script
         or the shell.
Including POST requests,
simulating cookies and all the
       other goodies :)
Which gives you an amazing
     amount of power
Say you want to build an API
     that doesn’t exist.
At
       Mashed08 I
       got bored.



http://www.flickr.com/photos/37996583811@N01/2600265124/
Someone came to me and
asked if I knew of a currency
      conversion API.
And I didn’t as there is none.
        (everybody pays good money for that data)
So I went to Yahoo Finance.
http://finance.yahoo.com/currency/convert?
 amt=1&from=USD&to=JPY&submit=Convert
Simple, predictable URL :)
I found the location of the
result by viewing the source
         of the page.
function convert($from,$to){
   $url= 'http://finance.yahoo.com/currency/convert?
 amt=1&from='.$from.'&to='.$to.'&submit=...
Turning this into an API was
           easy:
header('Content-type:text/javascript');
$from = $_GET['from'];
$to = $_GET['to'];
$callback = $_GET['callback'];
if(preg_m...
preg_match_all(quot;/tabledata1quot;>([^<]+)</
td>/quot;,$feed,$cells);
  if(is_numeric($cells[1][1])){
    $out = ‘{quot;...
} else {
    $out = ‘{quot;errorquot;:quot;Invalid callback
method namequot;}’;
  }
}
echo $out;
Using the API is as easy...
<script type=quot;text/javascriptquot;>
  function converted(obj){
    if(obj.error){
      alert(obj.error);
    } else {...
View Source is a good start.
However, much more win is
        Firebug.
It has never been easier to
find things to get with cURL.
Say twitter information...
How about showing this as a
       cool chart?
We all like to show off what
    we do on the web.
Charts can be tricky...
Good thing that there’s
   Google Charts.
        (yeah and YUI charts)
http://www.wait-till-i.com/2008/11/23/show-the-world-your-
         twitter-type-using-php-and-google-charts/
<?php
$user = $_GET['user'];
$isjs = quot;/^[a-z|A-Z|_|-|$|0-9|.]+$/quot;;
if(preg_match($isjs,$user)){
  $info = array();...
preg_match_all('/<span id=quot;update_countquot;
class=quot;stats_count numericquot;>([^>]+)</span>/
msi',$cont,$updates);...
}
  if($type === 'followed'){
    $t = ' is being ';
  }
  $title = $user . $t . $type;
  $out = array();
  foreach($info ...
function get($url){
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, $url);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER,...
What if you need to mix and
  match and filter data?
http://www.wait-till-i.com/2008/09/28/useful-
            tweets-with-pipe/
https://pipes.yahoo.com/
https://pipes.yahoo.com/
And *dum de dum de*...
http://developer.yahoo.com/yql/console/
/*
  Useful tweets badge by Christian Heilmann
*/
var tweets = function(){
   var x = document.getElementById('mytweet');
...
var now = data.value.items[i];
                   var li = document.createElement('li');
                   var a = docume...
One of my favourite toys:
Using GreaseMonkey you can
 change any web page out
    there with DOM and
         JavaScript.
You can for example
prototype an enhancement
and show it to people with a
        single link.
By playing with the web you
 can do jobs that until now
    cost a lot of money.
Say you want to help your
clients find good keywords to
promote their product online.
You can do some research,
surf all the competitors’ sites
      and note down the
 descriptions, keywords and
            ...
Or you can be a man and use
 cURL to write a script to do
            that.
Or you can be a clever man
and keep your eyes open and
 check if there is an API for
            that.
http://developer.yahoo.com/search/boss/
http://boss.yahooapis.com/ysearch/web/v1/donkeys?
                format=xml&appid=...
http://boss.yahooapis.com/ysearch/web/v1/donkeys?
        format=xml&view=keyterms&appid=...
All you need to do is getting
 the top 20, analyzing the
  keyword frequency and
      create a top 20.
http://developer.yahoo.net/blog/archives/2008/11/
               boss_keywords.html
Then you take YUI CSS grids,
and spend 30 minutes playing
   with colours and fonts.
And you have a
   product:
 http://keywordfinder.org
This is all cool,
but does it
bring us
anywhere?
Yes, if you get excited about
the web and its opportunities
    you can move things.
It takes determination!
And the outcome can be
rewarding beyond anything
    you ever imagined.
http://www.youtube.com/watch?v=CwsDKaalgq8&
http://www.youtube.com/watch?v=QiuT0y0KR6I
So by all means, put all your
 wonderful products on the
            web.
Especially when they cater a
    very specific need.
The prime number shitting
            bear...
http://alpha61.com/primenumbershittingbear/
And never stop to fiddle,
tweak and poke at the things
people offer you on the web.
So thank you for having me
    here and listening.
And I hope you will have an
   awesome Christmas!




                 Christian Heilmann
http://scriptingenabled.org | ht...
Playing With The Web
Playing With The Web
Playing With The Web
Playing With The Web
Playing With The Web
Playing With The Web
Playing With The Web
Playing With The Web
Upcoming SlideShare
Loading in...5
×

Playing With The Web

7,626

Published on

My second talk at geekmeet sweden talking about the tools you can use to hack and remix the web.

Published in: Technology, Business
1 Comment
7 Likes
Statistics
Notes
No Downloads
Views
Total Views
7,626
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
105
Comments
1
Likes
7
Embeds 0
No embeds

No notes for slide

Playing With The Web

  1. 1. Playing with the web or “The geek shall inherit the earth” Christian Heilmann | http://wait-till-i.com | http://twitter.com/codepo8 Geek Meet Stockholm, December 2008
  2. 2. The web is an awesome opportunity and media.
  3. 3. I learnt that pretty early and left an old media to work for the new one.
  4. 4. One thing that keeps amazing me about it is how simple the technologies driving it are.
  5. 5. You don’t need to be a rocket scientist to wreak havoc on the web.
  6. 6. I am not talking about malicious intent here.
  7. 7. I am talking about ethical hacking.
  8. 8. So today I am going to show some tools I love to use to play and mess with the web.
  9. 9. Disrupting the process to foster and drive innovation. Mr. Buzzword will see you now!
  10. 10. Let’s start with the first one – a true Swedish product!
  11. 11. Their product?
  12. 12. cURL
  13. 13. cURL allows you to get raw text data from any server.
  14. 14. You can create the full range of HTTP requests from a script or the shell.
  15. 15. Including POST requests, simulating cookies and all the other goodies :)
  16. 16. Which gives you an amazing amount of power
  17. 17. Say you want to build an API that doesn’t exist.
  18. 18. At Mashed08 I got bored. http://www.flickr.com/photos/37996583811@N01/2600265124/
  19. 19. Someone came to me and asked if I knew of a currency conversion API.
  20. 20. And I didn’t as there is none. (everybody pays good money for that data)
  21. 21. So I went to Yahoo Finance.
  22. 22. http://finance.yahoo.com/currency/convert? amt=1&from=USD&to=JPY&submit=Convert
  23. 23. Simple, predictable URL :)
  24. 24. I found the location of the result by viewing the source of the page.
  25. 25. function convert($from,$to){ $url= 'http://finance.yahoo.com/currency/convert? amt=1&from='.$from.'&to='.$to.'&submit=Convert'; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $feed = curl_exec($ch); curl_close($ch); preg_match_all(quot;/tabledata1quot;>([^<]+)</td>/quot;, $feed,$cells); return $cells[1][1]; } echo convert(’USD’,'GBP’); http://www.wait-till-i.com/2008/06/21/currency- conversion-api/
  26. 26. Turning this into an API was easy:
  27. 27. header('Content-type:text/javascript'); $from = $_GET['from']; $to = $_GET['to']; $callback = $_GET['callback']; if(preg_match(quot;/[A-Z|a-z]{3}/quot;,$to) && preg_match(quot;/[A-Z|a-z]{3}/quot;,$from)){ $to = strToUpper($to); $from = strToUpper($from); $url= ‘http://finance.yahoo.com/currency/ convert?’ . ‘amt=1&from=’.$from.’&to=’. $to.’&submit=Convert’; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $feed = curl_exec($ch); curl_close($ch);
  28. 28. preg_match_all(quot;/tabledata1quot;>([^<]+)</ td>/quot;,$feed,$cells); if(is_numeric($cells[1][1])){ $out = ‘{quot;fromquot;:quot;’.$from.’quot;,quot;toquot;:quot;’. $to.’quot;,quot;factorquot;:quot;’.$cells[1][1].’quot;}’; } else { $out = ‘{quot;errorquot;:quot;Could not convert currencies, are you sure about the names?quot;}’; } } else { $out = ‘{quot;errorquot;:quot;Invalid Currency format, must be three lettersquot;}’; } if(isset($callback)){ if(preg_match(quot;/[a-z|A-Z|_|-|$|0-9|.]/quot;, $callback)){ $out = $callback.’(’.$out.’)';
  29. 29. } else { $out = ‘{quot;errorquot;:quot;Invalid callback method namequot;}’; } } echo $out;
  30. 30. Using the API is as easy...
  31. 31. <script type=quot;text/javascriptquot;> function converted(obj){ if(obj.error){ alert(obj.error); } else { alert('one ' + obj.from + ' is ' + obj.factor + ' ' + obj.to); } } </script> <script type=quot;text/javascriptquot; src=quot;convert.php? from=gbp&to=usd&callback=convertedquot;> </script>
  32. 32. View Source is a good start.
  33. 33. However, much more win is Firebug.
  34. 34. It has never been easier to find things to get with cURL.
  35. 35. Say twitter information...
  36. 36. How about showing this as a cool chart?
  37. 37. We all like to show off what we do on the web.
  38. 38. Charts can be tricky...
  39. 39. Good thing that there’s Google Charts. (yeah and YUI charts)
  40. 40. http://www.wait-till-i.com/2008/11/23/show-the-world-your- twitter-type-using-php-and-google-charts/
  41. 41. <?php $user = $_GET['user']; $isjs = quot;/^[a-z|A-Z|_|-|$|0-9|.]+$/quot;; if(preg_match($isjs,$user)){ $info = array(); $cont = get('http://twitter.com/'.$user); preg_match_all('/<span id=quot;following_countquot; class=quot;stats_count numericquot;>([^>]+)</span>/msi',$cont, $follow); $info['follower'] = convert($follow[1] [0]); preg_match_all('/<span id=quot;follower_countquot; class=quot;stats_count numericquot;>([^>]+)</span>/ msi',$cont,$follower); $info['followed'] = convert($follower[1] [0]);
  42. 42. preg_match_all('/<span id=quot;update_countquot; class=quot;stats_count numericquot;>([^>]+)</span>/ msi',$cont,$updates); $info['updater'] = convert($updates[1] [0]); $max = max($info); $convert = 100 / $max ; foreach($info as $k=>$f){ if($f === $max){ $type = $k; } $disp[$k] = $f * $convert; } if($type === 'updater'){ $t = ' is an '; } if($type === 'follower'){ $t = ' is a ';
  43. 43. } if($type === 'followed'){ $t = ' is being '; } $title = $user . $t . $type; $out = array(); foreach($info as $k=>$i){ $out[] = $k.'+('.$i.')'; } $labels = join($out,'|'); $values = join($disp,','); header('location:http:// chart.apis.google.com/chart? cht=p3&chco=336699&'. 'chtt='.urlencode($title).'&chd=t:'.$values. '&chs=350x100&chl='.$labels); }
  44. 44. function get($url){ $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $feed = curl_exec($ch); curl_close($ch); return $feed; } function convert($x){ $x = str_replace(',','',$x); $x = (int)$x; return $x; } ?>
  45. 45. What if you need to mix and match and filter data?
  46. 46. http://www.wait-till-i.com/2008/09/28/useful- tweets-with-pipe/
  47. 47. https://pipes.yahoo.com/
  48. 48. https://pipes.yahoo.com/
  49. 49. And *dum de dum de*...
  50. 50. http://developer.yahoo.com/yql/console/
  51. 51. /* Useful tweets badge by Christian Heilmann */ var tweets = function(){ var x = document.getElementById('mytweet'); if(x){ var twitterUserId = x.className.replace('user-',''); var s = document.createElement('script'); s.type = 'text/javascript'; s.src = 'http://pipes.yahoo.com/pipes/pipe.run?' + '_id=f7229d01b79e508d543fb84e8a0abb0d&_render=json' + '&id=' + twitterUserId + '&_callback=tweets.tweet'; document.getElementsByTagName('head')[0].appendChild(s); }; function tweet(data){ if(data && data.value && data.value.items){ if(typeof data.value.items.length !== 'undefined'){ var ul = document.createElement('ul'); var all = data.value.items.length; var end = all > 5 ? 5 : all; for(var i=0;i < end;i++){
  52. 52. var now = data.value.items[i]; var li = document.createElement('li'); var a = document.createElement('a'); a.href = now.link; a.appendChild( document.createTextNode(now.title) ); li.appendChild(a); ul.appendChild(li); } x.appendChild(ul); } } }; return{ tweet:tweet } }();
  53. 53. One of my favourite toys:
  54. 54. Using GreaseMonkey you can change any web page out there with DOM and JavaScript.
  55. 55. You can for example prototype an enhancement and show it to people with a single link.
  56. 56. By playing with the web you can do jobs that until now cost a lot of money.
  57. 57. Say you want to help your clients find good keywords to promote their product online.
  58. 58. You can do some research, surf all the competitors’ sites and note down the descriptions, keywords and titles.
  59. 59. Or you can be a man and use cURL to write a script to do that.
  60. 60. Or you can be a clever man and keep your eyes open and check if there is an API for that.
  61. 61. http://developer.yahoo.com/search/boss/
  62. 62. http://boss.yahooapis.com/ysearch/web/v1/donkeys? format=xml&appid=...
  63. 63. http://boss.yahooapis.com/ysearch/web/v1/donkeys? format=xml&view=keyterms&appid=...
  64. 64. All you need to do is getting the top 20, analyzing the keyword frequency and create a top 20.
  65. 65. http://developer.yahoo.net/blog/archives/2008/11/ boss_keywords.html
  66. 66. Then you take YUI CSS grids, and spend 30 minutes playing with colours and fonts.
  67. 67. And you have a product: http://keywordfinder.org
  68. 68. This is all cool, but does it bring us anywhere?
  69. 69. Yes, if you get excited about the web and its opportunities you can move things.
  70. 70. It takes determination!
  71. 71. And the outcome can be rewarding beyond anything you ever imagined.
  72. 72. http://www.youtube.com/watch?v=CwsDKaalgq8& http://www.youtube.com/watch?v=QiuT0y0KR6I
  73. 73. So by all means, put all your wonderful products on the web.
  74. 74. Especially when they cater a very specific need.
  75. 75. The prime number shitting bear... http://alpha61.com/primenumbershittingbear/
  76. 76. And never stop to fiddle, tweak and poke at the things people offer you on the web.
  77. 77. So thank you for having me here and listening.
  78. 78. And I hope you will have an awesome Christmas! Christian Heilmann http://scriptingenabled.org | http://wait-till-i.com twitter/flickr: codepo8
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×