Your SlideShare is downloading. ×
0
Coding on the
      Shoulders of Giants


                              Matt Biddulph


                                  ...
DOPPLR
                                   DOPPLR
Dopplr is a service for frequent travellers and their friends. It lets yo...
A quot;platformquot; is a system that can
           be programmed and therefore
           customized by outside develope...
A quot;platformquot; is a system that can
           be programmed and therefore
           customized by outside develope...
Dopplr is a very targeted web application, and we have no wish to reinvent the wheel on any
technology or function that is...
Everything in this talk is based on live code that’s running on dopplr.com today. I’ll be
pointing out what libraries we u...
The most basic and most important principle of integrating web apps today is sharing data. A
good webapp helps its users m...
Sharing Data
The most basic and most important principle of integrating web apps today is sharing data. A
good webapp help...
The simplest way to expose data in a webapp is to supply feeds in machine-readable
formats. Dopplr gives its users a “my t...
The simplest way to expose data in a webapp is to supply feeds in machine-readable
formats. Dopplr gives its users a “my t...
The simplest way to expose data in a webapp is to supply feeds in machine-readable
formats. Dopplr gives its users a “my t...
It’s also easy to transform our data into KML and view it in Google Earth.
And NetNewsWire lets you import trip details in iCal because of the hCalendar.
We also mark up our contact lists using hCard.
We also mark up our contact lists using hCard.
http://www.hauntedcastle.org/castle/
And via our API, users are starting to create interesting tools like this Carbon Calc...
If we want to integrate your Dopplr data with your data from elsewhere, we need to know
who you are on other sites.
User Identity
If we want to integrate your Dopplr data with your data from elsewhere, we need to know
who you are on other...
DOPPLR
                         DOPPLR
                DOPPLR
People have different identities spread across many sites.

...
OpenID is the obvious solution to this mess - if you use the same URL everywhere then sites
sharing data can be sure that ...
OpenID is the obvious solution to this mess - if you use the same URL everywhere then sites
sharing data can be sure that ...
“OpenId is the
                                                           game changer for
                               ...
Once we know who you are on other sites, we’d like to help you import your social network
from those sites to Dopplr.
Social Network
Once we know who you are on other sites, we’d like to help you import your social network
from those sites ...
For example, Twitter mark up their contact lists with XFN and hCard. Using the heuristic of “if
you use the same username ...
http://mofo.rubyforge.org




           http://code.whytheluckystiff.net/hpricot/




We use the Mofo library to parse mi...
class Traveller < ActiveRecord::Base
         matches_identities
       end




              http://identity-matcher.goog...
http://identity-matcher.googlecode.com/

This is an example of running the “match_twitter” call on Chris Messina’s profile.
>> Traveller.match_twitter(quot;factoryjoequot;)
          [0].map(&:name)




              http://identity-matcher.googl...
>> Traveller.match_twitter(quot;factoryjoequot;)
          [0].map(&:name)
       => [quot;Alexander Ljungquot;, quot;Geor...
“the best way for you
                                                       to manage your
                              ...
To make interesting mashups, we often need to delegate authority to 3rd-party automated
services. Code running elsewhere n...
Delegating Authority
To make interesting mashups, we often need to delegate authority to 3rd-party automated
services. Cod...
The worst way to do this (we’re guilty too) is for the 3rd-party to ask for the users login
details and fake a login as th...
BBAuth
                                        AuthSub
                                        Authentication
            ...
“An open protocol to
                allow secure API
                authentication in a
                simple and stand...
There are many ways to get a piece of screen real-estate on other sites.
Widgets & Plugins
There are many ways to get a piece of screen real-estate on other sites.
Dopplr has a Facebook app that puts a Dopplr profile box on your profile page. It doesn’t try
to cram the Dopplr user experi...
It also publishes cute little items in your mini-feed.
The Facebook F8 platform is actually very nice to work with. There are a few gotchas:

1. Pages served under apps.facebook...
proxies directly to your server




The Facebook F8 platform is actually very nice to work with. There are a few gotchas:
...
proxies directly to your server

                       rewrites your HTML, CSS and JS




The Facebook F8 platform is act...
proxies directly to your server

                       rewrites your HTML, CSS and JS


                       caches wha...
proxies directly to your server

                       rewrites your HTML, CSS and JS


                       caches wha...
http://rfacebook.rubyforge.org




We use the RFacebook library to abstract a lot of the low-level details away. It helps.
We also have a blog badge that follows the same style as the Facebook profilebox.
<div id=quot;dopplr-blog-badgequot;>
            <script
              src=quot;dopplr.com/blogbadge/...quot;>
           ...
“Javascript is the
                                                  extra layer above the
                               ...
(function(i) {u = navigator.userAgent; e = /*@cc_on!@*/false; st = setTimeout;
      if(/webkit/i.test(u)){st(function(){d...
Even application hosting is starting to become an on-demand commodity.
Utility Computing
Even application hosting is starting to become an on-demand commodity.
We’re currently experimenting with using Amazon S3 for MySQL backups, and EC2 to run a
complete copy of the Dopplr applica...
S3 Data Storage




We’re currently experimenting with using Amazon S3 for MySQL backups, and EC2 to run a
complete copy o...
S3 Data Storage


         Pre-built EC2 Rails



We’re currently experimenting with using Amazon S3 for MySQL backups, an...
S3 Data Storage


         Pre-built EC2 Rails


                         MySQL Slave
We’re currently experimenting with u...
Sharing Data
       User Identity
     Social Network
Delegating Authority
  Widgets & Plugins
  Utility Computing
Thank you


    Matt Biddulph




                    DOPPLR
           DOPPLR
  DOPPLR




Where next?
Coding on the Shoulders of Giants
Coding on the Shoulders of Giants
Coding on the Shoulders of Giants
Upcoming SlideShare
Loading in...5
×

Coding on the Shoulders of Giants

19,824

Published on

A developer talk from FOWA 2007 on how Dopplr works as a Small Piece, Loosely Joined.

Published in: Technology
3 Comments
58 Likes
Statistics
Notes
No Downloads
Views
Total Views
19,824
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
748
Comments
3
Likes
58
Embeds 0
No embeds

No notes for slide

Transcript of "Coding on the Shoulders of Giants"

  1. 1. Coding on the Shoulders of Giants Matt Biddulph DOPPLR DOPPLR Dopplr is a service for frequent travellers and their friends. It lets you share your travel plans with a group of trusted people that you choose. DOPPLR This worldmap shows where everyone on Dopplr came to London from on October 4th 2007 (the day this talk was given at FOWA) Where next?
  2. 2. DOPPLR DOPPLR Dopplr is a service for frequent travellers and their friends. It lets you share your travel plans with a group of trusted people that you choose. DOPPLR This worldmap shows where everyone on Dopplr came to London from on October 4th 2007 (the day this talk was given at FOWA) Where next?
  3. 3. A quot;platformquot; is a system that can be programmed and therefore customized by outside developers – users – and in that way, adapted to countless needs and niches that the platform's original developers could not have possibly contemplated, much less had time to accommodate. –Marc Andreessen, http://blog.pmarca.com/2007/09/the-three-kinds.html Today I’m going to talk about using the internet as a platform that your webapps can build on top of.
  4. 4. A quot;platformquot; is a system that can be programmed and therefore customized by outside developers – users – and in that way, adapted to countless needs and niches that the platform's original developers could not have possibly contemplated, much less had time to accommodate. –Marc Andreessen, http://blog.pmarca.com/2007/09/the-three-kinds.html It’s impossible to predict all possible uses that your users might want to make of your app and its data, so we’re going to look at the technologies that developers can use today that maximise possibilities for integration and reuse.
  5. 5. Dopplr is a very targeted web application, and we have no wish to reinvent the wheel on any technology or function that isn’t part of our core mission to find serendipity in travel. We designed Dopplr from the start to be a Small Piece, Loosely Joined. We believe that your data belongs to you, and that you should be able to have it anywhere you want it - even if you don’t frequently visit dopplr.com
  6. 6. Everything in this talk is based on live code that’s running on dopplr.com today. I’ll be pointing out what libraries we used. Because Dopplr’s written in Ruby on Rails, that’s where all the examples will come from.
  7. 7. The most basic and most important principle of integrating web apps today is sharing data. A good webapp helps its users make distinctive structured data (whether directly or indirectly) and does useful things with it.
  8. 8. Sharing Data The most basic and most important principle of integrating web apps today is sharing data. A good webapp helps its users make distinctive structured data (whether directly or indirectly) and does useful things with it.
  9. 9. The simplest way to expose data in a webapp is to supply feeds in machine-readable formats. Dopplr gives its users a “my trips” feed. As a newsfeed, it’s not all that useful as it tells you things you already know (the trips that you’ve added). As a lightweight read-only API, it works very well. We mark up our feed with all the appropriate machine-readable formats that we can find: GeoRSS, hCalendar and Google Calendar GData. This means it’s trivial to plug it into other systems like Yahoo Pipes, and mash it up with other data.
  10. 10. The simplest way to expose data in a webapp is to supply feeds in machine-readable formats. Dopplr gives its users a “my trips” feed. As a newsfeed, it’s not all that useful as it tells you things you already know (the trips that you’ve added). As a lightweight read-only API, it works very well. We mark up our feed with all the appropriate machine-readable formats that we can find: GeoRSS, hCalendar and Google Calendar GData. This means it’s trivial to plug it into other systems like Yahoo Pipes, and mash it up with other data.
  11. 11. The simplest way to expose data in a webapp is to supply feeds in machine-readable formats. Dopplr gives its users a “my trips” feed. As a newsfeed, it’s not all that useful as it tells you things you already know (the trips that you’ve added). As a lightweight read-only API, it works very well. We mark up our feed with all the appropriate machine-readable formats that we can find: GeoRSS, hCalendar and Google Calendar GData. This means it’s trivial to plug it into other systems like Yahoo Pipes, and mash it up with other data.
  12. 12. It’s also easy to transform our data into KML and view it in Google Earth.
  13. 13. And NetNewsWire lets you import trip details in iCal because of the hCalendar.
  14. 14. We also mark up our contact lists using hCard.
  15. 15. We also mark up our contact lists using hCard.
  16. 16. http://www.hauntedcastle.org/castle/ And via our API, users are starting to create interesting tools like this Carbon Calculator.
  17. 17. If we want to integrate your Dopplr data with your data from elsewhere, we need to know who you are on other sites.
  18. 18. User Identity If we want to integrate your Dopplr data with your data from elsewhere, we need to know who you are on other sites.
  19. 19. DOPPLR DOPPLR DOPPLR People have different identities spread across many sites. Where next? Where next? Where next?
  20. 20. OpenID is the obvious solution to this mess - if you use the same URL everywhere then sites sharing data can be sure that you are the same person. It’s not just for login. OpenID can also be used to prove you own an identity. You may not want to use your AOL Instant Messenger identity as a login, but Dopplr can use the AOL OpenID provider to let you prove what your AIM ID is. This lets us write an AIM bot that talks to you by IM secure in the knowledge that we’re not leaking your information to an impostor.
  21. 21. OpenID is the obvious solution to this mess - if you use the same URL everywhere then sites sharing data can be sure that you are the same person. It’s not just for login. OpenID can also be used to prove you own an identity. You may not want to use your AOL Instant Messenger identity as a login, but Dopplr can use the AOL OpenID provider to let you prove what your AIM ID is. This lets us write an AIM bot that talks to you by IM secure in the knowledge that we’re not leaking your information to an impostor.
  22. 22. “OpenId is the game changer for social networks, allowing for portability of the social graph and preferences.” –Simon Willison ©2007 Julian Cash flickr.com/photos/juliancash/673891099/ For more about OpenID, read Simon Willison’s many talks and writings. He knows.
  23. 23. Once we know who you are on other sites, we’d like to help you import your social network from those sites to Dopplr.
  24. 24. Social Network Once we know who you are on other sites, we’d like to help you import your social network from those sites to Dopplr.
  25. 25. For example, Twitter mark up their contact lists with XFN and hCard. Using the heuristic of “if you use the same username and real name on two systems, you’re probably the same person”, we can show our users a suggested list of users who might be the same on Twitter and Dopplr.
  26. 26. http://mofo.rubyforge.org http://code.whytheluckystiff.net/hpricot/ We use the Mofo library to parse microformats, and Hpricot to scrape raw HTML.
  27. 27. class Traveller < ActiveRecord::Base matches_identities end http://identity-matcher.googlecode.com/ We’re releasing the code we use to match identities across sites as a Rails plugin.
  28. 28. http://identity-matcher.googlecode.com/ This is an example of running the “match_twitter” call on Chris Messina’s profile.
  29. 29. >> Traveller.match_twitter(quot;factoryjoequot;) [0].map(&:name) http://identity-matcher.googlecode.com/ This is an example of running the “match_twitter” call on Chris Messina’s profile.
  30. 30. >> Traveller.match_twitter(quot;factoryjoequot;) [0].map(&:name) => [quot;Alexander Ljungquot;, quot;George Kellyquot;, quot;Amy Raymondquot;, quot;Andrew Crowquot;, quot;Jonathan Greenequot;, quot;Aubrey Sabalaquot;, quot;Matt Jonesquot;, quot;Blaine Cookquot;, quot;Brian Oberkirchquot;, quot;Buzz Andersenquot;, quot;David Ulevitchquot;, quot;Michael Buffingtonquot;, quot;Eric Costelloquot;, quot;Kaustubh Srikanthquot;, quot;Brian Del Vecchioquot;, quot;Jack Dorseyquot;, quot;joshua schachterquot;, quot;Kevin Lawverquot;, quot;Scott Bealequot;, quot;Lisa McMillanquot;, quot;Mary Hodderquot;, quot;Dan Safferquot;, quot;Rob Hayesquot;, quot;Thomas Vander Walquot;, quot;James Walkerquot;] http://identity-matcher.googlecode.com/ This is an example of running the “match_twitter” call on Chris Messina’s profile.
  31. 31. “the best way for you to manage your network is to stop thinking about all of the little pieces and to start focusing on the big picture” As Gavin Bell has been telling us for a while, the tools for social network portability are reaching maturity. David Recordon (pictured) is working on open systems at Six Apart to aggregate and share social network information.
  32. 32. To make interesting mashups, we often need to delegate authority to 3rd-party automated services. Code running elsewhere needs to have access to users’ accounts to act on their behalf.
  33. 33. Delegating Authority To make interesting mashups, we often need to delegate authority to 3rd-party automated services. Code running elsewhere needs to have access to users’ accounts to act on their behalf.
  34. 34. The worst way to do this (we’re guilty too) is for the 3rd-party to ask for the users login details and fake a login as them. This is not only bad for privacy but also teaches users terrible habits.
  35. 35. BBAuth AuthSub Authentication OpenAuth There are several standards for delegating API access to a 3rd-party but they all work slightly differently.
  36. 36. “An open protocol to allow secure API authentication in a simple and standard method from desktop and web applications.” http://oauth.net So the OAuth project is standardising a single protocol that everyone can use. This will mean one client library per language (rather than one per language per site) and an easier landscape for users to understand.
  37. 37. There are many ways to get a piece of screen real-estate on other sites.
  38. 38. Widgets & Plugins There are many ways to get a piece of screen real-estate on other sites.
  39. 39. Dopplr has a Facebook app that puts a Dopplr profile box on your profile page. It doesn’t try to cram the Dopplr user experience into a small box; it just displays a useful summary of your travel information.
  40. 40. It also publishes cute little items in your mini-feed.
  41. 41. The Facebook F8 platform is actually very nice to work with. There are a few gotchas: 1. Pages served under apps.facebook.com are proxied directly to your server at request time. This makes development versions a bit of a pain as your dev server must be visible on the internet. We do this with an SSH tunnel. 2. FBML is a subset of XHTML with some extra elements in the Facebook namespace. Facebook rewrite all your IDs and classes in the HTML and CSS so that you can’t accidentally change another app’s style.
  42. 42. proxies directly to your server The Facebook F8 platform is actually very nice to work with. There are a few gotchas: 1. Pages served under apps.facebook.com are proxied directly to your server at request time. This makes development versions a bit of a pain as your dev server must be visible on the internet. We do this with an SSH tunnel. 2. FBML is a subset of XHTML with some extra elements in the Facebook namespace. Facebook rewrite all your IDs and classes in the HTML and CSS so that you can’t accidentally change another app’s style.
  43. 43. proxies directly to your server rewrites your HTML, CSS and JS The Facebook F8 platform is actually very nice to work with. There are a few gotchas: 1. Pages served under apps.facebook.com are proxied directly to your server at request time. This makes development versions a bit of a pain as your dev server must be visible on the internet. We do this with an SSH tunnel. 2. FBML is a subset of XHTML with some extra elements in the Facebook namespace. Facebook rewrite all your IDs and classes in the HTML and CSS so that you can’t accidentally change another app’s style.
  44. 44. proxies directly to your server rewrites your HTML, CSS and JS caches what it can The Facebook F8 platform is actually very nice to work with. There are a few gotchas: 1. Pages served under apps.facebook.com are proxied directly to your server at request time. This makes development versions a bit of a pain as your dev server must be visible on the internet. We do this with an SSH tunnel. 2. FBML is a subset of XHTML with some extra elements in the Facebook namespace. Facebook rewrite all your IDs and classes in the HTML and CSS so that you can’t accidentally change another app’s style.
  45. 45. proxies directly to your server rewrites your HTML, CSS and JS caches what it can needs you to respond quickly The Facebook F8 platform is actually very nice to work with. There are a few gotchas: 1. Pages served under apps.facebook.com are proxied directly to your server at request time. This makes development versions a bit of a pain as your dev server must be visible on the internet. We do this with an SSH tunnel. 2. FBML is a subset of XHTML with some extra elements in the Facebook namespace. Facebook rewrite all your IDs and classes in the HTML and CSS so that you can’t accidentally change another app’s style.
  46. 46. http://rfacebook.rubyforge.org We use the RFacebook library to abstract a lot of the low-level details away. It helps.
  47. 47. We also have a blog badge that follows the same style as the Facebook profilebox.
  48. 48. <div id=quot;dopplr-blog-badgequot;> <script src=quot;dopplr.com/blogbadge/...quot;> </script> </div> The user only has to paste a tiny piece of code into their template to enable it. It looks for its div and insert HTML and CSS there.
  49. 49. “Javascript is the extra layer above the mark-up ‘what is this text’ and the CSS ‘how should it be displayed’. It adds a new dimension, ‘how should this element behave.’” –Christian Heilmann This follows the principle of unobstrusive Javascript.
  50. 50. (function(i) {u = navigator.userAgent; e = /*@cc_on!@*/false; st = setTimeout; if(/webkit/i.test(u)){st(function(){dr = document.readyState;if(dr==quot;loadedquot;|| dr==quot;completequot;){i();}else{st(arguments.callee,10);}},10);}else if((/mozilla/i. test(u)&&!/(compati)/.test(u))||(/opera/i.test(u))){document.addEventListener (quot;DOMContentLoadedquot;,i,false);} else if(e){(function(){t=document.createElement ('doc:ready');try{t.doScroll('left');i();t= null;}catch(e){st(arguments.callee, 0);}})();}else{window.onload = i;}})(function() { // make the badge }); http://www.kryogenix.org/days/2007/09/26/shortloaded We also avoid the need for pulling in a full Javascript library such as jQuery just to get a proper ‘page is loaded’ event, using this code.
  51. 51. Even application hosting is starting to become an on-demand commodity.
  52. 52. Utility Computing Even application hosting is starting to become an on-demand commodity.
  53. 53. We’re currently experimenting with using Amazon S3 for MySQL backups, and EC2 to run a complete copy of the Dopplr application and a MySQL slave replicated over an SSH tunnel from live. This is looking like a good way to scale, and it’s a great way to run stats reports and live backups without disturbing our main servers.
  54. 54. S3 Data Storage We’re currently experimenting with using Amazon S3 for MySQL backups, and EC2 to run a complete copy of the Dopplr application and a MySQL slave replicated over an SSH tunnel from live. This is looking like a good way to scale, and it’s a great way to run stats reports and live backups without disturbing our main servers.
  55. 55. S3 Data Storage Pre-built EC2 Rails We’re currently experimenting with using Amazon S3 for MySQL backups, and EC2 to run a complete copy of the Dopplr application and a MySQL slave replicated over an SSH tunnel from live. This is looking like a good way to scale, and it’s a great way to run stats reports and live backups without disturbing our main servers.
  56. 56. S3 Data Storage Pre-built EC2 Rails MySQL Slave We’re currently experimenting with using Amazon S3 for MySQL backups, and EC2 to run a complete copy of the Dopplr application and a MySQL slave replicated over an SSH tunnel from live. This is looking like a good way to scale, and it’s a great way to run stats reports and live backups without disturbing our main servers.
  57. 57. Sharing Data User Identity Social Network Delegating Authority Widgets & Plugins Utility Computing
  58. 58. Thank you Matt Biddulph DOPPLR DOPPLR DOPPLR Where next?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×