2024: Domino Containers - The Next Step. News from the Domino Container commu...
Â
Coding on the Shoulders of Giants
1. Coding on the
Shoulders of Giants
Matt Biddulph
DOPPLR
DOPPLR
Dopplr is a service for frequent travellers and their friends. It lets you share your travel plans
with a group of trusted people that you choose.
DOPPLR
This worldmap shows where everyone on Dopplr came to London from on October 4th 2007
(the day this talk was given at FOWA)
Where next?
2. DOPPLR
DOPPLR
Dopplr is a service for frequent travellers and their friends. It lets you share your travel plans
with a group of trusted people that you choose.
DOPPLR
This worldmap shows where everyone on Dopplr came to London from on October 4th 2007
(the day this talk was given at FOWA)
Where next?
3. A quot;platformquot; is a system that can
be programmed and therefore
customized by outside developers
â users â and in that way, adapted
to countless needs and niches
that the platform's original
developers could not have
possibly contemplated, much less
had time to accommodate.
âMarc Andreessen, http://blog.pmarca.com/2007/09/the-three-kinds.html
Today Iâm going to talk about using the internet as a platform that your webapps can build
on top of.
4. A quot;platformquot; is a system that can
be programmed and therefore
customized by outside developers
â users â and in that way, adapted
to countless needs and niches
that the platform's original
developers could not have
possibly contemplated, much less
had time to accommodate.
âMarc Andreessen, http://blog.pmarca.com/2007/09/the-three-kinds.html
Itâs impossible to predict all possible uses that your users might want to make of your app
and its data, so weâre going to look at the technologies that developers can use today that
maximise possibilities for integration and reuse.
5. Dopplr is a very targeted web application, and we have no wish to reinvent the wheel on any
technology or function that isnât part of our core mission to ďŹnd serendipity in travel. We
designed Dopplr from the start to be a Small Piece, Loosely Joined.
We believe that your data belongs to you, and that you should be able to have it anywhere
you want it - even if you donât frequently visit dopplr.com
6. Everything in this talk is based on live code thatâs running on dopplr.com today. Iâll be
pointing out what libraries we used. Because Dopplrâs written in Ruby on Rails, thatâs where
all the examples will come from.
7. The most basic and most important principle of integrating web apps today is sharing data. A
good webapp helps its users make distinctive structured data (whether directly or indirectly)
and does useful things with it.
8. Sharing Data
The most basic and most important principle of integrating web apps today is sharing data. A
good webapp helps its users make distinctive structured data (whether directly or indirectly)
and does useful things with it.
9. The simplest way to expose data in a webapp is to supply feeds in machine-readable
formats. Dopplr gives its users a âmy tripsâ feed. As a newsfeed, itâs not all that useful as it
tells you things you already know (the trips that youâve added). As a lightweight read-only
API, it works very well. We mark up our feed with all the appropriate machine-readable
formats that we can ďŹnd: GeoRSS, hCalendar and Google Calendar GData.
This means itâs trivial to plug it into other systems like Yahoo Pipes, and mash it up with
other data.
10. The simplest way to expose data in a webapp is to supply feeds in machine-readable
formats. Dopplr gives its users a âmy tripsâ feed. As a newsfeed, itâs not all that useful as it
tells you things you already know (the trips that youâve added). As a lightweight read-only
API, it works very well. We mark up our feed with all the appropriate machine-readable
formats that we can ďŹnd: GeoRSS, hCalendar and Google Calendar GData.
This means itâs trivial to plug it into other systems like Yahoo Pipes, and mash it up with
other data.
11. The simplest way to expose data in a webapp is to supply feeds in machine-readable
formats. Dopplr gives its users a âmy tripsâ feed. As a newsfeed, itâs not all that useful as it
tells you things you already know (the trips that youâve added). As a lightweight read-only
API, it works very well. We mark up our feed with all the appropriate machine-readable
formats that we can ďŹnd: GeoRSS, hCalendar and Google Calendar GData.
This means itâs trivial to plug it into other systems like Yahoo Pipes, and mash it up with
other data.
12. Itâs also easy to transform our data into KML and view it in Google Earth.
17. If we want to integrate your Dopplr data with your data from elsewhere, we need to know
who you are on other sites.
18. User Identity
If we want to integrate your Dopplr data with your data from elsewhere, we need to know
who you are on other sites.
19. DOPPLR
DOPPLR
DOPPLR
People have different identities spread across many sites.
Where next?
Where next?
Where next?
20. OpenID is the obvious solution to this mess - if you use the same URL everywhere then sites
sharing data can be sure that you are the same person.
Itâs not just for login. OpenID can also be used to prove you own an identity. You may not
want to use your AOL Instant Messenger identity as a login, but Dopplr can use the AOL
OpenID provider to let you prove what your AIM ID is. This lets us write an AIM bot that talks
to you by IM secure in the knowledge that weâre not leaking your information to an impostor.
21. OpenID is the obvious solution to this mess - if you use the same URL everywhere then sites
sharing data can be sure that you are the same person.
Itâs not just for login. OpenID can also be used to prove you own an identity. You may not
want to use your AOL Instant Messenger identity as a login, but Dopplr can use the AOL
OpenID provider to let you prove what your AIM ID is. This lets us write an AIM bot that talks
to you by IM secure in the knowledge that weâre not leaking your information to an impostor.
22. âOpenId is the
game changer for
social networks,
allowing for
portability of the
social graph and
preferences.â
âSimon Willison
Š2007 Julian Cash
ďŹickr.com/photos/juliancash/673891099/
For more about OpenID, read Simon Willisonâs many talks and writings. He knows.
23. Once we know who you are on other sites, weâd like to help you import your social network
from those sites to Dopplr.
24. Social Network
Once we know who you are on other sites, weâd like to help you import your social network
from those sites to Dopplr.
25. For example, Twitter mark up their contact lists with XFN and hCard. Using the heuristic of âif
you use the same username and real name on two systems, youâre probably the same
personâ, we can show our users a suggested list of users who might be the same on Twitter
and Dopplr.
26. http://mofo.rubyforge.org
http://code.whytheluckystiff.net/hpricot/
We use the Mofo library to parse microformats, and Hpricot to scrape raw HTML.
27. class Traveller < ActiveRecord::Base
matches_identities
end
http://identity-matcher.googlecode.com/
Weâre releasing the code we use to match identities across sites as a Rails plugin.
29. >> Traveller.match_twitter(quot;factoryjoequot;)
[0].map(&:name)
http://identity-matcher.googlecode.com/
This is an example of running the âmatch_twitterâ call on Chris Messinaâs proďŹle.
30. >> Traveller.match_twitter(quot;factoryjoequot;)
[0].map(&:name)
=> [quot;Alexander Ljungquot;, quot;George Kellyquot;, quot;Amy
Raymondquot;, quot;Andrew Crowquot;, quot;Jonathan
Greenequot;, quot;Aubrey Sabalaquot;, quot;Matt Jonesquot;,
quot;Blaine Cookquot;, quot;Brian Oberkirchquot;, quot;Buzz
Andersenquot;, quot;David Ulevitchquot;, quot;Michael
BufďŹngtonquot;, quot;Eric Costelloquot;, quot;Kaustubh
Srikanthquot;, quot;Brian Del Vecchioquot;, quot;Jack Dorseyquot;,
quot;joshua schachterquot;, quot;Kevin Lawverquot;, quot;Scott
Bealequot;, quot;Lisa McMillanquot;, quot;Mary Hodderquot;, quot;Dan
Safferquot;, quot;Rob Hayesquot;, quot;Thomas Vander Walquot;,
quot;James Walkerquot;]
http://identity-matcher.googlecode.com/
This is an example of running the âmatch_twitterâ call on Chris Messinaâs proďŹle.
31. âthe best way for you
to manage your
network is to stop
thinking about all of
the little pieces and
to start focusing on
the big pictureâ
As Gavin Bell has been telling us for a while, the tools for social network portability are
reaching maturity. David Recordon (pictured) is working on open systems at Six Apart to
aggregate and share social network information.
32. To make interesting mashups, we often need to delegate authority to 3rd-party automated
services. Code running elsewhere needs to have access to usersâ accounts to act on their
behalf.
33. Delegating Authority
To make interesting mashups, we often need to delegate authority to 3rd-party automated
services. Code running elsewhere needs to have access to usersâ accounts to act on their
behalf.
34. The worst way to do this (weâre guilty too) is for the 3rd-party to ask for the users login
details and fake a login as them. This is not only bad for privacy but also teaches users
terrible habits.
35. BBAuth
AuthSub
Authentication
OpenAuth
There are several standards for delegating API access to a 3rd-party but they all work slightly
differently.
36. âAn open protocol to
allow secure API
authentication in a
simple and standard
method from desktop
and web applications.â
http://oauth.net
So the OAuth project is standardising a single protocol that everyone can use. This will mean
one client library per language (rather than one per language per site) and an easier
landscape for users to understand.
37. There are many ways to get a piece of screen real-estate on other sites.
39. Dopplr has a Facebook app that puts a Dopplr proďŹle box on your proďŹle page. It doesnât try
to cram the Dopplr user experience into a small box; it just displays a useful summary of
your travel information.
41. The Facebook F8 platform is actually very nice to work with. There are a few gotchas:
1. Pages served under apps.facebook.com are proxied directly to your server at request time.
This makes development versions a bit of a pain as your dev server must be visible on the
internet. We do this with an SSH tunnel.
2. FBML is a subset of XHTML with some extra elements in the Facebook namespace.
Facebook rewrite all your IDs and classes in the HTML and CSS so that you canât accidentally
change another appâs style.
42. proxies directly to your server
The Facebook F8 platform is actually very nice to work with. There are a few gotchas:
1. Pages served under apps.facebook.com are proxied directly to your server at request time.
This makes development versions a bit of a pain as your dev server must be visible on the
internet. We do this with an SSH tunnel.
2. FBML is a subset of XHTML with some extra elements in the Facebook namespace.
Facebook rewrite all your IDs and classes in the HTML and CSS so that you canât accidentally
change another appâs style.
43. proxies directly to your server
rewrites your HTML, CSS and JS
The Facebook F8 platform is actually very nice to work with. There are a few gotchas:
1. Pages served under apps.facebook.com are proxied directly to your server at request time.
This makes development versions a bit of a pain as your dev server must be visible on the
internet. We do this with an SSH tunnel.
2. FBML is a subset of XHTML with some extra elements in the Facebook namespace.
Facebook rewrite all your IDs and classes in the HTML and CSS so that you canât accidentally
change another appâs style.
44. proxies directly to your server
rewrites your HTML, CSS and JS
caches what it can
The Facebook F8 platform is actually very nice to work with. There are a few gotchas:
1. Pages served under apps.facebook.com are proxied directly to your server at request time.
This makes development versions a bit of a pain as your dev server must be visible on the
internet. We do this with an SSH tunnel.
2. FBML is a subset of XHTML with some extra elements in the Facebook namespace.
Facebook rewrite all your IDs and classes in the HTML and CSS so that you canât accidentally
change another appâs style.
45. proxies directly to your server
rewrites your HTML, CSS and JS
caches what it can
needs you to respond quickly
The Facebook F8 platform is actually very nice to work with. There are a few gotchas:
1. Pages served under apps.facebook.com are proxied directly to your server at request time.
This makes development versions a bit of a pain as your dev server must be visible on the
internet. We do this with an SSH tunnel.
2. FBML is a subset of XHTML with some extra elements in the Facebook namespace.
Facebook rewrite all your IDs and classes in the HTML and CSS so that you canât accidentally
change another appâs style.
47. We also have a blog badge that follows the same style as the Facebook proďŹlebox.
48.
49.
50. <div id=quot;dopplr-blog-badgequot;>
<script
src=quot;dopplr.com/blogbadge/...quot;>
</script>
</div>
The user only has to paste a tiny piece of code into their template to enable it. It looks for its
div and insert HTML and CSS there.
51. âJavascript is the
extra layer above the
mark-up âwhat is this
textâ and the CSS
âhow should it be
displayedâ. It adds a
new dimension,
âhow should this
element behave.ââ
âChristian Heilmann
This follows the principle of unobstrusive Javascript.
52. (function(i) {u = navigator.userAgent; e = /*@cc_on!@*/false; st = setTimeout;
if(/webkit/i.test(u)){st(function(){dr = document.readyState;if(dr==quot;loadedquot;||
dr==quot;completequot;){i();}else{st(arguments.callee,10);}},10);}else if((/mozilla/i.
test(u)&&!/(compati)/.test(u))||(/opera/i.test(u))){document.addEventListener
(quot;DOMContentLoadedquot;,i,false);} else if(e){(function(){t=document.createElement
('doc:ready');try{t.doScroll('left');i();t= null;}catch(e){st(arguments.callee,
0);}})();}else{window.onload = i;}})(function() {
// make the badge
});
http://www.kryogenix.org/days/2007/09/26/shortloaded
We also avoid the need for pulling in a full Javascript library such as jQuery just to get a
proper âpage is loadedâ event, using this code.
55. Weâre currently experimenting with using Amazon S3 for MySQL backups, and EC2 to run a
complete copy of the Dopplr application and a MySQL slave replicated over an SSH tunnel
from live. This is looking like a good way to scale, and itâs a great way to run stats reports
and live backups without disturbing our main servers.
56. S3 Data Storage
Weâre currently experimenting with using Amazon S3 for MySQL backups, and EC2 to run a
complete copy of the Dopplr application and a MySQL slave replicated over an SSH tunnel
from live. This is looking like a good way to scale, and itâs a great way to run stats reports
and live backups without disturbing our main servers.
57. S3 Data Storage
Pre-built EC2 Rails
Weâre currently experimenting with using Amazon S3 for MySQL backups, and EC2 to run a
complete copy of the Dopplr application and a MySQL slave replicated over an SSH tunnel
from live. This is looking like a good way to scale, and itâs a great way to run stats reports
and live backups without disturbing our main servers.
58. S3 Data Storage
Pre-built EC2 Rails
MySQL Slave
Weâre currently experimenting with using Amazon S3 for MySQL backups, and EC2 to run a
complete copy of the Dopplr application and a MySQL slave replicated over an SSH tunnel
from live. This is looking like a good way to scale, and itâs a great way to run stats reports
and live backups without disturbing our main servers.
59.
60. Sharing Data
User Identity
Social Network
Delegating Authority
Widgets & Plugins
Utility Computing
61. Thank you
Matt Biddulph
DOPPLR
DOPPLR
DOPPLR
Where next?