1. Building Things Fast
and getting approval
Simon Willison, An Event Apart Chicago 2009

2. Web development
four years ago
sucked

3. This talk
• Individual techniques and tools
• From one developer to a group
• From a group to an organisation

4. • Because the tools have never been better
• Because the sooner you get your stuff in
front of users the sooner you’ll realise
how wrong you were
• Because it’s cheaper
• Because it’s (a lot) more fun
Why?

5. 1 hour kick off meeting
x 6 people
6 man-hours
4 hours prototying
x 1 developer
4 man-hours

6. • Interactive development
• Client-side prototyping
• Web frameworks
• Web APIs
• Open source
Rapid development trends

7. Interactive
development

8. Have a good idea if
your code will work
before you even
save it to a file

11. JavaScript Firebug
CSS “edit styles” bookmarklet
Python python, ipython
Ruby irb
PHP phpsh (by Facebook)
Java BeanShell
Perl Devel::REPL
Search for “language REPL” or “language shell”

12. http://xkcd.com/303/

13. Client-side
prototyping

14. Server-side =
deployment headaches

15. The same
origin policy
http://en.wikipedia.org/wiki/Same_origin_policy

16. Evil website
Your bank
iframe
JavaScript

17. JSON-P
JSON with Padding

18. http://twitter.com/statuses/user_timeline/simonw.json?callback=foo
foo([
{
"favorited": false,
"in_reply_to_status_id": 4420410018,
"in_reply_to_screen_name": "andrewgodwin",
"in_reply_to_user_id": 8749512,
"source": "web",
"text": "@andrewgodwin CMS that do full revision tracking are
pretty useful - I've had success with MediaWiki as a CMS backend
in the past",
"created_at": "Sun Sep 27 17:31:24 +0000 2009",
"truncated": false,
"id": 4421168167,
"user": {
"id": 12497,
"verified": false,
"profile_sidebar_fill_color": "FFFFFF",
"profile_text_color": "000000",
"followers_count": 5664,
"protected": false,

19. LEARN JQUERY NOW!
is a new kind of JavaScript Library.
jQuery is a fast and concise JavaScript Library that simplifies HTML
document traversing, event handling, animating, and Ajax interactions for
rapid web development. jQuery is designed to change the way that you
write JavaScript.
PRODUCTION (19KB, Minified and Gzipped)
DEVELOPMENT (120KB, Uncompressed Code)
Current Release: v.1.3.2
GRAB THE LATEST VERSION!
CHOOSE YOUR COMPRESSION LEVEL:
jQuery Plugins UI Blog About Donate
Download Documentation Tutorials Bug Tracker Discussion
Lightweight Footprint CSS3 Compliant Cross-browser
WHO'S USING JQUERY?
JQUERY RESOURCES

20. jQuery.getJSON(
“http://twitter.com/statuses/
user_timeline/simonw.json?callback=?”,
function(data) {
alert(data);
});

21. http://j.mp/afeedapart

22. <div id="feed">
<ul>
<li class="tweet group" id="template">
<a href="http://twitter.com/simonw"
title="simonw"
style="background-image: url('http://
a1.twimg.com/profile_images/408476120/django-
cowboy_normal.jpg');"
class="profile-image"></a>
<div class="tweet-text">
<span>Here come the tweets...</span>
<div class="posted">posted
<span>0 minutes ago</span> by
<a href="http://twitter.com/simonw">
simonw</a>
</div>
</div>
</li>
</ul>
</div>

23. var search_url = 'http://search.twitter.com/
search.json?q=%23aea&callback=?';
var last_tweet_id = null;
jQuery(function($) {
var template = $('#template').clone();
template.attr('id', '');
$('#activity-indicator').hide();
function refresh() {
var url = search_url;
if (last_tweet_id) {
url += '&since_id=' + last_tweet_id;
}
$('#activity-indicator').show();
jQuery.getJSON(url, function(data) {
$('#template').remove();
$('#activity-indicator').hide();
last_tweet_id = data.max_id;
$('#feed li').removeClass('new-tweet');
data.results.reverse();

24. var search_url = 'http://search.twitter.com/
search.json?q=%23aea&callback=?';
var last_tweet_id = null;
jQuery(function($) {
var template = $('#template').clone();
template.attr('id', '');
$('#activity-indicator').hide();
function refresh() {
var url = search_url;
if (last_tweet_id) {
url += '&since_id=' + last_tweet_id;
}
$('#activity-indicator').show();
jQuery.getJSON(url, function(data) {
$('#template').remove();
$('#activity-indicator').hide();
last_tweet_id = data.max_id;
$('#feed li').removeClass('new-tweet');
data.results.reverse();

25. jQuery.getJSON(url, function(data) {
$('#template').remove();
$('#activity-indicator').hide();
last_tweet_id = data.max_id;
$('#feed li').removeClass('new-tweet');
data.results.reverse();
$.each(data.results, function() {
var tweet = template.clone();
tweet.find(
'.tweet-text span:first'
).html(this.text);
tweet.find('a:first').css(
'background-image',
'url('+this.profile_image_url+')'
);
// ...
tweet.hide();
$('#feed ul').prepend(tweet);
tweet.slideDown();
});
});

26. jQuery.getJSON(url, function(data) {
$('#template').remove();
$('#activity-indicator').hide();
last_tweet_id = data.max_id;
$('#feed li').removeClass('new-tweet');
data.results.reverse();
$.each(data.results, function() {
var tweet = template.clone();
tweet.find(
'.tweet-text span:first'
).html(this.text);
tweet.find('a:first').css(
'background-image',
'url('+this.profile_image_url+')'
);
// ...
tweet.hide();
$('#feed ul').prepend(tweet);
tweet.slideDown();
});
});

27. jQuery.getJSON(url, function(data) {
$('#template').remove();
$('#activity-indicator').hide();
last_tweet_id = data.max_id;
$('#feed li').removeClass('new-tweet');
data.results.reverse();
$.each(data.results, function() {
var tweet = template.clone();
tweet.find(
'.tweet-text span:first'
).html(this.text);
tweet.find('a:first').css(
'background-image',
'url('+this.profile_image_url+')'
);
// ...
tweet.hide();
$('#feed ul').prepend(tweet);
tweet.slideDown();
});
});

28. jQuery.getJSON(url, function(data) {
$('#template').remove();
$('#activity-indicator').hide();
last_tweet_id = data.max_id;
$('#feed li').removeClass('new-tweet');
data.results.reverse();
$.each(data.results, function() {
var tweet = template.clone();
tweet.find(
'.tweet-text span:first'
).html(this.text);
tweet.find('a:first').css(
'background-image',
'url('+this.profile_image_url+')'
);
// ...
tweet.hide();
$('#feed ul').prepend(tweet);
tweet.slideDown();
});
});

29. jQuery.getJSON(url, function(data) {
$('#template').remove();
$('#activity-indicator').hide();
last_tweet_id = data.max_id;
$('#feed li').removeClass('new-tweet');
data.results.reverse();
$.each(data.results, function() {
var tweet = template.clone();
tweet.find(
'.tweet-text span:first'
).html(this.text);
tweet.find('a:first').css(
'background-image',
'url('+this.profile_image_url+')'
);
// ...
tweet.hide();
$('#feed ul').prepend(tweet);
tweet.slideDown();
});
});

30. jQuery, JSON-P and
45 lines of JavaScript

31. HOME DOCUMENTATION MY PROJECTSsimonwillison Sign Out
YDN Yahoo! Query Language Documentation Console
Recent Queries
get my profile data
get my friends
get all my friends profiles
get my friends nicknames
get my last added friend
get my friends sorted by nickname
get 10 flickr "cat" photos
get a flickr photo by photo ID
get the flickr image url by photo ID
get san francisco geo data
get san francisco woeid
get geo details about san francisco
find sushi restaurants in san francisco
find sushi restaurants in san francisco wi
find events in north beach
find all upcoming events in north beach s
find the next upcoming event in north bea
get rss feed from yahoo news top stories
search the web using the title of the top
extract the headlines from yahoo finance
aggregate and filter multiple rss feeds
Example Queries
Show Community Tables
social
social.connections
social.connections.updates
social.contacts
social.presence
social.profile
social.profile.image
social.profile.status
social.updates
flickr
geo
local
maps
meme
music
mybloglog
Data Tables (94)
Whats this?
Your YQL Statement
show tables
XML JSON TEST
Permalink
The REST query
http://query.yahooapis.com/v1/public/y
ql?q=show%20tables&format=xml
How do I use this?
COPY URLFlash
TREE VIEW
<?xml version="1.0" encoding="UTF-8"?>
<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng" yahoo:count="94" yahoo:created="2009-09-27T06:03:36Z" yahoo:lang="
<diagnostics>
<publiclyCallable>true</publiclyCallable>
<user-time>1</user-time>
<service-time>0</service-time>
<build-version>3130</build-version>
</diagnostics>
<results>
<table>atom</table>
<table>csv</table>
<table>feed</table>
<table>flickr.photos.exif</table>
<table>flickr.photos.info</table>
<table>flickr.photos.interestingness</table>
<table>flickr.photos.recent</table>
<table>flickr.photos.search</table>
<table>flickr.photos.sizes</table>
<table>flickr.photosets.photos</table>
<table>flickr.places</table>
<table>flickr.places.info</table>
<table>geo.placemaker</table>
<table>geo.places</table>
<table>geo.places.ancestors</table>
<table>geo.places.belongtos</table>
<table>geo.places.children</table>
<table>geo.places.neighbors</table>
<table>geo.places.parent</table>
<table>geo.places.siblings</table>
<table>geo.placetypes</table>
FORMATTED VIEW
Version 1.0 | Copyright © 2009 Yahoo! Inc. All rights reserved. Copyright | Privacy Policy | Forum
Building a toolboxYQL

32. But...

33. Beware the
Phantom Web
Or JavaScript will destroy us all

34. Server-side
prototyping

35. Modern web frameworks
• Ruby on Rails, Symfony, CakePHP...
• “Full-stack” - everything you need
• DRY; eliminate boiler-plate
• Model layer, separate templates

37. “Web development on
journalism deadlines”

38. The MP
expenses
scandal

39. 2005 FOI request

46. • Prototype that became a product
• One week from concept to launch
• Design, ops, QA involvement
• Load tested “live” on EC2

47. • Database models / ORM
• Automatic admin interface
• Display / validate / redisplay forms
• Templates, and template inheritance
• Well designed URLs!
Django features

48. Simon Willison's Weblog
New entry / Comments
Home › Entries
Select entry to change
Welcome, simon. Documentation / Change password / Log out
Add entry
2002 2003 2004 2005 2006 2007 2008
2009
Go
Title Created Comments open
Django ponies: Proposals for Django 1.2 Sept. 28, 2009, 11:32 p.m.
Hack Day tools for non-developers July 28, 2009, 2:23 p.m.
Teaching users to be secure is a shared responsibility July 16, 2009, 8:04 p.m.
Facebook Usernames and OpenID June 13, 2009, 5:01 p.m.
djng - a Django powered microframework May 19, 2009, 12:13 a.m.
rev=canonical bookmarklet and designing shorter URLs April 11, 2009, 5:37 p.m.
List of SxSW 2009 panels with "social" in the title March 14, 2009, 11:02 p.m.
A few notes on the Guardian Open Platform March 10, 2009, 2:28 p.m.
Pragmatism, purity and JSON content types Feb. 6, 2009, 10:19 a.m.
Rate limiting with memcached Jan. 7, 2009, 10:27 p.m.
DjangoCon and PyCon UK Sept. 15, 2008, 3:20 p.m.
Announcing dmigrations Sept. 3, 2008, 7:17 p.m.
Back to full-time employment Aug. 22, 2008, 5:26 p.m.
The point of "Open" in OpenID June 24, 2008, 8:12 a.m.
Debugging Django May 22, 2008, 12:35 a.m.
jQuery style chaining with the Django ORM May 1, 2008, 12:31 p.m.
wikinear.com, OAuth and Fire Eagle March 22, 2008, 2:34 p.m.
Django People: OpenID and microformats Jan. 24, 2008, 2:02 a.m.
Django People Jan. 23, 2008, 2 a.m.
Yahoo!, Flickr, OpenID and Identity Projection Jan. 7, 2008, 11:33 p.m.
Comet works, and it's easier than you think Dec. 5, 2007, 4:22 p.m.
Figuring out OpenSocial Nov. 2, 2007, 10:29 a.m.
Questioning Steve Ballmer Oct. 1, 2007, 11:57 p.m.
Filter
By created
Any date
Today
Past 7 days
This month
This year
By comments open
All
Yes
No

49. Title:
Slug:
Body:
Created:
Django ponies: Proposals for Django 1.2
ponies
<p>I've decided to step up my involvement in Django development in the run-up to Django 1.2, so I'm
currently going through several years worth of accumulated pony requests figuring out which ones are worth
advocating for. I'm also ensuring I have the code to back them up - my innocent <a
href="http://code.djangoproject.com/wiki/AutoEscaping">AutoEscaping proposal</a> a few years ago
resulted in an enormous amount of work by Malcolm and I don't think he'd appreciate a repeat
performance.</p>
<p>I'm not a big fan of branches when it comes to exploratory development - they're fine for doing the final
implementation once an approach has been agreed, but I don't think they are a very effective way of
discussing proposals. I'd much rather see working code in a separate application - that way I can try it out
with an existing project without needing to switch to a new Django branch. Keeping code out of a branch also
means people can start using it for real development work, making the API much easier to evaluate. Most of
my proposals here have accompanying applications on GitHub.</p>
<p>I've recently got in to the habit of including an "examples" directory with each of my experimental
applications. This is a full Django project (with settings.py, urls.py and manage.py files) which serves two
purposes. Firstly, it allows developers to run the application's unit tests without needing to install it in to their
own pre-configured project, simply by changing in to the examples directory and running
<samp>./manage.py test</samp>. Secondly, it gives me somewhere to put demonstration code that can be
viewed in a browser using the runserver command - a further way of making the code easier to evaluate. <a
href="http://github.com/simonw/django-safeform">django-safeform</a> is a good example of this
pattern.</p>
<p>Here's my current list of ponies, in rough order of priority.</p>
<h4>Signing and signed cookies</h4>
<p>Signing strings to ensure they have not yet been tampered with is a crucial technique in web application
security. As with all cryptography, it's also surprisingly difficult to do correctly. <a
href="http://vnhacker.blogspot.com/2009/09/flickrs-api-signature-forgery.html">A vulnerability in the
signing implementation</a> used to protect the Flickr API was revealed just today.</p>
<p>One of the many uses of signed strings is to implement signed cookies. Signed cookies are fantastically
powerful - they allow you to send cookies safe in the knowledge that your user will not be able to alter them
without you knowing. This dramatically reduces the need for sessions - most web apps use sessions for
security rather than for storing large amounts of data, so moving that "logged in user ID" value to a signed
cookie eliminates the need for session storage entirely, saving a round-trip to persistent storage on every
request.</p>
<p>This has particularly useful implications for scaling - you can push your shared secret out to all of your
front end web servers and scale horizontally, with no need for shared session storage just to handle simple
authentication and "You are logged in as X" messages.</p>
Date: 2009-09-28 Today |
Time: 23:32:04 Now |
Comments open
Simon Willison's Weblog
New entry / Comments
Home › Entries › Django ponies: Proposals for Django 1.2
Change entry
Welcome, simon. Documentation / Change password / Log out
History View on site

50. Rapid
development
in a group

51. /dev/fort

55. e-mail
password
or OpenID
what is this?
desired
username
Register now
Already a member? » LOG IN
By registering with Wildlife Near You, you are
agreeing to our terms and conditions of use.
» Go for it
This is alpha.wildlifenearyou.com - please don't link to or Twitter this yet, we're still fixing up some loose ends
Welcome back!
Wildlife Near You is a site for you to share your
passion for animals. Search for a place you've
been, add in which animals you spotted, and add
any photos you have of them!
REGISTER LOGIN FEEDBACK
Try tigers, llamas near brighton, new zealand or your favourite animal
LOOK FOR » Search

56. General information
Address: Regent's Park, London, England, NW1 4RY, United Kingdom
(map)
Phone: 020 7722 3333
VISIT THEIR OFFICIAL WEBSITE
Animals you might see
Abyssinian Ground-hornbill
African Brush-tailed Porcupine African Penguin African Wild Dog
Alpaca American Black Vulture
Bactrian camel Bearded Pig
Black-and-white-casqued Hornbill
Black-cheeked Lovebird
See all species that have been spotted here.
Have you been here?
Average rating
Based on ratings of 3 trips by 3 spotters.
Have you been here? Why not add
your trip?
Have we got our facts right?
Suggest changes to our information
Photos
This is alpha.wildlifenearyou.com - please don't link to or Twitter this yet, we're still fixing up some loose ends
London Zoo
REGISTER LOGIN FEEDBACK
LOOK FOR search

57. It was great!
simon went to Les Géants du Ciel
in August 2009
Sightings
Griffon Vulture
Gyps fulvus
Barn Owl
Tyto alba
Eurasian Eagle Owl
Bubo bubo
Trip rating
Photos
Comments
You need to log in to post comments.
This is alpha.wildlifenearyou.com - please don't link to or Twitter this yet, we're still fixing up some loose ends
My trip to Les Géants du Ciel in August
2009
REGISTER LOGIN FEEDBACK
LOOK FOR search

58. owls near chicago
Search
Alternatively, search for “owls” everywhere, or everything near “chicago”.
owls near Chicago, IL, USA
Show results on a map
National Zoo
593 miles away, in Washington, United States. Visit their official
website.
Oregon Zoo
1757 miles away, in Portland, United States. Visit their official website.
Fowlmere RSPB Bird Sanctuary
3939 miles away, in Fowlmere, United Kingdom. Visit their official
website.
London Zoo
3949 miles away, in London, United Kingdom. Visit their official
Matching species
Spectacled Owl
Pulsatrix perspicillata
Burrowing Owl
Athene cunicularia
Tawny Owl
Strix aluco
Barn Owl
Tyto alba
Snowy Owl
Bubo scandiacus
More species matching “owls”…
This is alpha.wildlifenearyou.com - please don't link to or Twitter this yet, we're still fixing up some loose ends
Search: “owls” near “chicago”
REGISTER LOGIN FEEDBACK

59. Pair programming

60. The KJ method

61. Great for learning
Not so great for shipping

62. Making your
organisation
more hackable

64. Innovation

65. Hack days

66. Every Wednesday
is Hack Day
Dev Labs

67. APIs

68. RSS feeds

69. Screen scraping
(scrAPIs)

70. (You don’t have to use regular
expressions any more)

71. JavaScript YQL
Python BeautifulSoup, html5lib
Ruby hpricot + many others
PHP PHP Simple HTML DOM Parser

72. With YQL
select * from data.html.cssselect
where url="www.yahoo.com" and
css="#news a"
Example query: http://j.mp/IhF1b
http://developer.yahoo.net/blog/archives/2009/04/yql_execute.html

73. Open
source

74. ?

75. Open source is an ecosystem

76. Open source
• It’s (mostly) not about the ongoing price
• Free to evaluate
• No lockin
• No “we’ve paid for it, we have to use it”
• Awesome community and support*
* Depending on both the project and your own attitude

77. Should you release
your own code?

78. An Open Source project is for life, not just for Christmas

79. simonw Dashboard Inbox 28 Account Settings Log Out
Search GitHub…Explore GitHub Gist Blog Help
simonw / bugle_project
Description: Group collaboration tools for hackers in forts. edit
Homepage: Click to edit edit
Public Clone URL: git://github.com/simonw/bugle_project.git
Your Clone URL: git@github.com:simonw/bugle_project.git
Added BSD license file
simonw (author)
June 08, 2009
commit 741fff120b583c78d6860e28f9c51db618f48862
tree bd0268213a3db68168e987dac00aa1e01ff05522
parent 30528491762acf27fae07a36084e7e3189475ff8
history
bugle_project /
name age message
.gitignore May 31, 2009 local_manage.py is now supported to make the py... [simonw]
LICENSE.txt June 08, 2009 Added BSD license file [simonw]
README.txt June 08, 2009 Clarified dependencies [simonw]
__init__.py May 30, 2009 Created project [simonw]
bugle/ June 08, 2009 New shared _blasts.html template, re-implemente... [simonw]
ext/ May 30, 2009 Added ridiculously simple registration system, ... [simonw]
manage.py May 31, 2009 local_manage.py is now supported to make the py... [simonw]
registration/ June 08, 2009 Added reserved usernames [simonw]
screenshot.png June 08, 2009 Tweaked readme, added screenshot [simonw]
Source Commits Network (2) Fork Queue Issues (0) Downloads (0) Wiki (1) Graphs Admin
master all branches all tags
228

80. GitHub: open source
for developers who are
scared of commitment

81. Calls To Action!
• Use interactive development
• Build client-side prototypes with JSON-P
• Learn a modern web framework
• Hire a fort
• Organise hack days

82. Further reading
• http://simonwillison.net/
• http://news.ycombinator.com/
• http://programming.reddit.com/