Google Developer Day 2007 - Nestoria: From Mash-up To Business

this space left intentionally blank

why perl is awesome and all other languages suck Mike Astle flamebait@lokku.com

from mash-up to business Mike Astle - Lokku Ltd. Google Developer Day 31 May 2007 - London

i like questions

Milestones • Start dev April 2006 • Launch London June 2006 • Launch UK Sept 2006 • Launch Spain May 2007

Numbers • In the UK, have 850,000 listings refreshed continually • Boils down to about 500,000 properties • 100,000+ users per month • 4 programmers • No dedicated operations staff

Houses on a map…so what?

this is harder than it looks

The Plan • Talk about some of the technical decisions we made at Nestoria • Go over a few challenges that we have faced • Hopefully there are lessons to learn from our experiences

first, you design

Things to worry about • Architecture • Services vs. APIs • How to scale? • i18n • Future products

then you choose a platform

MS vs. Everything Else

That Leaves LAMP

Ruby? LAMR?

Python?

Perl Utility Belt == CPAN

Apache

GNU/Linux (Debian)

now choose a development environment

Source Control • SVN…duh • Does anybody use CVS anymore? • Love the trac

Coders • A few veterans • As many interns / junior coders as fit in the room

Highly Collaborative (but not XP or Agile)

Emphasis • Collective decision-making • Individual responsibility for deadlines • Many small success rather than few large projects

Why write tests? • Find bugs, of course • Increase confidence to make changes to unfamiliar systems or libraries • Promote good design / expose bad design

Design • Informal process (no UML, minimal design patterns) • Concentrate on the things that will really hurt - e.g. i18n • Not everything has to be an object

Documentation • POD for interfaces • twiki for systems • Schedule regular refreshers

Outsource • If you can spec it faster than you can write it, consider farming it out • Many, many site full of hungry coders • Have had good results with scriptlance.com but there are lots of others out there

how to deploy?

Virtualization? • Sounds like voodoo to me • Extra layer to worry about • Easy enough to do everything in user space

Wigwam • 12 subdomains • Live data • 22 services • Many playpens per box • Releases several times a week

big ISP vs. small ISP • Bigger - more consistent • Smaller - more personal • Bigger - discrete choices • Smaller - continuous advice • We went small

Hardware • Know your downtime limits • Be reasonable! • Load balancing - expensive and complex - we do it with DNS through a vendor • Have a failure plan

The Hawkeye Principle

but is anybody using it?

Logging • Need your own, and it needs to be flexible • Use the free tools as a sanity check

Reporting • An important area of expertise, not an annoying afterthought • Glossary • Every mean needs a variance • A/B Testing (a.k.a Bucket Testing) is hard to do right

Also… • Build tools for ad hoc queries, they >will< get used • Keep an audit trail of changes • Effective tools for drilling down on sessions allows non-techs to help debugging

problems we have solved

http://www.nestoria.co.uk/covent- garden/flat/buy/bedrooms-2/maxprice- 1000000/minprice-250000/bathrooms-2

• Alderly Edge => Alderley Edge • Bayswater London => Bayswater • Berkhampsted => Berkhamsted • Birminham => Birmingham • Bletchingly => Bletchingley • Broxbourne Station => Broxbourne • Cambuslank => Cambuslang • Canary Warf => Canary Wharf • Claphamjuction => Clapham Junction • Covent Garden London => Covent Garden

Geocoding • Transform address into lat/long coordinates • 1,000+ calls a minute • Very high data variance (few cache hits) • Royal Mail data for UK • Teleatlas data for Spain

• Albert Villas, Gilbert Mews, LEIGHTON BUZZARD, • Bedfordshire, LU7 1NF • Lake Lock Drive, Wakefield West Yorkshire • Church Lane, London • C ELMINGTON ROAD, LONDON • Fountain Road (Flat 2), Edgbaston • London SE18, UK • Lee, London, UK • Free Trade Wharf, 340 The Highway, Wapping/Limehouse London • Stunning 3 Storey Contemporary Barn Conversion In The Popular Village Of Utkinton • A.4-1 GREAT NORTHERN TOWER,WATSON STREET,CITY CENTRE, • MANCHESTER GREATER MANCHESTER • 15, ASTLEY HOUSE, LONDON

(((^[BEGLMNS][1-9]d?) | (^W[2-9] ) | ( ^( A[BL] | B[ABDHLNRST] | C[ABFHMORTVW] | D[ADEGHLNTY] | E[HNX] | F[KY] | G[LUY] | H[ADGPRSUX] | I[GMPV] | JE | K[ATWY] | L[ADELNSU] | M[EKL] | N[EGNPRW] | O[LX] | P[AEHLOR] | R[GHM] | S[AEGKL-PRSTWY] | T[ADFNQRSW] | UB | W[ADFNRSV] | YO | ZE ) dd?) | (^W1[A-HJKSTUW0-9]) | (( (^WC[1-2]) | (^EC[1-4]) | (^SW1) ) [ABEHMNPRVWXY] ) ) (s*)? ([0-9][ABD-HJLNP-UW-Z]{2}))

what next?

Future Projects • Expose more data via API (e.g. historical aggregate data) • Implicit personalisation (the system learns from users) • More international expansion • Systemic testing to improve user experience

like what you see?

http://www.lokku.com/jobs

thanks! astle@lokku.com http://www.lokku.com http://www.nestoria.co.uk http://www.nestoria.es http://www.lokku.com/lpw06_geocoding/ http://www.lokku.com/lpw06_wigwam/

Google Developer Day 2007 - Nestoria: From Mash-up To Business

by lokku

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 12

Statistics