this space left intentionally blank
why perl is awesome and
all other languages suck
         Mike Astle
    flamebait@lokku.com
from mash-up to business

    Mike Astle - Lokku Ltd.
    Google Developer Day
    31 May 2007 - London
i like questions
Milestones
•   Start dev April 2006
•   Launch London June 2006
•   Launch UK Sept 2006
•   Launch Spain May 2007
Numbers
• In the UK, have 850,000 listings
  refreshed continually
• Boils down to about 500,000 properties
• 100,000+ use...
Houses on a map…so what?
this is harder than it looks
The Plan
• Talk about some of the technical
  decisions we made at Nestoria
• Go over a few challenges that we have
  face...
first, you design
Things to worry about
•   Architecture
•   Services vs. APIs
•   How to scale?
•   i18n
•   Future products
then you choose a platform
MS vs. Everything Else
Java?
That Leaves LAMP
PHP?
Ruby?

LAMR?
Python?
Perl
Perl


Utility Belt == CPAN
Apache
MySQL
GNU/Linux
 (Debian)
now choose a
development environment
Source Control
• SVN…duh
• Does anybody use CVS anymore?
• Love the trac
Coders
• A few veterans
• As many interns / junior coders as fit in
  the room
Highly Collaborative
(but not XP or Agile)
Emphasis
• Collective decision-making
• Individual responsibility for deadlines
• Many small success rather than few
  lar...
Why write tests?
• Find bugs, of course
• Increase confidence to make changes
  to unfamiliar systems or libraries
• Promo...
Design
• Informal process (no UML, minimal
  design patterns)
• Concentrate on the things that will really
  hurt - e.g. i...
Documentation
• POD for interfaces
• twiki for systems
• Schedule regular refreshers
Outsource
• If you can spec it faster than you can
  write it, consider farming it out
• Many, many site full of hungry co...
how to deploy?
Virtualization?
• Sounds like voodoo to me
• Extra layer to worry about
• Easy enough to do everything in user
  space
Wigwam


•   12 subdomains
•   Live data
•   22 services
•   Many playpens per box
•   Releases several times a week
big ISP vs. small ISP
•   Bigger - more consistent
•   Smaller - more personal
•   Bigger - discrete choices
•   Smaller -...
Hardware
• Know your downtime limits
• Be reasonable!
• Load balancing - expensive and
  complex - we do it with DNS throu...
The
Hawkeye
Principle
but is anybody using it?
Logging
• Need your own, and it needs to be
  flexible
• Use the free tools as a sanity check
Reporting
• An important area of expertise, not an
  annoying afterthought
• Glossary
• Every mean needs a variance
• A/B ...
Also…
• Build tools for ad hoc queries, they
  >will< get used
• Keep an audit trail of changes
• Effective tools for dril...
problems we have solved
http://www.nestoria.co.uk/covent-
 garden/flat/buy/bedrooms-2/maxprice-
1000000/minprice-250000/bathrooms-2
•   Alderly Edge => Alderley Edge
•   Bayswater London => Bayswater
•   Berkhampsted => Berkhamsted
•   Birminham => Birmi...
Geocoding
• Transform address into lat/long
  coordinates
• 1,000+ calls a minute
• Very high data variance (few cache hit...
•   Albert Villas, Gilbert Mews, LEIGHTON BUZZARD,
•   Bedfordshire, LU7 1NF
•   Lake Lock Drive, Wakefield West Yorkshire...
(((^[BEGLMNS][1-9]d?) | (^W[2-9] ) | ( ^( A[BL] | B[ABDHLNRST] |
C[ABFHMORTVW] | D[ADEGHLNTY] | E[HNX] | F[KY] | G[LUY] | ...
what next?
Future Projects
• Expose more data via API (e.g.
  historical aggregate data)
• Implicit personalisation (the system
  lea...
like what you see?
http://www.lokku.com/jobs
thanks!

            astle@lokku.com
          http://www.lokku.com
        http://www.nestoria.co.uk
         http://www....
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Google Developer Day 2007 - Nestoria: From Mash-up To Business
Upcoming SlideShare
Loading in...5
×

Google Developer Day 2007 - Nestoria: From Mash-up To Business

3,326

Published on

A case study of Nestoria. Discuss the difference between a simple mash-up and a business that runs on mash-ups. Also touch on some challenges we have faced.

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
3,326
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
84
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Google Developer Day 2007 - Nestoria: From Mash-up To Business

  1. 1. this space left intentionally blank
  2. 2. why perl is awesome and all other languages suck Mike Astle flamebait@lokku.com
  3. 3. from mash-up to business Mike Astle - Lokku Ltd. Google Developer Day 31 May 2007 - London
  4. 4. i like questions
  5. 5. Milestones • Start dev April 2006 • Launch London June 2006 • Launch UK Sept 2006 • Launch Spain May 2007
  6. 6. Numbers • In the UK, have 850,000 listings refreshed continually • Boils down to about 500,000 properties • 100,000+ users per month • 4 programmers • No dedicated operations staff
  7. 7. Houses on a map…so what?
  8. 8. this is harder than it looks
  9. 9. The Plan • Talk about some of the technical decisions we made at Nestoria • Go over a few challenges that we have faced • Hopefully there are lessons to learn from our experiences
  10. 10. first, you design
  11. 11. Things to worry about • Architecture • Services vs. APIs • How to scale? • i18n • Future products
  12. 12. then you choose a platform
  13. 13. MS vs. Everything Else
  14. 14. Java?
  15. 15. That Leaves LAMP
  16. 16. PHP?
  17. 17. Ruby? LAMR?
  18. 18. Python?
  19. 19. Perl
  20. 20. Perl Utility Belt == CPAN
  21. 21. Apache
  22. 22. MySQL
  23. 23. GNU/Linux (Debian)
  24. 24. now choose a development environment
  25. 25. Source Control • SVN…duh • Does anybody use CVS anymore? • Love the trac
  26. 26. Coders • A few veterans • As many interns / junior coders as fit in the room
  27. 27. Highly Collaborative (but not XP or Agile)
  28. 28. Emphasis • Collective decision-making • Individual responsibility for deadlines • Many small success rather than few large projects
  29. 29. Why write tests? • Find bugs, of course • Increase confidence to make changes to unfamiliar systems or libraries • Promote good design / expose bad design
  30. 30. Design • Informal process (no UML, minimal design patterns) • Concentrate on the things that will really hurt - e.g. i18n • Not everything has to be an object
  31. 31. Documentation • POD for interfaces • twiki for systems • Schedule regular refreshers
  32. 32. Outsource • If you can spec it faster than you can write it, consider farming it out • Many, many site full of hungry coders • Have had good results with scriptlance.com but there are lots of others out there
  33. 33. how to deploy?
  34. 34. Virtualization? • Sounds like voodoo to me • Extra layer to worry about • Easy enough to do everything in user space
  35. 35. Wigwam • 12 subdomains • Live data • 22 services • Many playpens per box • Releases several times a week
  36. 36. big ISP vs. small ISP • Bigger - more consistent • Smaller - more personal • Bigger - discrete choices • Smaller - continuous advice • We went small
  37. 37. Hardware • Know your downtime limits • Be reasonable! • Load balancing - expensive and complex - we do it with DNS through a vendor • Have a failure plan
  38. 38. The Hawkeye Principle
  39. 39. but is anybody using it?
  40. 40. Logging • Need your own, and it needs to be flexible • Use the free tools as a sanity check
  41. 41. Reporting • An important area of expertise, not an annoying afterthought • Glossary • Every mean needs a variance • A/B Testing (a.k.a Bucket Testing) is hard to do right
  42. 42. Also… • Build tools for ad hoc queries, they >will< get used • Keep an audit trail of changes • Effective tools for drilling down on sessions allows non-techs to help debugging
  43. 43. problems we have solved
  44. 44. http://www.nestoria.co.uk/covent- garden/flat/buy/bedrooms-2/maxprice- 1000000/minprice-250000/bathrooms-2
  45. 45. • Alderly Edge => Alderley Edge • Bayswater London => Bayswater • Berkhampsted => Berkhamsted • Birminham => Birmingham • Bletchingly => Bletchingley • Broxbourne Station => Broxbourne • Cambuslank => Cambuslang • Canary Warf => Canary Wharf • Claphamjuction => Clapham Junction • Covent Garden London => Covent Garden
  46. 46. Geocoding • Transform address into lat/long coordinates • 1,000+ calls a minute • Very high data variance (few cache hits) • Royal Mail data for UK • Teleatlas data for Spain
  47. 47. • Albert Villas, Gilbert Mews, LEIGHTON BUZZARD, • Bedfordshire, LU7 1NF • Lake Lock Drive, Wakefield West Yorkshire • Church Lane, London • C ELMINGTON ROAD, LONDON • Fountain Road (Flat 2), Edgbaston • London SE18, UK • Lee, London, UK • Free Trade Wharf, 340 The Highway, Wapping/Limehouse London • Stunning 3 Storey Contemporary Barn Conversion In The Popular Village Of Utkinton • A.4-1 GREAT NORTHERN TOWER,WATSON STREET,CITY CENTRE, • MANCHESTER GREATER MANCHESTER • 15, ASTLEY HOUSE, LONDON
  48. 48. (((^[BEGLMNS][1-9]d?) | (^W[2-9] ) | ( ^( A[BL] | B[ABDHLNRST] | C[ABFHMORTVW] | D[ADEGHLNTY] | E[HNX] | F[KY] | G[LUY] | H[ADGPRSUX] | I[GMPV] | JE | K[ATWY] | L[ADELNSU] | M[EKL] | N[EGNPRW] | O[LX] | P[AEHLOR] | R[GHM] | S[AEGKL-PRSTWY] | T[ADFNQRSW] | UB | W[ADFNRSV] | YO | ZE ) dd?) | (^W1[A-HJKSTUW0-9]) | (( (^WC[1-2]) | (^EC[1-4]) | (^SW1) ) [ABEHMNPRVWXY] ) ) (s*)? ([0-9][ABD-HJLNP-UW-Z]{2}))
  49. 49. what next?
  50. 50. Future Projects • Expose more data via API (e.g. historical aggregate data) • Implicit personalisation (the system learns from users) • More international expansion • Systemic testing to improve user experience
  51. 51. like what you see?
  52. 52. http://www.lokku.com/jobs
  53. 53. thanks! astle@lokku.com http://www.lokku.com http://www.nestoria.co.uk http://www.nestoria.es http://www.lokku.com/lpw06_geocoding/ http://www.lokku.com/lpw06_wigwam/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×