Startup Architecture: How to Lean on Others to Get Stuff DoneUntitled

@robb1e
how to lean on others to get stuﬀ done
Software Engineer, Pivotal Labs
Robbie Clutton
Startup Architecture

InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/last-moment-design

Presented at QCon London
www.qconlondon.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

@robb1e
with Rails applications
Robbie Clutton

@robb1e
Simple, small web application
Service oriented RESTful/pubsub/LMAX/P2P
distributed architecture
Gray area

@robb1e
Our codebase is 5 years
old and too hard to change

@robb1e
We’ve allowed our design
to evolve into a big
ball of mud

@robb1e
We’ll probably create
services at some point,
might as well start there

@robb1e
I’m going to design
everything up front based on
unvaldiated assumptions

@robb1e
Kent Beck
Make it work,
make it right,
make it fast

@robb1e
• Features that have hypotheses
• Hypotheses that can be easily validated
• Code that is always production ready
• Code that is easy to change
Goals

@robb1e
Creating sustainable small architectures
Robbie Clutton

@robb1e
Real stories from
colleagues and myself

@robb1e
Names have been
changed to protect
the innocent

@robb1e
Some stories are
pre-production, others
are in production

@robb1e
Users clicking headers
that are not links

@robb1e
You could feel the
users frustration

@robb1e
Simple user testing can pay dividends
Lesson

@robb1e
• CrazyEgg.com
• UserTesting.com
• SliverbackApp.com
• LeanLaunchLab.com
• Trello.com
Tools

@robb1e
Funnels, user testing,
hypotheses and validations
Story

@robb1e
Product with wizard like
pages which pre-selected
default services

@robb1e
Changes to the basket
updates price in real-time

@robb1e
Funnel showed massive
drop oﬀ at a certain step

@robb1e
In person user testing
to discover why the
drop oﬀ was occurring

@robb1e
Create hypothesis to
stop users leaving at
this junction

@robb1e
Implement change:
allow users to use
default or create own

@robb1e
Review funnel
after deployment

@robb1e
Learn what is blocking the users
Lesson

@robb1e
• KissMetrics.com
• StatsD (Etsy)
• Cube (Square)
Tools

@robb1e
Always be validating
Take away

@robb1e
You’re gonna need
a bigger boat
Story

@robb1e
Dave walks into a new job
START-UP 3.0

@robb1e
We need more
RAM for the
database
Product manager tells you

@robb1e
This report takes
20 minutes
to run.

@robb1e
No primary or
foreign keys

@robb1e
Needed more RAM so
the whole database
could ﬁt in memory

@robb1e
Dave cleans up a bit,
report now takes 10
seconds to run

@robb1e
Use tools to discover simple mistakes
Lesson

@robb1e
Passing tests don’t
imply production quality
Bonus

@robb1e
• Rails Best Practices
• SQL Explain
• NewRelic.com
Tools

@robb1e
Instrument, refactor, repeat
Story

@robb1e
Client moving from
ColdFusion to Ruby

@robb1e
Yes, there are people
still using ColdFusion

@robb1e
Ruby is slow and
we’re going to
production next
week.

@robb1e
We’ve made a
terrible mistake...

@robb1e
You say...
Hold on a minute,
let’s take a look

@robb1e
Instrument to ﬁnd slow
requests/queries

@robb1e
Refactor slowest query
until more performant
with green tests

@robb1e
Rinse and repeat until
performance has
improved enough

@robb1e
Paul Hammond, 2012
Every scaling story:
1. Find the biggest problem
2. Fix the biggest problem
3. Repeat

@robb1e
‘Friday afternoon’ performance
refactoring can build upon itself
Bonus

@robb1e
• NewRelic.com
• CodeClimate.com
• Emma, FindBugs
Tools

@robb1e
Use tools to discover improvements
Take away

@robb1e
Distributed cache
Story

@robb1e
Website was growing
and gaining visitors

@robb1e
Scaling strategy was to
add app servers

@robb1e
Each server had the web
app and a local cache

@robb1e
Spinning up a new server
meant more pressure
on the database

@robb1e
Using a distributed cache
bought the team time to
make improvements

@robb1e
Caching can buy signiﬁcant
performance improvements
Lesson

@robb1e
• MemcacheD.org
• Varnish-Cache.org
• Squid-Cache.org
Tools

@robb1e
To cache, or not to cache?
Story

@robb1e
Sometimes code
speaks to you
Yo.

@robb1e
This part is slow,
let’s cache it.
Problem solved

@robb1e
But I’m going to
invalidate that
elsewhere

@robb1e
Collection of widgets
being rendered with
new and old design

@robb1e
Can’t replicate on
staging or locally

@robb1e
Changing the template had
not invalidated the entry

@robb1e
- Phil Karlton
"There are only two hard things in
Computer Science: cache
invalidation and naming things."

@robb1e
Caching can obsure
poorly written code
Bonus

@robb1e
Be careful what you cache
Take away

@robb1e
Non-essential work
during a request
Story

@robb1e
User registration
stopped working

@robb1e
Mailing list provider
was down

@robb1e
Exception bubbled up
and prevented
registering new user

@robb1e
Put mailing list
subscription in
background job

@robb1e
Shorten the request/response cycle
Lesson

@robb1e
When dealing with integrations, some
healthy paranoia is a good thing
Bonus

@robb1e
• Background workers
• Message Queues
• Threads
Tools

@robb1e
A tale of two websites
Story

@robb1e
www.guardian.co.uk
125 requests
1.2MB
HTML: 3.7s
Loaded: 8.4s

@robb1e
m.guardian.co.uk
44 requests
340KB
HTML: 1.68s
Loaded: 3.32s

@robb1e
That’s not the result of
better SQL or server
optimizations

@robb1e
Result of highly tuned
client-side Javascript
and CSS

@robb1e
No (large)
Javascript libraries

@robb1e
Conditional loading of
secondary content

@robb1e
- Steve Saunders, 2007
“Optimize front-end performance
ﬁrst, that's where 80% or more of
the end-user response time is
spent”

@robb1e
• Firebug
• Chrome Developer Tools
• Compass
• YSlow
• YUI Compressor
Tools

@robb1e
Perceived performance is more
important than actual performance
Take away

@robb1e
Was that really the
best use of your time?
Story

@robb1e
During technical due
diligence for an acquisition

@robb1e
The company had built
their own message queue

@robb1e
Didn’t use standard
protocol like AMPQ

@robb1e
Not explicitly sending a
terminating character
would eventually result
in the queue crashing

@robb1e
Almost all transactions
passed through this queue

@robb1e
Not buying a message
queue company

@robb1e
- Joel Spolsky, 2001
"If it's a core business
function - do it yourself,
no matter what."

@robb1e
Time is the most expensive out going
Bonus

@robb1e
Real-time vs near-time
Story

@robb1e
Trading system which
updates users’ screen
every 10 seconds

@robb1e
Lots of number crunching
and message queues

@robb1e
Did some in the
ﬁeld research

@robb1e
Traders only checked
values every few minutes

@robb1e
This was not high
volatile trading
http://www.boldjack.com/wp-content/uploads/2012/01/wall_street4.jpg

@robb1e
Removed message
queues and moved
to publishing updates
to web server directly

@robb1e
Reduced complexity
of the product

@robb1e
Ron Jeﬀeries, ~2005
Always implement things when
you actually need them, never
when you just foresee that you
need them

@robb1e
‘Real-time’ can mean diﬀerent things
depending on who you talk too
Bonus

@robb1e
$50 a month is really
expensive for this
hosted service

@robb1e
We can build it ourselves
and get exactly the
features we need

@robb1e
Can you build the
widget service yourself
in that time?

@robb1e
Are you in the
widget business?

@robb1e
Francis Hwang, 2012
The biggest expense for a startup
is your time. Not your laptop, not
your hosting bill, not your ofﬁce,
but the hours in your day.

@robb1e
Focus on your diﬀerentiators
Bonus

@robb1e
Over engineering is a form of waste
Take away

@robb1e
Horizontal Scalability
Story

@robb1e
Guardian Content API
is read only and
eventually consistent

@robb1e
Used by m., iPhone app,
parts of www. and more

@robb1e
Just a simple API over
an indexed data store

@robb1e
Each server has
it’s own data store

@robb1e
Each data store is a replica
of an internal master

@robb1e
Simple, elegant design can prevent
complex architecture creep
Lesson

@robb1e
• Solr
• Elastic Search
• MongoDB
Tools

@robb1e
Use of feature switches
at Guardian enable
‘super happy fun mode’

@robb1e
Turn features oﬀ when
site under increased load

@robb1e
Content is king and must
be readable at all times

@robb1e
Page pressing enables
zero downtime and
last fallback

@robb1e
Feature ﬂags can oﬀer resilience as
well as a way to roll out new features
Lesson

@robb1e
Complex should be lots of simple
Take away

@robb1e
Allow architecture to evolve
Spend your time wisely
Refactor continuously

@robb1e
Sandi Metz
“The wrong abstraction is far more
damaging than no abstraction at
all. Waiting trumps guessing every
time”

Startup Architecture: How to Lean on Others to Get Stuff DoneUntitled

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Startup Architecture: How to Lean on Others to Get Stuff DoneUntitled

Similar to Startup Architecture: How to Lean on Others to Get Stuff DoneUntitled (20)

More from C4Media

More from C4Media (20)

Recently uploaded

Recently uploaded (20)

Startup Architecture: How to Lean on Others to Get Stuff DoneUntitled