Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
How we Build Vox April 4 2007
Six Apart <ul><li>Movable Type </li></ul><ul><li>TypePad </li></ul><ul><li>LiveJournal </li></ul><ul><li>Vox </li></ul>
How we Build Vox: a Web 2.0, Large-scale, Fast, Internationalized website
How we Build Vox: a  Web 2.0 , Large-scale, Fast, Internationalized website
Web 2.0 <ul><li>Overused… </li></ul><ul><li>But useful </li></ul>
Vox talks to web services
APIs: Tools <ul><li>We use our own custom libraries </li></ul><ul><ul><li>No Net::Amazon, Net::Flickr, etc </li></ul></ul>...
APIs: Open Media Profile <ul><li><entry> </li></ul><ul><li><title>Foo bar baz</title> </li></ul><ul><li><link href=&quot;h...
GData, OpenSearch, Media RSS… GData OpenSearch Media RSS
Open Media Profile GData OpenSearch Media RSS
APIs: Outbound <ul><li>Atom Publishing Protocol </li></ul><ul><li>Everything is Atom/RSS </li></ul><ul><li>Cool URIs </li>...
Ajax <ul><li>JSON serialization </li></ul><ul><ul><li>Lightweight </li></ul></ul><ul><ul><li>Normal data types (no need to...
How we Build Vox: a Web 2.0,  Large-scale , Fast, Internationalized website
Large-scale <ul><li>We started with this: </li></ul>
We added some stuff.
Data::ObjectDriver <ul><li>Movable Type and TypePad: custom ORM </li></ul><ul><li>We wanted more: </li></ul><ul><ul><li>Bu...
Data::ObjectDriver: Caching <ul><li>Built-in support for memcached </li></ul><ul><li>All primary key data maintained for y...
One line of code (basically): <ul><li>Data::ObjectDriver::Driver::Cache::Memcached->new( </li></ul><ul><li>cache => Cache:...
Data::ObjectDriver: Partitioning <ul><li>Sharded data </li></ul><ul><li>Based on arbitrary criteria </li></ul><ul><li>Comp...
One line of code: <ul><li>Data::ObjectDriver::Driver::Cache::Cache->new( </li></ul><ul><li>cache  => Cache::Memcached->new...
Partitioning: traffic
Example: loading user’s posts <ul><li>my $user = ArcheType::M::User->lookup_by_email( </li></ul><ul><li>'ben@sixapart.com’...
Example <ul><li>Loading $user hits the global </li></ul><ul><li>Loading @assets then does: </li></ul><ul><ul><li>Get the p...
ID Allocation: Issues <ul><li>Partitioned databases -> no more auto_increment </li></ul><ul><li>Master/master </li></ul><u...
ID Allocation: yuidd <ul><li>IDs unique a datacenter </li></ul><ul><li>64-bit integers (fit in a BIGINT column) </li></ul>...
Job Queueing <ul><li>Offload processing from Apache </li></ul><ul><li>It’s big and heavy </li></ul>
Job Queueing: TheSchwartz <ul><li>We’ll probably rename it.  </li></ul><ul><li>asynchronous, reliable job queue </li></ul>...
How we Build Vox: a Web 2.0, Large-scale,  Fast , Internationalized website
Fast <ul><li>Need both large-scale  and  fast </li></ul>
Catalyst <ul><li>Vox uses Catalyst </li></ul><ul><li>Does what we want, allows us to do everything else </li></ul><ul><li>...
Is Catalyst fast? <ul><li>A common question on the mailing list! </li></ul><ul><li>It’s fast enough (more on that later). ...
Template Toolkit <ul><li>Pretty fast… </li></ul><ul><li>But we’re probably overloading it. </li></ul>
Template Toolkit: profile <ul><li>[info] Request took 0.244932s (4.083/s) </li></ul><ul><li>.-----------------------------...
Template Toolkit: profile <ul><li>Wow! Template Toolkit takes 60% of the request time. </li></ul><ul><li>4 times as long a...
Template Toolkit: versioned caching <ul><li>On-disk cache </li></ul><ul><li>Versioned with application version </li></ul><...
Versioned caching <ul><li>Template->new({ </li></ul><ul><li>..., </li></ul><ul><li>COMPILE_DIR => '/tmp/tt-cache-' . Vox->...
Template Toolkit: syscalls <ul><li>Lots of syscalls for files that don’t exist! </li></ul><ul><li>But we patched it. </li>...
Caching <ul><li>Data caching </li></ul><ul><li>Automatic caching using Data::ObjectDriver </li></ul><ul><li>Saves millions...
Caching: lists <ul><li>Lists of things: tags on an asset. </li></ul><ul><li>Tag objects are automatically cached </li></ul...
Like this: <ul><li>asset<assetid>-tags => [ <tagid1>, <tagid2>, … ] </li></ul>
Caching: lists <ul><li>Grab list of tag IDs from memcached </li></ul><ul><li>Use get_multi to get back the tags </li></ul>
Get back the tag objects: <ul><li>get_multi <tagid1> <tagid2> … </li></ul>
That’s not all! In bulk: <ul><li>get_multi asset<assetid1>-tags asset<assetid2>-tags … </li></ul>
Caching: lists <ul><li>In a database: N one-to-many queries </li></ul><ul><li>In memcached: 2 queries </li></ul><ul><li>Us...
Perlbal <ul><li>Reverse-proxy setup </li></ul><ul><li>Like Apache 2/mod_proxy in front of mod_perl… </li></ul><ul><li>But ...
Perlbal: webserver mode <ul><li>Serves CSS, images, JavaScript </li></ul><ul><li>Static stuff </li></ul><ul><li>Really fas...
Perlbal: Serving JS and CSS <ul><li>We use a lot of JS and CSS! </li></ul><ul><li>20 JS files per page, 10 CSS files per p...
Perlbal: Serving JS and CSS <ul><li>Added file concatenation support in a plugin </li></ul><ul><li>(it’s now core) </li></ul>
Used to be this: <ul><li><script type=&quot;text/javascript&quot; src=&quot;/.shared:v25.3:vox:en/js/Core.jsc&quot;></scri...
And now it’s this! <ul><li><script type=&quot;text/javascript&quot; src=&quot;/.shared:v25.3:vox:en/js/Core.jsc?/js/DOM.js...
Perlbal: Concatenation <ul><li>Much faster </li></ul><ul><li>Much less latency </li></ul><ul><li>Perlbal handles Last-Modi...
Lots of good stuff! <ul><li>Tools to play with: </li></ul><ul><ul><li>Perlbal </li></ul></ul><ul><ul><li>MogileFS </li></u...
All available at… <ul><li>http://code.sixapart.com/ </li></ul>
Upcoming SlideShare
Loading in …5
×

How we Build Vox: a How we build Vox

20,621 views

Published on

How we Build Vox: a Web 2.0 , Large-scale, Fast, Internationalized website

Published in: Technology

×