Mongodb and Totsy: An e-commerce case study

MongoDB E-Commerce Case Study: Totsy Mitch Pirtle MongoDC - November 18, 2010 Washington, D.C.

About this session

About the speaker New Yorker Contributor to many FOSS projects (Joomla! founder) Budding author MongoDB for Web Development Extreme Joomla! Performance High-trafﬁc projects (Jetsetter, MTV, Food Networks) Husband, dad, skate punk, metalcore bassist, coach

About the team

What is MongoDB?

What is MongoDB? Document database

What is MongoDB? Document database Wicked fast speed

What is MongoDB? Document database Wicked fast speed Great for scale

What is MongoDB? Document database Wicked fast speed Great for scale Low administration requirements

What is MongoDB? Document database Wicked fast speed Great for scale Low administration requirements Encourages simple code, fewer bugs

Community Resources http://www.mongodb.org/display/DOCS/ Community

MongoDB Language Center http://www.mongodb.org/display/DOCS/Drivers

Development Tracker http://jira.mongodb.org

MongoDB Cookbook http://cookbook.mongodb.org/

Explore MongoDB @Github http://github.com/search?q=mongo

www.totsy.com Private sale website catering to parents with small kids

What is Totsy? Launched last year Relies heavily on organic growth Stampede trafﬁc patterns - high peaks, deep valleys Leading player in their market segment

The challenge Get out of initial outsourced platform Didn’t scale Didn’t perform Didn’t work Brittle, difﬁcult to manage or enhance Essentially a case study on how not to use RoR, PostgreSQL and Heroku

The Original Plan™ Use Joomla! as an application container Use MongoDB for all non-Joomla! data Build custom Joomla! extensions, all using MongoDB

“Oh yeah, we’re probably dead in a couple weeks without the new platform.” :-)

Hey, wait a minute.

How about a framework? One layer lower, with much less overhead, and only using what we wanted and nothing more...

The New Plan™

The New Plan™ Primary need was speed and scale, of which the previous effort was an #epicfail

The New Plan™ Primary need was speed and scale, of which the previous effort was an #epicfail Tap into existing personal network of PHP talent

The New Plan™ Primary need was speed and scale, of which the previous effort was an #epicfail Tap into existing personal network of PHP talent Leverage relationship with MongoDB and Lithium projects

The New Plan™ Primary need was speed and scale, of which the previous effort was an #epicfail Tap into existing personal network of PHP talent Leverage relationship with MongoDB and Lithium projects Fully showcase MongoDB, and ﬁnally have a site with no relational engines needed

What was expected

What was expected Rapid development

What was expected Rapid development Fast execution of code

What was expected Rapid development Fast execution of code Spartan hosting requirements

The Result

The Result Extremely rapid development

The Result Extremely rapid development Extremely speed

The Result Extremely rapid development Extremely speed Extremely minimal hosting requirements

The Result Extremely rapid development Extremely speed Extremely minimal hosting requirements K.I.S.S. and all the goodness that entails

Development proﬁle

Development proﬁle Small team: 1 designer, 1 front end developer, 2 engineers and 1 contract engineer

Development proﬁle Small team: 1 designer, 1 front end developer, 2 engineers and 1 contract engineer Three months active development

Development proﬁle Small team: 1 designer, 1 front end developer, 2 engineers and 1 contract engineer Three months active development Redmine + Git for project management + version control

Development proﬁle Small team: 1 designer, 1 front end developer, 2 engineers and 1 contract engineer Three months active development Redmine + Git for project management + version control Local, Development, Test, Production environments

Stack proﬁle

Stack proﬁle PHP: fast execution, quick prototyping

Stack proﬁle PHP: fast execution, quick prototyping Lithium: PHP 5.3+ RAD framework

Stack proﬁle PHP: fast execution, quick prototyping Lithium: PHP 5.3+ RAD framework MongoDB: whack-daddy database from tomorrow

Stack proﬁle PHP: fast execution, quick prototyping Lithium: PHP 5.3+ RAD framework MongoDB: whack-daddy database from tomorrow LightTPD: fcgi lightweight webserver

Stack proﬁle PHP: fast execution, quick prototyping Lithium: PHP 5.3+ RAD framework MongoDB: whack-daddy database from tomorrow LightTPD: fcgi lightweight webserver Xcache: PHP opcode cache

Stack proﬁle PHP: fast execution, quick prototyping Lithium: PHP 5.3+ RAD framework MongoDB: whack-daddy database from tomorrow LightTPD: fcgi lightweight webserver Xcache: PHP opcode cache Linux: defacto standard operating system in datacenters

Atomic Operations with MongoDB

Your relational data Whoah Nelly! Good reason for a DBA Many foreign keys Need for triggers, stored procedures, cronjobs...

With a side of MongoDB K.I.S.S. Simple to manage Simple to access Little or no need for foreign constraints

What it looks like now 2.5GB data on disk 1.0GB data in memory

What MongoDB did for me.

What MongoDB did for me. Encouraged extremely simple data model, producing tighter, faster, bug-free code.

What MongoDB did for me. Eliminated the need for cache. Really. No kidding.

What MongoDB did for me. Used GridFS for all ﬁle storage. Lithium happily provided a media adapter allowing us to create a GridFS plugin to transparently store ﬁles in the database.

What MongoDB did for me. Removed worries about a complicated hosting environment

What MongoDB did for me. Bought us a ton of time to carefully deliberate next steps, and come out of permanent crisis mode

What MongoDB did for me. Allowed us to commit every possible crime required for survival in a frantic, frenetic startup environment

Crimes? What crimes?

My shameful story

My shameful story code code code commit code pray commit code code

My shameful story code code code commit code pray commit code code Models had no schema deﬁned

My shameful story code code code commit code pray commit code code Models had no schema deﬁned No time for optimization or performance testing

My shameful story code code code commit code pray commit code code Models had no schema deﬁned No time for optimization or performance testing Not a single line of code was run through a harness

So what is Lithium?

Lithium PHP framework http://lithify.me http://rad-dev.org/lithium/wiki Projects demonstrating MongoDB support: http://rad-dev.org/lithium_mongo/source http://rad-dev.org/lithium_blog/source

Why Lithium?

Why Lithium? Leverages the latest and greatest features and practices found in PHP 5.3+

Why Lithium? Leverages the latest and greatest features and practices found in PHP 5.3+ Extremely modular, everything in Lithium is a plugin

Why Lithium? Leverages the latest and greatest features and practices found in PHP 5.3+ Extremely modular, everything in Lithium is a plugin Native hooks into MongoDB and other non-relational databases

Why Lithium? Leverages the latest and greatest features and practices found in PHP 5.3+ Extremely modular, everything in Lithium is a plugin Native hooks into MongoDB and other non-relational databases Simple hooks for additional customization (GridFS...)

Why Lithium?

Why Lithium? Promiscuously opinionated framework

Why Lithium? Promiscuously opinionated framework Test harness provided

Why Lithium? Promiscuously opinionated framework Test harness provided Simpliﬁed prototyping

Why Lithium? Promiscuously opinionated framework Test harness provided Simpliﬁed prototyping li3_docs - Dynamic code documentation

Why Lithium? Promiscuously opinionated framework Test harness provided Simpliﬁed prototyping li3_docs - Dynamic code documentation Many other helpful plugins available and growing

What kind of speed?

What kind of speed? One evening right when a bunch of major sales launched, I saw 2,800+ operations per second on the MongoDB master server.

What kind of speed? One evening right when a bunch of major sales launched, I saw 2,800+ operations per second on the MongoDB master server. This server had a load of 0.68.

What kind of speed? One evening right when a bunch of major sales launched, I saw 2,800+ operations per second on the MongoDB master server. This server had a load of 0.68. This translates to 17% capacity on a four core machine.

What kind of scale?

What kind of scale? We’re using multiple servers for redundancy only.

What kind of scale? We’re using multiple servers for redundancy only. When we need to add servers, the effort will be greatly simpliﬁed by having sharding already provided.

What kind of scale? We’re using multiple servers for redundancy only. When we need to add servers, the effort will be greatly simpliﬁed by having sharding already provided. Probably could have launched on this laptop with identical performance.

A quick note on single server durability.

</rant>

Questions and Excuses

Thanks! Email: spacemonkey@mongodb.org Twitter: @mitchitized Web: http://www.mitchitized.com/ Slides: http://www.slideshare.net/ spacemonkeylabs

http://www.nosqldatabases.com	251
http://www.mongodb.org	64
http://www.mitchitized.com	5
http://translate.googleusercontent.com	3
http://webcache.googleusercontent.com	1
http://a0.twimg.com	1
http://www.twylah.com	1

Mongodb and Totsy: An e-commerce case study

by Mitch Pirtle

Accessibility

Categories

Upload Details

Usage Rights

7 Embeds 326

Statistics

Mongodb and Totsy: An e-commerce case study Presentation Transcript