Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
E-COMMERCE AND        COUCHBASECreating an e-commerce platform with Couchbase 2.0                                         ...
A STORY ABOUT A FAILURE
OUR GOAL
To sell unique, exclusively designedproducts by small, inspiring designers          all over the world
WHO AM I?• PM   for a big, international consulting firm• CTO    at layers.com / perso.na• Co-founder    at Mobile Genius, ...
HARDCORESQL - PL/SQL GUY
OR_C_E
PRESIVE.COM   Apache 2    FastCGI    Satchmo    Django     Python   PostgreSQL
BUT...• Django   didn’t provide any of the beautiful HTML5 features• HTML     had super-rigid structure• Templates   were ...
PL/SQL Stored ProcedureXML      RecordSet                  ResultSet
JSON
ALIENS
LOGIN
{    ...    "last_login": "2012-08-22T21:31:34+00:00",    ...}
LOGIN• Typically: PL/SQL function that takes username and password, modifies the last login field and returns the user “row”...
LOGIN• Multiple   queries:   Retrieve the user using its ID   Alter the data of the user   Save the user•Exactly what Djan...
LOGIN     47% fasterwith couchbase, nodejs and baseview    https://github.com/PatrickHeneise/baseview
LET’S GO COUCHBASE!
WE ONLY HAD MONEY FOR 7 MORE WEEKS
YOU HAVE 5 WEEKS TOCREATE THE NEW STORE
CONCEPTUAL MODEL
CONCEPTUAL MODEL
CONCEPTUAL MODEL
CONCEPTUAL MODEL               User Address 1   Address 2   Address N   City 1     City 2     City N
{    "_id": "44fb07ca-69ee-4fc1-bb44-e0b13f3d08d8",    "_rev": "56-000a6a6984fca8a10000065300000000",    "$flags": 0,    "...
USER MIGRATION
CLUSTER - JOIN APPROACH
SAME WORKED FOR STORES                  Store    Product 1   Product 2   Product N   Designer 1 Designer 1    Designer 1
ONE QUERY       =ONE PAGE LOADED
DIFFERENT TYPES OFDOCUMENTS DEPENDINGON THE REQUESTED URL
DOCUMENTCEPTION
COMPLEX WHEN UPDATINGA DESIGNER, FOR INSTANCE
var designer = req.body.designer;var productDesigner = JSON.parse( JSON.stringify(designer) );Common.baseview.view("produc...
STOCK CONTROL
{    ...    "id" : "bussoga-ceramic-tile",    "name" : "Ceramic Tile",    "initialStock": "5",    ...}
15 - MINUTE RESERVATION
CART{    "_id": "011de442eace65a9c0de7ec1a1b86ad1",    "_rev": "1-0010fc329c910b380000014100000000",    "$flags": 0,    "$...
DOCUMENT SHOULD BERECOVERED ON DEMAND
“$EXPIRATION” FIELD DIDN’T       WORK FOR US
Map: function (doc) {    if(doc.jsonType == cart) {        for(item in doc.items) {           emit([item, doc.updated_at],...
QUERYING VIEWS                 @hardlifeofapo
@hardlifeofapo
@hardlifeofapo
@hardlifeofapo
Common.baseview.view(stock, reservations,    {      startkey: [aKey, from],      endkey: [aKey, to],      group: true,    ...
Common.baseview.view(stock, reservations,    {      startkey: [aKey, from],      endkey: [aKey, to],      group: true,    ...
STALE = FALSE• We couldn’t rely on an out-of-date index for stock control• Wehad to force stale=false in order to get the ...
ONE THING WE DIDN’T LIKE                     @hardlifeofapo
DEPLOYMENT             @hardlifeofapo
DEPLOYMENT     nginx    forever    express    node.js   baseview   Couchbase               @hardlifeofapo
DEPLOYMENT• One-machine     deployment• 16GB   RAM on GoGrid• Later, we   had to move to 4GB RAM Linode• 4GB   were enough...
BACK-UPS           @hardlifeofapo
COUCHBASE 2.0-DP 4DIDN’T PROVIDE A VALID BACK-UP         MECHANISM                          @hardlifeofapo
WE USED AMAZON S3   FOR BACK-UPS                    @hardlifeofapo
 def run(self):        url = /usr/bin/curl http://%s:%s/%s/_all_docs?include_docs=true% (SERVER_NAME, 8092, CB_BUCKET_NAME...
LESSONS LEARNED                  @hardlifeofapo
THE GOOD• Couchbase     make it simpler to create, evolve and maintain your database• Schema-less   makes new requirements...
THE BAD• Some   troubles when it came to massive inserts (10K+ rows)• Node.js   is absolutely not our preferred language• ...
ONE MORE THING...                    @hardlifeofapo
90% OF DEVELOPMENTCOSTS WERE RELATED      TO UX/UI                 @hardlifeofapo
THANK YOU!@mayorova   @faliru   @JosifovicS   @swilera   @hardlifeofapo
ANY QUESTIONS?                 @hardlifeofapo
CCB12 Creating an e-commerce platform with Couchbase Server 2.0
CCB12 Creating an e-commerce platform with Couchbase Server 2.0
CCB12 Creating an e-commerce platform with Couchbase Server 2.0
CCB12 Creating an e-commerce platform with Couchbase Server 2.0
CCB12 Creating an e-commerce platform with Couchbase Server 2.0
CCB12 Creating an e-commerce platform with Couchbase Server 2.0
Upcoming SlideShare
Loading in …5
×

CCB12 Creating an e-commerce platform with Couchbase Server 2.0

2,529 views

Published on

  • Be the first to comment

CCB12 Creating an e-commerce platform with Couchbase Server 2.0

  1. 1. E-COMMERCE AND COUCHBASECreating an e-commerce platform with Couchbase 2.0 @hardlifeofapo
  2. 2. A STORY ABOUT A FAILURE
  3. 3. OUR GOAL
  4. 4. To sell unique, exclusively designedproducts by small, inspiring designers all over the world
  5. 5. WHO AM I?• PM for a big, international consulting firm• CTO at layers.com / perso.na• Co-founder at Mobile Genius, LLC• Developer at presive.com• Lecturer at BarcelonaTech• Lead developer at a new stealth-mode start-up
  6. 6. HARDCORESQL - PL/SQL GUY
  7. 7. OR_C_E
  8. 8. PRESIVE.COM Apache 2 FastCGI Satchmo Django Python PostgreSQL
  9. 9. BUT...• Django didn’t provide any of the beautiful HTML5 features• HTML had super-rigid structure• Templates were complex to edit• Satchmo was too tedius to administrate for non-techs• Reporting / BI / BPM didn’t exist at all• FastCGI not that fast
  10. 10. PL/SQL Stored ProcedureXML RecordSet ResultSet
  11. 11. JSON
  12. 12. ALIENS
  13. 13. LOGIN
  14. 14. { ... "last_login": "2012-08-22T21:31:34+00:00", ...}
  15. 15. LOGIN• Typically: PL/SQL function that takes username and password, modifies the last login field and returns the user “row”• No PL/SQL now
  16. 16. LOGIN• Multiple queries: Retrieve the user using its ID Alter the data of the user Save the user•Exactly what Django does undercover
  17. 17. LOGIN 47% fasterwith couchbase, nodejs and baseview https://github.com/PatrickHeneise/baseview
  18. 18. LET’S GO COUCHBASE!
  19. 19. WE ONLY HAD MONEY FOR 7 MORE WEEKS
  20. 20. YOU HAVE 5 WEEKS TOCREATE THE NEW STORE
  21. 21. CONCEPTUAL MODEL
  22. 22. CONCEPTUAL MODEL
  23. 23. CONCEPTUAL MODEL
  24. 24. CONCEPTUAL MODEL User Address 1 Address 2 Address N City 1 City 2 City N
  25. 25. { "_id": "44fb07ca-69ee-4fc1-bb44-e0b13f3d08d8", "_rev": "56-000a6a6984fca8a10000065300000000", "$flags": 0, "$expiration": 0, "email": "p.casado.arias@gmail.com", "password": "password_in_encrypted_in_some_magical_way", "tos": "yes", "name": { "givenName": "Pablo", "familyName": "Casado" }, "jsonType": "user", "last_login": "2012-08-22T21:31:34+00:00", "lang": "en", "time_zone": "Europe/Madrid", "displayName": "Pablo Casado", "billingAddresses": [ { "addressee": "Pablo Casado", "street": "Some Place", "postalCode": "08840", "city": "Viladecans", "stateProvince": "Barcelona", "country": "Spain", "phoneNumber": "34690916113" } ], "created_at": "2012-03-02T20:03:57+00:00"}
  26. 26. USER MIGRATION
  27. 27. CLUSTER - JOIN APPROACH
  28. 28. SAME WORKED FOR STORES Store Product 1 Product 2 Product N Designer 1 Designer 1 Designer 1
  29. 29. ONE QUERY =ONE PAGE LOADED
  30. 30. DIFFERENT TYPES OFDOCUMENTS DEPENDINGON THE REQUESTED URL
  31. 31. DOCUMENTCEPTION
  32. 32. COMPLEX WHEN UPDATINGA DESIGNER, FOR INSTANCE
  33. 33. var designer = req.body.designer;var productDesigner = JSON.parse( JSON.stringify(designer) );Common.baseview.view("products", "by_designer", {"key": designer.id, include_docs: true}, updateAllProductsContainingDesigner );Common.baseview.view(store, by_Designer, {"key": designer.id, "include_docs": true}, function(error, result){ updateAllStoresContainingDesigner( error, result, productDesigner ); } );
  34. 34. STOCK CONTROL
  35. 35. { ... "id" : "bussoga-ceramic-tile", "name" : "Ceramic Tile", "initialStock": "5", ...}
  36. 36. 15 - MINUTE RESERVATION
  37. 37. CART{ "_id": "011de442eace65a9c0de7ec1a1b86ad1", "_rev": "1-0010fc329c910b380000014100000000", "$flags": 0, "$expiration": 0, "jsonType": "cart", "user": "852da85a-a730-4d40-ac67-b3e28059345b", "created_at": "2012-07-12T09:11:21+00:00", "updated_at": "2012-07-12T09:11:21+00:00", "items": { "bussoga-ceramic-tile-mural": { "added_at": "2012-07-12T09:11:21+00:00", "quantity": 1 } }}
  38. 38. DOCUMENT SHOULD BERECOVERED ON DEMAND
  39. 39. “$EXPIRATION” FIELD DIDN’T WORK FOR US
  40. 40. Map: function (doc) { if(doc.jsonType == cart) { for(item in doc.items) { emit([item, doc.updated_at], doc.items[item].quantity); } } }Reduce: _sum
  41. 41. QUERYING VIEWS @hardlifeofapo
  42. 42. @hardlifeofapo
  43. 43. @hardlifeofapo
  44. 44. @hardlifeofapo
  45. 45. Common.baseview.view(stock, reservations, { startkey: [aKey, from], endkey: [aKey, to], group: true, group_level: 1 stale: false }, function(error, booked) { ... } ); @hardlifeofapo
  46. 46. Common.baseview.view(stock, reservations, { startkey: [aKey, from], endkey: [aKey, to], group: true, group_level: 1 stale: false }, function(error, booked) { ... } ); @hardlifeofapo
  47. 47. STALE = FALSE• We couldn’t rely on an out-of-date index for stock control• Wehad to force stale=false in order to get the most recent data from the view• Eventually, we got wrong results anyway. Mostly related with browser cache• Anyideas on how to do this better will be really appreciated. @hardlifeofapo
  48. 48. ONE THING WE DIDN’T LIKE @hardlifeofapo
  49. 49. DEPLOYMENT @hardlifeofapo
  50. 50. DEPLOYMENT nginx forever express node.js baseview Couchbase @hardlifeofapo
  51. 51. DEPLOYMENT• One-machine deployment• 16GB RAM on GoGrid• Later, we had to move to 4GB RAM Linode• 4GB were enough to handle 1000 concurrent users @hardlifeofapo
  52. 52. BACK-UPS @hardlifeofapo
  53. 53. COUCHBASE 2.0-DP 4DIDN’T PROVIDE A VALID BACK-UP MECHANISM @hardlifeofapo
  54. 54. WE USED AMAZON S3 FOR BACK-UPS @hardlifeofapo
  55. 55.  def run(self):        url = /usr/bin/curl http://%s:%s/%s/_all_docs?include_docs=true% (SERVER_NAME, 8092, CB_BUCKET_NAME )        print url        p = subprocess.Popen(url, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)        output, errors = p.communicate()        output = json.loads(output)        items = output[rows]        # Create does not create again a bucket if that bucket already exists.        s3_bucket = self.conn.create_bucket(self.s3BucketName)                for item in items:            k = Key(s3_bucket)            k.key = item["doc"]["_id"]            print "Saving %s" % (item["doc"]["_id"])            # json.dumps is needed to prevent storage of pythons unicode representation of strings, like ua_string            k.set_contents_from_string( json.dumps(item["doc"], sort_keys=True) )https://github.com/hardlifeofapo/couchbase_backup_s3 Thanks go to Francis Varga (nerd@crowdpark.com) @hardlifeofapo
  56. 56. LESSONS LEARNED @hardlifeofapo
  57. 57. THE GOOD• Couchbase make it simpler to create, evolve and maintain your database• Schema-less makes new requirements to be satified faster• Migrations are very, very easy• Easier to operate and back-up• As fast (or even faster) than any other SQL engine• Lower CPU usage, smaller and cheaper machines @hardlifeofapo
  58. 58. THE BAD• Some troubles when it came to massive inserts (10K+ rows)• Node.js is absolutely not our preferred language• Good ideas, good teams and good technology can also become a massive failure @hardlifeofapo
  59. 59. ONE MORE THING... @hardlifeofapo
  60. 60. 90% OF DEVELOPMENTCOSTS WERE RELATED TO UX/UI @hardlifeofapo
  61. 61. THANK YOU!@mayorova @faliru @JosifovicS @swilera @hardlifeofapo
  62. 62. ANY QUESTIONS? @hardlifeofapo

×