E-COMMERCE AND        COUCHBASECreating an e-commerce platform with Couchbase 2.0                                         ...
A STORY ABOUT A FAILURE
OUR GOAL
To sell unique, exclusively designedproducts by small, inspiring designers          all over the world
WHO AM I?• PM   for a big, international consulting firm• CTO    at layers.com / perso.na• Co-founder    at Mobile Genius, ...
HARDCORESQL - PL/SQL GUY
OR_C_E
PRESIVE.COM   Apache 2    FastCGI    Satchmo    Django     Python   PostgreSQL
BUT...• Django   didn’t provide any of the beautiful HTML5 features• HTML     had super-rigid structure• Templates   were ...
PL/SQL Stored ProcedureXML      RecordSet                  ResultSet
JSON
ALIENS
LOGIN
{    ...    "last_login": "2012-08-22T21:31:34+00:00",    ...}
LOGIN• Typically: PL/SQL function that takes username and password, modifies the last login field and returns the user “row”...
LOGIN• Multiple   queries:   Retrieve the user using its ID   Alter the data of the user   Save the user•Exactly what Djan...
LOGIN     47% fasterwith couchbase, nodejs and baseview    https://github.com/PatrickHeneise/baseview
LET’S GO COUCHBASE!
WE ONLY HAD MONEY FOR 7 MORE WEEKS
YOU HAVE 5 WEEKS TOCREATE THE NEW STORE
CONCEPTUAL MODEL
CONCEPTUAL MODEL
CONCEPTUAL MODEL
CONCEPTUAL MODEL               User Address 1   Address 2   Address N   City 1     City 2     City N
{    "_id": "44fb07ca-69ee-4fc1-bb44-e0b13f3d08d8",    "_rev": "56-000a6a6984fca8a10000065300000000",    "$flags": 0,    "...
USER MIGRATION
CLUSTER - JOIN APPROACH
SAME WORKED FOR STORES                  Store    Product 1   Product 2   Product N   Designer 1 Designer 1    Designer 1
ONE QUERY       =ONE PAGE LOADED
DIFFERENT TYPES OFDOCUMENTS DEPENDINGON THE REQUESTED URL
DOCUMENTCEPTION
COMPLEX WHEN UPDATINGA DESIGNER, FOR INSTANCE
var designer = req.body.designer;var productDesigner = JSON.parse( JSON.stringify(designer) );Common.baseview.view("produc...
STOCK CONTROL
{    ...    "id" : "bussoga-ceramic-tile",    "name" : "Ceramic Tile",    "initialStock": "5",    ...}
15 - MINUTE RESERVATION
CART{    "_id": "011de442eace65a9c0de7ec1a1b86ad1",    "_rev": "1-0010fc329c910b380000014100000000",    "$flags": 0,    "$...
DOCUMENT SHOULD BERECOVERED ON DEMAND
“$EXPIRATION” FIELD DIDN’T       WORK FOR US
Map: function (doc) {    if(doc.jsonType == cart) {        for(item in doc.items) {           emit([item, doc.updated_at],...
QUERYING VIEWS                 @hardlifeofapo
@hardlifeofapo
@hardlifeofapo
@hardlifeofapo
Common.baseview.view(stock, reservations,    {      startkey: [aKey, from],      endkey: [aKey, to],      group: true,    ...
Common.baseview.view(stock, reservations,    {      startkey: [aKey, from],      endkey: [aKey, to],      group: true,    ...
STALE = FALSE• We couldn’t rely on an out-of-date index for stock control• Wehad to force stale=false in order to get the ...
ONE THING WE DIDN’T LIKE                     @hardlifeofapo
DEPLOYMENT             @hardlifeofapo
DEPLOYMENT     nginx    forever    express    node.js   baseview   Couchbase               @hardlifeofapo
DEPLOYMENT• One-machine     deployment• 16GB   RAM on GoGrid• Later, we   had to move to 4GB RAM Linode• 4GB   were enough...
BACK-UPS           @hardlifeofapo
COUCHBASE 2.0-DP 4DIDN’T PROVIDE A VALID BACK-UP         MECHANISM                          @hardlifeofapo
WE USED AMAZON S3   FOR BACK-UPS                    @hardlifeofapo
 def run(self):        url = /usr/bin/curl http://%s:%s/%s/_all_docs?include_docs=true% (SERVER_NAME, 8092, CB_BUCKET_NAME...
LESSONS LEARNED                  @hardlifeofapo
THE GOOD• Couchbase     make it simpler to create, evolve and maintain your database• Schema-less   makes new requirements...
THE BAD• Some   troubles when it came to massive inserts (10K+ rows)• Node.js   is absolutely not our preferred language• ...
ONE MORE THING...                    @hardlifeofapo
90% OF DEVELOPMENTCOSTS WERE RELATED      TO UX/UI                 @hardlifeofapo
THANK YOU!@mayorova   @faliru   @JosifovicS   @swilera   @hardlifeofapo
ANY QUESTIONS?                 @hardlifeofapo
CCB12 Creating an e-commerce platform with Couchbase Server 2.0
CCB12 Creating an e-commerce platform with Couchbase Server 2.0
CCB12 Creating an e-commerce platform with Couchbase Server 2.0
CCB12 Creating an e-commerce platform with Couchbase Server 2.0
CCB12 Creating an e-commerce platform with Couchbase Server 2.0
CCB12 Creating an e-commerce platform with Couchbase Server 2.0
Upcoming SlideShare
Loading in …5
×

CCB12 Creating an e-commerce platform with Couchbase Server 2.0

1,828
-1

Published on

0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,828
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
16
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • CCB12 Creating an e-commerce platform with Couchbase Server 2.0

    1. 1. E-COMMERCE AND COUCHBASECreating an e-commerce platform with Couchbase 2.0 @hardlifeofapo
    2. 2. A STORY ABOUT A FAILURE
    3. 3. OUR GOAL
    4. 4. To sell unique, exclusively designedproducts by small, inspiring designers all over the world
    5. 5. WHO AM I?• PM for a big, international consulting firm• CTO at layers.com / perso.na• Co-founder at Mobile Genius, LLC• Developer at presive.com• Lecturer at BarcelonaTech• Lead developer at a new stealth-mode start-up
    6. 6. HARDCORESQL - PL/SQL GUY
    7. 7. OR_C_E
    8. 8. PRESIVE.COM Apache 2 FastCGI Satchmo Django Python PostgreSQL
    9. 9. BUT...• Django didn’t provide any of the beautiful HTML5 features• HTML had super-rigid structure• Templates were complex to edit• Satchmo was too tedius to administrate for non-techs• Reporting / BI / BPM didn’t exist at all• FastCGI not that fast
    10. 10. PL/SQL Stored ProcedureXML RecordSet ResultSet
    11. 11. JSON
    12. 12. ALIENS
    13. 13. LOGIN
    14. 14. { ... "last_login": "2012-08-22T21:31:34+00:00", ...}
    15. 15. LOGIN• Typically: PL/SQL function that takes username and password, modifies the last login field and returns the user “row”• No PL/SQL now
    16. 16. LOGIN• Multiple queries: Retrieve the user using its ID Alter the data of the user Save the user•Exactly what Django does undercover
    17. 17. LOGIN 47% fasterwith couchbase, nodejs and baseview https://github.com/PatrickHeneise/baseview
    18. 18. LET’S GO COUCHBASE!
    19. 19. WE ONLY HAD MONEY FOR 7 MORE WEEKS
    20. 20. YOU HAVE 5 WEEKS TOCREATE THE NEW STORE
    21. 21. CONCEPTUAL MODEL
    22. 22. CONCEPTUAL MODEL
    23. 23. CONCEPTUAL MODEL
    24. 24. CONCEPTUAL MODEL User Address 1 Address 2 Address N City 1 City 2 City N
    25. 25. { "_id": "44fb07ca-69ee-4fc1-bb44-e0b13f3d08d8", "_rev": "56-000a6a6984fca8a10000065300000000", "$flags": 0, "$expiration": 0, "email": "p.casado.arias@gmail.com", "password": "password_in_encrypted_in_some_magical_way", "tos": "yes", "name": { "givenName": "Pablo", "familyName": "Casado" }, "jsonType": "user", "last_login": "2012-08-22T21:31:34+00:00", "lang": "en", "time_zone": "Europe/Madrid", "displayName": "Pablo Casado", "billingAddresses": [ { "addressee": "Pablo Casado", "street": "Some Place", "postalCode": "08840", "city": "Viladecans", "stateProvince": "Barcelona", "country": "Spain", "phoneNumber": "34690916113" } ], "created_at": "2012-03-02T20:03:57+00:00"}
    26. 26. USER MIGRATION
    27. 27. CLUSTER - JOIN APPROACH
    28. 28. SAME WORKED FOR STORES Store Product 1 Product 2 Product N Designer 1 Designer 1 Designer 1
    29. 29. ONE QUERY =ONE PAGE LOADED
    30. 30. DIFFERENT TYPES OFDOCUMENTS DEPENDINGON THE REQUESTED URL
    31. 31. DOCUMENTCEPTION
    32. 32. COMPLEX WHEN UPDATINGA DESIGNER, FOR INSTANCE
    33. 33. var designer = req.body.designer;var productDesigner = JSON.parse( JSON.stringify(designer) );Common.baseview.view("products", "by_designer", {"key": designer.id, include_docs: true}, updateAllProductsContainingDesigner );Common.baseview.view(store, by_Designer, {"key": designer.id, "include_docs": true}, function(error, result){ updateAllStoresContainingDesigner( error, result, productDesigner ); } );
    34. 34. STOCK CONTROL
    35. 35. { ... "id" : "bussoga-ceramic-tile", "name" : "Ceramic Tile", "initialStock": "5", ...}
    36. 36. 15 - MINUTE RESERVATION
    37. 37. CART{ "_id": "011de442eace65a9c0de7ec1a1b86ad1", "_rev": "1-0010fc329c910b380000014100000000", "$flags": 0, "$expiration": 0, "jsonType": "cart", "user": "852da85a-a730-4d40-ac67-b3e28059345b", "created_at": "2012-07-12T09:11:21+00:00", "updated_at": "2012-07-12T09:11:21+00:00", "items": { "bussoga-ceramic-tile-mural": { "added_at": "2012-07-12T09:11:21+00:00", "quantity": 1 } }}
    38. 38. DOCUMENT SHOULD BERECOVERED ON DEMAND
    39. 39. “$EXPIRATION” FIELD DIDN’T WORK FOR US
    40. 40. Map: function (doc) { if(doc.jsonType == cart) { for(item in doc.items) { emit([item, doc.updated_at], doc.items[item].quantity); } } }Reduce: _sum
    41. 41. QUERYING VIEWS @hardlifeofapo
    42. 42. @hardlifeofapo
    43. 43. @hardlifeofapo
    44. 44. @hardlifeofapo
    45. 45. Common.baseview.view(stock, reservations, { startkey: [aKey, from], endkey: [aKey, to], group: true, group_level: 1 stale: false }, function(error, booked) { ... } ); @hardlifeofapo
    46. 46. Common.baseview.view(stock, reservations, { startkey: [aKey, from], endkey: [aKey, to], group: true, group_level: 1 stale: false }, function(error, booked) { ... } ); @hardlifeofapo
    47. 47. STALE = FALSE• We couldn’t rely on an out-of-date index for stock control• Wehad to force stale=false in order to get the most recent data from the view• Eventually, we got wrong results anyway. Mostly related with browser cache• Anyideas on how to do this better will be really appreciated. @hardlifeofapo
    48. 48. ONE THING WE DIDN’T LIKE @hardlifeofapo
    49. 49. DEPLOYMENT @hardlifeofapo
    50. 50. DEPLOYMENT nginx forever express node.js baseview Couchbase @hardlifeofapo
    51. 51. DEPLOYMENT• One-machine deployment• 16GB RAM on GoGrid• Later, we had to move to 4GB RAM Linode• 4GB were enough to handle 1000 concurrent users @hardlifeofapo
    52. 52. BACK-UPS @hardlifeofapo
    53. 53. COUCHBASE 2.0-DP 4DIDN’T PROVIDE A VALID BACK-UP MECHANISM @hardlifeofapo
    54. 54. WE USED AMAZON S3 FOR BACK-UPS @hardlifeofapo
    55. 55.  def run(self):        url = /usr/bin/curl http://%s:%s/%s/_all_docs?include_docs=true% (SERVER_NAME, 8092, CB_BUCKET_NAME )        print url        p = subprocess.Popen(url, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)        output, errors = p.communicate()        output = json.loads(output)        items = output[rows]        # Create does not create again a bucket if that bucket already exists.        s3_bucket = self.conn.create_bucket(self.s3BucketName)                for item in items:            k = Key(s3_bucket)            k.key = item["doc"]["_id"]            print "Saving %s" % (item["doc"]["_id"])            # json.dumps is needed to prevent storage of pythons unicode representation of strings, like ua_string            k.set_contents_from_string( json.dumps(item["doc"], sort_keys=True) )https://github.com/hardlifeofapo/couchbase_backup_s3 Thanks go to Francis Varga (nerd@crowdpark.com) @hardlifeofapo
    56. 56. LESSONS LEARNED @hardlifeofapo
    57. 57. THE GOOD• Couchbase make it simpler to create, evolve and maintain your database• Schema-less makes new requirements to be satified faster• Migrations are very, very easy• Easier to operate and back-up• As fast (or even faster) than any other SQL engine• Lower CPU usage, smaller and cheaper machines @hardlifeofapo
    58. 58. THE BAD• Some troubles when it came to massive inserts (10K+ rows)• Node.js is absolutely not our preferred language• Good ideas, good teams and good technology can also become a massive failure @hardlifeofapo
    59. 59. ONE MORE THING... @hardlifeofapo
    60. 60. 90% OF DEVELOPMENTCOSTS WERE RELATED TO UX/UI @hardlifeofapo
    61. 61. THANK YOU!@mayorova @faliru @JosifovicS @swilera @hardlifeofapo
    62. 62. ANY QUESTIONS? @hardlifeofapo
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×