• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Real time ecommerce analytics with MongoDB at Gilt Groupe (Michael Bryzek & Michael Nutt)
 

Real time ecommerce analytics with MongoDB at Gilt Groupe (Michael Bryzek & Michael Nutt)

on

  • 14,417 views

 

Statistics

Views

Total Views
14,417
Views on SlideShare
13,929
Embed Views
488

Actions

Likes
43
Downloads
248
Comments
0

8 Embeds 488

http://www.slideshare.net 299
http://rg443blog.wordpress.com 61
http://www.10gen.com 52
http://lanyrd.com 47
https://twitter.com 12
http://www.techgig.com 10
http://www.mongodb.org 6
http://localhost:8080 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Real time ecommerce analytics with MongoDB at Gilt Groupe (Michael Bryzek & Michael Nutt) Real time ecommerce analytics with MongoDB at Gilt Groupe (Michael Bryzek & Michael Nutt) Presentation Transcript

    • Real Time Ecommerce Analytics at Gilt Groupe
      Michael Bryzek, CTO & Founder
      Michael Nutt, Senior Engineer
      Mongo SF - April 30, 2010
      We’re hiring: michael@gilt.com
      www.gilt.com/invite/michael
    • What is Gilt Groupe?
      The world’s best brands at up to 70% off
      Sales start every day at noon
      Simple, luxurious online experience
      Relentless focus on the customer
      . . .
      A fast growing young company
    • What does noon look like in Tech?
    • What does noon look like?
    • MongoDB at Gilt Groupe
      Real time analytics is a sweet spot for MongoDB
      Two production examples we’ll share today at Gilt Groupe:
      Selecting product to sell based on real time data
      Hummingbird: Real time visualization of site traffic
    • Using MongoDB for Real Time Analytics
      Goal: Improve conversion of our gifts section (www.gilt.com/gifts) by ensuring good products are being promoted at the right time
      Challenge: High traffic makes it hard to collect and analyze data in a scalable and fast way
      Approach:
      Capture data in real time in MongoDB
      Analyze w/ Map Reduce
      Update txn systems
      Repeat
    • Step 1: Data Capture
      • Java server speaks JSON/HTTP, writes to MongoDB
      • Each page view receive a list of every item on the page and its position via AJAX
      • Purchase data sent by background job post purchase
    • Step 1: Data Capture
      Storing Data in Java
      _db = new Mongo().getDB("gifts");
      _listing_visits = _db.getCollection("listing_visits");
      --------------------------------------------------
      BasicDBObject record = new BasicDBObject();
      record.put("gift_product_look_guid", de.giftProductLookGuid);
      record.put("product_look_guid", de.productLookGuid);
      record.put("sale_id", info.saleId);
      record.put("user_guid", info.userGuid);
      record.put("subsite_id", info.subsiteId);
      record.put("created_at", info.createdAt);
      record.put("position", position);
      _listing_visits.save(record);
    • Step 2: Map
      Calculate a score for each item based on page views, conversion, inventory, and merchandising input
      m = function(){
      [snip]
      if ( hourly.visits > 0 && this.quantity_sold > 0 ) {
      var rate = this.quantity_sold / hourly.visits;
      points = parseInt(100*rate);
      v += points;
      explanation += "Conversion rate of " + points + "% ”;
      } else if ( hourly.visits == null || hourly.visits == 0 ) {
      v += 500;
      explanation += "Product has never been seen (500 points). ";
      }
      [snip]
      emit( { gift_product_look_guid : this._id },
      { score : v,
      explanation : explanation}});
      }
    • Step 2: Reduce
      Reduce is a passthrough
      r = function( pid , values ){
      return values[0];
      }
      Map Reduce run every 15 minutes via CRON – results stored in a collection named “scores”
      res = db.gift_product_looks.mapReduce( m , r , { out : "scores" } );
    • Step 3: Update Transactional Systems
      • Mongo + server run on ec2
      • Send “scores” collection back to our primary data center, storing latest scores in our primary relational database
      • Gift items are always sorted by score – transactional system only needed an “order by score desc” clause
    • Questions before we move on to Hummingbird?
    • Tracking Pixels
      /tracking.gif?events=&prop1=women&server=www.gilt.com&products=&pageName=sales%3A+women&channel=sale&prop4=sale+category+page&u=http%3A%2F%2Fwww.gilt.com%2Fsale%2Fwomen&guid=418237ca-2bc6-932e-84c2-d4f02d9fd5bf&gen=f&uid=25423567&cb=443460396
    • Omniture
      GILT
      Data
      Warehouse
      Users
    • Omniture
      24 hours
      later...
      GILT
      Data
      Warehouse
      Users
    • Node.js
      Asynchronous, evented web framework
      http://nodejs.org
    • var mongo = require(’lib/mongodb’);
      var db = new mongo.Db('hummingbird', new mongo.Server('localhost', 27017, {}), {});
      db.createCollection('visits', function(err, collection) {
      db.collection('visits', function(err, collection) {
      collection.insert(env);
      });
      });
    • DEMO