Real time ecommerce analytics with MongoDB at Gilt Groupe (Michael Bryzek & Michael Nutt)
Upcoming SlideShare
Loading in...5
×
 

Real time ecommerce analytics with MongoDB at Gilt Groupe (Michael Bryzek & Michael Nutt)

on

  • 14,619 views

 

Statistics

Views

Total Views
14,619
Views on SlideShare
14,130
Embed Views
489

Actions

Likes
43
Downloads
248
Comments
0

8 Embeds 489

http://www.slideshare.net 299
http://rg443blog.wordpress.com 61
http://www.10gen.com 52
http://lanyrd.com 47
https://twitter.com 13
http://www.techgig.com 10
http://www.mongodb.org 6
http://localhost:8080 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Real time ecommerce analytics with MongoDB at Gilt Groupe (Michael Bryzek & Michael Nutt) Real time ecommerce analytics with MongoDB at Gilt Groupe (Michael Bryzek & Michael Nutt) Presentation Transcript

  • Real Time Ecommerce Analytics at Gilt Groupe
    Michael Bryzek, CTO & Founder
    Michael Nutt, Senior Engineer
    Mongo SF - April 30, 2010
    We’re hiring: michael@gilt.com
    www.gilt.com/invite/michael
  • What is Gilt Groupe?
    The world’s best brands at up to 70% off
    Sales start every day at noon
    Simple, luxurious online experience
    Relentless focus on the customer
    . . .
    A fast growing young company
  • What does noon look like in Tech?
  • What does noon look like?
  • MongoDB at Gilt Groupe
    Real time analytics is a sweet spot for MongoDB
    Two production examples we’ll share today at Gilt Groupe:
    Selecting product to sell based on real time data
    Hummingbird: Real time visualization of site traffic
  • Using MongoDB for Real Time Analytics
    Goal: Improve conversion of our gifts section (www.gilt.com/gifts) by ensuring good products are being promoted at the right time
    Challenge: High traffic makes it hard to collect and analyze data in a scalable and fast way
    Approach:
    Capture data in real time in MongoDB
    Analyze w/ Map Reduce
    Update txn systems
    Repeat
  • Step 1: Data Capture
    • Java server speaks JSON/HTTP, writes to MongoDB
    • Each page view receive a list of every item on the page and its position via AJAX
    • Purchase data sent by background job post purchase
  • Step 1: Data Capture
    Storing Data in Java
    _db = new Mongo().getDB("gifts");
    _listing_visits = _db.getCollection("listing_visits");
    --------------------------------------------------
    BasicDBObject record = new BasicDBObject();
    record.put("gift_product_look_guid", de.giftProductLookGuid);
    record.put("product_look_guid", de.productLookGuid);
    record.put("sale_id", info.saleId);
    record.put("user_guid", info.userGuid);
    record.put("subsite_id", info.subsiteId);
    record.put("created_at", info.createdAt);
    record.put("position", position);
    _listing_visits.save(record);
  • Step 2: Map
    Calculate a score for each item based on page views, conversion, inventory, and merchandising input
    m = function(){
    [snip]
    if ( hourly.visits > 0 && this.quantity_sold > 0 ) {
    var rate = this.quantity_sold / hourly.visits;
    points = parseInt(100*rate);
    v += points;
    explanation += "Conversion rate of " + points + "% ”;
    } else if ( hourly.visits == null || hourly.visits == 0 ) {
    v += 500;
    explanation += "Product has never been seen (500 points). ";
    }
    [snip]
    emit( { gift_product_look_guid : this._id },
    { score : v,
    explanation : explanation}});
    }
  • Step 2: Reduce
    Reduce is a passthrough
    r = function( pid , values ){
    return values[0];
    }
    Map Reduce run every 15 minutes via CRON – results stored in a collection named “scores”
    res = db.gift_product_looks.mapReduce( m , r , { out : "scores" } );
  • Step 3: Update Transactional Systems
    • Mongo + server run on ec2
    • Send “scores” collection back to our primary data center, storing latest scores in our primary relational database
    • Gift items are always sorted by score – transactional system only needed an “order by score desc” clause
  • Questions before we move on to Hummingbird?
  • Tracking Pixels
    /tracking.gif?events=&prop1=women&server=www.gilt.com&products=&pageName=sales%3A+women&channel=sale&prop4=sale+category+page&u=http%3A%2F%2Fwww.gilt.com%2Fsale%2Fwomen&guid=418237ca-2bc6-932e-84c2-d4f02d9fd5bf&gen=f&uid=25423567&cb=443460396
  • Omniture
    GILT
    Data
    Warehouse
    Users
  • Omniture
    24 hours
    later...
    GILT
    Data
    Warehouse
    Users
  • Node.js
    Asynchronous, evented web framework
    http://nodejs.org
  • var mongo = require(’lib/mongodb’);
    var db = new mongo.Db('hummingbird', new mongo.Server('localhost', 27017, {}), {});
    db.createCollection('visits', function(err, collection) {
    db.collection('visits', function(err, collection) {
    collection.insert(env);
    });
    });
  • DEMO