Introducing social networking into an e commerce platform - (delver) sears holdings IL
Upcoming SlideShare
Loading in...5
×
 

Introducing social networking into an e commerce platform - (delver) sears holdings IL

on

  • 3,443 views

 

Statistics

Views

Total Views
3,443
Views on SlideShare
2,929
Embed Views
514

Actions

Likes
0
Downloads
56
Comments
1

8 Embeds 514

http://www.gigaspaces.com 503
http://translate.googleusercontent.com 3
http://gstil.com 2
http://www.gigaspaces-old.com 2
http://twitter.com 1
http://www.gigaspaces.rsvp1.com 1
http://wiki.gigaspaces.com 1
http://old.gigaspaces.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Image sources (also linked):* Facebook US traffic estimates (http://www.insidefacebook.com/2011/01/03/november-2010-facebook-traffic/)* Top 20 visited websites (http://www.hitwise.com/us/datacenter/main/)
  • Sources:Sumeet Jain: http://vator.tv/news/2010-12-27-2011-location-and-social-will-rule-commerceAndy Leaver: http://www.currybet.net/cbet_blog/2009/11/notes-and-quotes-from-ecommerc-2.phpGordon Gould: http://socialcommercetoday.com/will-2010-be-the-year-of-social-commerce/
  • Case in point:Discovery and recommendation features: when was the last time YOU “might be interested in this product”? How accurate are the typical recommendation systems for you?Just how relevant is the typical ad or marketing campaign? When was the last time you went into Amazon and got a coupon for a truly relevant occasion or product? How tired are you of flashing banners?
  • Public data source: http://bizinformation.ca/www.sears.com#visitors
  • Massive amounts of data:* Imagine modeling every person on the planet (say, 6 billion). Now say each person is connected to just 100 others (a conservative estimate)Image source: http://networkweaver.blogspot.com/2010/03/overlapping-boards.html
  • SPAR presentation: http://www.slideshare.net/jmpujol/the-little-engines-that-could-scaling-online-social-networksImage source: http://paysa09.wikispaces.com/Networks
  • Time distribution image source: http://blog.nielsen.com/nielsenwire/global/social-norms-twitter-users-follow-the-797-rule-in-the-u-k/Twitter following distribution image source: http://www.personalizemedia.com/twitter-long-tail-broadcastization-pre-twitter-reputation/
  • Inevitably, someone will ask: what are the problems you encountered?Barrier of entry:Ops: setting up a GigaSpaces cluster is not a hassle-free affair. Lots of work went into a robust, efficient bootstrapping procedure and we had to content with quite a few unexpected snags. I believe things are a lot better with the current version than they were a while ago. Furthermore, the overall cost of setting up and deploying GigaSpaces is significantly less than the total overhead for using specific products to tackle our various needs (compared to a traditional system, the cost of setting up e.g. MySQL+RHCS+client configuration; more likely we’d have had to use some sort of 3rd party graph storage, clustering and persistence solution)Devs: working against GigaSpacesis considerably harder than vanilla, commonplace RDBMS. To counter the barrier of entry we modeled our organization so that a core team of developers handle graph storage and data analysis, with most other teams either integrating with this subsystem or handling their own requirements with regular Hibernate/NHibernate over MySQL.Hard to handle migration paths, zero-time deployment and schema evolution. Features in 8.0 should help remedy the situation (cue Nati Shalom)

Introducing social networking into an e commerce platform - (delver) sears holdings IL Introducing social networking into an e commerce platform - (delver) sears holdings IL Presentation Transcript

  • Introducing Social Networking Into an e-commerce Platform
    Tomer Gabel |Sears Holdings IL
    03.02.2011
  • Social Commerce: An Introduction
    The last few years have seen tremendous growth in social networks
    Some estimates place Facebook above Google
    Even if not, we’re talking millions of daily unique visitors
    So the obvious question is… where’s the money?
    2
  • Social Commerce: An Introduction
    3
  • Social Commerce: Business Case
    What’s wrong with traditional e-commerce?
    Discovery/recommendation features are extremely hard to get right
    Overly broad market targeting means lost sales and disgruntled, ad-weary customers
    The trust model is inherently broken
    Impossible to gauge truth and accuracy in customer reviews
    “Wisdom of the masses” does not always apply
    Not fun!
    Shopping is a social experience (going to the mall, holiday shopping sprees)
    This does not translate to existing e-commerce sites!
    4
  • Social Commerce: Business Case
    “Social commerce” aims to address these deficiencies
    Correlating interests and products is more accurate and significantly easier when based on social context
    Social circles are inherently constructed on shared interests and perspectives
    A customer’s social network is much smaller in scope than generating a global, statistical recommendation model
    More accurate personalized data exposes new opportunities
    Personalized discovery allows more opportunity to tap the long tail
    Social interaction makes it easy to identify domain experts
    A single opinion provided by a friend, family member or acquaintance is more trustworthy than dozens of unrelated product reviews/ratings
    5
  • Social Commerce: Business Case
    Most crucially, social commerce is all about user engagement and collaboration:
    Should I buy an iPhone, Blackberry or Android phone?
    Which wedding dress looks best?
    Which video games are suitable for a preschooler?
    6
    Ask your friends!
  • Social Commerce: The Axiom
    Social features increase user engagement
    Increased conversion
    Profit!
    7
  • 8
    Enter: Delver
  • Enter: Delver
    The Delver team has two products on the market
    Two sides of the same coin, really:
    sears.com is a traditional e-commerce website with a social twist
    delver.com is a traditional social website with an e-commerce twist
    9
  • The Technical Challenge
    sears.com is a fully blown commercial retail site
    Over 1 million page-views daily
    Over 270,000 visitors daily
    Traffic can easily spike up to ten times in the holiday season!
    10
  • The Technical Challenge
    Processing social networks is not an easy proposition
    Massive amounts of branching data
    No data locality
    Very few assumptions can be made about the data
    Let’s address each of these in turn
    11
    Source: NetworkWeaver
  • The Technical Challenge
    Massive amounts of branching data:
    Imagine every Facebook user (500 million)
    Imagine each person is only connected to 100 others (conservative estimate)
    How is user X connected with Y?
    X has 100 friends
    Each of them has 100 friends
    10,001 nodes visited!
    101 reads from the underlying storage system!
    12
    X
    Y
  • The Technical Challenge
    No data locality:
    Any object may be connected to any other object in no particular order
    How to split the data?
    Some research is being done in the area (SPAR)
    13
  • The Technical Challenge
    No easy assumptions:
    No “typical user”
    Not enough data to draw archetypes
    Significant, unavoidable long tail
    Difficult to pre-tune data structures
    14
  • The Technical Challenge
    The crux of the problem:
    High branch factor necessitates many loads to serve even a simple request
    No data locality + high branch factor means very high random I/O
    Traditional storage models (RDBMS, flat files etc.) are a poor fit
    Serious research into graph storage, social network composition etc. only dates back a few years
    No best practices or “accepted truths” to build on
    15
  • Use Case for GigaSpaces
    To solve the graph storage and traversal problem, we arrived at the following requirements:
    Completely in-memory storage
    No data locality means caching is inefficient
    Massive amounts of random I/O cannot scale vertically, and hardware (basically, spindle count) cost quickly becomes prohibitive
    If data access is sufficiently fast, data can be randomly partitioned
    Horizontal scaling with a well-known scale-up strategy
    Add more memory or more nodes to handle data growth
    Add more CPUs or additional nodes to handle load growth
    16
  • Use Case for GigaSpaces
    Additional requirements include:
    Map/Reduce execution framework
    Graph traversal and data analysis requirements lend well to the map/reduce paradigm
    Code execution on the data nodes
    Because of the massive amounts of data involved, the network interface will be quickly saturated by retrievals
    Memory retrieval is at least two orders of magnitude faster than network throughput (DDR2-800 on a dual channel memory controller has a theoretical throughput maximum of 102.4Gb/s)
    17
  • Use Case for GigaSpaces
    As an operations tech I had a few things to add to the list, namely…
    Nonfunctional requirements:
    Built-in fault tolerance and high availability
    Zero-configuration (or as close to it as it gets) setup; in particular, component discovery and assignment must be automated
    Well-documented deployment, configuration and tuning process
    Monitoring API
    Administrative client for diagnosis, trouble resolution and manual intervention
    18
  • Use Case for GigaSpaces
    GigaSpaces features map well to our requirements
    Data grid
    Compute grid
    High availability
    Horizontal data and load scaling
    Management API
    Very few viable alternatives:
    Hadoop, neo4j are disk-based
    Terracotta is overly simplistic and has no execution framework
    Oracle Coherence is expensive and has a limited feature set
    19
  • Delver Architecture
    We ended up with a hybrid platform:
    GigaSpaces for graph storage, traversal and analysis
    MySQL for traditional, “simple” data as well as a backing store for GigaSpaces
    .NET-based front-end, Java-based back-end
    We had to factor our organization accordingly
    Data access team provides abstracted interfaces on top of GigaSpaces and MySQL
    Back-end “heavy lifting” services (e.g. recommendation engine) work directly against GigaSpaces
    Most other components either use the abstracted DAL or are simple enough to work directly against MySQL using (N)Hibernate
    20
  • Delver Architecture
    21
  • Key Benefits
    Significantly reduced integration costs
    GigaSpaces does a lot of what we need out of the box
    An alternative solution would require integrating several products, incurring significant integration and development overhead
    Broad feature set
    Social commerce is an emerging, dynamic market requiring rapid experimentation and adaptation
    The large feature set allows us to introduce new features into the system at a furious pace
    While primarily intended for graph storage, we also use GigaSpaces as a message queue, distributed lock server and distributed scheduler
    22
  • Now is a good time for…
    Questions?COMMENTS?
    23
  • Endgame
    Experience our work!
    Visit Delver at http://www.delver.com/in?invite=friends-and-family
    Visit Sears Social at http://catalog.sears.com
    Read about our work at http://blog.delver.com
    Have anything to discuss?
    Contact me at tomer@delver.com
    Visit my blog at http://www.tomergabel.com
    Follow me on Twitter at http://www.twitter.com/tomerg
    Thank you for your time!
    24