Your SlideShare is downloading. ×
Introducing social networking into an e commerce platform - (delver) sears holdings IL
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Introducing social networking into an e commerce platform - (delver) sears holdings IL

3,028
views

Published on

Published in: Technology, Business

1 Comment
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total Views
3,028
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
58
Comments
1
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Image sources (also linked):* Facebook US traffic estimates (http://www.insidefacebook.com/2011/01/03/november-2010-facebook-traffic/)* Top 20 visited websites (http://www.hitwise.com/us/datacenter/main/)
  • Sources:Sumeet Jain: http://vator.tv/news/2010-12-27-2011-location-and-social-will-rule-commerceAndy Leaver: http://www.currybet.net/cbet_blog/2009/11/notes-and-quotes-from-ecommerc-2.phpGordon Gould: http://socialcommercetoday.com/will-2010-be-the-year-of-social-commerce/
  • Case in point:Discovery and recommendation features: when was the last time YOU “might be interested in this product”? How accurate are the typical recommendation systems for you?Just how relevant is the typical ad or marketing campaign? When was the last time you went into Amazon and got a coupon for a truly relevant occasion or product? How tired are you of flashing banners?
  • Public data source: http://bizinformation.ca/www.sears.com#visitors
  • Massive amounts of data:* Imagine modeling every person on the planet (say, 6 billion). Now say each person is connected to just 100 others (a conservative estimate)Image source: http://networkweaver.blogspot.com/2010/03/overlapping-boards.html
  • SPAR presentation: http://www.slideshare.net/jmpujol/the-little-engines-that-could-scaling-online-social-networksImage source: http://paysa09.wikispaces.com/Networks
  • Time distribution image source: http://blog.nielsen.com/nielsenwire/global/social-norms-twitter-users-follow-the-797-rule-in-the-u-k/Twitter following distribution image source: http://www.personalizemedia.com/twitter-long-tail-broadcastization-pre-twitter-reputation/
  • Inevitably, someone will ask: what are the problems you encountered?Barrier of entry:Ops: setting up a GigaSpaces cluster is not a hassle-free affair. Lots of work went into a robust, efficient bootstrapping procedure and we had to content with quite a few unexpected snags. I believe things are a lot better with the current version than they were a while ago. Furthermore, the overall cost of setting up and deploying GigaSpaces is significantly less than the total overhead for using specific products to tackle our various needs (compared to a traditional system, the cost of setting up e.g. MySQL+RHCS+client configuration; more likely we’d have had to use some sort of 3rd party graph storage, clustering and persistence solution)Devs: working against GigaSpacesis considerably harder than vanilla, commonplace RDBMS. To counter the barrier of entry we modeled our organization so that a core team of developers handle graph storage and data analysis, with most other teams either integrating with this subsystem or handling their own requirements with regular Hibernate/NHibernate over MySQL.Hard to handle migration paths, zero-time deployment and schema evolution. Features in 8.0 should help remedy the situation (cue Nati Shalom)
  • Transcript

    • 1. Introducing Social Networking Into an e-commerce Platform
      Tomer Gabel |Sears Holdings IL
      03.02.2011
    • 2. Social Commerce: An Introduction
      The last few years have seen tremendous growth in social networks
      Some estimates place Facebook above Google
      Even if not, we’re talking millions of daily unique visitors
      So the obvious question is… where’s the money?
      2
    • 3. Social Commerce: An Introduction
      3
    • 4. Social Commerce: Business Case
      What’s wrong with traditional e-commerce?
      Discovery/recommendation features are extremely hard to get right
      Overly broad market targeting means lost sales and disgruntled, ad-weary customers
      The trust model is inherently broken
      Impossible to gauge truth and accuracy in customer reviews
      “Wisdom of the masses” does not always apply
      Not fun!
      Shopping is a social experience (going to the mall, holiday shopping sprees)
      This does not translate to existing e-commerce sites!
      4
    • 5. Social Commerce: Business Case
      “Social commerce” aims to address these deficiencies
      Correlating interests and products is more accurate and significantly easier when based on social context
      Social circles are inherently constructed on shared interests and perspectives
      A customer’s social network is much smaller in scope than generating a global, statistical recommendation model
      More accurate personalized data exposes new opportunities
      Personalized discovery allows more opportunity to tap the long tail
      Social interaction makes it easy to identify domain experts
      A single opinion provided by a friend, family member or acquaintance is more trustworthy than dozens of unrelated product reviews/ratings
      5
    • 6. Social Commerce: Business Case
      Most crucially, social commerce is all about user engagement and collaboration:
      Should I buy an iPhone, Blackberry or Android phone?
      Which wedding dress looks best?
      Which video games are suitable for a preschooler?
      6
      Ask your friends!
    • 7. Social Commerce: The Axiom
      Social features increase user engagement
      Increased conversion
      Profit!
      7
    • 8. 8
      Enter: Delver
    • 9. Enter: Delver
      The Delver team has two products on the market
      Two sides of the same coin, really:
      sears.com is a traditional e-commerce website with a social twist
      delver.com is a traditional social website with an e-commerce twist
      9
    • 10. The Technical Challenge
      sears.com is a fully blown commercial retail site
      Over 1 million page-views daily
      Over 270,000 visitors daily
      Traffic can easily spike up to ten times in the holiday season!
      10
    • 11. The Technical Challenge
      Processing social networks is not an easy proposition
      Massive amounts of branching data
      No data locality
      Very few assumptions can be made about the data
      Let’s address each of these in turn
      11
      Source: NetworkWeaver
    • 12. The Technical Challenge
      Massive amounts of branching data:
      Imagine every Facebook user (500 million)
      Imagine each person is only connected to 100 others (conservative estimate)
      How is user X connected with Y?
      X has 100 friends
      Each of them has 100 friends
      10,001 nodes visited!
      101 reads from the underlying storage system!
      12
      X
      Y
    • 13. The Technical Challenge
      No data locality:
      Any object may be connected to any other object in no particular order
      How to split the data?
      Some research is being done in the area (SPAR)
      13
    • 14. The Technical Challenge
      No easy assumptions:
      No “typical user”
      Not enough data to draw archetypes
      Significant, unavoidable long tail
      Difficult to pre-tune data structures
      14
    • 15. The Technical Challenge
      The crux of the problem:
      High branch factor necessitates many loads to serve even a simple request
      No data locality + high branch factor means very high random I/O
      Traditional storage models (RDBMS, flat files etc.) are a poor fit
      Serious research into graph storage, social network composition etc. only dates back a few years
      No best practices or “accepted truths” to build on
      15
    • 16. Use Case for GigaSpaces
      To solve the graph storage and traversal problem, we arrived at the following requirements:
      Completely in-memory storage
      No data locality means caching is inefficient
      Massive amounts of random I/O cannot scale vertically, and hardware (basically, spindle count) cost quickly becomes prohibitive
      If data access is sufficiently fast, data can be randomly partitioned
      Horizontal scaling with a well-known scale-up strategy
      Add more memory or more nodes to handle data growth
      Add more CPUs or additional nodes to handle load growth
      16
    • 17. Use Case for GigaSpaces
      Additional requirements include:
      Map/Reduce execution framework
      Graph traversal and data analysis requirements lend well to the map/reduce paradigm
      Code execution on the data nodes
      Because of the massive amounts of data involved, the network interface will be quickly saturated by retrievals
      Memory retrieval is at least two orders of magnitude faster than network throughput (DDR2-800 on a dual channel memory controller has a theoretical throughput maximum of 102.4Gb/s)
      17
    • 18. Use Case for GigaSpaces
      As an operations tech I had a few things to add to the list, namely…
      Nonfunctional requirements:
      Built-in fault tolerance and high availability
      Zero-configuration (or as close to it as it gets) setup; in particular, component discovery and assignment must be automated
      Well-documented deployment, configuration and tuning process
      Monitoring API
      Administrative client for diagnosis, trouble resolution and manual intervention
      18
    • 19. Use Case for GigaSpaces
      GigaSpaces features map well to our requirements
      Data grid
      Compute grid
      High availability
      Horizontal data and load scaling
      Management API
      Very few viable alternatives:
      Hadoop, neo4j are disk-based
      Terracotta is overly simplistic and has no execution framework
      Oracle Coherence is expensive and has a limited feature set
      19
    • 20. Delver Architecture
      We ended up with a hybrid platform:
      GigaSpaces for graph storage, traversal and analysis
      MySQL for traditional, “simple” data as well as a backing store for GigaSpaces
      .NET-based front-end, Java-based back-end
      We had to factor our organization accordingly
      Data access team provides abstracted interfaces on top of GigaSpaces and MySQL
      Back-end “heavy lifting” services (e.g. recommendation engine) work directly against GigaSpaces
      Most other components either use the abstracted DAL or are simple enough to work directly against MySQL using (N)Hibernate
      20
    • 21. Delver Architecture
      21
    • 22. Key Benefits
      Significantly reduced integration costs
      GigaSpaces does a lot of what we need out of the box
      An alternative solution would require integrating several products, incurring significant integration and development overhead
      Broad feature set
      Social commerce is an emerging, dynamic market requiring rapid experimentation and adaptation
      The large feature set allows us to introduce new features into the system at a furious pace
      While primarily intended for graph storage, we also use GigaSpaces as a message queue, distributed lock server and distributed scheduler
      22
    • 23. Now is a good time for…
      Questions?COMMENTS?
      23
    • 24. Endgame
      Experience our work!
      Visit Delver at http://www.delver.com/in?invite=friends-and-family
      Visit Sears Social at http://catalog.sears.com
      Read about our work at http://blog.delver.com
      Have anything to discuss?
      Contact me at tomer@delver.com
      Visit my blog at http://www.tomergabel.com
      Follow me on Twitter at http://www.twitter.com/tomerg
      Thank you for your time!
      24