I get by with a little help
from my friends

Ido Shilon | 7/22/2013
Today’s programme

Who is LivePerson?
What was our journey?
Why was Couchbase chosen?
How did we build the new platform?
W...
@idoshilon

{
name: "Ido Shilon",
age: 37,
kids: [
"illy"
],
wife: "Oshrat",
Title: "Group Manager @ LivePerson (Heading t...
LivePerson is…

Mission

Customers

Creating Meaningful
Customer Connections

8,500
customers

Technology

SaaS pioneer si...
Optimize Customer Acquisition & Reduce Bounce Rate

Live engagement for
Rich multimedia to
lingering customer
drive sales ...
Technology @ LivePerson

Application Stack
JVM heavy - Java & Scala
Private cloud based on openstack
Linux on commodity se...
Data @ LP

13
VOLUME

TB

per	
  month

20

M

Engagements	
  per	
  month

1.8

B

Visits	
  per	
  month
Data stack

MONITORING

CHAT/VOICE
system

APACHE KAFKA

Batch track

Real-Time track

PERPETUAL STORE

COMPLEX EVENT
PROC...
The use case

Web agent console
Enables your agents to interact with
website visitors
Improve agent efficiency
Reduce chat...
The story - once upon a time

Agents console
(Java app)

Web Tier

Visitor’s
Events

Visitors
And then the story continues

Web Agent

???

Data center 1

Kafka & Strom
(Event bus)

Data center 2
Possible solutions we considered
Why did we pick Couchbase

Cassandra

Always on
Linear scale
Searchable
Document store
Key Value
High throughput (R/W)
XDC...
Architecture

REST API

Application server
Tomcat
Couchbase Java SDK

cluster

cluster

XDCR

M/R views

M/R views

Storm ...
Data flow

Agent

Visitor
(5) Get
visitors List
Every 3 sec

(7) Return
relevant
visitors

(1) Visitor
browsing
Visitor
Mo...
Data modeling - doc

Visitor type doc
Document = Active visitor
Contain session level attributes
Multi tenant bucket
Views...
Document structure

{
"accountId": "64302875",
"id": 121640710013,
"rtSessionId": "643028754295878498",
"eventSequence": 5...
Cross data center replication

Using Bi Directional replication (A/A)

Tips :
Key space is the same across the two
cluster...
Couchbase in production
What did we learn till now ?

Test in production
Use the Couchbase sizing guidelines
RAM and IO (SSD) are key factors in
D...
Data stack now with Couchbase

MONITORING

CHAT/VOICE
system

APACHE KAFKA

Batch track

Real-Time track

PERPETUAL STORE
...
Couchbase in LP - Strategic choice

Visitor session store/ Server side cookie
Fast RW persistent store
Generic caching lay...
Couchbase in LP – What are we looking for ?

N1QL
Inmemory indexing
Better security model
Summary

Who we are
Our journey
The problem we have encountered
We are not alone - Couchbase helped us
Thank You
We are hiring
Backup slides
Sizing

Key name size(bytes)
Average value size(bytes)

100	
  
10240	
  

Number of replicas

1	
  

Ram node size(GB)

6...
Couchbase in production - topology
Couchbase in production
Data modeling - doc

Non reduce views - used for
secondary indexes
Map reduce views – used for
calculating stats
Data sizing

•  Numbers
o  Avg doc size - 10 K
o  Average key size - 10 characters
o  5 2nd level indexes

•  Throughput (...
Document structure

{
"accountId": "64302875",
"id": 121640710013,
........................
"pages": [
{
"pageLastAccessTi...
The story - once upon a time

Visitors

Visitor’s
Events

Agents console
(Java app)

Web Tier
Insert/Update
public Visitor getVisitor(String visitorSessionId) {
dalMetrics.addCouchbaseReadTotalCount();
Visitor visito...
Views/ Design doc

Use the view to set the keys and sorting

function (doc, meta) {
order = ....
if (order, doc.accountId,...
Retrieve data

.../_view/by_accountid_state_timestamp?

limit=10&skip=0&startkey=["qa15020713", 0 ]&endkey=["qa15020713" ,...
The Moment of Truth: Close sales from any location
Orbitz Mobile:
Saving a Sale
Customer having difficulty

Customer is us...
Data @ LivePerson

•  1.8 billion visitors monitored (sessions)
per month

•  20 million connections per month
•  ~0.5TB c...
Why did we picked Couchbase

High throughput, really fast
Linear scale
Searchable (Views and M/R)
Supports both K/V & Docu...
Upcoming SlideShare
Loading in...5
×

Couchbase_TLV_2014_Couchbase_at_LivePerson

22,678

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
22,678
On Slideshare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
31
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Couchbase_TLV_2014_Couchbase_at_LivePerson

  1. 1. I get by with a little help from my friends Ido Shilon | 7/22/2013
  2. 2. Today’s programme Who is LivePerson? What was our journey? Why was Couchbase chosen? How did we build the new platform? What can you take with you?
  3. 3. @idoshilon { name: "Ido Shilon", age: 37, kids: [ "illy" ], wife: "Oshrat", Title: "Group Manager @ LivePerson (Heading the platform group)", Lived_Worked_At: [ "Silicon Wadi (Israel)", "Silicon Alley (NYC)", "Silicon Valley (Bay Area)" ] }
  4. 4. LivePerson is… Mission Customers Creating Meaningful Customer Connections 8,500 customers Technology SaaS pioneer since 1998
  5. 5. Optimize Customer Acquisition & Reduce Bounce Rate Live engagement for Rich multimedia to lingering customer drive sales closure Recommended use case
  6. 6. Technology @ LivePerson Application Stack JVM heavy - Java & Scala Private cloud based on openstack Linux on commodity servers Client side – js, backbone, iOS and Android
  7. 7. Data @ LP 13 VOLUME TB per  month 20 M Engagements  per  month 1.8 B Visits  per  month
  8. 8. Data stack MONITORING CHAT/VOICE system APACHE KAFKA Batch track Real-Time track PERPETUAL STORE COMPLEX EVENT PROCESSING STORM ANALYTICAL DB RT REPOSITORY BUSINESS INTELLIGENCE Cassandra LiveEngage DASHBOARD
  9. 9. The use case Web agent console Enables your agents to interact with website visitors Improve agent efficiency Reduce chat time
  10. 10. The story - once upon a time Agents console (Java app) Web Tier Visitor’s Events Visitors
  11. 11. And then the story continues Web Agent ??? Data center 1 Kafka & Strom (Event bus) Data center 2
  12. 12. Possible solutions we considered
  13. 13. Why did we pick Couchbase Cassandra Always on Linear scale Searchable Document store Key Value High throughput (R/W) XDCR
  14. 14. Architecture REST API Application server Tomcat Couchbase Java SDK cluster cluster XDCR M/R views M/R views Storm Topology Storm Topology Couchbase Java SDK Couchbase Java SDK
  15. 15. Data flow Agent Visitor (5) Get visitors List Every 3 sec (7) Return relevant visitors (1) Visitor browsing Visitor Monitoring Service Visitor Feed API (2) Visitor events Kafka (6) Return relevant visitors Couchbase (4) Write event to visitor document Visitor Feed Storm Topology (3) Analyze relevant events
  16. 16. Data modeling - doc Visitor type doc Document = Active visitor Contain session level attributes Multi tenant bucket Views Used for secondary indexes
  17. 17. Document structure { "accountId": "64302875", "id": 121640710013, "rtSessionId": "643028754295878498", "eventSequence": 5104, "ipAddress": { "fieldValue": "194.39.63.10", "seq": 1 }, Multi tenant DB "browser": { "fieldValue": "Chrome 27.0.1453.116", "seq": 1 }, Basic visitor "state": { information "fieldValue": "LEFT_SITE", "seq": 5104 } ...................................... Sequence use due to Kafka }
  18. 18. Cross data center replication Using Bi Directional replication (A/A) Tips : Key space is the same across the two clusters (avoid conflicts) Impact on the sizing
  19. 19. Couchbase in production
  20. 20. What did we learn till now ? Test in production Use the Couchbase sizing guidelines RAM and IO (SSD) are key factors in Disk write queue for views
  21. 21. Data stack now with Couchbase MONITORING CHAT/VOICE system APACHE KAFKA Batch track Real-Time track PERPETUAL STORE COMPLEX EVENT PROCESSING STORM ANALYTICAL DB RT REPOSITORY BUSINESS INTELLIGENCE Cassandra LiveEngage DASHBOARD
  22. 22. Couchbase in LP - Strategic choice Visitor session store/ Server side cookie Fast RW persistent store Generic caching layer (Memcached style) RDMB replacement (MySQL)
  23. 23. Couchbase in LP – What are we looking for ? N1QL Inmemory indexing Better security model
  24. 24. Summary Who we are Our journey The problem we have encountered We are not alone - Couchbase helped us
  25. 25. Thank You We are hiring
  26. 26. Backup slides
  27. 27. Sizing Key name size(bytes) Average value size(bytes) 100   10240   Number of replicas 1   Ram node size(GB) 64   Number of keys 2,000,000   Working Set (% of total dataset) 100%   Read ops/sec (sustained) 3,000   Write (Ins / Upd ) (sustained) 3,000   Deletes / Expirations / sec Assumptions 3,000   Same for remote cluster in case of bi-directional XDCR. Same key space across the two clusters.
  28. 28. Couchbase in production - topology
  29. 29. Couchbase in production
  30. 30. Data modeling - doc Non reduce views - used for secondary indexes Map reduce views – used for calculating stats
  31. 31. Data sizing •  Numbers o  Avg doc size - 10 K o  Average key size - 10 characters o  5 2nd level indexes •  Throughput (Final rollout) o  ~ 1 M concurrent documents/visitors o  ~ 100K ops/sec (heavy on insert/update)
  32. 32. Document structure { "accountId": "64302875", "id": 121640710013, ........................ "pages": [ { "pageLastAccessTime": "1372319894409", "pageTitle": "Visit Site 64302875", "pageUrl": "http://www.exaple.com/createSite.jsp? server=hc1&siteid=64302875", "seq": 4 List of pages }, the visitor has { browsed "pageLastAccessTime": "1372323616982", "pageTitle": "Visit Site 64302875", Lists of "pageUrl": "http://hc1d/createSite.jsp? server=hc1&siteid=64302875", custom "seq": 980 }, variables ]}
  33. 33. The story - once upon a time Visitors Visitor’s Events Agents console (Java app) Web Tier
  34. 34. Insert/Update public Visitor getVisitor(String visitorSessionId) { dalMetrics.addCouchbaseReadTotalCount(); Visitor visitor = null; try { String visitorDoc = (String) client.get(visitorSessionId); visitor = gson.fromJson(visitorDoc, Visitor.class); } catch (Exception e) { LOG.debug("Failed to retrieve or convert visitor: " + e.getMessage()); dalMetrics.addCouchbaseReadErrorCount(); throw e; } return visitor; } public void setVisitorWithFields(String rtSessionId, Visitor visitor) { try { client.set(rtSessionId, defaultTtl, gson.toJson(visitor)); } catch (Exception e) { LOG.error("Error occurred while updating visitor fields: " + e.getMessage()); } }
  35. 35. Views/ Design doc Use the view to set the keys and sorting function (doc, meta) { order = .... if (order, doc.accountId, doc.visitStartTime.fieldValue) { doc.visitStartTime.fieldValue],null); } } emit([doc.accountId, order,
  36. 36. Retrieve data .../_view/by_accountid_state_timestamp? limit=10&skip=0&startkey=["qa15020713", 0 ]&endkey=["qa15020713" , 9 ] ComplexKey startKey = null; // Create a new View Query Query query = new Query(); query.setIncludeDocs(true); // Include the full document as if (startValueToFilterBy != null) { startKey = ComplexKey.of(accountId, startValueToFilterBy); } if (endValueToFilterBy == null && startKey != null) { query.setKey(startKey); } else { ComplexKey endKey = ComplexKey.of(accountId, endValueToFilterBy); query.setRange(startKey, endKey); } if (limit > 0) { query.setLimit(limit); } if (skip > 0) { query.setSkip(skip); }
  37. 37. The Moment of Truth: Close sales from any location Orbitz Mobile: Saving a Sale Customer having difficulty Customer is using wifi at the airport to change flight details (no global calling ability) booking a flight on a mobile Send a message to our live service reps for immediate help Customer initiates chat to get assistance and close the purchase 25-30% of retail traffic is now coming from mobile devices (source: LivePerson Connecting with Customers Research 2013) Me: Hi. I’m having trouble with reserving this flight. Mitch: Hello. No problem, I can help you with that.
  38. 38. Data @ LivePerson •  1.8 billion visitors monitored (sessions) per month •  20 million connections per month •  ~0.5TB compressed data is loading into Hadoop each day Storm Kafka
  39. 39. Why did we picked Couchbase High throughput, really fast Linear scale Searchable (Views and M/R) Supports both K/V & Document store Cross data center replication “Always on”, Resilience solution
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×