LivePerson uses CouchBase for real-time analytics of visitor data to provide visibility to customers on their online visitors. Previously, visitor state was stored in memory on stateful web servers, limiting scalability. CouchBase was chosen for its performance, resilience, linear scalability, schema flexibility, and ability to handle LivePerson's high throughput of over 1 million concurrent visitors and 100k operations per second. It is used to store visitor documents containing events and is queried to return relevant visitors to agents. Cross data center replication is also used to improve resilience. LivePerson has found CouchBase easy to develop on and has expanded its use to additional cases like session state and caching.

1. CouchBase @
LivePerson
Ido Shilon | 7/22/2013 | idos@liveperson.com

2. Agenda
● Connection before content
● Who LivePerson is
● Data & Technology @ LivePerson
● What is the use case (Real time analytics)
● The Story
● Why CouchBase was chosen
● Architecture & Code
● Production
● What did we learn so far
● Q&A

3. Company
● The leading customer intelligent
engagement platform
● 8,500 customers around the globe
● 8 of the top 10 Fortune 500 companies
● Doing SaaS from 1999 (when it was
called ASP)
Mission
Creating
Meaningful
Customer
Connections
LivePerson is…

4. Typical Engagements

5. Technology @ LivePerson
● Application Stack
○ JVM heavy
○ Linux on commodity servers
○ Private cloud based on openstack

6. Data @ LivePerson
● 1.8 billion visitors monitored (sessions) per
month
● 20 million connections per month
● ~0.5TB compressed data is loading into
Hadoop each day
KafkaStorm

7. The LivePerson use case
Visitor List
Real time Analytics for
LivePerson's customers

8. What do we need it for ?
● Provide visibility to LivePerson's customers on
their online visitors
● Been used by thousands of agents (call centers)
around the world

9. The story - once upon a time
Agents
Console
(Java app)
Visitor events/data
Web site Visitors
Monitoring
Events
● Stateful web servers (vertical
scalability)
● Multi purpose servers
● Hold all visitors state in memory

10. DC 2
DC1
And then the story continue
Web Agent
(Web app) ??????
Web site Visitors
Monitoring
Events
● Multiple web servers (horizontal scalability)
● SOA architecture
● All visitors states are streamed to event bus
Kafka & Strom
(Event bus)
Send
events
Send
events

11. Possible solutions we considered

12. Why did we picked Couchbase
● Performance, high throughput, really fast
● Resilience solution
● Linear scale
● Schemaless !!!
● Searchable (Queries)
● Supports both K/V & document store
● Cross data center replication
● Simplicity (quick dev and roll out)
● We needed a solution !!!

13. System architecture
Visitor
(Browser/Mobile)
Stream Event Processing
Visitor Feed -
Storm Topology
Customer
Representative
Kafka
Couchbase
Visitor Monitoring
Service
(1) Visitor
browsing
(2) Visitor
events
(4) Write event to
user document
(6) Return relevant
visitors
(7) Return relevant
visitors
(5) Get
visitors
List
Every 3
sec
Visitor Feed API
(3) Analyze relevant
events and persist

14. Data design & numbers
● Document = User
● Document Structure :
○ Each document contains 15-20 attributes, in
addition to 3 lists of sub attributes
○ Each doc contain the account id (multi
tenant db)

15. Data design & numbers
● Numbers
○ Avg doc size - 10 K
○ Average key size - 10 characters
○ 5 2nd level indexes
● Throughput (Final rollout)
○ ~ 1 M concurrent documents/visitors
○ ~ 100K ops/sec (heavy on insert/update)

16. Insert/Update
public Visitor getVisitor(String visitorSessionId) {
dalMetrics.addCouchbaseReadTotalCount();
Visitor visitor = null;
try {
String visitorDoc = (String) client.get(visitorSessionId);
visitor = gson.fromJson(visitorDoc, Visitor.class);
} catch (Exception e) {
LOG.debug("Failed to retrieve or convert visitor: "+ e.getMessage());
dalMetrics.addCouchbaseReadErrorCount();
throw e;
}
return visitor;
}
public void setVisitorWithFields(String rtSessionId, Visitor visitor) {
try {
client.set(rtSessionId, defaultTtl, gson.toJson(visitor));
} catch (Exception e) {
LOG.error("Error occurred while updating visitor fields: "+ e.getMessage());
}
}

17. Views/ Design doc
Use the view to set the keys and sorting
function (doc, meta) {
order = ....
if (order, doc.accountId, doc.visitStartTime.fieldValue) {
emit([doc.accountId, order, doc.visitStartTime.fieldValue],null);
}
}

18. Retrieve data
.../_view/by_accountid_state_timestamp?limit=10&skip=0&startkey=
["qa15020713", 0 ]&endkey=["qa15020713" , 9 ]
ComplexKey startKey = null;
// Create a new View Query
Query query = new Query();
query.setIncludeDocs(true); // Include the full document as
if (startValueToFilterBy != null) {
startKey = ComplexKey.of(accountId, startValueToFilterBy);
}
if (endValueToFilterBy == null && startKey != null) {
query.setKey(startKey);
} else {
ComplexKey endKey= ComplexKey.of(accountId, endValueToFilterBy);
query.setRange(startKey, endKey);
}
if (limit > 0) {
query.setLimit(limit);
}
if (skip > 0) {
query.setSkip(skip);
}

19. Cross data center replication options
1. Unidirectional replication to replicate the
data to our DR data center
2. (Future) Bi Directional replication, each data
center holds portion of entire the traffic
Insights :
○ Keyspace is the same (in both scenarios) -
avoid conflicts
○ Impact on the cluster size - from 5 nodes to
7-8 nodes in each cluster

20. What did we learn till now ?
● Delete docs
○ Use TTL instead instead of delete
○ Use longer TTL if possible
● In our use case the working set is around 100 %
- RAM and SSD are key factors in scalability
● Move to production ASAP, even for staging !!

21. Couchbase in LP - Additional Use cases
● Session state
● Cross Session state
● Caching layer - Memcached style

22. Thank You