Myself• Many years @ 888 an online Casino & Poker company. Finished as Gaming Architect• CTO @ UMOO, a financial market gaming platform• CTO @ Playtika
Playtika• Founded mid 2010• Social gaming company• Multiple social networks and sites• Multiple platforms (SNs, destination, Android, iOS)• Acquired by by Caesar’s Interactive Entertainment mid 2011
Our Games• A few million daily active users.• A few (more) million monthly active users• Our biggest game today Slotomania – 200K+ concurrent users at peak hours. No less than 100K during off-peak• At least one request per user every 5 seconds• At least one read-modify write on user state per request• In many cases multiple read-modify-writes
Keep MySQL – Why?• Fear is the mind killer• Site to site replication• Can still “SQL Query” the data• Easy as pie…• BUT, not everyone can keep everything on one node.
What do we keep in couchbase?• Everything key value• Anything we need to access fast and frequent – User Score – User Save – User Messages• Only things where true ACIDness is not 100% mandatory or can be faked.
Keys and Values• Be creative with keys and objects• Most common: user_id• E.g. user_id & session_id• E.g. user_id & game_type
“Schema” – SQL vs. Document based• Sometimes you need to turn the data on its head• 2 Table join in SQL = multi get in Couchbase• Excessive ‘multigetting’ = network saturation• Our Lesson: Don’t migrate as is. Refactor/reimagine the schema for a document-oriented database
Serialization• Right now, some JSON some Binary• Intend to move most if not all to JSON – 2.0! – Browse keys without having to have entire context – Easier to support object versioning• Consider other serialization methods (thrift, protobuf)
Data Services or Direct Access?• Encapsulate data access?• Direct Access => High Performance• Data layer Encapsulation => mitigate complex SOA-ish environments
Data Services vs. Direct Access Couchbase Couchbase Data Services DAL Lib DAL Lib RPC Lib RPC Lib Service A Service B Service A Service B
Locks• Pessimistic vs. Optimistic• We use 2 different lock types in different cases:• CAS + spin X times (and fail)• Lock object with timeout (actually can be done better than we do it!)• Avoid contention if possible (especially cross- process ) by old-school tricks – Queue stuff – IP based stickiness (you lose nothing by this) – Don’t be a nerd! Cut corners where possible.
Our cluster(s)• 12 Couchbase buckets + 4 memcache buckets On two clusters• The 2 “heavy” buckets are on their own cluster• 40K ops/s on couchbase, 20K ops/s on memcache• 30-35% Couchbase ops are writes• We still live in a pre-2.0 world (1.8.1)
Ops/s (off-peak)• “Small cluster” (actually with a bug on prod)• “Big cluster”