An overview of experiences of moving from MongoDB to Cassandra from the team at metabroadcast.

1. MongoDB to Cassandra
The Atlas Odyssey
Fred van den Driessche Tom McAdam Adam Horwich
Engineer CTO Systems Engineer
@fredvdd @tfm @Mmmkayness

3. http://flickr.com/photos/dhammza/88644497/

4. Our platform - late 2012
tbc tbc
MetaBroadcast platform
Video and audio metadata Profiles and activity from video and
from 20+ sources Analytic requests and groupings
audio products, social networks

5. ?

6. Main clients Main Partners
Data Partners

7. What is Atlas?
/content
BBC
/schedules
/topics
PA
ATLAS
C4
sitemaps
radioplayer
etc... DB
interlinking

8. DEMO

9. Atlas Data Model
brand item
series version
broadcast location

10. MongoDB
• flexible
• features
• really simple
• shell

11. Where MongoDB falls short
• too simple
• lack of control
• sharding
• embedding

12. Where to?

13. Where to?
• add a cache?

14. Atlas API
• content
• http://atlas.metabroadcast.com/3.0/content.json?uri=http://www.bbc.co.uk/programmes/
b0074g7p&annotations=description,brand_summary,locations&apiKey=6ed2a984627daff816198acde82
• http://atlas.metabroadcast.com/3.0/content.json?apiKey=aaaa&uri=http://www.bbc.co.uk/programmes/
b0074g7p&annotations=description,brand_summary,locations
• schedules
• http://atlas.metabroadcast.com/3.0/schedule.json?from=now&to=now.plus.
3h&channel=bbcone&publisher=bbc.co.uk
• http://atlas.metabroadcast.com/3.0/schedule.json?
from=1948-12-24&to=1948-12-25&channel=radio4&publisher=bbc.co.uk
• api explorer http://atlas.metabroadcast.com/#apiExplorer

16. Atlas API
• content
• http://atlas.metabroadcast.com/3.0/content.json?uri=http://www.bbc.co.uk/programmes/
b0074g7p&annotations=description,brand_summary,locations&apiKey=6ed2a984627daff816198acde82
• http://atlas.metabroadcast.com/3.0/content.json?apiKey=aaaa&uri=http://www.bbc.co.uk/programmes/
b0074g7p&annotations=description,brand_summary,locations
• schedules
• http://atlas.metabroadcast.com/3.0/schedule.json?from=now&to=now.plus.
3h&channel=bbcone&publisher=bbc.co.uk
• http://atlas.metabroadcast.com/3.0/schedule.json?
from=1948-12-24&to=1948-12-25&channel=radio4&publisher=bbc.co.uk
• api explorer http://atlas.metabroadcast.com/#apiExplorer

17. Why Cassandra?
•scalability/performance
• row caches
• consistency control
• column-based model matches our use case

18. And?
• ElasticSearch
• messaging
• tooling: bootstraps

19. What is Atlas?
BBC
Data ingest
server DB
PA
C4
Update bus HTTP server
etc...
ES

20. Data model
• columns to model annotations
• secondary indexes
• index.direct(keyspace, SEGMENT_URI_INDEX_CF, ConsistencyLevel.CL_QUORUM).
from(segment.getCanonicalUri()).
to(segment.getIdentifier()).
index().execute(requestTimeout, TimeUnit.MILLISECONDS);

21. ID generation
• give external data our own ID on ingest
• needs to be user-friendly:
http://www.radiotimes.com/programme/cf2/eastenders
• mongo: findAndModify()
• solution: uses Astyanax client with its distributed locking
• more details: http://metabroadcast.com/blog/let-
cassandra-identify-your-data

22. Where we’re at
• already live with some data
• alpha release of schedule endpoint coming soon
• later: roll out across other endpoints

23. Ops

24. Ops in Cassandra
• we love Puppet
• it’s great for automation and deployment
• MongoDB: 1 file
• Cassandra: 2 files!
• oh... tokens

25. Cassandra Tokens
• define where data is written to
in a cluster
• therefore balanced tokens =
balanced cluster
• tokens should be rack aware
• tools available to provide appropriate tokens
for you

26. Cassandra plays nicely with AWS
• datacentre / rack aware
• AWS Region = Datacentre
• AWS Availability Zone = Rack
• only recently introduced in MongoDB but simple to
implement in Cassandra
• horizontally (and vertically) scalable

27. Monitoring
• Nagios is a little threadbare for Cassandra
• basic TCP service check
• stats from API not very helpful
• nodetool and CLI tools useful
• manual effort to integrate them
• if only there was some useful service...

28. OpsCenter
• wonderful for an overview
• not so much for alerting ;)
• ohai API
• can integrate metrics into Nagios

29. Disaster Recovery
• we operate a 4 node cluster presently
• replication factor of 3 with quorum read/writes
• DR complicated by tokens
• cluster should be balanced
• snapshot + S3 Backups

30. Cluster Happiness and Headaches
• little maintenance overhead
• cluster rebalancing
• uncommon maintenance procedure
• schema changes are cumbersome
• little scope for rollback, can put cluster in unrecoverable state

31. Summary
• Mongo is good, Atlas has outgrown it
• Cassandra isn’t a drop-in replacement
• Ops more complex but so far so good

32. Questions?