This document discusses migrating data from MongoDB to Cassandra for the Atlas metadata platform. Some key points: - MongoDB was becoming too simple and lacked features like control and sharding needed for Atlas' large scale. Cassandra offers better scalability, performance, and consistency controls. - The data model is being adapted to Cassandra's column-based model with secondary indexes to support annotations. Unique IDs will be generated during ingest rather than using MongoDB's findAndModify(). - Ops processes like deployment, monitoring, and disaster recovery require adjustments for Cassandra's distributed architecture and use of tokens but integrate well with tools like Puppet, OpsCenter, and AWS. - Overall the migration is progress

1. MongoDB to Cassandra
The Atlas Odyssey
Fred van den Driessche Tom McAdam Adam Horwich
Engineer CTO Systems Engineer
@fredvdd @tfm @Mmmkayness

3. http://flickr.com/photos/dhammza/88644497/

4. Our platform - late 2012
tbc tbc
MetaBroadcast platform
Video and audio metadata Profiles and activity from video and
from 20+ sources Analytic requests and groupings
audio products, social networks

5. ?

6. Main clients Main Partners
Data Partners

7. What is Atlas?
/content
BBC
/schedules
/topics
PA
ATLAS
C4
sitemaps
radioplayer
etc... DB
interlinking

8. DEMO

9. Atlas Data Model
brand item
series version
broadcast location

10. MongoDB
• flexible
• features
• really simple
• shell

11. Where MongoDB falls short
• too simple
• lack of control
• sharding
• embedding

12. Where to?

13. Where to?
• add a cache?

14. Atlas API
• content
• http://atlas.metabroadcast.com/3.0/content.json?uri=http://www.bbc.co.uk/programmes/
b0074g7p&annotations=description,brand_summary,locations&apiKey=6ed2a984627daff816198acde82
• http://atlas.metabroadcast.com/3.0/content.json?apiKey=aaaa&uri=http://www.bbc.co.uk/programmes/
b0074g7p&annotations=description,brand_summary,locations
• schedules
• http://atlas.metabroadcast.com/3.0/schedule.json?from=now&to=now.plus.
3h&channel=bbcone&publisher=bbc.co.uk
• http://atlas.metabroadcast.com/3.0/schedule.json?
from=1948-12-24&to=1948-12-25&channel=radio4&publisher=bbc.co.uk
• api explorer http://atlas.metabroadcast.com/#apiExplorer

16. Atlas API
• content
• http://atlas.metabroadcast.com/3.0/content.json?uri=http://www.bbc.co.uk/programmes/
b0074g7p&annotations=description,brand_summary,locations&apiKey=6ed2a984627daff816198acde82
• http://atlas.metabroadcast.com/3.0/content.json?apiKey=aaaa&uri=http://www.bbc.co.uk/programmes/
b0074g7p&annotations=description,brand_summary,locations
• schedules
• http://atlas.metabroadcast.com/3.0/schedule.json?from=now&to=now.plus.
3h&channel=bbcone&publisher=bbc.co.uk
• http://atlas.metabroadcast.com/3.0/schedule.json?
from=1948-12-24&to=1948-12-25&channel=radio4&publisher=bbc.co.uk
• api explorer http://atlas.metabroadcast.com/#apiExplorer

17. Why Cassandra?
•scalability/performance
• row caches
• consistency control
• column-based model matches our use case

18. And?
• ElasticSearch
• messaging
• tooling: bootstraps

19. What is Atlas?
BBC
Data ingest
server DB
PA
C4
Update bus HTTP server
etc...
ES

20. Data model
• columns to model annotations
• secondary indexes
• index.direct(keyspace, SEGMENT_URI_INDEX_CF, ConsistencyLevel.CL_QUORUM).
from(segment.getCanonicalUri()).
to(segment.getIdentifier()).
index().execute(requestTimeout, TimeUnit.MILLISECONDS);

21. ID generation
• give external data our own ID on ingest
• needs to be user-friendly:
http://www.radiotimes.com/programme/cf2/eastenders
• mongo: findAndModify()
• solution: uses Astyanax client with its distributed locking
• more details: http://metabroadcast.com/blog/let-
cassandra-identify-your-data

22. Where we’re at
• already live with some data
• alpha release of schedule endpoint coming soon
• later: roll out across other endpoints

23. Ops

24. Ops in Cassandra
• we love Puppet
• it’s great for automation and deployment
• MongoDB: 1 file
• Cassandra: 2 files!
• oh... tokens

25. Cassandra Tokens
• define where data is written to
in a cluster
• therefore balanced tokens =
balanced cluster
• tokens should be rack aware
• tools available to provide appropriate tokens
for you

26. Cassandra plays nicely with AWS
• datacentre / rack aware
• AWS Region = Datacentre
• AWS Availability Zone = Rack
• only recently introduced in MongoDB but simple to
implement in Cassandra
• horizontally (and vertically) scalable

27. Monitoring
• Nagios is a little threadbare for Cassandra
• basic TCP service check
• stats from API not very helpful
• nodetool and CLI tools useful
• manual effort to integrate them
• if only there was some useful service...

28. OpsCenter
• wonderful for an overview
• not so much for alerting ;)
• ohai API
• can integrate metrics into Nagios

29. Disaster Recovery
• we operate a 4 node cluster presently
• replication factor of 3 with quorum read/writes
• DR complicated by tokens
• cluster should be balanced
• snapshot + S3 Backups

30. Cluster Happiness and Headaches
• little maintenance overhead
• cluster rebalancing
• uncommon maintenance procedure
• schema changes are cumbersome
• little scope for rollback, can put cluster in unrecoverable state

31. Summary
• Mongo is good, Atlas has outgrown it
• Cassandra isn’t a drop-in replacement
• Ops more complex but so far so good

32. Questions?