Cozy with Cassandra<br />Getting to know the Cassandra Codebase<br />Gary Dusbabek • Rackspace<br />@gdusbabek<br />Cassan...
Outline<br />Code Themes<br />Startup Sequence<br />Key Classes<br />Read Path<br />Write Path<br />Stages & Threading<br ...
Themes and Patterns<br />Layers<br />
Themes and Patterns<br />Services<br />
Themes and Patterns<br />Singletons <br />and <br />statics<br />
Themes and Patterns<br />Stages &<br />Thread pools<br />
StartupProcess<br />
CassandraDaemon<br />Loads configuration<br />Transport initialization<br />Storage (Keyspace initialization)<br />CommitL...
CassandraServer<br />Implements IDL interface methods (cassandra.thrift, cassandra.genavro)<br />Good place to start divin...
Configuration<br />DatabaseDescriptor<br />Via CassandraDaemon.setup()<br />Looks for config path, loads yaml<br />Doesn’t...
Code<br />Some<br />Classes<br />
Main Controllers<br />End with *Service or *Manager<br />StorageService, MessagingService<br />CompactionManager, HintedHa...
StorageProxy<br />Put & Get methods<br />Collection of static methods<br />Merges local and distributed operations<br />Tr...
StorageService<br />initServer()—Starts services<br />Registers verb handlers (in MessagingService)<br />Main event respon...
MessagingService<br />Verb handlers reside here<br />Sets up socket listeners<br />Gateway for outbound messages<br />MS.s...
Table & ColumnFamilyStore<br />Also RowMutation<br />Low-level storage operations<br />o.a.c.db.*<br />SSTable<br />Local ...
Read+Write Paths<br />
Reading	<br />Socket->CassandraServer<br />Permissions<br />Request validation<br />Marshalling<br />
Reading	<br />StorageProxy<br />Ranges<br />Collectors<br />Local & remote branches<br />
Reading	<br />StorageProxy local<br />Table, ColumnFamilyStore<br />CFS<br />Make QueryFilter<br />Query Memtables<br />Qu...
Reading	<br />StorageProxy remote<br />read command<br />Response handler<br />Send to remote nodes<br />
Writing<br />Socket->CassandraServer<br />Validation<br />Convert to Mutation (IDL object)<br />Penalties!<br />
Writing<br />StorageProxy<br />blocking/non-blocking mutate <br />local/remote branch<br />RowMutation<br />one ColumnFami...
Writing<br />RM.apply->Table.apply<br />Write to CL<br />Iterate over RM CFs<br />CFS.apply()<br />Overwrites results on p...
Writing<br />RM is serialized into a Message and sent to other nodes<br />Waits for ACKs depending on CL<br />
Stages & Threading<br />
Stages<br />SEDA<br />http://www.eecs.harvard.edu/~mdw/papers/seda-sosp01.pdf<br />o.a.c.concurrent.StageManager<br />Read...
Threads<br />MessagingService.listen() spawns thread.<br />Each incoming connection spawns a new short-lived thread (Incom...
Bootstraping!<br />Any interest?<br />//FIXMETODOCRAP<br />
<=0.6 Bootstraping<br />A wants data, B has data.<br />StreamingRequestMessage A->B<br />Handled on B by StreamRequestVerb...
0.7 Bootstrapping<br />A wants data, B has data<br />StreamRequestMessage A->B<br />On B, StreamRequestVerbHandler<br />If...
Tests<br />Testable & Untestable<br />Unit tests<br />ant clean build test<br />System tests<br />ant gen-thrift-py<br />n...
IDE<br />Configuration file must be in the classpath<br />Treat as source<br />lib vs build/lib<br />Log at debug<br />
IDE<br />-ea -Xms128M –Xmx2G <br />-Dcom.sun.management.jmxremote.port=8081 <br />-Dcom.sun.management.jmxremote.ssl=false...
Adding API methods<br />Same goes for modifying<br />Define method and structures in IDL<br />interface/cassandra.thrift<b...
Questions?<br />gdusbabek@gmail.com<br />@gdusbabek<br />
Upcoming SlideShare
Loading in …5
×

Getting to Know the Cassandra Codebase

6,574 views

Published on

Slide notes from a presentation I gave at the Cassandra Summit held on 10 August 2010.

Published in: Technology
0 Comments
23 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,574
On SlideShare
0
From Embeds
0
Number of Embeds
97
Actions
Shares
0
Downloads
0
Comments
0
Likes
23
Embeds 0
No embeds

No notes for slide
  • Talk about Cassandra processes and the classes that relate to them.
  • Data operated on,then passed off to another layer.Evident on R/W pathOnion analogy
  • DisjointCoupled when neccessary
  • Model of class/object designSuck it upNot a class project
  • ServicesGossiperMessagingServiceMigrationManagerBootstrapPreloaded cacheHandle to server-related tasks.
  • layers
  • Getting to Know the Cassandra Codebase

    1. 1. Cozy with Cassandra<br />Getting to know the Cassandra Codebase<br />Gary Dusbabek • Rackspace<br />@gdusbabek<br />Cassandra Summit • Mission Bay Conference Center • San Francisco • 10 August 2010<br />
    2. 2. Outline<br />Code Themes<br />Startup Sequence<br />Key Classes<br />Read Path<br />Write Path<br />Stages & Threading<br />Bootstrap & Streaming<br />Tests & IDE considerations<br />Adding API methods<br />Questions<br />
    3. 3. Themes and Patterns<br />Layers<br />
    4. 4. Themes and Patterns<br />Services<br />
    5. 5. Themes and Patterns<br />Singletons <br />and <br />statics<br />
    6. 6. Themes and Patterns<br />Stages &<br />Thread pools<br />
    7. 7. StartupProcess<br />
    8. 8. CassandraDaemon<br />Loads configuration<br />Transport initialization<br />Storage (Keyspace initialization)<br />CommitLogrecovery<br />StorageService.initServer()<br />Initializes CassandraServer<br />Passes it off to transport<br />
    9. 9. CassandraServer<br />Implements IDL interface methods (cassandra.thrift, cassandra.genavro)<br />Good place to start diving when adding or troubleshooting API methods<br />
    10. 10. Configuration<br />DatabaseDescriptor<br />Via CassandraDaemon.setup()<br />Looks for config path, loads yaml<br />Doesn’t spin anything up<br />Defines system tables<br />KS and CF described by CFMetaData and KSMetaData<br />
    11. 11. Code<br />Some<br />Classes<br />
    12. 12. Main Controllers<br />End with *Service or *Manager<br />StorageService, MessagingService<br />CompactionManager, HintedHandoffManager, StageManager, StreamInManager<br />
    13. 13. StorageProxy<br />Put & Get methods<br />Collection of static methods<br />Merges local and distributed operations<br />Tracks latency<br />Exposed via StorageProxyMBean<br />
    14. 14. StorageService<br />initServer()—Starts services<br />Registers verb handlers (in MessagingService)<br />Main event responders<br />Repository of replication strategies and TokenMetadata<br />Ring topology & token information<br />
    15. 15. MessagingService<br />Verb handlers reside here<br />Sets up socket listeners<br />Gateway for outbound messages<br />MS.sendRR()<br />MS.sendOneWay()<br />Inbound too<br />MS.receive()<br />
    16. 16. Table & ColumnFamilyStore<br />Also RowMutation<br />Low-level storage operations<br />o.a.c.db.*<br />SSTable<br />Local operations<br />
    17. 17. Read+Write Paths<br />
    18. 18. Reading <br />Socket->CassandraServer<br />Permissions<br />Request validation<br />Marshalling<br />
    19. 19. Reading <br />StorageProxy<br />Ranges<br />Collectors<br />Local & remote branches<br />
    20. 20. Reading <br />StorageProxy local<br />Table, ColumnFamilyStore<br />CFS<br />Make QueryFilter<br />Query Memtables<br />Query SSTables<br />Coalesce in iterators<br />o.a.c.db package<br />o.a.c.db.filter<br />
    21. 21. Reading <br />StorageProxy remote<br />read command<br />Response handler<br />Send to remote nodes<br />
    22. 22. Writing<br />Socket->CassandraServer<br />Validation<br />Convert to Mutation (IDL object)<br />Penalties!<br />
    23. 23. Writing<br />StorageProxy<br />blocking/non-blocking mutate <br />local/remote branch<br />RowMutation<br />one ColumnFamily per<br />ColumnFamily<br />Collection of column modifications<br />
    24. 24. Writing<br />RM.apply->Table.apply<br />Write to CL<br />Iterate over RM CFs<br />CFS.apply()<br />Overwrites results on pre-existing column families<br />
    25. 25. Writing<br />RM is serialized into a Message and sent to other nodes<br />Waits for ACKs depending on CL<br />
    26. 26. Stages & Threading<br />
    27. 27. Stages<br />SEDA<br />http://www.eecs.harvard.edu/~mdw/papers/seda-sosp01.pdf<br />o.a.c.concurrent.StageManager<br />Read, mutation, stream, gossip, response, anti entropy, load balance, migration<br />Thread pools that consume tasks<br />
    28. 28. Threads<br />MessagingService.listen() spawns thread.<br />Each incoming connection spawns a new short-lived thread (IncomingTcpConnection)<br />Non-stream ops go to MS.messageDeserializerExecutor_<br />Stream ops handled there.<br />Anti-entropy repair<br />
    29. 29. Bootstraping!<br />Any interest?<br />//FIXMETODOCRAP<br />
    30. 30. <=0.6 Bootstraping<br />A wants data, B has data.<br />StreamingRequestMessage A->B<br />Handled on B by StreamRequestVerbHandler<br />For each range StreamOut.transferRanges()<br />Flush, anticompaction<br />StreamInitiateMessage B->A for each range transfer<br />Meanwhile, back on A…<br />StreamInitiateVerbHandler gets the SIM from B, does some nesting.<br />StreamInitiateDone A->B<br />Back to B…<br />StreamInitiateDoneHandler gets the SID from A<br />Calls StreamOutManager.startNext() which sends a single file to A<br />MessagingService on A picks this up and the file is streamed.<br />Sstable is created<br />STREAM_FINISHED A->B<br />B gets rid of the file, calls SOM.startNext()<br />
    31. 31. 0.7 Bootstrapping<br />A wants data, B has data<br />StreamRequestMessage A->B<br />On B, StreamRequestVerbHandler<br />If single file, sends it.<br />If range, StreamOut.transferRangesForRequest()<br />Send next file (first will contain meta data about all files)<br />On A, IncomingStreamReader.read() <br />Data is received, sstable created<br />Ack, request next file<br />
    32. 32. Tests<br />Testable & Untestable<br />Unit tests<br />ant clean build test<br />System tests<br />ant gen-thrift-py<br />nosetests test/system/test_thrift_server.py<br />
    33. 33. IDE<br />Configuration file must be in the classpath<br />Treat as source<br />lib vs build/lib<br />Log at debug<br />
    34. 34. IDE<br />-ea -Xms128M –Xmx2G <br />-Dcom.sun.management.jmxremote.port=8081 <br />-Dcom.sun.management.jmxremote.ssl=false <br />-Dcom.sun.management.jmxremote.authenticate=false<br />-Dcassandra-foreground=yes <br />-Dlog4j.configuration=log4j-server.properties<br />-Dmx4jport=9081<br />
    35. 35. Adding API methods<br />Same goes for modifying<br />Define method and structures in IDL<br />interface/cassandra.thrift<br />Regenerate files<br />ant gen-thrift-java gen-thrift-py<br />Implement methods in o.a.c.thrift.CassandraServer<br />Create a system test (tests/system/test_thrift_server.py)<br />
    36. 36. Questions?<br />gdusbabek@gmail.com<br />@gdusbabek<br />

    ×