Codebase 2011<br />Getting to know the codebase<br />Gary Dusbabek<br />@gdusbabek<br />
Questions?<br />
Outline<br />How to contribute<br />Internals<br />Some thoughts<br />
How to Contribute<br />
How to Contribute<br />http://wiki.apache.org/cassandra/HowToContribute<br />JIRA: “lhf” label (Low hanging fruit)<br />Sc...
How to Contribute<br />Run the tests<br />ant test<br />nosetests test/system/test_thrift_server.py<br />
How to Contribute<br />http://wiki.apache.org/cassandra/CodeStyle<br />Avoid:<br />Reformatting white space<br />Renaming ...
How to Contribute<br />Use git<br />Attach patches<br />git format-patch as jira attachments.<br />Group them sensibly<br />
How to Contribute<br />Someone will review your code<br />Usually a committer<br />Persistence helps<br />Don’t get your f...
How to Contribute<br />Participate!<br />#cassandra-dev on freenode<br />dev@cassandra.apache.org<br />
Internals<br />
Services<br />Ring Operations (StorageService)<br />Storage Operations (StorageProxy)<br />
Startup Sequence<br />bin/cassandra<br />Finds cassandra.in.sh<br />$CLASSPATH (mandatory)<br />$CASSANDRA_HOME<br />$CASS...
o.a.c.thrift.CassandraDaemon<br />
AbstractCassandraDaemon<br />ACD.setup():<br />Reads configuration: DatabaseDescriptor<br />Loads schema: DD.loadSchemas()...
Attn Tinkerers!<br />Abstracted initialization of transport.<br />Handy if you’re experimenting with transports/RPC<br />J...
o.a.c.thrift.CassandraServer<br />Implements thrift interface methods (the API).<br />Start here when trying to understand...
Configuration<br />DatabaseDescriptor<br />Side-effect of ACD.setup()<br />Reads config settings from yaml<br />Defines sy...
Main Singletons<br />StorageService<br />StorageProxy<br />MessagingService<br />CompactionManager<br />StageManager<br />...
Did you just say ‘Singletons?’<br />
Main Singletons<br />StorageService<br />StorageProxy<br />MessagingService<br />CompactionManager<br />StageManager<br />...
JMX MBeans<br />Tooling supplied by Mbeans<br />Anything that does measureable/configurable work is tooled<br />Thread poo...
StorageService<br />initServer() -> joinTokenRing()<br />Starts gossip<br />Starts MessagingService<br />Negotiates bootst...
MessagingService<br />Verb handlers live here (initialized from SS).<br />Main event handlers, haven’t changed much.<br />...
StorageProxy<br />Top level of all read/write operations<br />Called from o.a.c.thrift.CassandraServer<br />Write path cha...
StageManager<br />Fancy java ThreadPoolExecutor<br />SEDA:  http://www.eecs.harvard.edu/~mdw/papers/seda-sosp01.pdf<br />c...
Adding API Methods<br />Define method+structures in IDL<br />interface/cassandra.thrift<br />Regenerate files<br />ant gen...
Reading<br />Socket->CassandraServer<br />Permissions<br />Request validation<br />Marshalling<br />ReadCommands created i...
Reading<br />StorageProxy.read(), fetchRows()<br />For each ReadCommand<br />Determine endpoints<br />Local & remote branc...
Reading<br />StorageProxy local<br />READ stage executes a LocalReadRunnable<br />True read vs digest<br />Table, ColumnFa...
Reading<br />StorageProxy remote<br />read command<br />Response handler<br />Send to remote nodes<br />Read repair happen...
Writing<br />CS.doInsert()<br />Marshalling, creates RMs<br />StorageProxy<br />local/remote branch<br />SP.sendToHintedEn...
Writing<br />RM.apply->Table.apply<br />Write to CL<br />Iterate over RM CFs<br />CFS.apply()<br />Overwrites results on p...
Writing<br />RM is serialized into a Message and sent to other nodes<br />Waits for ACKs depending on CL<br />
Challenges<br />
Challenges<br />To have an in-depth understanding of everything.<br />Hard for hobbyist/part-timers<br />Outside of Datast...
Challenge: Lines of Code<br />0.4 (Sep 2009)<br />52 kloc<br />0.5 (Jan 2010)<br />59 kloc<br />0.6 (Apr 2010)<br />73 klo...
Challenges<br />Codewise Growing pains<br />Software maturity<br />Decisions made early on<br />
Upcoming SlideShare
Loading in …5
×

Cassandra Codebase 2011

5,339 views

Published on

Published in: Technology, Education
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,339
On SlideShare
0
From Embeds
0
Number of Embeds
154
Actions
Shares
0
Downloads
263
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide
  • Who was here last year?Very good presentations on data modeling and capacity planning.
  • Turn it around.Ask questions first.
  • Transport still not initialized though.DD getting loaded is just a side-effect
  • This is actually a good exercise.
  • Good place to extend and experiment on your own.
  • Cassandra Codebase 2011

    1. 1. Codebase 2011<br />Getting to know the codebase<br />Gary Dusbabek<br />@gdusbabek<br />
    2. 2. Questions?<br />
    3. 3. Outline<br />How to contribute<br />Internals<br />Some thoughts<br />
    4. 4. How to Contribute<br />
    5. 5. How to Contribute<br />http://wiki.apache.org/cassandra/HowToContribute<br />JIRA: “lhf” label (Low hanging fruit)<br />Scratch your itch<br />
    6. 6. How to Contribute<br />Run the tests<br />ant test<br />nosetests test/system/test_thrift_server.py<br />
    7. 7. How to Contribute<br />http://wiki.apache.org/cassandra/CodeStyle<br />Avoid:<br />Reformatting white space<br />Renaming things everywhere<br />Unrelated changes<br />
    8. 8. How to Contribute<br />Use git<br />Attach patches<br />git format-patch as jira attachments.<br />Group them sensibly<br />
    9. 9. How to Contribute<br />Someone will review your code<br />Usually a committer<br />Persistence helps<br />Don’t get your feelings hurt<br />It usually takes a few rounds<br />
    10. 10. How to Contribute<br />Participate!<br />#cassandra-dev on freenode<br />dev@cassandra.apache.org<br />
    11. 11. Internals<br />
    12. 12. Services<br />Ring Operations (StorageService)<br />Storage Operations (StorageProxy)<br />
    13. 13. Startup Sequence<br />bin/cassandra<br />Finds cassandra.in.sh<br />$CLASSPATH (mandatory)<br />$CASSANDRA_HOME<br />$CASSANDRA_CONF (mandatory)<br />Executes $CASSANDRA_CONF/cassandra-env.sh<br />Sets heap sizes (gc tuning goes here!)<br />
    14. 14. o.a.c.thrift.CassandraDaemon<br />
    15. 15. AbstractCassandraDaemon<br />ACD.setup():<br />Reads configuration: DatabaseDescriptor<br />Loads schema: DD.loadSchemas()<br />Scrub directories<br />Initialize storage (keyspaces + CFs)<br />Commit log recovery: CL.recover()<br />StorageService.initServer() -> StorageService.joinTokenRing()<br />
    16. 16. Attn Tinkerers!<br />Abstracted initialization of transport.<br />Handy if you’re experimenting with transports/RPC<br />Just extend AbstractCassandraDaemon and make sure that class is started up via bin/cassandra.<br />
    17. 17. o.a.c.thrift.CassandraServer<br />Implements thrift interface methods (the API).<br />Start here when trying to understand the read/write path and RPC.<br />
    18. 18. Configuration<br />DatabaseDescriptor<br />Side-effect of ACD.setup()<br />Reads config settings from yaml<br />Defines system tables<br />Changes regularly<br />I hate this code. Please fix it.<br />
    19. 19. Main Singletons<br />StorageService<br />StorageProxy<br />MessagingService<br />CompactionManager<br />StageManager<br />MigrationManager<br />
    20. 20. Did you just say ‘Singletons?’<br />
    21. 21. Main Singletons<br />StorageService<br />StorageProxy<br />MessagingService<br />CompactionManager<br />StageManager<br />MigrationManager<br />
    22. 22. JMX MBeans<br />Tooling supplied by Mbeans<br />Anything that does measureable/configurable work is tooled<br />Thread pools<br />Compaction<br />Hinted handoff<br />Streaming<br />Storage<br />Commit log<br />
    23. 23. StorageService<br />initServer() -> joinTokenRing()<br />Starts gossip<br />Starts MessagingService<br />Negotiates bootstrap<br />Many ring operations live here.<br />Repository of ring topology<br />TokenMetadata (quasi-singleton via SS.tokenMetadata_)<br />Partitioner instance is also here<br />
    24. 24. MessagingService<br />Verb handlers live here (initialized from SS).<br />Main event handlers, haven’t changed much.<br />Socket listener<br />2 threads per ring node<br />Message gateway<br />emitted from MessageProducerimpls<br />MS.sendRR()<br />MS.sendOneWay()<br />MS.receive()<br />Messages are versioned now (0.8)<br />IncomingTCPConnection<br />
    25. 25. StorageProxy<br />Top level of all read/write operations<br />Called from o.a.c.thrift.CassandraServer<br />Write path changed because of counters<br />Notion of WritePerformer<br />Eventually to Table and ColumnFamilyStore<br />Further, to SSTable and related classes.<br />
    26. 26. StageManager<br />Fancy java ThreadPoolExecutor<br />SEDA: http://www.eecs.harvard.edu/~mdw/papers/seda-sosp01.pdf<br />consumes callables from a queue.<br />Manages concurrency.<br />Hasn’t changed much.<br />
    27. 27. Adding API Methods<br />Define method+structures in IDL<br />interface/cassandra.thrift<br />Regenerate files<br />ant gen-thrift-java gen-thrift-py<br />Implement stubs:<br />o.a.c.thrift.CassandraServer<br />Create a system test<br />tests/system/test_thrift_server.py<br />
    28. 28. Reading<br />Socket->CassandraServer<br />Permissions<br />Request validation<br />Marshalling<br />ReadCommands created in CS.multigetSliceInternal, passed to StorageProxy<br />1 per key<br />
    29. 29. Reading<br />StorageProxy.read(), fetchRows()<br />For each ReadCommand<br />Determine endpoints<br />Local & remote branches<br />
    30. 30. Reading<br />StorageProxy local<br />READ stage executes a LocalReadRunnable<br />True read vs digest<br />Table, ColumnFamilyStore<br />CFS.getTopLevelColumns<br />Make QueryFilter<br />Query Memtables<br />Query SSTables<br />Coalesce in iterators<br />
    31. 31. Reading<br />StorageProxy remote<br />read command<br />Response handler<br />Send to remote nodes<br />Read repair happens in SP.fetchRows().<br />
    32. 32. Writing<br />CS.doInsert()<br />Marshalling, creates RMs<br />StorageProxy<br />local/remote branch<br />SP.sendToHintedEndpoints()<br />RowMutation<br />one Key per (several CFs)<br />ColumnFamily<br />Collection of column modifications<br />
    33. 33. Writing<br />RM.apply->Table.apply<br />Write to CL<br />Iterate over RM CFs<br />CFS.apply()<br />Overwrites results on pre-existing column families<br />
    34. 34. Writing<br />RM is serialized into a Message and sent to other nodes<br />Waits for ACKs depending on CL<br />
    35. 35. Challenges<br />
    36. 36. Challenges<br />To have an in-depth understanding of everything.<br />Hard for hobbyist/part-timers<br />Outside of Datastax, little support for full-timers<br />Still changing fast<br />Keeping up<br />
    37. 37. Challenge: Lines of Code<br />0.4 (Sep 2009)<br />52 kloc<br />0.5 (Jan 2010)<br />59 kloc<br />0.6 (Apr 2010)<br />73 kloc<br />0.7 (Jan 2011)<br />122 kloc<br />0.8 (Jun 2011)<br />146 kloc<br />Trunk (yesterday)<br />149 kloc<br />Average:<br />4,500 lines per month<br />
    38. 38. Challenges<br />Codewise Growing pains<br />Software maturity<br />Decisions made early on<br />

    ×