• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Cassandra Codebase 2011
 

Cassandra Codebase 2011

on

  • 4,501 views

 

Statistics

Views

Total Views
4,501
Views on SlideShare
4,354
Embed Views
147

Actions

Likes
8
Downloads
211
Comments
0

10 Embeds 147

http://digitalboys.wordpress.com 58
http://www.scoop.it 58
https://twitter.com 15
url_unknown 6
http://twitter.com 3
http://www.linkedin.com 3
http://paper.li 1
http://tweetedtimes.com 1
http://www.techgig.com 1
https://twimg0-a.akamaihd.net 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Who was here last year?Very good presentations on data modeling and capacity planning.
  • Turn it around.Ask questions first.
  • Transport still not initialized though.DD getting loaded is just a side-effect
  • This is actually a good exercise.
  • Good place to extend and experiment on your own.

Cassandra Codebase 2011 Cassandra Codebase 2011 Presentation Transcript

  • Codebase 2011
    Getting to know the codebase
    Gary Dusbabek
    @gdusbabek
  • Questions?
  • Outline
    How to contribute
    Internals
    Some thoughts
  • How to Contribute
  • How to Contribute
    http://wiki.apache.org/cassandra/HowToContribute
    JIRA: “lhf” label (Low hanging fruit)
    Scratch your itch
  • How to Contribute
    Run the tests
    ant test
    nosetests test/system/test_thrift_server.py
  • How to Contribute
    http://wiki.apache.org/cassandra/CodeStyle
    Avoid:
    Reformatting white space
    Renaming things everywhere
    Unrelated changes
  • How to Contribute
    Use git
    Attach patches
    git format-patch as jira attachments.
    Group them sensibly
  • How to Contribute
    Someone will review your code
    Usually a committer
    Persistence helps
    Don’t get your feelings hurt
    It usually takes a few rounds
  • How to Contribute
    Participate!
    #cassandra-dev on freenode
    dev@cassandra.apache.org
  • Internals
  • Services
    Ring Operations (StorageService)
    Storage Operations (StorageProxy)
  • Startup Sequence
    bin/cassandra
    Finds cassandra.in.sh
    $CLASSPATH (mandatory)
    $CASSANDRA_HOME
    $CASSANDRA_CONF (mandatory)
    Executes $CASSANDRA_CONF/cassandra-env.sh
    Sets heap sizes (gc tuning goes here!)
  • o.a.c.thrift.CassandraDaemon
  • AbstractCassandraDaemon
    ACD.setup():
    Reads configuration: DatabaseDescriptor
    Loads schema: DD.loadSchemas()
    Scrub directories
    Initialize storage (keyspaces + CFs)
    Commit log recovery: CL.recover()
    StorageService.initServer() -> StorageService.joinTokenRing()
  • Attn Tinkerers!
    Abstracted initialization of transport.
    Handy if you’re experimenting with transports/RPC
    Just extend AbstractCassandraDaemon and make sure that class is started up via bin/cassandra.
  • o.a.c.thrift.CassandraServer
    Implements thrift interface methods (the API).
    Start here when trying to understand the read/write path and RPC.
  • Configuration
    DatabaseDescriptor
    Side-effect of ACD.setup()
    Reads config settings from yaml
    Defines system tables
    Changes regularly
    I hate this code. Please fix it.
  • Main Singletons
    StorageService
    StorageProxy
    MessagingService
    CompactionManager
    StageManager
    MigrationManager
  • Did you just say ‘Singletons?’
  • Main Singletons
    StorageService
    StorageProxy
    MessagingService
    CompactionManager
    StageManager
    MigrationManager
  • JMX MBeans
    Tooling supplied by Mbeans
    Anything that does measureable/configurable work is tooled
    Thread pools
    Compaction
    Hinted handoff
    Streaming
    Storage
    Commit log
  • StorageService
    initServer() -> joinTokenRing()
    Starts gossip
    Starts MessagingService
    Negotiates bootstrap
    Many ring operations live here.
    Repository of ring topology
    TokenMetadata (quasi-singleton via SS.tokenMetadata_)
    Partitioner instance is also here
  • MessagingService
    Verb handlers live here (initialized from SS).
    Main event handlers, haven’t changed much.
    Socket listener
    2 threads per ring node
    Message gateway
    emitted from MessageProducerimpls
    MS.sendRR()
    MS.sendOneWay()
    MS.receive()
    Messages are versioned now (0.8)
    IncomingTCPConnection
  • StorageProxy
    Top level of all read/write operations
    Called from o.a.c.thrift.CassandraServer
    Write path changed because of counters
    Notion of WritePerformer
    Eventually to Table and ColumnFamilyStore
    Further, to SSTable and related classes.
  • StageManager
    Fancy java ThreadPoolExecutor
    SEDA: http://www.eecs.harvard.edu/~mdw/papers/seda-sosp01.pdf
    consumes callables from a queue.
    Manages concurrency.
    Hasn’t changed much.
  • Adding API Methods
    Define method+structures in IDL
    interface/cassandra.thrift
    Regenerate files
    ant gen-thrift-java gen-thrift-py
    Implement stubs:
    o.a.c.thrift.CassandraServer
    Create a system test
    tests/system/test_thrift_server.py
  • Reading
    Socket->CassandraServer
    Permissions
    Request validation
    Marshalling
    ReadCommands created in CS.multigetSliceInternal, passed to StorageProxy
    1 per key
  • Reading
    StorageProxy.read(), fetchRows()
    For each ReadCommand
    Determine endpoints
    Local & remote branches
  • Reading
    StorageProxy local
    READ stage executes a LocalReadRunnable
    True read vs digest
    Table, ColumnFamilyStore
    CFS.getTopLevelColumns
    Make QueryFilter
    Query Memtables
    Query SSTables
    Coalesce in iterators
  • Reading
    StorageProxy remote
    read command
    Response handler
    Send to remote nodes
    Read repair happens in SP.fetchRows().
  • Writing
    CS.doInsert()
    Marshalling, creates RMs
    StorageProxy
    local/remote branch
    SP.sendToHintedEndpoints()
    RowMutation
    one Key per (several CFs)
    ColumnFamily
    Collection of column modifications
  • Writing
    RM.apply->Table.apply
    Write to CL
    Iterate over RM CFs
    CFS.apply()
    Overwrites results on pre-existing column families
  • Writing
    RM is serialized into a Message and sent to other nodes
    Waits for ACKs depending on CL
  • Challenges
  • Challenges
    To have an in-depth understanding of everything.
    Hard for hobbyist/part-timers
    Outside of Datastax, little support for full-timers
    Still changing fast
    Keeping up
  • Challenge: Lines of Code
    0.4 (Sep 2009)
    52 kloc
    0.5 (Jan 2010)
    59 kloc
    0.6 (Apr 2010)
    73 kloc
    0.7 (Jan 2011)
    122 kloc
    0.8 (Jun 2011)
    146 kloc
    Trunk (yesterday)
    149 kloc
    Average:
    4,500 lines per month
  • Challenges
    Codewise Growing pains
    Software maturity
    Decisions made early on