At Instagram, our mission is to capture and share the world's moments. Our app is used by over 400M people monthly; this creates a lot of challenging data needs. We use Cassandra heavily, as a general key-value storage. In this presentation, I will talk about how we use Cassandra to serve our critical use cases; the improvements/patches we made to make sure Cassandra can meet our low latency, high scalability requirements; and some pain points we have.
About the Speaker
Dikang Gu Software Engineer, Facebook
I'm a software engineer at Instagram core infra team, working on scaling Instagram infrastructure, especially on building a generic key-value store based on Cassandra. Prior to this, I worked on the development of HDFS in Facebook. I got the master degree of Computer Science in Shanghai Jiao Tong university in China.
• CPU usage +30% when bootstrapping new nodes.
• Client requests latency jumps and timeouts
• Multimap<Range<Token>, InetAddress> PendingRange
• In-efﬁcient O(n) pendingRanges lookup for request
• Use two NavigableMaps to implement the pending ranges
• We can expand or shrink the cluster without affecting requests
• Thanks Branimir Lambov for patch review and feedbacks.
• Track the write ampliﬁcation. (CASSANDRA-11420)
• Optimize the overlapping lookup. (CASSANDRA-11571)
• Optimize the isEOF() checking. (CASSANDRA-12013)
• Avoid searching for column index. (CASSANDRA-11450)
• Persist last compacted key per level. (CASSANDRA-6216)
• Compact tables before making available in L0. (CASSANDRA-10862)