Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Apache cassandra nio

Apache cassandra 4.0 nio Report

  • Be the first to comment

  • Be the first to like this

Apache cassandra nio

  1. 1. Apache Cassandra 4.0 NIO MessagingService Japan Cassandra Community Cassandra Study Kazutaka Tomita
  2. 2. SELF INTRODUCTION Kazutaka Tomita (@railute) • INTHEFOREST Co., Ltd. CEO/CTO • Consulting for Apache Cassandra and Apache Spark Systems • Supporting for Apache Cassandra in Japan • An organizer of Cassandra Summit Tokyo Specialty • RDBMS (Oracle,SQLServer,MySQL,PostgreSQL) • Apache Cassandra • Apache Spark • Apache Hadoop with YARN • And the other NoSQL • NLP and Text mining for Japanese
  3. 3. Background The internode networking on Apache Cassandra Changed Points
  4. 4. Introduction Background PURPOS E In Apache Cassandra 4.0, the first major changes to the network layer have been implemented. The purpose of this slide is reviewing this change in order to consider how it will affect Apache Cassandra.
  5. 5. Background Background Background for Changed As the number of nodes in the cluster increases, the performance impact from the context switch becomes more apparent. Value Percentile TotalCount 1/(1-Percentile) 0.000 0.000000000000 5554 1.00 ~snip~ 1634.303 0.987500000000 1229643 80.00 1688.575 0.989062500000 1231596 91.43 1756.159 0.990625000000 1233523 106.67 ~snip~ 148176371.711 0.999999237061 1245183 1310720.00 148176371.711 1.000000000000 1245183 Value Percentile TotalCount 1/(1-Percentile) 0.000 0.000000000000 21883 1.00 ~snip~ 875.519 0.987500000000 3494765 80.00 935.423 0.989062500000 3500258 91.43 1010.175 0.990625000000 3505780 106.67 ~snip~ 186562641.919 0.999999761581 3538943 4194304.00 186562641.919 1.000000000000 3538943 *From CASSANDRA-8457 75% of messages within 100 microseconds and 99% within 1 millisecond. 50% of messages are sent within 100 microseconds of each other and 92% are sent within one millisecond. Before adaptation After adaptation
  6. 6. NetworkService Networking Structure The internode networking on Apache Cassandra C* C* C* Inbound Outbound Gossip Small Large (URGENT) Client The Networking Structure on Apache Cassandra
  7. 7. Non-Blocking IO The internode networking on Apache Cassandra REQUESTS EVENT LOOP SINGLE THREAD DELEGATE ASYNC THREADS C*
  8. 8. Notification The internode networking on Apache Cassandra C* C* C* Client The implement of NIO in 4.0 applies only to inter nodes. Already applied between client and coordinator nodes when implementing native protocol The benefit is not available without proper OS configuration beforehand, such as proper IRQ configuration or jumbo frame adaptation.
  9. 9. Before 3.X Changed Points /** * Listen on the specified port. * * @param localEp InetAddress whose port to listen on. */ private void listen(InetAddress localEp) throws ConfigurationException { for (ServerSocket ss : getServerSockets(localEp)) { SocketThread th = new SocketThread(ss, "ACCEPT-" + localEp); th.start(); socketThreads.add(th); } } 3.11.10 /** * Listen on the specified port. * @param localEp InetAddress whose port to listen on. */ public void listen(InetAddress localEp) throws IOException { ServerSocketChannel serverChannel = ServerSocketChannel.open(); final ServerSocket ss = serverChannel.socket(); ss.setReuseAddress(true); ss.bind(new InetSocketAddress(localEp, DatabaseDescriptor.getStoragePort())); socketThread = new SocketThread(ss, "ACCEPT-" + localEp); socketThread.start(); listenGate.signalAll(); } /** * Listen on the specified port. * @param localEp InetAddress whose port to listen on. */public void listen(InetAddress localEp) throws IOException, ConfigurationException { callbacks.reset(); // hack to allow tests to stop/restart MS for (ServerSocket ss: getServerSocket(localEp)) { SocketThread th = new SocketThread(ss, "ACCEPT-" + localEp); th.start(); socketThreads.add(th); } listenGate.signalAll(); } 1.0.12 0.6.13 org.apache.cassandra.net.MessagingService
  10. 10. After 4.0 Changed Points public void listen() { inboundSockets.open(); } 4.0-beta4 org.apache.cassandra.net.InboundSocket org.apache.cassandra.net.MessagingService
  11. 11. Summary  The bottom layer has been completely changed, so the impact is significant.  When used in public clouds including AWS, we should take care in tuning for servers.

×