Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

a wild Supposition: can MySQL be Kafka ?

1,216 views

Published on

This is an idea which i presented at percona live 2015.

Is MySQL an avatar of Apache Kafka ?

Can it be Kafka ?

Yes, it can.

This talk takes a shot at modeling MySQL as Kafka.

PS: it's unconventional, hence a WILD supposition :)

http://www.percona.com/live/mysql-conference-2015/sessions/wild-supposition-can-mysql-be-kafka

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

a wild Supposition: can MySQL be Kafka ?

  1. 1. a  Supposi)on:   “  can  MySQL  be  Ka5a  ?  ”   vishnu  rao  –  pocketmath.com  
  2. 2. Apache  Ka5a  
  3. 3. Apache  Ka5a   •  A  publish-­‐subscribe  messaging  system   implemented  as  distributed  commit  log.  
  4. 4. Apache  Ka5a   •  A  publish-­‐subscribe  messaging  system   implemented  as  distributed  commit  log.   •  No)ons:  
  5. 5. Apache  Ka5a   •  A  publish-­‐subscribe  messaging  system   implemented  as  distributed  commit  log.   •  No)ons:   – Producer  
  6. 6. Apache  Ka5a   •  A  publish-­‐subscribe  messaging  system   implemented  as  distributed  commit  log.   •  No)ons:   – Producer   – Topic  
  7. 7. Apache  Ka5a   •  A  publish-­‐subscribe  messaging  system   implemented  as  distributed  commit  log.   •  No)ons:   – Producer   – Topic   – Consumer  
  8. 8. Apache  Ka5a   •  A  publish-­‐subscribe  messaging  system   implemented  as  distributed  commit  log.   •  No)ons:   – Producer   – Topic   – Consumer   – Message  offset    
  9. 9. Apache  Ka5a   •  A  publish-­‐subscribe  messaging  system   implemented  as  distributed  commit  log.   •  No)ons:   – Producer   – Topic   – Consumer   – Message  offset   – Broker    
  10. 10. •  Producers  publish  messages  to  a  topic.      
  11. 11. •  Producers  publish  messages  to  a  topic.       •  Topic  is  just  a  par))oned  write-­‐ahead  log.   Producers  APPEND  records  to  these  logs.    
  12. 12. •  Producers  publish  messages  to  a  topic.       •  Topic  is  just  a  par))oned  write-­‐ahead  log.   Producers  APPEND  records  to  these  logs.   •  Every  message  is  at  a  OFFSET  in  the  log.  
  13. 13. •  Producers  publish  messages  to  a  topic.       •  Topic  is  just  a  par))oned  write-­‐ahead  log.   Producers  APPEND  records  to  these  logs.   •  Every  message  is  at  a  OFFSET  in  the  log.   •  Consumers  consume  messages  from  logs  using   offset  
  14. 14. What  about  …   •  Replicas?  
  15. 15. What  about  …   •  Replicas?   – MySQL  Master  –  Slave  
  16. 16. What  about  …   •  Replicas?   – MySQL  Master  –  Slave   •  Horizontally  Scalable  ?  
  17. 17. What  about  …   •  Replicas?   – MySQL  Master  –  Slave   •  Horizontally  Scalable  ?   – Add  more  MySQL  instances  (split  write)  
  18. 18. What  about  …   •  Replicas?   – MySQL  Master  –  Slave   •  Horizontally  Scalable  ?   – Add  more  MySQL  instances  (split  write)   •  Mul)ple  Par))ons  in  a  ka5a  broker  Versus  
  19. 19. What  about  …   •  Replicas?   – MySQL  Master  –  Slave   •  Horizontally  Scalable  ?   – Add  more  MySQL  instances  (split  write)   •  Mul)ple  Par))ons  in  a  ka5a  broker  Versus   – Single  par))on(binlog)  in  a  MySQL  instance  
  20. 20. What  about  …   •  Replicas?   – MySQL  Master  –  Slave   •  Horizontally  Scalable  ?   – Add  more  MySQL  instances  (split  write)   •  Mul)ple  Par))ons  in  a  ka5a  broker  Versus   – Single  par))on(binlog)  in  a  MySQL  instance   •  Data  Reten)on  ?  
  21. 21. What  about  …   •  Replicas?   – MySQL  Master  –  Slave   •  Horizontally  Scalable  ?   – Add  more  MySQL  instances  (split  write)   •  Mul)ple  Par))ons  in  a  ka5a  broker  Versus   – Single  par))on(binlog)  in  a  MySQL  instance   •  Data  Reten)on  ?   –  How  long  you  keep  the  binlogs.  
  22. 22. What  about  …   •  Replicas?   – MySQL  Master  –  Slave   •  Horizontally  Scalable  ?   – Add  more  MySQL  instances  (split  write)   •  Mul)ple  Par))ons  in  a  ka5a  broker  Versus   – Single  par))on(binlog)  in  a  MySQL  instance   •  Data  Reten)on  ?   –  How  long  you  keep  the  binlogs.   •  Transac)ons  for  free  versus  no  txn  in  ka5a  
  23. 23.     This  approach  might  seem   unconven)onal  &  unorthodox.        
  24. 24.     This  approach  might  seem   unconven)onal  &  unorthodox.         hence   It’s  a  WILD  supposi)on   “MySQL  can  be  Ka5a.”  
  25. 25. Thank  you.   We  are  Hiring!   h]p://www.pocketmath.com/jobs.html     vishnuhr@pocketmath.com   jaihind213@gmail.com   sweetweet213@twi]er  

×