Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tuning Kafka for Fun and Profit

5,414 views

Published on

This presentation was given at the ApacheCon 2015 Kafka Meetup.

These slides go into some detail on how to tune and scale Kafka clusters and the components involved. The slides themselves are bullet points, and all the detail is in the slide notes, so please download the original presentation and review those.

Published in: Data & Analytics
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @Guido Serra Sorry, there was no video taken at this event. If you download the original presentation, there are slide notes for each slide.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • hi, do you have a video of ur presentation?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Tuning Kafka for Fun and Profit

  1. 1. ORGANIZATION NAME©2013 LinkedIn Corporation. All Rights Reserved. Tuning Kafka for Fun and Profit
  2. 2. ORGANIZATION NAME©2013 LinkedIn Corporation. All Rights Reserved. Zookeeper  5-node vs. 3-node Ensembles  Solid State Disks – Use good SSDs – Transaction logs only – Significant improvement in latency and outstanding requests 2
  3. 3. ORGANIZATION NAME©2013 LinkedIn Corporation. All Rights Reserved. Kafka Broker Disks  Disk Layout  JBOD vs. RAID – JBOD and RAID-0 are similar – RAID-5/6 has significant performance overhead – RAID-10 still offers the best performance and protection  Filesystem – New testing shows XFS has a clear benefit – No tuning required – Will be continuing testing with more production traffic 3
  4. 4. ORGANIZATION NAME©2013 LinkedIn Corporation. All Rights Reserved. Scaling Kafka Clusters  Disk Capacity  Network Capacity  Partition Counts – Per-Cluster – Per-Broker  Limitations – Topic list length 4
  5. 5. ORGANIZATION NAME©2013 LinkedIn Corporation. All Rights Reserved. Topic Configuration  Retention Settings  Partition Counts – Balance over consumers – Balance over brokers – Partition size on disk – Application-specific requirements 5
  6. 6. ORGANIZATION NAME©2013 LinkedIn Corporation. All Rights Reserved. Mirror Maker  Network Locality  Consumer Tuning – Number of streams – Partition assignment strategy  Producer Tuning – Number of streams – In flight requests – Linger time 6

×