Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
KafkaA little introduction
Pub-Sub Messaging System
Distributed
Performance
Disk/Memory Performance                     1000M                       100M                        10M                   ...
Disk/Memory Performance                     1000M                       100M                        10M                   ...
Disk/Memory Performance                     1000M                       100M                        10M                   ...
Disk/Memory Performance                     1000M                       100M                        10M                   ...
Persistent
Length    Magic Value Checksum   Payload4 bytes     1 byte     4 bytes   n bytes
TokenOffset: 0             InputBroker: kafka.localTopic: Testing                                       MR Job            ...
TokenOffset: 0             InputBroker: kafka.localTopic: Testing                                       MR Job            ...
Useful Things• http://incubator.apache.org/kafka/• https://github.com/pingles/clj-kafka
Kafka - A little introduction
Kafka - A little introduction
Kafka - A little introduction
Kafka - A little introduction
Kafka - A little introduction
Kafka - A little introduction
Kafka - A little introduction
Kafka - A little introduction
Kafka - A little introduction
Kafka - A little introduction
Kafka - A little introduction
Kafka - A little introduction
Kafka - A little introduction
Upcoming SlideShare
Loading in …5
×

Kafka - A little introduction

3,242 views

Published on

A brief run through of Kafka and some of it's interesting characteristics that make it a great messaging system for collecting and aggregating data.

Published in: Technology
  • Be the first to comment

Kafka - A little introduction

  1. 1. KafkaA little introduction
  2. 2. Pub-Sub Messaging System
  3. 3. Distributed
  4. 4. Performance
  5. 5. Disk/Memory Performance 1000M 100M 10M 1MRead values/second 100,000 10,000 1,000 100 10 1 Disk SSD Memory Random access Sequential Access Source: http://queue.acm.org/detail.cfm?id=1563874
  6. 6. Disk/Memory Performance 1000M 100M 10M 1MRead values/second 100,000 10,000 1,000 100 10 1 Disk SSD Memory Random access Sequential Access Source: http://queue.acm.org/detail.cfm?id=1563874
  7. 7. Disk/Memory Performance 1000M 100M 10M 1MRead values/second 100,000 10,000 1,000 100 10 1 Disk SSD Memory Random access Sequential Access Source: http://queue.acm.org/detail.cfm?id=1563874
  8. 8. Disk/Memory Performance 1000M 100M 10M 1MRead values/second 100,000 Sequential disk read 10,000 faster than random 1,000 100 memory read 10 1 Disk SSD Memory Random access Sequential Access Source: http://queue.acm.org/detail.cfm?id=1563874
  9. 9. Persistent
  10. 10. Length Magic Value Checksum Payload4 bytes 1 byte 4 bytes n bytes
  11. 11. TokenOffset: 0 InputBroker: kafka.localTopic: Testing MR Job Output Output Offset: 130098 Broker: kafka.local Topic: Testing Sequence File
  12. 12. TokenOffset: 0 InputBroker: kafka.localTopic: Testing MR Job Output Output Offset: 130098 Broker: kafka.local Topic: Testing Sequence File
  13. 13. Useful Things• http://incubator.apache.org/kafka/• https://github.com/pingles/clj-kafka

×