Your SlideShare is downloading. ×
An introduction to Apache Chukwa
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

An introduction to Apache Chukwa

960
views

Published on

A introduction to Apache Chukwa, what is it and …

A introduction to Apache Chukwa, what is it and
how does it work ? Why is it important to monitor
Hadoop DFS and how can it help us ?

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
960
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
22
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Apache Chukwa ● What is it ? ● How does it work ? ● What can we collect ? ● Architecture www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  • 2. Chukwa – What is it ? ● For log collection and analysis ● Designed for big data ● Designed for Hadoop ● Uses HDFS and MapReduce ● Scaleable ● Robust ● Provides a tool kit to analyse logs www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  • 3. Chukwa – How does it work ? ● Chukwa agents on source nodes ● Transfer data to collectors which save data to HDFS ● Data sinks contain raw unsorted data ● Data sinks clean data ● Demux adds structure to create Chukwa records ● Chukwa records go to database ● Are ready to be analysed www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  • 4. Chukwa – What can we collect ? ● Metrics ● System logs – Defined format – Undefined format ● Low latency – Access to log data www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  • 5. Chukwa – Architecture ? www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  • 6. Chukwa – Architecture ? ● Chukwa agents – Reside on the Hadoop machines – Collect raw data – Use adaptors for data sources – Use http to transmit data – Operate on data chunks – Can fail over between collectors www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  • 7. Contact Us ● Feel free to contact us at – www.semtech-solutions.co.nz – info@semtech-solutions.co.nz ● We offer IT project consultancy ● We are happy to hear about your problems ● You can just pay for those hours that you need ● To solve your problems