An introduction to Apache Falcon

1,637 views

Published on

A short introduction to Apache Falcon, what is it and what is it used for ?
How can it help with Hadoop based data life cycle management ? What is it's
architecture and what are the benefits of using it ?

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,637
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
43
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

An introduction to Apache Falcon

  1. 1. Apache Falcon ● What is it ? ● Benefits ● Architecture ● Example www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  2. 2. Apache Falcon – What is it ? ● A data life cycle management framework ● Created for Hadoop ● Logic based in Falcon rather than apps ● Simplifies data management ● Developed by InMobi and HortonWorks ● Falcon can manage – Work flows – Replication – Provides data abstraction www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  3. 3. Apache Falcon – What is it ? ● Falcon provides services – Data import / replication – Scheduling / coordination – Lifecycle policies – Cluster management – SLA Management ● An enterprise solution for data lifecycle management ● Currently an Apache incubator project www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  4. 4. Apache Falcon – Benefits ● Reduce workflow / ETL development time ● Reduce costs ● No need to re implement functionality – – ● Already in Falcon Already tested Use a single Falcon configuration file to – Define replication points – Define data processing pipeline www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  5. 5. Apache Falcon – Architecture www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  6. 6. Apache Falcon – BI Example ● Falcon used to manage work flow ● Falcon used to manage Cluster data replication ● BI example – Staged and presented data replicated – Presented data visible for Reporting ● Analytics ● See next slide ..... ● www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  7. 7. Apache Falcon – BI Example www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  8. 8. Contact Us ● Feel free to contact us at – www.semtech-solutions.co.nz – info@semtech-solutions.co.nz ● We offer IT project consultancy ● We are happy to hear about your problems ● You can just pay for those hours that you need ● To solve your problems

×