SlideShare a Scribd company logo

Block replication on HDFS

•Download as PPTX, PDF•
•
Koos van Strien
Koos van Strien

The replication of data on a Hadoop node is explained in three slides.

Technology
1 of 3
Files are broken up in ‘blocks’ Log file a.txt Block 1 Essentials of Hadoop for Big Data Block 2 Log file b.txt Block 3 Block 4 1 Log file e.txt Block 5
Blocks are replicated (at least 3x) throughout the cluster on Data Nodes Block 5 Node 1 Node 2 Node 3 Node 4 Block 4 Block 3 Block 2 Block 1 Essentials of Hadoop for Big Data 2
Blocks are replicated (at least 3x) throughout the cluster on Data Nodes Block 5 Block 1 Block 2 Block 1 Essentials of Hadoop for Big Data Block 3 Block 1 Block 2 Block 4 Block 4 Block 3 Node 2 Node 1 Block 5 Block 4 Block 5 Block 1 Block 2 Block 3 Block 5 Node 3 Node 4 Block 2 Block 3 Block 4 3

Recommended

Big data and data mining
Big data and data miningBig data and data mining
Big data and data mining
Polash Halder
 
Hadoop YARN overview
Hadoop YARN overviewHadoop YARN overview
Hadoop YARN overview
Arnon Rotem-Gal-Oz
 
003 - JavaFX Tutorial - Layouts
003 - JavaFX Tutorial - Layouts003 - JavaFX Tutorial - Layouts
003 - JavaFX Tutorial - Layouts
Mohammad Hossein Rimaz
 
Sphinx autodoc - automated api documentation - PyCon.MY 2015
Sphinx autodoc - automated api documentation - PyCon.MY 2015Sphinx autodoc - automated api documentation - PyCon.MY 2015
Sphinx autodoc - automated api documentation - PyCon.MY 2015
Takayuki Shimizukawa
 
Criação de Data Warehouse em Banco de Dados NoSQL com Cassandra, Spark e Python
Criação de Data Warehouse em Banco de Dados NoSQL com Cassandra, Spark e PythonCriação de Data Warehouse em Banco de Dados NoSQL com Cassandra, Spark e Python
Criação de Data Warehouse em Banco de Dados NoSQL com Cassandra, Spark e Python
Leandro Mendes Ferreira
 
Data cubes
Data cubesData cubes
Data cubes
Mohammed
 
Software engineering 101 - The basics you should hear about at least once
Software engineering 101 - The basics you should hear about at least onceSoftware engineering 101 - The basics you should hear about at least once
Software engineering 101 - The basics you should hear about at least once
Alexey (Mr_Mig) Migutsky
 
Beyond syllabus for web technology
Beyond syllabus for web technologyBeyond syllabus for web technology
Beyond syllabus for web technology
Durga Bhargavi Yarrabally
 

Recommended

Big data and data mining
Big data and data miningBig data and data mining
Big data and data mining
Polash Halder
 
Hadoop YARN overview
Hadoop YARN overviewHadoop YARN overview
Hadoop YARN overview
Arnon Rotem-Gal-Oz
 
003 - JavaFX Tutorial - Layouts
003 - JavaFX Tutorial - Layouts003 - JavaFX Tutorial - Layouts
003 - JavaFX Tutorial - Layouts
Mohammad Hossein Rimaz
 
Sphinx autodoc - automated api documentation - PyCon.MY 2015
Sphinx autodoc - automated api documentation - PyCon.MY 2015Sphinx autodoc - automated api documentation - PyCon.MY 2015
Sphinx autodoc - automated api documentation - PyCon.MY 2015
Takayuki Shimizukawa
 
Criação de Data Warehouse em Banco de Dados NoSQL com Cassandra, Spark e Python
Criação de Data Warehouse em Banco de Dados NoSQL com Cassandra, Spark e PythonCriação de Data Warehouse em Banco de Dados NoSQL com Cassandra, Spark e Python
Criação de Data Warehouse em Banco de Dados NoSQL com Cassandra, Spark e Python
Leandro Mendes Ferreira
 
Data cubes
Data cubesData cubes
Data cubes
Mohammed
 
Software engineering 101 - The basics you should hear about at least once
Software engineering 101 - The basics you should hear about at least onceSoftware engineering 101 - The basics you should hear about at least once
Software engineering 101 - The basics you should hear about at least once
Alexey (Mr_Mig) Migutsky
 
Beyond syllabus for web technology
Beyond syllabus for web technologyBeyond syllabus for web technology
Beyond syllabus for web technology
Durga Bhargavi Yarrabally
 
Big data
Big dataBig data
Big data
Ami Redwan Haq
 
Big Data Ppt PowerPoint Presentation Slides
Big Data Ppt PowerPoint Presentation Slides Big Data Ppt PowerPoint Presentation Slides
Big Data Ppt PowerPoint Presentation Slides
SlideTeam
 
Big Data Analytics with Hadoop
Big Data Analytics with HadoopBig Data Analytics with Hadoop
Big Data Analytics with Hadoop
Philippe Julio
 
Distributed DBMS - Unit 4 - Data Distribution Alternatives:
Distributed DBMS - Unit 4 - Data Distribution Alternatives:Distributed DBMS - Unit 4 - Data Distribution Alternatives:
Distributed DBMS - Unit 4 - Data Distribution Alternatives:
Gyanmanjari Institute Of Technology
 
Linux basic commands with examples
Linux basic commands with examplesLinux basic commands with examples
Linux basic commands with examples
abclearnn
 
Lecture1 introduction to big data
Lecture1 introduction to big dataLecture1 introduction to big data
Lecture1 introduction to big data
hktripathy
 
Hadoop HDFS Concepts
Hadoop HDFS ConceptsHadoop HDFS Concepts
Hadoop HDFS Concepts
tutorialvillage
 
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
Simplilearn
 
Mining high speed data streams: Hoeffding and VFDT
Mining high speed data streams: Hoeffding and VFDTMining high speed data streams: Hoeffding and VFDT
Mining high speed data streams: Hoeffding and VFDT
Davide Gallitelli
 
Introduction to Information Retrieval
Introduction to Information RetrievalIntroduction to Information Retrieval
Introduction to Information Retrieval
Roi Blanco
 
introduction of Java beans
introduction of Java beansintroduction of Java beans
introduction of Java beans
shravan kumar upadhayay
 
Text Classification
Text ClassificationText Classification
Text Classification
RAX Automation Suite
 
5.1 mining data streams
5.1 mining data streams5.1 mining data streams
5.1 mining data streams
Krish_ver2
 
Files concepts.53
Files concepts.53Files concepts.53
Files concepts.53
myrajendra
 
Language modelling and its use cases
Language modelling and its use casesLanguage modelling and its use cases
Language modelling and its use cases
Khrystyna Skopyk
 
THE 3V's OF BIG DATA: VARIETY, VELOCITY, AND VOLUME from Structure:Data 2012
THE 3V's OF BIG DATA: VARIETY, VELOCITY, AND VOLUME from Structure:Data 2012THE 3V's OF BIG DATA: VARIETY, VELOCITY, AND VOLUME from Structure:Data 2012
THE 3V's OF BIG DATA: VARIETY, VELOCITY, AND VOLUME from Structure:Data 2012
Gigaom
 
JavaScript Interview Questions 2023
JavaScript Interview Questions 2023JavaScript Interview Questions 2023
JavaScript Interview Questions 2023
Laurence Svekis ✔
 
Big Data
Big DataBig Data
Big Data
Vinayak Kamath
 
Big Data
Big DataBig Data
Big Data
Subhavinolin Raja
 
Introducción a la informática, sistema binario y codificación
Introducción a la informática, sistema binario y codificaciónIntroducción a la informática, sistema binario y codificación
Introducción a la informática, sistema binario y codificación
Tm-CS
 
Introduction to hadoop and hdfs
Introduction to hadoop and hdfsIntroduction to hadoop and hdfs
Introduction to hadoop and hdfs
shrey mehrotra
 
Trdk 2016-2-id-bloom
Trdk 2016-2-id-bloomTrdk 2016-2-id-bloom
Trdk 2016-2-id-bloom
Vladimir Kukharenko
 

More Related Content

What's hot

Big Data Ppt PowerPoint Presentation Slides
Big Data Ppt PowerPoint Presentation Slides Big Data Ppt PowerPoint Presentation Slides
Big Data Ppt PowerPoint Presentation Slides
SlideTeam
 
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
Simplilearn
 
Files concepts.53
Files concepts.53Files concepts.53
Files concepts.53
myrajendra
 
JavaScript Interview Questions 2023
JavaScript Interview Questions 2023JavaScript Interview Questions 2023
JavaScript Interview Questions 2023
Laurence Svekis ✔
 

What's hot (20)

Big data
Big dataBig data
Big data
 
Big Data Ppt PowerPoint Presentation Slides
Big Data Ppt PowerPoint Presentation Slides Big Data Ppt PowerPoint Presentation Slides
Big Data Ppt PowerPoint Presentation Slides
 
Big Data Analytics with Hadoop
Big Data Analytics with HadoopBig Data Analytics with Hadoop
Big Data Analytics with Hadoop
 
Distributed DBMS - Unit 4 - Data Distribution Alternatives:
Distributed DBMS - Unit 4 - Data Distribution Alternatives:Distributed DBMS - Unit 4 - Data Distribution Alternatives:
Distributed DBMS - Unit 4 - Data Distribution Alternatives:
 
Linux basic commands with examples
Linux basic commands with examplesLinux basic commands with examples
Linux basic commands with examples
 
Lecture1 introduction to big data
Lecture1 introduction to big dataLecture1 introduction to big data
Lecture1 introduction to big data
 
Hadoop HDFS Concepts
Hadoop HDFS ConceptsHadoop HDFS Concepts
Hadoop HDFS Concepts
 
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
 
Mining high speed data streams: Hoeffding and VFDT
Mining high speed data streams: Hoeffding and VFDTMining high speed data streams: Hoeffding and VFDT
Mining high speed data streams: Hoeffding and VFDT
 
Introduction to Information Retrieval
Introduction to Information RetrievalIntroduction to Information Retrieval
Introduction to Information Retrieval
 
introduction of Java beans
introduction of Java beansintroduction of Java beans
introduction of Java beans
 
Text Classification
Text ClassificationText Classification
Text Classification
 
5.1 mining data streams
5.1 mining data streams5.1 mining data streams
5.1 mining data streams
 
Files concepts.53
Files concepts.53Files concepts.53
Files concepts.53
 
Language modelling and its use cases
Language modelling and its use casesLanguage modelling and its use cases
Language modelling and its use cases
 
THE 3V's OF BIG DATA: VARIETY, VELOCITY, AND VOLUME from Structure:Data 2012
THE 3V's OF BIG DATA: VARIETY, VELOCITY, AND VOLUME from Structure:Data 2012THE 3V's OF BIG DATA: VARIETY, VELOCITY, AND VOLUME from Structure:Data 2012
THE 3V's OF BIG DATA: VARIETY, VELOCITY, AND VOLUME from Structure:Data 2012
 
JavaScript Interview Questions 2023
JavaScript Interview Questions 2023JavaScript Interview Questions 2023
JavaScript Interview Questions 2023
 
Big Data
Big DataBig Data
Big Data
 
Big Data
Big DataBig Data
Big Data
 
Introducción a la informática, sistema binario y codificación
Introducción a la informática, sistema binario y codificaciónIntroducción a la informática, sistema binario y codificación
Introducción a la informática, sistema binario y codificación
 

Viewers also liked

Hadoop scalability
Hadoop scalabilityHadoop scalability
Hadoop scalability
WANdisco Plc
 
HBase and HDFS: Understanding FileSystem Usage in HBase
HBase and HDFS: Understanding FileSystem Usage in HBaseHBase and HDFS: Understanding FileSystem Usage in HBase
HBase and HDFS: Understanding FileSystem Usage in HBase
enissoz
 

Viewers also liked (6)

Introduction to hadoop and hdfs
Introduction to hadoop and hdfsIntroduction to hadoop and hdfs
Introduction to hadoop and hdfs
 
Trdk 2016-2-id-bloom
Trdk 2016-2-id-bloomTrdk 2016-2-id-bloom
Trdk 2016-2-id-bloom
 
Hadoop Internals
Hadoop InternalsHadoop Internals
Hadoop Internals
 
Hadoop HDFS Detailed Introduction
Hadoop HDFS Detailed IntroductionHadoop HDFS Detailed Introduction
Hadoop HDFS Detailed Introduction
 
Hadoop scalability
Hadoop scalabilityHadoop scalability
Hadoop scalability
 
HBase and HDFS: Understanding FileSystem Usage in HBase
HBase and HDFS: Understanding FileSystem Usage in HBaseHBase and HDFS: Understanding FileSystem Usage in HBase
HBase and HDFS: Understanding FileSystem Usage in HBase
 

Recently uploaded

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Orbitshub
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
UiPathCommunity
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Orbitshub
 

Recently uploaded (20)

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKSpring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 

Block replication on HDFS

  • 1. Files are broken up in ‘blocks’ Log file a.txt Block 1 Essentials of Hadoop for Big Data Block 2 Log file b.txt Block 3 Block 4 1 Log file e.txt Block 5
  • 2. Blocks are replicated (at least 3x) throughout the cluster on Data Nodes Block 5 Node 1 Node 2 Node 3 Node 4 Block 4 Block 3 Block 2 Block 1 Essentials of Hadoop for Big Data 2
  • 3. Blocks are replicated (at least 3x) throughout the cluster on Data Nodes Block 5 Block 1 Block 2 Block 1 Essentials of Hadoop for Big Data Block 3 Block 1 Block 2 Block 4 Block 4 Block 3 Node 2 Node 1 Block 5 Block 4 Block 5 Block 1 Block 2 Block 3 Block 5 Node 3 Node 4 Block 2 Block 3 Block 4 3