Downloaded 28 times
Uploaded byelliando dias
Hadoop - Simple. Scalable.
This document provides an overview of Hadoop, including: 1) Hadoop solves the problems of analyzing massively large datasets by distributing data storage and analysis across multiple machines to tolerate node failure. 2) Hadoop uses HDFS for distributed data storage, which shards massive files across data nodes with replication for fault tolerance, and MapReduce for distributed data analysis by sending code to the data. 3) The document demonstrates MapReduce concepts like map, reduce, and their composition with an example job.
More Related Content
Similar to Hadoop - Simple. Scalable.
Hadoop - Simple. Scalable.
- 1.
- 2.
- 3. Java. Clojure. Ruby. Cloudera Certified
- 4.
- 5. Agenda Overview MassivelyLarge Data Sets and the problems therein Distributed File System MapReduce Pig
- 6.
- 7. Doug Cutting Genius
- 8. Favorite Hadoop Story New York Times
- 9. 4 Terabytes ofSource Articles.
- 10.
- 11. 5.5 Terabytes ofPDFs.
- 12.
- 13.
- 14. Infoporn from Yahoo 73 hours 490 TB Shuffling 280 TB Output 4000 Nodes 16 PB Disk Space 32K Cores 64 TB RAM
- 15.
- 16. Analyzing Massively Large Datasets
- 17. Two Problems You haveto distribute.
- 18. Data Storage Capacityhas increased rapidly beyond read speeds. Datasets won't fit on one disk. Tolerate node failure.
- 19. Data Analysis Combine data from many machines. Tolerate node failure.
- 20. How Hadoop solvesthese problems.
- 21. Send Code toData. Not Data to Code.
- 22. Data Storage HDFS
- 23. Name Node. DataNodes. Master - Slave Relationship
- 24. Shard massive filesacross multiple machines. MB, GB, and TB
- 25. Tolerant of NodeFailure Files replicated across at least 3 nodes.
- 26. HDFS behaves likea normal file system. No true appends yet.
- 27.
- 28. Data Analysis MapReduce
- 29. Job Tracker. TaskNodes. Master - Slave Relationship.
- 30.
- 31.
- 32.
- 33.
- 34.
- 35.
- 36.
- 37.
- 38. MapReduce Java
- 39. Nobody likes it. :-)
- 40. MapReduce Ruby. Python. UnixUtilities.
- 41.
- 42. Hadoop Ecosystem Pigkeeper. Hive.Cascading.
- 43.
- 44.