Download to read offline
Uploaded byJavier Lafora Rey
Understanding big data-drupalcamp
The document discusses the rapid growth of data, highlighting that 90% of the world's data was created in the last two years and how traditional methods of data management are becoming insufficient. It introduces distributed file systems and parallelized computing as solutions to efficiently handle large data volumes. Notably, it mentions Hadoop and MapReduce as effective tools for managing data variety and computing scalability.
Understanding big data-drupalcamp
- 1. Understanding Big Data... ...ortrying to understand a new awesome paradigm
- 2.
- 5. A small walkthrough historyA small walk through history
- 10. Data Growth “90% ofthe data in the world today has been created in the last two years alone.” - IBM http://www-01.ibm.com/software/in/data/bigdata/
- 11. Data growth “There were5 Exabytes of information created between the dawn of civilization through 2003, but that much information is now created every 2 days”. - Eric Schmidt, Google CEO, 2010
- 17. Traditional approach One powerful computerfor all the data...
- 18. Traditional approach ...until itreaches storage limit...
- 19. Traditional approach ...or untilit reaches process limit
- 23.
- 24.
- 25.
- 26. Distributed filesystems A DFSmanages files and folders across multiple computers. It serves the same purpose as a traditional file system, but is designed to provide file storage and controlled access to files over local and wide area networks.
- 27. You don't wantto worry about where a file is located
- 28. You don't wantto worry about replicating data
- 29. You don't wantto worry about managing failures
- 30.
- 31. You don't wantto worry about breaking computation into pieces
- 32. You don't wantto worry about scaling your code
- 33. Hadoop to therescue
- 34.
- 35.
- 37.
- 38.
- 39.
- 40. Streaming / realtime
- 41.
- 42.
- 43.
- 44.
- 49.
- 50.
Editor's Notes
- #12 1 EB = 1024 PB 1 PB = 1024 TB 1 TB = 1024 GB