Key big data terms you should know

951 views

Published on

Listing of key Big Data terms that you should know and a very brief explanation of what it is in simple language. Hope you find it useful.

Published in: Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
951
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Key big data terms you should know

  1. 1. Key Big Data Terms You Should Know Harish Kotadia, Ph.D. Blog: http://HKotadia.com Twitter: http://twitter.com/HKotadia LinkedIn: http://www.linkedin.com/in/HKotadia © 2013 Harish Kotadia, Ph.D. 1
  2. 2. Key Big Data Terms You Should Know1. Hadoop: System for processing very large data sets2. HDFS or Hadoop Distributed File System: For storage of large volume of data (key elements – Datanodes, Namenode and Tasktracker)3. MapReduce: Think of it as Assembly level language for distributed computing. Used for computation in Hadoop4. Pig: Developed by Yahoo. It is a higher level language than MapReduce5. Hive: Higher level language developed by Facebook with SQL like syntax6. Apache HBase: For real-time access to Hadoop data7. Accumulo: Improved HBase with new features like cell level security8. AVRO: New data serialization format (protocol buffers etc.)9. Apache ZooKeeper: Distributed co-ordination system © 2013 Harish Kotadia, Ph.D. 2
  3. 3. Key Big Data Terms You Should Know10. HCatalog: For combining meta store of Hive and merging with what Pig does11. Oozie: Scheduling system developed by Yahoo12. Flume: Log aggregation system13. Whirr: For automating hadoop cluster processing14. Sqoop: For transfering structured data to Hadoop15. Mahout: Machine learning on top of MapReduce16. Bigtop: Integrate multiple Hadoop sub-systems into one that works as a whole17. Crunch: Runs on top of MapReduce, Java API for tedious tasks like joining18. Giraph: Used for large scale distributed graph processing © 2013 Harish Kotadia, Ph.D. 3
  4. 4. for more, check out my blog: http://hkotadia.com/ © 2013 Harish Kotadia, Ph.D. 4

×