Your SlideShare is downloading. ×
0
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Hadoop Conference Japan 2011 Fallに行ってきました

4,439

Published on

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,439
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
21
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. @just_do_neet 1
  • 2. Hadoop Conference Japan 2011 fall• 2011/9/26( ) http://hadoop-conference-japan-2011-fall.eventbrite.com/• Hadoop 1000• Cloudera HortonWorks MapR• NTT mixi Yahoo 2
  • 3. http://itpro.nikkeibp.co.jp/article/NEWS/20110926/369421/?SS=imgview&FD=-821521671&ST=cloud 3
  • 4. 4
  • 5. 5
  • 6. Hadoop T 6
  • 7. Hadoop 7
  • 8. Hadoop• Apache Software Foundation OSS• • :MapReduce • :HDFS/HBase 8
  • 9. Hadoop• Google GFS(ACM ’03) “The Google File System” → HDFS MapReduce(OSDI ’04) “Simplified Data Processing on Large Clusters” → MapReduce BigTable(OSDI ’06) “A Distributed Storage System for Structured Data” → HBase• Yahoo! Inc. Cloudera Doug Cutting Google 2004 Java OSS ”Hadoop” Doug 9
  • 10. Hadoop• index index etc... ※Google ※ Doug Lucene Nutch OSS• Hadoop• Hadoop 10
  • 11. Hadoop• HBase BigTable• ZooKeeper Chubby• Hive HDFS SQL• Flume• and more.... 11
  • 12. • sorry, confidential... 12
  • 13. 13
  • 14. Hadoop Conference Japan 2011 Fall• Hadoop 0.23•• 14
  • 15. Hadoop 0.23• 2011 Q4 beta Integration Test Pig• Hadoop NameNode SPoF (NameNode• MapReduce v2 : MapReduce MPI(C Giraph Hama Graph Spark etc• MapReduce shuffle 30% / 10,000 4000 / etc 15
  • 16. Giraph• Hadoop https://github.com/aching/Giraph Graph → Google PageRank Google Pregel(SIGMOD ’10)• BSP(Bulk Synchronous Parallel 16
  • 17. Spark• Scala http://www.spark-project.org/ → ex.• Mesos http://www.mesosproject.org/ Hadoop 17
  • 18. H• Cloudera http://www.cloudera.com/ OSS (CDH Sqoop Hue• HortonWorks http://www.hortonworks.com/ Yahoo! Inc. Hadoop• MapR http://www.mapr.com/ Hadoop• Hadoop 18
  • 19. H Cloudera• CDH Linux• OSS Sqoop SQL HDFS Hue Web Console etc..• SCM Express Web Hadoop 50 free• Cloudera Enterprise Cloudera Management Suite• 19
  • 20. H HortonWorks• Yahoo! Inc. Hadoop 22 Hadoop ※ ( 500,000• 2011/7• Yahoo! Inc. Yahoo! Mail 42,000 Hadoop• Horton the Elephant http://en.wikipedia.org/wiki/Horton_the_Elephant 20
  • 21. H MapR• Hadoop EMC OEM Greenplum http://www.greenplum.com/• HDFS C++ HDFS random read/write NIC bonding snapshot etc.. NFS Hadoop R• OSS Hadoop 21
  • 22. • Hadoop LSP etc..• Asakusa Framework OSS https://github.com/asakusafw DSL 22
  • 23. Yahoo! Japan• Hadoop Geohash Hadoop Iterable Twitter ※ MPI ... 23
  • 24. 24
  • 25. • Hadoop BtoC Hadoop 0.23 MapReduce v2 Hadoop• Hadoop Hadoop Map/Reduce + Graph BI 25
  • 26. 26

×