3 Hadoop


Hadoop MR + Cassandra


              2010/06/28
             @ryu_kobayashi
• Hadoop MapReduce
• Cassandra
• MapReduce + Cassandra
• Cassandra       MapReduce

•
•
•
     •                       ...

•                              ...

•   Twitter @ryu_kobayashi
Hadoop MapReduce



         Google MapReduce
Cassandra

Facebook       KVS NoSQL       DB
               0.6.2       0.6.3
Cassandra

Facebook       KVS NoSQL       DB
               0.6.2       0.6.3
Cassandra

Facebook         KVS NoSQL       DB
                 0.6.2       0.6.3




  •   Facebook
Cassandra

Facebook         KVS NoSQL       DB
                 0.6.2       0.6.3




  •   Facebook
  •   Twitter
Cassandra

Facebook         KVS NoSQL       DB
                 0.6.2       0.6.3




  •   Facebook
  •   Twitter
  •
Cassandra

Facebook         KVS NoSQL       DB
                 0.6.2       0.6.3




  •   Facebook
  •   Twitter
  •
  •
Cassandra

Facebook         KVS NoSQL       DB
                 0.6.2       0.6.3




  •   Facebook
  •   Twitter
  •
  •...
Hadoop MR + Cassandra
Hadoop MR + Cassandra
Hadoop MR + Cassandra
Hadoop MR + Cassandra


Cassandra Contributer
Hadoop MR + Cassandra
Hadoop MR + Cassandra
Hadoop MR + Cassandra
Hadoop MR + Cassandra




   Hadoop    Cassandra   H
H
Hadoop MR + Cassandra
Hadoop MR + Cassandra
•   MR HDFS
Hadoop MR + Cassandra
•   MR HDFS
•    HDFS       30 40
Hadoop MR + Cassandra
•        MR HDFS
•         HDFS                30 40


•   Cassandra 0.6   Hadoop
                  ...
Hadoop MR + Cassandra
•        MR HDFS
•         HDFS                  30 40


•   Cassandra 0.6      Hadoop
             ...
Hadoop MR + Cassandra
•        MR HDFS
•         HDFS                        30 40


•   Cassandra 0.6      Hadoop
       ...
Hadoop MR + Cassandra
Hadoop MR + Cassandra
Hadoop MR + Cassandra
Hadoop MR + Cassandra
Cassandra               HDFS   +TaskTracker
            Cassandra
Hadoop MR + Cassandra
Cassandra               HDFS   +TaskTracker
            Cassandra

                  Contribute
Cassandra                     MapReduce

•   org.apache.cassandra.hadoop

•                               storage-conf.xml...
Cassandra            MapReduce




            github


    http://bit.ly/9nwgSu
Cassandra-GUI
http://code.google.com/p/cassandra-gui/
Hadoopソースコードリーディング第3回 Hadopo MR + Cassandra
Hadoopソースコードリーディング第3回 Hadopo MR + Cassandra
Upcoming SlideShare
Loading in …5
×

Hadoopソースコードリーディング第3回 Hadopo MR + Cassandra

7,418 views

Published on

Published in: Technology
1 Comment
9 Likes
Statistics
Notes
No Downloads
Views
Total views
7,418
On SlideShare
0
From Embeds
0
Number of Embeds
267
Actions
Shares
0
Downloads
116
Comments
1
Likes
9
Embeds 0
No embeds

No notes for slide
































  • Hadoopソースコードリーディング第3回 Hadopo MR + Cassandra

    1. 1. 3 Hadoop Hadoop MR + Cassandra 2010/06/28 @ryu_kobayashi
    2. 2. • Hadoop MapReduce • Cassandra • MapReduce + Cassandra • Cassandra MapReduce •
    3. 3. • • • ... • ... • Twitter @ryu_kobayashi
    4. 4. Hadoop MapReduce Google MapReduce
    5. 5. Cassandra Facebook KVS NoSQL DB 0.6.2 0.6.3
    6. 6. Cassandra Facebook KVS NoSQL DB 0.6.2 0.6.3
    7. 7. Cassandra Facebook KVS NoSQL DB 0.6.2 0.6.3 • Facebook
    8. 8. Cassandra Facebook KVS NoSQL DB 0.6.2 0.6.3 • Facebook • Twitter
    9. 9. Cassandra Facebook KVS NoSQL DB 0.6.2 0.6.3 • Facebook • Twitter •
    10. 10. Cassandra Facebook KVS NoSQL DB 0.6.2 0.6.3 • Facebook • Twitter • •
    11. 11. Cassandra Facebook KVS NoSQL DB 0.6.2 0.6.3 • Facebook • Twitter • • @shot6
    12. 12. Hadoop MR + Cassandra
    13. 13. Hadoop MR + Cassandra
    14. 14. Hadoop MR + Cassandra
    15. 15. Hadoop MR + Cassandra Cassandra Contributer
    16. 16. Hadoop MR + Cassandra
    17. 17. Hadoop MR + Cassandra
    18. 18. Hadoop MR + Cassandra
    19. 19. Hadoop MR + Cassandra Hadoop Cassandra H
    20. 20. H
    21. 21. Hadoop MR + Cassandra
    22. 22. Hadoop MR + Cassandra • MR HDFS
    23. 23. Hadoop MR + Cassandra • MR HDFS • HDFS 30 40
    24. 24. Hadoop MR + Cassandra • MR HDFS • HDFS 30 40 • Cassandra 0.6 Hadoop (API )
    25. 25. Hadoop MR + Cassandra • MR HDFS • HDFS 30 40 • Cassandra 0.6 Hadoop (API ) • KVS HDFS Hbase
    26. 26. Hadoop MR + Cassandra • MR HDFS • HDFS 30 40 • Cassandra 0.6 Hadoop (API ) • KVS HDFS Hbase • Cassandra InputFormat RDBS AWS EMR(Elastic MapReduce + S3
    27. 27. Hadoop MR + Cassandra
    28. 28. Hadoop MR + Cassandra
    29. 29. Hadoop MR + Cassandra
    30. 30. Hadoop MR + Cassandra Cassandra HDFS +TaskTracker Cassandra
    31. 31. Hadoop MR + Cassandra Cassandra HDFS +TaskTracker Cassandra Contribute
    32. 32. Cassandra MapReduce • org.apache.cassandra.hadoop • storage-conf.xml # Thrift storage-conf.xml • Jar Cassadra Thrift classpath • OutputFormat HDFS (0.7 ) • 1000 Map # FAQ Issue http://goo.gl/5Q3Z
    33. 33. Cassandra MapReduce github http://bit.ly/9nwgSu
    34. 34. Cassandra-GUI http://code.google.com/p/cassandra-gui/

    ×