LDeNA 峠とログ収集・集計の話

4,885 views
4,808 views

Published on

Published in: Technology, Health & Medicine
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,885
On SlideShare
0
From Embeds
0
Number of Embeds
2,171
Actions
Shares
0
Downloads
27
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide

LDeNA 峠とログ収集・集計の話

  1. 1. 12 1 30
  2. 2. 12 1 30
  3. 3. 12 1 30
  4. 4. ( )12 1 30
  5. 5. ( )12 1 30
  6. 6. 12 1 30
  7. 7. 12 1 30
  8. 8. 12 1 30
  9. 9. 12 1 30
  10. 10. 12 1 30
  11. 11. 12 1 30
  12. 12. 12 1 30
  13. 13. 12 1 30
  14. 14. 12 1 30
  15. 15. 12 1 30
  16. 16. 12 1 30
  17. 17. 12 1 30
  18. 18. 12 1 30
  19. 19. (2000 2011) 1. scp /URL (bot/ PV ) 2. 1 3. gluster fs (gluster)12 1 30
  20. 20. 12 1 30
  21. 21. 12 1 30
  22. 22. 12 1 30
  23. 23. 12 1 30
  24. 24. (2011) 1. scribe push ( ) (scribed) 2. HDFS 3. parse Shib Hadoop Hadoop Hadoop Hive (scribed) Hadoop Cluster Hadoop Cluster Hadoop RAID Cluster Hadoop (scribed) Cluster Cluster RAID Cluster 4.12 1 30
  25. 25. …12 1 30
  26. 26. ………12 1 30
  27. 27. 12 1 30
  28. 28. 12 1 30
  29. 29. 12 1 30
  30. 30. (NOW!) (scribed) (scribed) RAID (scribed) (scribed) RAID Fluentd Hadoop Hadoop Hadoop Hadoop Hadoop Hadoop Shib Cluster Hadoop Cluster Hadoop Cluster Hadoop Cluster Fluentd Cluster Hadoop Cluster Hadoop Hadoop Hive Cluster Cluster Cluster Cluster Cluster Cluster HDFS12 1 30
  31. 31. Shirahne -> Shib (scribed) (scribed) RAID (scribed) (scribed) RAID Hadoop Hadoop Hadoop Hadoop Hadoop Hadoop Shib Cluster Hadoop Cluster Hadoop Cluster Hadoop Cluster Fluentd Cluster Hadoop Cluster Hadoop Hadoop Hive Cluster Cluster Cluster Cluster Cluster Cluster Shirahne12 1 30
  32. 32. 12 1 30
  33. 33. Yahoo! HTTP parse Fluentd IRC12 1 30
  34. 34. 12 1 30
  35. 35. 12 1 30

×