Hadoop 5
Upcoming SlideShare
Loading in...5
×
 

Hadoop 5

on

  • 1,699 views

Hadoop本輪読会第4回の発表資料です。

Hadoop本輪読会第4回の発表資料です。

Statistics

Views

Total Views
1,699
Views on SlideShare
1,693
Embed Views
6

Actions

Likes
1
Downloads
31
Comments
0

2 Embeds 6

http://www.slideshare.net 5
http://www.slashdocs.com 1

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />

Hadoop 5 Hadoop 5 Presentation Transcript

  • Hadoop 5 MapReduce
  • ❖ blog: http://brass.to/blog/ ❖ twitter: @akahigeg ❖ Rails ❖ Mac,Vim ❖ Linux(CentOS),Apache,MySQL ❖ Agile
  • ❖ X ❖ PHP CakePHP ❖ TDD ❖ 16 Puppet
  • 5.1 API P127 5.2 P130 5.3 P137 5.4 P140 5.5 P146 5.6 P161 5.7 MapReduce P164
  • 5.1 API P127 ❖ = XML 5-1 P128 ❖ ‣ ❖ ‣ ‣ final
  • 5.1 API P129 ❖ ‣ DRY ❖ ‣ ❖
  • 5.2 P130 ❖ Hadoop ❖ IDE ❖ P131 ‣ -conf P132 ‣ conf ‣ P133
  • 5.2 P133 ❖ Tool ❖ GenericOptionParser ToolRunner ❖ P135 ❖ P135 ‣ JVM P136 ‣ -conf
  • 5.3 P137 ❖ map reduce ‣ ‣ OutputCollector ‣ ❖ = ❖ JUnit Mockito TDD ‣ MapReduce MRUnit
  • 5.3 P137 ❖ MaxTemperatureMapper ‣ Test: 5-4 P137 ‣ mapper ver.1 5-5 P138 ‣ Test: P138 ‣ mapper ver.2 P139 ❖ MaxTemperatureReducer ‣ Test: P139 ‣ reducer 5-6 P140
  • 5.4 P140 ❖ ‣ map reducer ‣ Tool ❖ JobRunner ‣ MapReduce ‣
  • 5.4 P142 ❖ ‣ mapper - ❖ 5-8 P143 ❖ mapper 5-9 P144 ❖ TDD
  • 5.4 P144 ❖ JobRunner ‣ Tool ‣ 5-10 P145 ❖ P145 ‣ HDFS MapReduce ‣ Hadoop ‣
  • 5.5 P146 ❖ ❖ ‣ JobClient P147 ‣ ID = job_YYYYMMDDHHMM_0000 ‣ ID = ID + _m_000000 ‣ ID = ID + _0000
  • 5.5 P148 ❖ MapReduce Web UI http://jobtracker-host:50030/ ‣ jobtracker P148 - Hadoop - - - - ❖ P150 - -
  • 5.5 P152 ❖ ‣ reducer - MapReduce ‣ ‣ ‣
  • 5.5 P153 ❖ ‣ Web UI ‣ ‣ MapReduce ❖ Web UI ‣ ‣
  • 5.5 P158 ❖ P158 ‣ P157 ‣ ❖ Hadoop P157 ‣ Chukwa ‣ ‣
  • 5.5 P159 ❖ ‣ ‣ ‣ ‣ ‣ ‣ IDE
  • 5.6 P161 ❖ ✓ mapper ✓ reducer ✓ combiner ✓ ✓ ✓
  • 5.6 P161 ❖ ‣ JobRunner P162 ❖ HPROF P162 ‣ ‣ ‣
  • 5.7 MapReduce P164 ❖ MapReduce ❖ ‣ UNIX ❖ MapReduce Pig Hive
  • 5.7 MapReduce P166 ❖ ❖ 2 ❖ ❖ ❖ ‣ JobControl ‣ Hadoop Workflow Scheduler HWS ※
  • ❖ TDD ‣ ‣ ‣ ❖ ❖ ❖