SlideShare a Scribd company logo
1 of 116
Download to read offline
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
w
, +1(++( +
r4JF 5YNP 5RYa B YV R FRZV N_s
4ZNg 8@E
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
•
z
•
w
× w
w w ×
• 7MI w
7 Rk _ CH& 7 Rk _ 9I
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
v
• – ()/ )) ) n x v v k
–4JF Uaa 2 Nd &NZNg &P Z# vk
• – x zj 4JF
zu j4JF v k
• – v k x z n j
v k
• 7MI U Vd _ e WWVc SZ_UZ_X acZTV bf eVd( 7MI acZTZ_X Zd afS ZT j RgRZ RS V R_U Zd
dfS VTe e TYR_XV Z_ RTT cUR_TV hZeY eYV 7MI 9fde Vc 7XcVV V_e RgRZ RS V Re
Yeea4))Rhd(R Rk _(T )RXcVV V_e)( 7_j acZTZ_X Z_W c ReZ _ Z_T fUVU Z_ eYZd
U Tf V_e Zd ac gZUVU _ j Rd R_ VdeZ ReV W fdRXV TYRcXVd W c 7MI dVcgZTVd SRdVU
_ TVceRZ_ Z_W c ReZ _ eYRe j f YRgV ac gZUVU( C _eY j TYRcXVd hZ SV SRdVU _
j fc RTefR fdV W 7MI dVcgZTVd& R_U Rj gRcj Wc eYV VdeZ ReVd ac gZUVU(
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4TR QN
•
• 7 Rk _ CH
• 7 Rk _ CH w , +0)+ $
• 7 Rk _ CH .(i
• w 7 Rk _ CH
•
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
–~
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E
❏ , 3 w
• CRaHVUfTV p +
❏ >RU a ×
• FZX& >ZgV& >8RdV& FcVde & >fV&
>RU a ,(i& IaRc ,(i& Z_
❏ 7MI w
• Ia e& 7C& :j_R :8& LF9&
9 fUMReTY& 9B & AZ_VdZd& ACI
→ – @N ERQbPR
– vm
Storage
S3 (EMRFS), HDFS
YARN
Cluster Resource Management
Batch
MapReduce
Interactive
Tez
In Memory
Spark, Flink
Applications
Hive, Pig, Spark SQL/Streaming/ML, Mahout, Sqoop
HBase/Phoenix
Presto
Hue (SQL Interface/Metastore Management)
Zeppelin (Interactive Notebook)
Ganglia (Monitoring)
HiveServer2/Spark Thriftserver (JDBC/ODBC)
Amazon EMR service
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
n
S3
EMR
EMR Redshift
Machine
Learning
Collect Process Analyze
Store
Data Collection
and Storage
Data
Processing
Event
Processing
Data
Analysis
Data Answers
Athena
Quicksight
Kinesis
DynamoDB
AWS	IoT
AWS	Lambda
KCL	Apps Glue
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
❏ w
)
• CRdeVc)9 cV)JRd _ UV
• CH I
• 8 edecRa 7TeZ _& IeVad
n
❏
w
• >RU a& O7HD
• >ZgV& FZX& >8RdV& IaRc &
Z_ & FcVde & VeT(
4ZNg 8@E
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E
n
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Task Node
Task Instance Group
4ZNg 8@E n
security group
security group
Master Node
Master Instance Group
Amazon
S3
Amazon
DynamoDB
Amazon
Kinesis
Core Node
Core Instance Group
HDFS HDFS
HDFS HDFS
Task Node
Task Instance Group
w >: I
7MI w
AWS Glue
Data Catalog
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E @N aR_ aN PR :_ b
❏ CRdeVc D UV +
• RZ gVc
❏ w
• DR VD UV
HVd fcTVCR_RXVc
• 9 cV D UV JRd D UV
Master Node
Master Instance Group
Hadoop1: JobTracker
Hadoop2: ResourceManager
HDFS: NameNode
Hive: HiveServer2, MetaStore
Presto: Coordinator
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E 6 _R aN PR :_ b
❏ + 9 cV D UV
❏ w
• D UVCR_RXVc
• :ReRD UV w
>: I
• 8Ip
EA
HDFS
Core Node
Core Instance Group
Hadoop1: TaskTracker
Hadoop2: NodeManager
HDFS: DataNode
Presto: Worker
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E 6 _R aN PR :_ b
❏ 9 cV D UV
• >: I
• 9FK)H7C
❏ >: I
p
• .(+(
HDFS
Core Node
Core Instance Group
Hadoop1: TaskTracker
Hadoop2: NodeManager
HDFS: DataNode
Presto: Worker
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E GN aN PR :_ b
❏ >: I
9 cV
❏ >: I w 9 cV
D UV
❏ >: I
• >: I w
:VT ZddZ _$
Task Node
Task Instance Group
Hadoop1: TaskTracker
Hadoop2: NodeManager
Presto: Worker
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E GN aN PR :_ b
❏ =c fa
• Ia e SZU
• _deR_TV JjaV
❏ H
p
Ia e
Task Instance Group 2
Task Instance Group 1
c3.xlarge * 2
bid: $0.1
r3.xlarge * 2
bid: $0.5
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
F a aN PR
Task Instance GroupCore Instance Group
❏ ❏
❖ B QRZN Q
9 cV D UV
❖ ”
×
E_ UV R_U
❖ F a aN PR JRd
D UV
❖ E_ UV R_U
3
w
4 .(i RcXV 0 + +, p/ $
6 _R A QR B QRZN Q# 4 (-+2o+ Yo0
GN A QR F a# 4 (-+2o/Yo0 % ( .o/Yo0
, +1 ++ + $ w qIa e q
5 +3( 2 +
5 + (1. /
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E
l
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
8@E9F2 4ZNg F 79F w
❏ w
❏
• w
❏ w
• w
❏ w
EMR
EMR
Amazon
S3
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
8@E9F
❏ ld-4))n ”
❏ 7 Rk _ I-
• 4 w 7 Rk _ = RTZVc
❏ 7 Rk _ I- w s
• 7 Rk _ I- w
❏ w
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
8@E9F 6 V aR a IVRd
❏ 7 Rk _ I-
• FKJ
HVRU RWeVc hcZeV T _dZdeV_Tj$
• FKJ): B J gV_efR
T _dZdeV_Tj$
❏ CH I 9 _dZdeV_e LZVh
• 7 Rk _ :j_R :8 w
• BZde 9 _dZdeV_Tj
• :V VeV 9 _dZdeV_Tj
• HVRU RWeVc hcZeV 9 _dZdeV_Tj
•
Amazon S3 Amazon DynamoDB
CH I
w
Yeead4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)CR_RXV V_e=fZUV)V c a R_ T _dZdeV_e gZVh(Ye
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E – v
❏ ”
•
•
❏ w 7 Rk _ I-
• w
• w
t
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E 2 5 a a_N 4PaV
❏ D UV
• EA
• 8RdY& HfSj& FjeY _& VeT(
• 7 Rk _ I- p
• 7aRTYV 8ZXe a w
w p8 edecRa
Yeea4))Rhd(ejaVaRU(T )dR a), +0) 3)SfZ UZ_X R_U UVa jZ_X Tfde Raa ZTReZ _d hZeY RaRTYV 8ZXe a R_U R Rk _ V c(Ye
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E 2 FaR
❏ )
• p
• 4 JB >ZgVGB
❏ 7 Rk _ I- KH p p w
• I- Rc
• IecVR Z_X& >ZgV& FZX& IaRc CH w T R_U cf__Vc( Rc $
• dTcZae cf__Vc( Rc SRdY
❏ IeVa
7fe eVc Z_ReV$
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E
y
❏ IeVa
• 7 Rk _ CH
• w w 9B
• 7MI I:A
• 7C
• )
•
❏ 7MI w
• BR SUR
• :ReR FZaV Z_V
❏ w
Amazon EMR
Amazon EMR
Step API
AWS Data Pipeline
Airflow, Luigi, or other
schedulers on EC2
AWS Lambda
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E
y
❏ w w
• CRdeVc D UV ddY p p
IIC 7 Rk _ 9, IjdeV d CR_RXVc$ Hf_ T R_U
• >ZgVIVcgVc,& daRc dfS Ze
• PVaaV Z_& >fV& daRc SdVcgVc =K
• BZgj HVde IaRc
• E kZV w w
Web UIs: Hue SQL editor, Zeppelin
notebooks,
R Studio, and more!
Connect with ODBC / JDBC using
HiveServer2/Spark Thriftserver
Use Spark Actions in your Apache
Oozie workflow to create DAGs of jobs.
(start using
start-thriftserver.sh)
Or, use the native APIs and CLIs for
each application
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
C_VcNaR FbO Ra 8@E z
❏ IfS_Ve FcZgReV
• CH .(,(
❏ FfS ZT
• I-):j_R :8
LF9 V_Ua Z_e
• I-):j_R :8 w
D7J =M
• CH IVcgZTV
IVTfcZej =c fa
Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +/)+,) Rf_TY R Rk _ V c T fdeVcd Z_ R Rk _ gaT acZgReV dfS_Ved)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4 NPUR F N_ GRg NQ @N ERQbPR
C_R a n z
Yeea4))U Td(Rhd(R Rk _(T )V c) ReVde)HV VRdV=fZUV)V c UReR V_TcjaeZ _ aeZ _d(Ye
❏ Re cVde pZ_ ecR_dZe p
w
❏ Re cVde
• CH I& >: Ip p w
× w
❏ Z_ ecR_dZe
• JBI
❏ CH ×
• 9 fUW c ReZ _
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E n (). )( #
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E n (). )( #
❏ 7aRTYV Z_ w
❏ w
❏ w w w
❏ 7aRTYV >8RdV w I-
❏ 7 Rk _ 9 fUMReTY gV_ed
❏ w w
❏ 7C w w
❏ Z- dVcZVd 7 Rk _ 9, Z_deR_TVd w
❏ 7 Rk _ CH w
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4 NPUR 9YV n
Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0)++)_ h cf_ cVR eZ V decVR ac TVddZ_X Re dTR V hZeY RaRTYV W Z_
_ R Rk _ V c)
❏ HV VRdV /(+( w
❏ HV VRdV /(3( p7aRTYV Z_ +(-(,
• w p
• IaRc p pIGB
• 7 Rk _ AZ_VdZd IecVR d w
• w
• ViRTe j _TV ×
• wp × w
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
n
y z
Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0)++)R Rk _ V c _ h dfaa ced WZ_V XcRZ_VU RTTVdd T _ec
fdZ_X T fdeVc eRXd)
❏ CH w p 7C
w
– + +( :Vae5: L w p
×
– + ,( :Vae5: Lp pJVde
w pz
– , +( :Vae5: L p w
– , ,( :VaezLR fV pz
❏ w w 9 fdeVc :
– ( :VdTcZSV9 fdeVc pBZde9 fdeVcd
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
n n n
❏ HV VRdV .(i /(i w
❏ w
– HV VRdV /(+(
• p
p
w
p w
•
– HV VRdV /(+(
• p w
Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0)++)R Rk _ V c _ h dfaa ced Rfe dTR Z_X R_U T _WZXfcRS V dTR V
U h_ SVYRgZ c)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4 NPUR 5N R n
F z
❏ HV VRdV /(,( w
– w p>8RdV w
I- >: I
– w I- w
❏ HV VRdV /(1(
– >8RdV I-
w
❏ KF
– CH w w I-
– CH I 9 _dZdeV_e LZVh
– M7B Z_ V cj W c McZeV
– /=8 c Ie cV
– Z_ V cj) _ UZd W c HVRU
Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0)++)R Rk _ V c _ h dfaa ced fdZ_X R Rk _ d- Rd R UReR de cV
W c RaRTYV YSRdV)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 6Y bQJNaPU 8cR a
Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0)+,)R Rk _ V c _ h dfaa ced VgV_ed R_U _ eZWZTReZ _d hZeY R Rk _
T fUhReTY VgV_ed)
❏ CH
– w
• IJ7HJ D=& HKDD D=& M7 J D=& J HC D7J :& J HC D7J :QM J>Q HHEHI
– w
• Wc H I P D= e HKDD D=z & Wc HKDD D= e H I P D=z
– w w w
• F D: D=& 7JJ79> :& : J79> :& 7 B :
– ×
• F D: D=& 97D9 BQF D: D=& HKDD D=& 9ECFB J :& 97D9 BB :& 7 B :
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
n n
Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +1) -)R Rk _ V c R__ f_TVd Z_deR_TV W VVed W c R Rk _ VT, da e
Z_deR_TVd R_U da e S T d)
❏ p CH w
❏
– w p / r
× r
–
– p+ 0
– C9 × g9FK
• g9FK p 9, g9FK p
CH O7HD D UVCR_RXVc gT cV
– w w
• w × p w w
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +1) -)R Rk _ V c R__ f_TVd Z_deR_TV W VVed W c R Rk _ VT, da e
Z_deR_TVd R_U da e S T d)
n n
+( ×
,(
-(
.( c ×
/( 8I w
0(
1( g9FK
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg V be 4@
n n
Yeead4))Rhd(R Rk _(T ) a)RS fe Rhd)hYRed _Vh), +1) 1)R Rk _ V c _ h dfaa ced Rf_TYZ_X T fdeVcd hZeY Tfde
R Rk _ Z_fi R Zd)
❏ 7 Rk _ BZ_fi 7 Rk _ CRTYZ_V
RXV 7C $ 7 Rk _ CH w
❏ 7C w p
w 7C
❏ w /(1( 7 Rk _ CH w
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
V n 4ZNg 86 V aN PR n
BPa OR_ ) ()/#
Yeead4))Rhd(R Rk _(T ) a)RS fe Rhd)hYRed _Vh), +1)+ )R Rk _ V c _ h dfaa ced Z- Z_deR_TVd)
❏ w 7 Rk _ 9, w
Z-
❏ Z- pD _ L ReZ V CV cj iacVdd
DLCV$ II: w p E
E
❏ w /(3( 7 Rk _ CH w
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E n
Yeead4))Rhd(R Rk _(T ) a)RS fe Rhd)hYRed _Vh), +1)+ )R Rk _ V c _ h dfaa ced aVc dVT _U SZ Z_X)
❏ , +1 + , p 7MI w
❏ w
w
❏ w +
❏ p+
+ $
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E ,&e
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E ERYRN R ()- / #
• HV VRdV .( ( m , +/) 1
• HV VRdV .(+( m , +/) 3
• HV VRdV .(,( m , +/)++
• HV VRdV .(-( m , +0) +
• HV VRdV .(.( m , +0) -
• HV VRdV .(/( m , +0) .
• HV VRdV .(0( m , +0) .
• HV VRdV .(1(i m , +0) 0 1
• HV VRdV .(2(i m , +0) 2 , +1) ,
• HV VRdV .(3(+ m , +1) -
• HV VRdV .(3(, m , +1) 1
7aRTYV 8ZXe a w w ×
+, +, w
>RU a ×
Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)HV VRdV=fZUV)V c cV VRdV T a _V_ed(Ye
• HV VRdV /( ( m , +0) 2
• HV VRdV /( (- m , +0)+
• HV VRdV /(+( m , +0)++
• HV VRdV /(,(i m , +0)+,
• HV VRdV /(-(i m , +1) + ,
• HV VRdV /(.( m , +1) ,
• HV VRdV /(/( m , +1) .
• HV VRdV /(0( m , +1) 0
• HV VRdV /(1( m , +1) 1
• HV VRdV /(2( m , +1) 2
• HV VRdV /(3( m , +1)+
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E ERYRN R
❏ w
• 7 Rk _ CH p w w
❏ w
• p w
❏ w w p
• GfZT 9cVReV
• pCR_RXV V_e =fZUV HV VRdV =fZUV
7C w w p
http://docs.aws.amazon.com/ElasticMapReduce/latest/ReleaseGuide/emr-release-differences.html
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R2 n
❏ w w
• w
• 4
• jRc_(cVd fcTV R_RXVc(hVSRaa(RUUcVdd a ce 3 ,0 56 2 22
• )Y V)YRU a)YZgV 56 )fdc) ZS)YZgV& )VeT)YZgV)T _W& )gRc) X)YZgV
❏ w w w
• YRU a w ”
• jRc_ w pYZgV w
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R2 n
❏ 9 RddZWZTReZ _ % Fc aVceZVd
• 8 edecRa 7TeZ _
• w ”
Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)HV VRdV=fZUV)V c T _WZXfcV Raad(Ye
e.g. /etc/hadoop/conf/mapred-site.xml
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R2 n j
❏ w w GfZT 9cVReV
• p w p × - ”
• w = e RUgR_TVU aeZ _d
❏ w
• CR_RXV V_e =fZUV
• 7 Rk _ CH w w
• Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)CR_RXV V_e=fZUV
• HV VRdV =fZUV
• w w
• Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)HV VRdV=fZUV
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
n o-&1&(#
Yeea4))U Td(Rhd(R Rk _(T ))V c) ReVde)HV VRdV=fZUV)V c cV VRdV T a _V_ed(Ye
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&ey -&e
❏ w
• IaRc ,(i
• FPNYN &))
• >ZgV ,(i
• Z_
• BZgj
❏
• NcN 0
• JVk >ZgV& FZX$
❏
• IeVa
❏ IR_US i
• w
• IR_US i
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E 4 YVPNaV S _ -&1&(
• >RU a& >ZgV& JVk&
IaRc & FcVde & FZX&
CRY fe& Ib a& Z_
w w
• >8RdV& FY V_Zi
=K
• >fV& PVaaV Z_
w w
• E kZV
• =R_X ZR& >9ReR X&
P VVaVc& YVcf
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4 YVPNaV 2
❏ >RU a
• qO7HD w $%>: I w $ p
qCRaHVUfTV + q CH
O7HD p q
❏ >ZgV
• O7HD p IGB q
CRaHVUfTV qI- w q
❏ JVk
• O7HD :7= $ q
>ZgV ” p >ZgV q
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4 YVPNaV 2
❏ IaRc
• w w qO7HD pCRaHVUfTV
qIaRc IaRc IGB >ZgV $ IaRc
IecVR Z_X w $& CB ZS $ p
q
❏ Z_
• w w q O7HD p w
p qIaRc p
pIGB q
w ViRTe j _TV × q
❏ FcVde
• IGB qO7HD ” pIGB
q w E D qIGB
7DI IGB q
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4 YVPNaV 2
❏ FZX
• CRaHVUfTV :IB q
q
❏ CRY fe
• CRaHVUfTV IaRc q
q
❏ Ib a
• H:8CI p>: I I- w CRaHVUfTV
q
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4 YVPNaV 2 n n
❏ >8RdV
• >: I ALIq w
w q
❏ FY V_Zi
• >8RdV w IGB w q
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4 YVPNaV 2 :H n n
❏ >fV
• >ZgV ×w p =K q
❏ PVaaV Z_
• =K IaRc w IaRc IGB p
p s w q
❏ E kZV
• w w w qCRaHVUfTV
>ZgV q
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4 YVPNaV 2
❏ =R_X ZR
• w 9FK&CV cj&DVeh c $
p q
❏ >9ReR X
• >ZgV ×w FZX w q
❏ P VVaVc
• w w pALI p w w
q >8RdV $ q
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4 YVPNaV 2
❏ BZgj
• IaRc CH w H IJ w p
pHF9
w w
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
n 4ZNg 8@E
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E x n
❏ w w
•
” w
• HV VRdV BRSV )7C LVcdZ _
w p w
❏ w w
• 7aRTYV 8ZXe a
w w
p8 edecRa
• w
7C
Yeea4))Rhd(ejaVaRU(T )dR a), +0) 3)SfZ UZ_X R_U UVa jZ_X Tfde Raa ZTReZ _d hZeY RaRTYV 8ZXe a R_U R Rk _ V c(Ye
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
n 2 NQ
❏ O7HD 5 OVe 7_ eYVc HVd fcTV
DVX eZRe c
❏ >RU a, w
• SJcRT Vc
❏ HVd fcTV HVbfVde w w
• 7aa ZTReZ _CRdeVc
p w
HVd fcTVCR_RXVc
MapReduce
Pig Hive Other
Other
HDFS
Other
Map
ReducePig Hive
YARN
HDFS
Hadoop V1
Hadoop V2
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
K4EA2 ER b_PR@N NTR_
❏ w w
❏ w w
• 9FK& CV cj& VeT(
❏ w
❏
• 7aa ZTReZ _CRdeVc
Yeea4))YRU a(RaRTYV( cX)U Td)TfccV_e)YRU a jRc_)YRU a jRc_ dZeV)O7HD(Ye
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
K4EA2 A QR@N NTR_
❏ w w
❏ w w
HC
❏ w 9 _eRZ_Vc
• 9 _eRZ_Vc
Yeea4))YRU a(RaRTYV( cX)U Td)TfccV_e)YRU a jRc_)YRU a jRc_ dZeV)O7HD(Ye
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
K4EA2 6 aNV R_
❏ w w
❏ DC
❏
• :VWRf e9 _eRZ_Vc iVTfe c
•
• BZ_fi9 _eRZ_Vc iVTfe c
• TXc fad
• : T Vc9 _eRZ_Vc iVTfe c
• : T Vc
Yeea4))YRU a(RaRTYV( cX)U Td)TfccV_e)YRU a jRc_)YRU a jRc_ dZeV)O7HD(Ye
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
K4EA2 4 YVPNaV @N aR_
❏
9 _eRZ_Vc
• SJcRT Vc
• + 7C
❏
9 _eRZ_Vc
Yeea4))YRU a(RaRTYV( cX)U Td)TfccV_e)YRU a jRc_)YRU a jRc_ dZeV)O7HD(Ye
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg 8@E K4EA
❏ CRaHVUfTV& >ZgV& FZX& IaRc _ O7HD
• 7 Rk _ CH pO7HD w
❏ ×
• CH >ZgV IaRc p w
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
n 2 VcR
❏ IGB Z V p w
w
❏ w w w p
×w
• >: I w w
• 7 Rk _ CH CH I 7 Rk _ I-$
• p7 Rk _ AZ_VdZd& 7 Rk _ :j_R :8
❏
• CRaHVUfTV& JVk& IaRc
Yeead4))YZgV(RaRTYV( cX)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
VcR
Metastore
HDFS
Cluster
Amazon
S3
HiveServer2
CREATE
TABLE
SELECT
FROM
+4 ×w
>: I 7 Rk _ I-
w
,4 ×w
w
-4 ×w
.4 ×w
w
/4
04 w
w
Amazon
DynamoDB
Amazon
Kinesis
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
VcR @RaN a _R
❏ IGB
CVeRde cV
❏ CVeRde cV CjIGB
• 7 Rk _ CH CRdeVc
D UV w
• p7 Rk _ H:I
❏ w 7MI = fV
:ReR 9ReR X CVeRde cV
CVeRde cV
Amazon S3
HDFS
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
v n BE6 9VYR CN_ bRa
❏ w
• w
•
• EaeZ ZkVU H h 9 f _Rc EH9$ Z V ) FRcbfVe
❏ w
• +
• + p w
• J NJ B & I GK D9 B EH9 w
Yeead4)) cT(RaRTYV( cX)U Td)daVT Z_ec (Ye
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
n w
❏
• w
• w w
❏ w
❏ w p+ w +
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
VcR BE6 9VYR CN_ bRa v
❏ ×w
”
❏
CREATE TABLE t (
col1 STRING,
…
) STORED AS [ORC/PARQUET];
INSERT INTO t (…);
SELECT col1 FROM t;
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
VcR 4ZNg V R V n
❏ 7 Rk _ AZ_VdZd IecVR w p>ZgV ×w
• 7 Rk _ AZ_VdZd p w w
• 4 >ZgV p w
❏ IecVR IYRcU CRaaVc w
• 7 Rk _ AZ_VdZd w
• 7 Rk _ :j_R :8 $ p ”
w
IYRcU
IYRcU +
CRaaVc
CRaaVc
SELECT
FROM
123
id data
1 …
2 …
3 …
Amazon
Kinesis
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
VcR 4ZNg V R V n
VcR n
7 Rk _ AZ_VdZd
IecVR
D #
>GB ” q
7 Rk _ AZ_VdZd
w
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
n 2 F N_
❏
w w
• p
• $
❏ w w p
w
• IGB& w & &
http://spark.apache.org/
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
F N_ FD 7NaN Ra 4C F N_ & #
❏ IaRc IGB
• >ZgVGB p>ZgV CVeRde cV
• :89)E:89
• IGB p
:ReRdVe):ReR cR V
❏ :ReRdVe 7F
• w
• ITR R) RgR
• :ReR cR V
:ReRdVe
Yeead4))UReRScZT d(T )S X), +0) 1)+.)R eR V
W eYcVV RaRTYV daRc RaZd cUUd UReRWcR Vd R_U
UReRdVed(Ye
Yeea4))daRc (RaRTYV( cX)U Td) ReVde)db ac XcR Z_X XfZUV(Ye
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
❏ :ZdTcVeZkVU IecVR $ p
❏ HVdZ ZV_e :ZdecZSfeVU :ReRdVe $ w
• p
❏ pIaRc ,(, CH HV VRdV /(2( $ pIecfTefcVU
IecVR Z_X 7F =7
DStream
RDD@T1 RDD@T2
Messages
Receiver
F N_ Fa_RNZV T i
Yeea4))daRc (RaRTYV( cX)U Td) ReVde)decVR Z_X ac XcR Z_X XfZUV(Ye
Yeead4))daRc (RaRTYV( cX)U Td) ReVde)decfTefcVU decVR Z_X ac XcR Z_X XfZUV(Ye
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4ZNg V R V F N_ Fa_RNZV T
7 Rk _ AZ_VdZd IecVR p7 Rk _ CH IaRc
:IecVR
• IaRc 7 Rk _ AZ_VdZd
• pAZ_VdZd 9 ZV_e BZScRcjzA9B
x p
• w p
http://spark.apache.org/docs/latest/streaming-kinesis-integration.html
IYRcU
IYRcU +
A9B eYcVRU
A9B eYcVRUAmazon
Kinesis
DStream
RDD@T1 RDD@T2
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
F N_ 7f NZVP ER b_PR 4YY PNaV
❏ iVTfe c H::
• >VRa p
❏ :j_R ZT HVd fcTV 7 TReZ _ 5 iVTfe c
• IecVR Z_X 7aa ZTReZ _ w
• daRc dYV w
• .(.( pdaRc (Uj_R ZT7 TReZ _(V_RS VU
a_bR
• daRc (dYfWW V(dVcgZTV(V_RS VU a_bR
https://spark.apache.org/docs/latest/job-scheduling.html
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
❏ w
)
n
❏
w
4ZNg 8@E
w p + “ u
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
p q
• 7 Rk _ CH CR_RXV V_e =fZUV
Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)CR_RXV V_e=fZUV
• 7 Rk _ CH HV VRdV =fZUV
Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)HV VRdV=fZUV
• 7 Rk _ CH 7F HVWVcV_TV
Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)7F )
• 7 Rk _ CH :VgV aVc =fZUV ,(i R_U -(i $
Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde):VgV aVc=fZUV)
• 7MI 9B V c
Yeea4))U Td(Rhd(R Rk _(T )T Z) ReVde)cVWVcV_TV)V c)Z_UVi(Ye
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
D 4
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
n
7MI w
• Yeea4))Rhd(R Rk _(T ) a)Rhd a Z_ec UfTeZ _)
7MI I feZ _d 7cTYZeVTe
• p w G 7
• Yeea4))Rhd(ejaVaRU(T )dR a)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
GdVaaR_ 9NPRO
4JF x
3Nd PY bQMW
p p p
w v u
–
Uaa 2 &SO&ZR )cE0fJZ
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4JF jx v
7MI p p
”
Yeead4))Rhd(R Rk _(T ) a)T _eRTe fd)Rhd dR Vd)
7MI ”
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4 R QVe
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4@ E YR z
, +/ 0 - p 7C H V
7C H V , p
• CH c V
• 7 Rk _ CH p7 Rk _ 9,
• 9, Z_deR_TV ac WZ V
• 7 Rk _ 9, p7 Rk _ I-
Yeea4))Rhd(ejaVaRU(T )dR a), +/) 0)V c R_URe cj ZR faUReV(Ye
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&(
w w
• >RU a ,(0(
• >ZgV +( (
• FZX (+.(
• CRY fe (+ (
• IaRc +(.(+
7aRTYV 8ZXe a w w ×
• p
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&)
w
• IaRc +(/(
• >fV -(1(+ $
p w
• FcVde (++3
• PVaaV Z_ (0 I_RadY e$
• E kZV .( (+
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&
w
• IaRc +(/(,
• =R_X ZR -(0( $
• FcVde (+,/
• PVaaV Z_ (/(/
• E kZV .(,(
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&
w
• >RU a ,(1(+
• IaRc +(0(
• =R_X ZR -(1(,
• FcVde (+-
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&,
w
• >9ReR X +( ( $
• CRY fe (++(+
• Ib a +(.(0 $
• FcVde (+-0
• PVaaV Z_ (/(0
IaRc
• :j_R ZT R TReZ _
RgR 2
• 7L7Q>EC ”
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&-
w
• IaRc +(0(+
• >RU a ,(1(,
• FcVde (+.
CH I
• I- II ACI
• 7MI ACI 9CA
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&.
w
• >8RdV +(,( $ m D IGB _ >RU a
• P VVaVc -(.(2 $
• FcVde (+.-
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&/&) t zu ,&/&(–
w
• FY V_Zi .(1( $ m IGB _ >8RdV
• JVk (2(- $ m :7= V_XZ_V _ O7HD
• >8RdV +(,(+
• CRYRfe (+,(
• FcVde (+.1
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&/&
w
• CRY fe (+,(,
• IaRc +(0(,
• FcVde (+.2
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&0&(
w
• >8RdV +(,(,
• JVk (2(.
• PVaaV Z_ (0(+
• FcVde (+/+
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&0&
w
• >RU a ,(1(-
• FcVde (+/,(-
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&0&
w
• IaRc +(0(-
• FcVde (+/1(+
• P VVaVc -(.(3
• CH :j_R :8
9 __VTe c
– .(,(
– >ZgV& IaRc
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R ,&1&)
w
•
FjeY _ ,(0 w
• w
>ZgV
• > L 3310
• > L + + 0
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -&(&(
w
• >ZgV ,(+(
• FcVde (+/
• IaRc ,( (
• FPNYN &))
• >fV -(+ (
• FZX (+0(
• JVk (2(.
• >8RdV +(,(,
• PVaaV Z_ (0(+
• NcN 0
• JVk >ZgV& FZX$
• IeVa
IR_US i
• w
• IR_US i
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
85F z
j @, 6, n
CH 7F 8I
• >: I
• p
• 9 fU c ReZ _
8I I9+)IJ+ w
• >: I
C. 9.
Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0) ,)R Rk _ V c _ h dfaa ced R Rk _ VSd g f Vd R_U . R_U
T. R Rk _ VT, WR Z ZVd)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4 NPUR VcR 4 NPUR F N_ 8@E
7f NZ 75 n BFF
:j_R :8
:j_R :8 ×w p CH w
E D
=Ze>fS
• Yeead4))XZeYfS(T )Rhd RSd)V c Uj_R US T __VTe c
Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0) 3)eYV R Rk _ V c Uj_R US T __VTe c W c RaRTYV YZgV R_U
RaRTYV daRc Zd _ h aV_ d fcTV)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -&(&
w
• >RU a ,(1(-
• FcVde (+/,(-
• IaRc ,( (+
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -&)&(
w
• Z_ +(+(- $
• >8RdV +(,(-
• PVaaV Z_ (0(,
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -& &(
w
• IaRc ,( (,
>8RdV _ I-
• >8RdV w
I-
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -& &)
w
• FcVde (+/1(+
• P VVaVc -(.(3
• CH :j_R :8
9 __VTe c
– .(,(
– >ZgV& IaRc
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -& &(
w
• >ZgV ,(+(+
• >fV -(++(
• IaRc ,(+(
• E kZV .(-(
• Z_ +(+(.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -& &)
w
•
PVaaV Z_
• P FF B D +2/0
• P FF B D +311
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -&,&(
w
• FcVde (+00
• FY V_Zi .(3(
• >8RdV +(-(
• Z_ +(,(
• PVaaV Z_ (1(
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
E, n
Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +1) -)R Rk _ V c cV VRdV / . R_U dfaa ce W c c. Z_deR_TVd _ h
RgRZ RS V)
H.
• × w w
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -&-&(
w
• >fV -(+,
• FcVde (+1
• B:7F w
• PVaaV Z_ (1(+
• P AVVaVc -(.(+
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -&.&(
w
• >8RdV +(-(+
• IaRc ,(+(+
• Z_ +(,(+
• CRY fe (+-(
• IaRc ,(i w
FcVde Z_ ecR_dZe
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -&/&(
w
• Z_ +(-(
• FY V_Zi .(++(
• PVaaV Z_ (1(,
w p
7 Rk _ BZ_fi 7C
5N R
• w w
C_R a
• _ UV(ac aVceZVd
K4EA
• T _eRZ_Vc X. (ac aVceZVd
F
• IGEEF ,22 w
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -&0&(
w
• 7MI I:A +(++(+0
• Z_ +(-(+
• >ZgV ,(-(
• IaRc ,(,(
• IecfTefcVU IecVR Z_X =7
7MI = fV :ReR 9ReR X
• >ZgV IaRc IGB 7MI
= fV :ReR 9ReR X >ZgV
w
B gVR
• EEP ,1.2 w
bR
• >K /2/3 w
5N R
• CN $ >8RdV w
w w
• w
RgR CR_RXV V_e ieV_dZ _d
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -&1&(
w
• 7MI I:A W c RgR +(++(+2-
• Z_ +(-(,
• >fV .( (+
• >fV D eVS W c IaRc w
• FZX (+1(
• FcVde (+2.
• BZgj (.( Z_TfSReZ_X $
Z- dVcZVd 7 Rk _ 9, Z_deR_TVd
w ETe SVc +,& , +1$
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ERYRN R -&1&(
F N_
• w p
p7fe ITR Z_X
XcRTVWf
• IF7HA ,+.3. w
LR RYV
• P FF B D ,-11 w
ON R
• >87I +2/--
bR
• >ZgV bfVcj VUZe c
7MI = fV :ReR 9ReR X
w
•

More Related Content

What's hot

MLflowで学ぶMLOpsことはじめ
MLflowで学ぶMLOpsことはじめMLflowで学ぶMLOpsことはじめ
MLflowで学ぶMLOpsことはじめKenichi Sonoda
 
SaaS テナント毎のコストを把握するための「AWS Application Cost Profiler」のご紹介
SaaS テナント毎のコストを把握するための「AWS Application Cost Profiler」のご紹介SaaS テナント毎のコストを把握するための「AWS Application Cost Profiler」のご紹介
SaaS テナント毎のコストを把握するための「AWS Application Cost Profiler」のご紹介Amazon Web Services Japan
 
STAIR Lab Seminar 202105
STAIR Lab Seminar 202105STAIR Lab Seminar 202105
STAIR Lab Seminar 202105Sho Takase
 
flaws.cloudに挑戦しよう!
flaws.cloudに挑戦しよう!flaws.cloudに挑戦しよう!
flaws.cloudに挑戦しよう!zaki4649
 
AWS + MLflow + SageMakerの環境を動かしてみたお話
AWS + MLflow + SageMakerの環境を動かしてみたお話AWS + MLflow + SageMakerの環境を動かしてみたお話
AWS + MLflow + SageMakerの環境を動かしてみたお話ItohHiroki
 
ハイパースレッディングの並列化への影響
ハイパースレッディングの並列化への影響ハイパースレッディングの並列化への影響
ハイパースレッディングの並列化への影響Hiroshi Watanabe
 
情報検索の基礎
情報検索の基礎情報検索の基礎
情報検索の基礎Retrieva inc.
 
Amazon SageMaker 推論エンドポイントを利用したアプリケーション開発
Amazon SageMaker 推論エンドポイントを利用したアプリケーション開発Amazon SageMaker 推論エンドポイントを利用したアプリケーション開発
Amazon SageMaker 推論エンドポイントを利用したアプリケーション開発Amazon Web Services Japan
 
データサイエンティスト向け性能問題対応の基礎
データサイエンティスト向け性能問題対応の基礎データサイエンティスト向け性能問題対応の基礎
データサイエンティスト向け性能問題対応の基礎Tetsutaro Watanabe
 
分散トレーシングAWS:X-Rayとの上手い付き合い方
分散トレーシングAWS:X-Rayとの上手い付き合い方分散トレーシングAWS:X-Rayとの上手い付き合い方
分散トレーシングAWS:X-Rayとの上手い付き合い方Recruit Lifestyle Co., Ltd.
 
機械学習向けCGデータの量産手法の検討
機械学習向けCGデータの量産手法の検討機械学習向けCGデータの量産手法の検討
機械学習向けCGデータの量産手法の検討Silicon Studio Corporation
 
Amazon Game Tech Night #20 ゲームバックエンド開発関連セッションのre:cap
Amazon Game Tech Night #20 ゲームバックエンド開発関連セッションのre:capAmazon Game Tech Night #20 ゲームバックエンド開発関連セッションのre:cap
Amazon Game Tech Night #20 ゲームバックエンド開発関連セッションのre:capAmazon Web Services Japan
 
BMS Molecular Translation 3rd place solution
BMS Molecular Translation 3rd place solutionBMS Molecular Translation 3rd place solution
BMS Molecular Translation 3rd place solutionKazuki Fujikawa
 
20191001 AWS Black Belt Online Seminar AWS Lake Formation
20191001 AWS Black Belt Online Seminar AWS Lake Formation 20191001 AWS Black Belt Online Seminar AWS Lake Formation
20191001 AWS Black Belt Online Seminar AWS Lake Formation Amazon Web Services Japan
 
Kinesis Firehoseを使ってみた
Kinesis Firehoseを使ってみたKinesis Firehoseを使ってみた
Kinesis Firehoseを使ってみたdcubeio
 
Amazon S3による静的Webサイトホスティング
Amazon S3による静的WebサイトホスティングAmazon S3による静的Webサイトホスティング
Amazon S3による静的WebサイトホスティングYasuhiro Horiuchi
 
[DL輪読会]A closer look at few shot classification
[DL輪読会]A closer look at few shot classification[DL輪読会]A closer look at few shot classification
[DL輪読会]A closer look at few shot classificationDeep Learning JP
 
モデル高速化百選
モデル高速化百選モデル高速化百選
モデル高速化百選Yusuke Uchida
 

What's hot (20)

MLflowで学ぶMLOpsことはじめ
MLflowで学ぶMLOpsことはじめMLflowで学ぶMLOpsことはじめ
MLflowで学ぶMLOpsことはじめ
 
SaaS テナント毎のコストを把握するための「AWS Application Cost Profiler」のご紹介
SaaS テナント毎のコストを把握するための「AWS Application Cost Profiler」のご紹介SaaS テナント毎のコストを把握するための「AWS Application Cost Profiler」のご紹介
SaaS テナント毎のコストを把握するための「AWS Application Cost Profiler」のご紹介
 
STAIR Lab Seminar 202105
STAIR Lab Seminar 202105STAIR Lab Seminar 202105
STAIR Lab Seminar 202105
 
flaws.cloudに挑戦しよう!
flaws.cloudに挑戦しよう!flaws.cloudに挑戦しよう!
flaws.cloudに挑戦しよう!
 
AWS + MLflow + SageMakerの環境を動かしてみたお話
AWS + MLflow + SageMakerの環境を動かしてみたお話AWS + MLflow + SageMakerの環境を動かしてみたお話
AWS + MLflow + SageMakerの環境を動かしてみたお話
 
ハイパースレッディングの並列化への影響
ハイパースレッディングの並列化への影響ハイパースレッディングの並列化への影響
ハイパースレッディングの並列化への影響
 
情報検索の基礎
情報検索の基礎情報検索の基礎
情報検索の基礎
 
AWSからのメール送信
AWSからのメール送信AWSからのメール送信
AWSからのメール送信
 
Amazon SageMaker 推論エンドポイントを利用したアプリケーション開発
Amazon SageMaker 推論エンドポイントを利用したアプリケーション開発Amazon SageMaker 推論エンドポイントを利用したアプリケーション開発
Amazon SageMaker 推論エンドポイントを利用したアプリケーション開発
 
データサイエンティスト向け性能問題対応の基礎
データサイエンティスト向け性能問題対応の基礎データサイエンティスト向け性能問題対応の基礎
データサイエンティスト向け性能問題対応の基礎
 
分散トレーシングAWS:X-Rayとの上手い付き合い方
分散トレーシングAWS:X-Rayとの上手い付き合い方分散トレーシングAWS:X-Rayとの上手い付き合い方
分散トレーシングAWS:X-Rayとの上手い付き合い方
 
機械学習向けCGデータの量産手法の検討
機械学習向けCGデータの量産手法の検討機械学習向けCGデータの量産手法の検討
機械学習向けCGデータの量産手法の検討
 
Amazon Game Tech Night #20 ゲームバックエンド開発関連セッションのre:cap
Amazon Game Tech Night #20 ゲームバックエンド開発関連セッションのre:capAmazon Game Tech Night #20 ゲームバックエンド開発関連セッションのre:cap
Amazon Game Tech Night #20 ゲームバックエンド開発関連セッションのre:cap
 
BMS Molecular Translation 3rd place solution
BMS Molecular Translation 3rd place solutionBMS Molecular Translation 3rd place solution
BMS Molecular Translation 3rd place solution
 
20191001 AWS Black Belt Online Seminar AWS Lake Formation
20191001 AWS Black Belt Online Seminar AWS Lake Formation 20191001 AWS Black Belt Online Seminar AWS Lake Formation
20191001 AWS Black Belt Online Seminar AWS Lake Formation
 
Kinesis Firehoseを使ってみた
Kinesis Firehoseを使ってみたKinesis Firehoseを使ってみた
Kinesis Firehoseを使ってみた
 
Amazon S3による静的Webサイトホスティング
Amazon S3による静的WebサイトホスティングAmazon S3による静的Webサイトホスティング
Amazon S3による静的Webサイトホスティング
 
[DL輪読会]A closer look at few shot classification
[DL輪読会]A closer look at few shot classification[DL輪読会]A closer look at few shot classification
[DL輪読会]A closer look at few shot classification
 
Redis at LINE
Redis at LINERedis at LINE
Redis at LINE
 
モデル高速化百選
モデル高速化百選モデル高速化百選
モデル高速化百選
 

Viewers also liked

AWS Black Belt Online Seminar 2017 Amazon GameLift
AWS Black Belt Online Seminar 2017 Amazon GameLiftAWS Black Belt Online Seminar 2017 Amazon GameLift
AWS Black Belt Online Seminar 2017 Amazon GameLiftAmazon Web Services Japan
 
AWS Black Belt Online Seminar 2017 AWSへのネットワーク接続とAWS上のネットワーク内部設計
AWS Black Belt Online Seminar 2017 AWSへのネットワーク接続とAWS上のネットワーク内部設計AWS Black Belt Online Seminar 2017 AWSへのネットワーク接続とAWS上のネットワーク内部設計
AWS Black Belt Online Seminar 2017 AWSへのネットワーク接続とAWS上のネットワーク内部設計Amazon Web Services Japan
 
AWS Black Belt Online Seminar 2017 AWS X-Ray
AWS Black Belt Online Seminar 2017 AWS X-RayAWS Black Belt Online Seminar 2017 AWS X-Ray
AWS Black Belt Online Seminar 2017 AWS X-RayAmazon Web Services Japan
 
AWS Black Belt Online Seminar 2017 Amazon Aurora
AWS Black Belt Online Seminar 2017 Amazon AuroraAWS Black Belt Online Seminar 2017 Amazon Aurora
AWS Black Belt Online Seminar 2017 Amazon AuroraAmazon Web Services Japan
 
AWS Black Belt Online Seminar 2017 Amazon Pinpoint で始めるモバイルアプリのグロースハック
AWS Black Belt Online Seminar 2017 Amazon Pinpoint で始めるモバイルアプリのグロースハックAWS Black Belt Online Seminar 2017 Amazon Pinpoint で始めるモバイルアプリのグロースハック
AWS Black Belt Online Seminar 2017 Amazon Pinpoint で始めるモバイルアプリのグロースハックAmazon Web Services Japan
 
Amazon Athena 初心者向けハンズオン
Amazon Athena 初心者向けハンズオンAmazon Athena 初心者向けハンズオン
Amazon Athena 初心者向けハンズオンAmazon Web Services Japan
 
AWS Black Belt Online Seminar 2017 AWS Summit Tokyo 2017 まとめ
AWS Black Belt Online Seminar 2017 AWS Summit Tokyo 2017 まとめAWS Black Belt Online Seminar 2017 AWS Summit Tokyo 2017 まとめ
AWS Black Belt Online Seminar 2017 AWS Summit Tokyo 2017 まとめAmazon Web Services Japan
 
Cloud OnAir #04 今話題の機械学習・GCP で何ができるのか?
Cloud OnAir #04 今話題の機械学習・GCP で何ができるのか? Cloud OnAir #04 今話題の機械学習・GCP で何ができるのか?
Cloud OnAir #04 今話題の機械学習・GCP で何ができるのか? Google Cloud Platform - Japan
 
AWS Black Belt Techシリーズ AWS Direct Connect
AWS Black Belt Techシリーズ AWS Direct ConnectAWS Black Belt Techシリーズ AWS Direct Connect
AWS Black Belt Techシリーズ AWS Direct ConnectAmazon Web Services Japan
 
AWS Black Belt Online Seminar 2017 Amazon Connect
AWS Black Belt Online Seminar 2017 Amazon ConnectAWS Black Belt Online Seminar 2017 Amazon Connect
AWS Black Belt Online Seminar 2017 Amazon ConnectAmazon Web Services Japan
 
分散処理基盤ApacheHadoop入門とHadoopエコシステムの最新技術動向(OSC2015 Kansai発表資料)
分散処理基盤ApacheHadoop入門とHadoopエコシステムの最新技術動向(OSC2015 Kansai発表資料)分散処理基盤ApacheHadoop入門とHadoopエコシステムの最新技術動向(OSC2015 Kansai発表資料)
分散処理基盤ApacheHadoop入門とHadoopエコシステムの最新技術動向(OSC2015 Kansai発表資料)NTT DATA OSS Professional Services
 
Hadoop概要説明
Hadoop概要説明Hadoop概要説明
Hadoop概要説明Satoshi Noto
 
Azure PowerShell Internal 勉強会
Azure PowerShell Internal 勉強会Azure PowerShell Internal 勉強会
Azure PowerShell Internal 勉強会ShuheiUda
 
Azure サポート エンジニア直伝 ~ PowerShell 実践活用術 ~
Azure サポート エンジニア直伝 ~ PowerShell 実践活用術 ~Azure サポート エンジニア直伝 ~ PowerShell 実践活用術 ~
Azure サポート エンジニア直伝 ~ PowerShell 実践活用術 ~ShuheiUda
 
AWS Black Belt Online Seminar 2016 Amazon EMR
AWS Black Belt Online Seminar 2016 Amazon EMRAWS Black Belt Online Seminar 2016 Amazon EMR
AWS Black Belt Online Seminar 2016 Amazon EMRAmazon Web Services Japan
 
Serverless Ninja Warriors [panel]
Serverless Ninja Warriors [panel]Serverless Ninja Warriors [panel]
Serverless Ninja Warriors [panel]真吾 吉田
 
Redshift Spectrumを使ってみた話
Redshift Spectrumを使ってみた話Redshift Spectrumを使ってみた話
Redshift Spectrumを使ってみた話Yoshiki Kouno
 

Viewers also liked (20)

AWS Black Belt Online Seminar 2017 Amazon GameLift
AWS Black Belt Online Seminar 2017 Amazon GameLiftAWS Black Belt Online Seminar 2017 Amazon GameLift
AWS Black Belt Online Seminar 2017 Amazon GameLift
 
AWS BlackBelt AWS上でのDDoS対策
AWS BlackBelt AWS上でのDDoS対策AWS BlackBelt AWS上でのDDoS対策
AWS BlackBelt AWS上でのDDoS対策
 
AWS Black Belt Online Seminar 2017 AWSへのネットワーク接続とAWS上のネットワーク内部設計
AWS Black Belt Online Seminar 2017 AWSへのネットワーク接続とAWS上のネットワーク内部設計AWS Black Belt Online Seminar 2017 AWSへのネットワーク接続とAWS上のネットワーク内部設計
AWS Black Belt Online Seminar 2017 AWSへのネットワーク接続とAWS上のネットワーク内部設計
 
AWS Black Belt Online Seminar 2017 AWS X-Ray
AWS Black Belt Online Seminar 2017 AWS X-RayAWS Black Belt Online Seminar 2017 AWS X-Ray
AWS Black Belt Online Seminar 2017 AWS X-Ray
 
AWS Black Belt Online Seminar 2017 Amazon Aurora
AWS Black Belt Online Seminar 2017 Amazon AuroraAWS Black Belt Online Seminar 2017 Amazon Aurora
AWS Black Belt Online Seminar 2017 Amazon Aurora
 
AWS Black Belt Online Seminar 2017 Amazon Pinpoint で始めるモバイルアプリのグロースハック
AWS Black Belt Online Seminar 2017 Amazon Pinpoint で始めるモバイルアプリのグロースハックAWS Black Belt Online Seminar 2017 Amazon Pinpoint で始めるモバイルアプリのグロースハック
AWS Black Belt Online Seminar 2017 Amazon Pinpoint で始めるモバイルアプリのグロースハック
 
Amazon Athena 初心者向けハンズオン
Amazon Athena 初心者向けハンズオンAmazon Athena 初心者向けハンズオン
Amazon Athena 初心者向けハンズオン
 
20170621 aws-black belt-ads-sms
20170621 aws-black belt-ads-sms20170621 aws-black belt-ads-sms
20170621 aws-black belt-ads-sms
 
AWS Black Belt Online Seminar 2017 AWS Summit Tokyo 2017 まとめ
AWS Black Belt Online Seminar 2017 AWS Summit Tokyo 2017 まとめAWS Black Belt Online Seminar 2017 AWS Summit Tokyo 2017 まとめ
AWS Black Belt Online Seminar 2017 AWS Summit Tokyo 2017 まとめ
 
AWS Black Belt online seminar 2017 Snowball
AWS Black Belt online seminar 2017 SnowballAWS Black Belt online seminar 2017 Snowball
AWS Black Belt online seminar 2017 Snowball
 
Cloud OnAir #04 今話題の機械学習・GCP で何ができるのか?
Cloud OnAir #04 今話題の機械学習・GCP で何ができるのか? Cloud OnAir #04 今話題の機械学習・GCP で何ができるのか?
Cloud OnAir #04 今話題の機械学習・GCP で何ができるのか?
 
AWS Black Belt Techシリーズ AWS Direct Connect
AWS Black Belt Techシリーズ AWS Direct ConnectAWS Black Belt Techシリーズ AWS Direct Connect
AWS Black Belt Techシリーズ AWS Direct Connect
 
AWS Black Belt Online Seminar 2017 Amazon Connect
AWS Black Belt Online Seminar 2017 Amazon ConnectAWS Black Belt Online Seminar 2017 Amazon Connect
AWS Black Belt Online Seminar 2017 Amazon Connect
 
分散処理基盤ApacheHadoop入門とHadoopエコシステムの最新技術動向(OSC2015 Kansai発表資料)
分散処理基盤ApacheHadoop入門とHadoopエコシステムの最新技術動向(OSC2015 Kansai発表資料)分散処理基盤ApacheHadoop入門とHadoopエコシステムの最新技術動向(OSC2015 Kansai発表資料)
分散処理基盤ApacheHadoop入門とHadoopエコシステムの最新技術動向(OSC2015 Kansai発表資料)
 
Hadoop概要説明
Hadoop概要説明Hadoop概要説明
Hadoop概要説明
 
Azure PowerShell Internal 勉強会
Azure PowerShell Internal 勉強会Azure PowerShell Internal 勉強会
Azure PowerShell Internal 勉強会
 
Azure サポート エンジニア直伝 ~ PowerShell 実践活用術 ~
Azure サポート エンジニア直伝 ~ PowerShell 実践活用術 ~Azure サポート エンジニア直伝 ~ PowerShell 実践活用術 ~
Azure サポート エンジニア直伝 ~ PowerShell 実践活用術 ~
 
AWS Black Belt Online Seminar 2016 Amazon EMR
AWS Black Belt Online Seminar 2016 Amazon EMRAWS Black Belt Online Seminar 2016 Amazon EMR
AWS Black Belt Online Seminar 2016 Amazon EMR
 
Serverless Ninja Warriors [panel]
Serverless Ninja Warriors [panel]Serverless Ninja Warriors [panel]
Serverless Ninja Warriors [panel]
 
Redshift Spectrumを使ってみた話
Redshift Spectrumを使ってみた話Redshift Spectrumを使ってみた話
Redshift Spectrumを使ってみた話
 

Similar to AWS Black Belt Online Seminar 2017 Amazon EMR

AWS上でのオンラインゲームリリースガイド
AWS上でのオンラインゲームリリースガイドAWS上でのオンラインゲームリリースガイド
AWS上でのオンラインゲームリリースガイドAmazon Web Services Japan
 
Best Practice for Online Game Development on AWS
Best Practice for Online Game Development on AWSBest Practice for Online Game Development on AWS
Best Practice for Online Game Development on AWSAmazon Web Services Japan
 
AWS Migration - General
AWS Migration - GeneralAWS Migration - General
AWS Migration - GeneralKenji Morooka
 
[db tech showcase Tokyo 2018] #dbts2018 #C32 『Deep Dive on the Amazon Aurora ...
[db tech showcase Tokyo 2018] #dbts2018 #C32 『Deep Dive on the Amazon Aurora ...[db tech showcase Tokyo 2018] #dbts2018 #C32 『Deep Dive on the Amazon Aurora ...
[db tech showcase Tokyo 2018] #dbts2018 #C32 『Deep Dive on the Amazon Aurora ...Insight Technology, Inc.
 
AWS Migration - Datacenter
AWS Migration - DatacenterAWS Migration - Datacenter
AWS Migration - DatacenterKenji Morooka
 
AWS Migration - AWS service
AWS Migration - AWS serviceAWS Migration - AWS service
AWS Migration - AWS serviceKenji Morooka
 
20180802 game-data-analytics-on-aws-game-tech-night-#7
20180802 game-data-analytics-on-aws-game-tech-night-#720180802 game-data-analytics-on-aws-game-tech-night-#7
20180802 game-data-analytics-on-aws-game-tech-night-#7Amazon Web Services Japan
 
[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習
[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習
[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習Amazon Web Services Japan
 
Building High Performance Apps with In-memory Data
Building High Performance Apps with In-memory DataBuilding High Performance Apps with In-memory Data
Building High Performance Apps with In-memory DataAmazon Web Services
 
AiCore Brochure 27-Mar-2023-205529.pdf
AiCore Brochure 27-Mar-2023-205529.pdfAiCore Brochure 27-Mar-2023-205529.pdf
AiCore Brochure 27-Mar-2023-205529.pdfAjayRawat829497
 
AWS Migration - As-Is Tool
AWS Migration - As-Is ToolAWS Migration - As-Is Tool
AWS Migration - As-Is ToolKenji Morooka
 
A3 sec -_regular_expressions
A3 sec -_regular_expressionsA3 sec -_regular_expressions
A3 sec -_regular_expressionsa3sec
 
Alexa連携デバイスクラウドを構成するAWS ソリューション
Alexa連携デバイスクラウドを構成するAWS ソリューションAlexa連携デバイスクラウドを構成するAWS ソリューション
Alexa連携デバイスクラウドを構成するAWS ソリューションToshiaki Enami
 
Meteor - not just for rockstars
Meteor - not just for rockstarsMeteor - not just for rockstars
Meteor - not just for rockstarsStephan Hochhaus
 
AWS CodeStar 및 Cloud9을 통한 서버리스(Serverless) 앱 개발 길잡이 - 윤석찬 (AWS 테크에반젤리스트)
AWS CodeStar 및 Cloud9을 통한 서버리스(Serverless) 앱 개발 길잡이 - 윤석찬 (AWS 테크에반젤리스트)AWS CodeStar 및 Cloud9을 통한 서버리스(Serverless) 앱 개발 길잡이 - 윤석찬 (AWS 테크에반젤리스트)
AWS CodeStar 및 Cloud9을 통한 서버리스(Serverless) 앱 개발 길잡이 - 윤석찬 (AWS 테크에반젤리스트)Amazon Web Services Korea
 
AWS Migration - Pattern
AWS Migration - PatternAWS Migration - Pattern
AWS Migration - PatternKenji Morooka
 
Serverless ASP.NET Core 2.0 Applicationsを作ってみた
Serverless ASP.NET Core 2.0 Applicationsを作ってみたServerless ASP.NET Core 2.0 Applicationsを作ってみた
Serverless ASP.NET Core 2.0 Applicationsを作ってみたAmazon Web Services Japan
 
AWS re:Invent 2017 주요 신규 서비스 분야별 요약 - 윤석찬 (AWS테크에반젤리스트)
AWS re:Invent 2017 주요 신규 서비스 분야별 요약 - 윤석찬 (AWS테크에반젤리스트)AWS re:Invent 2017 주요 신규 서비스 분야별 요약 - 윤석찬 (AWS테크에반젤리스트)
AWS re:Invent 2017 주요 신규 서비스 분야별 요약 - 윤석찬 (AWS테크에반젤리스트)Amazon Web Services Korea
 

Similar to AWS Black Belt Online Seminar 2017 Amazon EMR (20)

AWS上でのオンラインゲームリリースガイド
AWS上でのオンラインゲームリリースガイドAWS上でのオンラインゲームリリースガイド
AWS上でのオンラインゲームリリースガイド
 
Best Practice for Online Game Development on AWS
Best Practice for Online Game Development on AWSBest Practice for Online Game Development on AWS
Best Practice for Online Game Development on AWS
 
AWS Migration - General
AWS Migration - GeneralAWS Migration - General
AWS Migration - General
 
[db tech showcase Tokyo 2018] #dbts2018 #C32 『Deep Dive on the Amazon Aurora ...
[db tech showcase Tokyo 2018] #dbts2018 #C32 『Deep Dive on the Amazon Aurora ...[db tech showcase Tokyo 2018] #dbts2018 #C32 『Deep Dive on the Amazon Aurora ...
[db tech showcase Tokyo 2018] #dbts2018 #C32 『Deep Dive on the Amazon Aurora ...
 
AWS Migration - Datacenter
AWS Migration - DatacenterAWS Migration - Datacenter
AWS Migration - Datacenter
 
AWS Migration - AWS service
AWS Migration - AWS serviceAWS Migration - AWS service
AWS Migration - AWS service
 
20180802 game-data-analytics-on-aws-game-tech-night-#7
20180802 game-data-analytics-on-aws-game-tech-night-#720180802 game-data-analytics-on-aws-game-tech-night-#7
20180802 game-data-analytics-on-aws-game-tech-night-#7
 
[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習
[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習
[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習
 
Building High Performance Apps with In-memory Data
Building High Performance Apps with In-memory DataBuilding High Performance Apps with In-memory Data
Building High Performance Apps with In-memory Data
 
AiCore Brochure 27-Mar-2023-205529.pdf
AiCore Brochure 27-Mar-2023-205529.pdfAiCore Brochure 27-Mar-2023-205529.pdf
AiCore Brochure 27-Mar-2023-205529.pdf
 
AWS Migration - As-Is Tool
AWS Migration - As-Is ToolAWS Migration - As-Is Tool
AWS Migration - As-Is Tool
 
A3 sec -_regular_expressions
A3 sec -_regular_expressionsA3 sec -_regular_expressions
A3 sec -_regular_expressions
 
Alexa連携デバイスクラウドを構成するAWS ソリューション
Alexa連携デバイスクラウドを構成するAWS ソリューションAlexa連携デバイスクラウドを構成するAWS ソリューション
Alexa連携デバイスクラウドを構成するAWS ソリューション
 
Meteor - not just for rockstars
Meteor - not just for rockstarsMeteor - not just for rockstars
Meteor - not just for rockstars
 
GraphQL, l'avenir du REST ?
GraphQL, l'avenir du REST ?GraphQL, l'avenir du REST ?
GraphQL, l'avenir du REST ?
 
AWS CodeStar 및 Cloud9을 통한 서버리스(Serverless) 앱 개발 길잡이 - 윤석찬 (AWS 테크에반젤리스트)
AWS CodeStar 및 Cloud9을 통한 서버리스(Serverless) 앱 개발 길잡이 - 윤석찬 (AWS 테크에반젤리스트)AWS CodeStar 및 Cloud9을 통한 서버리스(Serverless) 앱 개발 길잡이 - 윤석찬 (AWS 테크에반젤리스트)
AWS CodeStar 및 Cloud9을 통한 서버리스(Serverless) 앱 개발 길잡이 - 윤석찬 (AWS 테크에반젤리스트)
 
Database Backup
Database BackupDatabase Backup
Database Backup
 
AWS Migration - Pattern
AWS Migration - PatternAWS Migration - Pattern
AWS Migration - Pattern
 
Serverless ASP.NET Core 2.0 Applicationsを作ってみた
Serverless ASP.NET Core 2.0 Applicationsを作ってみたServerless ASP.NET Core 2.0 Applicationsを作ってみた
Serverless ASP.NET Core 2.0 Applicationsを作ってみた
 
AWS re:Invent 2017 주요 신규 서비스 분야별 요약 - 윤석찬 (AWS테크에반젤리스트)
AWS re:Invent 2017 주요 신규 서비스 분야별 요약 - 윤석찬 (AWS테크에반젤리스트)AWS re:Invent 2017 주요 신규 서비스 분야별 요약 - 윤석찬 (AWS테크에반젤리스트)
AWS re:Invent 2017 주요 신규 서비스 분야별 요약 - 윤석찬 (AWS테크에반젤리스트)
 

More from Amazon Web Services Japan

202205 AWS Black Belt Online Seminar Amazon VPC IP Address Manager (IPAM)
202205 AWS Black Belt Online Seminar Amazon VPC IP Address Manager (IPAM)202205 AWS Black Belt Online Seminar Amazon VPC IP Address Manager (IPAM)
202205 AWS Black Belt Online Seminar Amazon VPC IP Address Manager (IPAM)Amazon Web Services Japan
 
202205 AWS Black Belt Online Seminar Amazon FSx for OpenZFS
202205 AWS Black Belt Online Seminar Amazon FSx for OpenZFS202205 AWS Black Belt Online Seminar Amazon FSx for OpenZFS
202205 AWS Black Belt Online Seminar Amazon FSx for OpenZFSAmazon Web Services Japan
 
202204 AWS Black Belt Online Seminar AWS IoT Device Defender
202204 AWS Black Belt Online Seminar AWS IoT Device Defender202204 AWS Black Belt Online Seminar AWS IoT Device Defender
202204 AWS Black Belt Online Seminar AWS IoT Device DefenderAmazon Web Services Japan
 
202204 AWS Black Belt Online Seminar Amazon Connect を活用したオンコール対応の実現
202204 AWS Black Belt Online Seminar Amazon Connect を活用したオンコール対応の実現202204 AWS Black Belt Online Seminar Amazon Connect を活用したオンコール対応の実現
202204 AWS Black Belt Online Seminar Amazon Connect を活用したオンコール対応の実現Amazon Web Services Japan
 
202204 AWS Black Belt Online Seminar Amazon Connect Salesforce連携(第1回 CTI Adap...
202204 AWS Black Belt Online Seminar Amazon Connect Salesforce連携(第1回 CTI Adap...202204 AWS Black Belt Online Seminar Amazon Connect Salesforce連携(第1回 CTI Adap...
202204 AWS Black Belt Online Seminar Amazon Connect Salesforce連携(第1回 CTI Adap...Amazon Web Services Japan
 
Amazon Game Tech Night #25 ゲーム業界向け機械学習最新状況アップデート
Amazon Game Tech Night #25 ゲーム業界向け機械学習最新状況アップデートAmazon Game Tech Night #25 ゲーム業界向け機械学習最新状況アップデート
Amazon Game Tech Night #25 ゲーム業界向け機械学習最新状況アップデートAmazon Web Services Japan
 
20220409 AWS BLEA 開発にあたって検討したこと
20220409 AWS BLEA 開発にあたって検討したこと20220409 AWS BLEA 開発にあたって検討したこと
20220409 AWS BLEA 開発にあたって検討したことAmazon Web Services Japan
 
202202 AWS Black Belt Online Seminar AWS Managed Rules for AWS WAF の活用
202202 AWS Black Belt Online Seminar AWS Managed Rules for AWS WAF の活用202202 AWS Black Belt Online Seminar AWS Managed Rules for AWS WAF の活用
202202 AWS Black Belt Online Seminar AWS Managed Rules for AWS WAF の活用Amazon Web Services Japan
 
202203 AWS Black Belt Online Seminar Amazon Connect Tasks.pdf
202203 AWS Black Belt Online Seminar Amazon Connect Tasks.pdf202203 AWS Black Belt Online Seminar Amazon Connect Tasks.pdf
202203 AWS Black Belt Online Seminar Amazon Connect Tasks.pdfAmazon Web Services Japan
 
Amazon QuickSight の組み込み方法をちょっぴりDD
Amazon QuickSight の組み込み方法をちょっぴりDDAmazon QuickSight の組み込み方法をちょっぴりDD
Amazon QuickSight の組み込み方法をちょっぴりDDAmazon Web Services Japan
 
マルチテナント化で知っておきたいデータベースのこと
マルチテナント化で知っておきたいデータベースのことマルチテナント化で知っておきたいデータベースのこと
マルチテナント化で知っておきたいデータベースのことAmazon Web Services Japan
 
機密データとSaaSは共存しうるのか!?セキュリティー重視のユーザー層を取り込む為のネットワーク通信のアプローチ
機密データとSaaSは共存しうるのか!?セキュリティー重視のユーザー層を取り込む為のネットワーク通信のアプローチ機密データとSaaSは共存しうるのか!?セキュリティー重視のユーザー層を取り込む為のネットワーク通信のアプローチ
機密データとSaaSは共存しうるのか!?セキュリティー重視のユーザー層を取り込む為のネットワーク通信のアプローチAmazon Web Services Japan
 
パッケージソフトウェアを簡単にSaaS化!?既存の資産を使ったSaaS化手法のご紹介
パッケージソフトウェアを簡単にSaaS化!?既存の資産を使ったSaaS化手法のご紹介パッケージソフトウェアを簡単にSaaS化!?既存の資産を使ったSaaS化手法のご紹介
パッケージソフトウェアを簡単にSaaS化!?既存の資産を使ったSaaS化手法のご紹介Amazon Web Services Japan
 
202202 AWS Black Belt Online Seminar Amazon Connect Customer Profiles
202202 AWS Black Belt Online Seminar Amazon Connect Customer Profiles202202 AWS Black Belt Online Seminar Amazon Connect Customer Profiles
202202 AWS Black Belt Online Seminar Amazon Connect Customer ProfilesAmazon Web Services Japan
 
Amazon Game Tech Night #24 KPIダッシュボードを最速で用意するために
Amazon Game Tech Night #24 KPIダッシュボードを最速で用意するためにAmazon Game Tech Night #24 KPIダッシュボードを最速で用意するために
Amazon Game Tech Night #24 KPIダッシュボードを最速で用意するためにAmazon Web Services Japan
 
202202 AWS Black Belt Online Seminar AWS SaaS Boost で始めるSaaS開発⼊⾨
202202 AWS Black Belt Online Seminar AWS SaaS Boost で始めるSaaS開発⼊⾨202202 AWS Black Belt Online Seminar AWS SaaS Boost で始めるSaaS開発⼊⾨
202202 AWS Black Belt Online Seminar AWS SaaS Boost で始めるSaaS開発⼊⾨Amazon Web Services Japan
 
[20220126] JAWS-UG 2022初頭までに葬ったAWSアンチパターン大紹介
[20220126] JAWS-UG 2022初頭までに葬ったAWSアンチパターン大紹介[20220126] JAWS-UG 2022初頭までに葬ったAWSアンチパターン大紹介
[20220126] JAWS-UG 2022初頭までに葬ったAWSアンチパターン大紹介Amazon Web Services Japan
 
202111 AWS Black Belt Online Seminar AWSで構築するSmart Mirrorのご紹介
202111 AWS Black Belt Online Seminar AWSで構築するSmart Mirrorのご紹介202111 AWS Black Belt Online Seminar AWSで構築するSmart Mirrorのご紹介
202111 AWS Black Belt Online Seminar AWSで構築するSmart Mirrorのご紹介Amazon Web Services Japan
 
202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...
202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...
202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...Amazon Web Services Japan
 

More from Amazon Web Services Japan (20)

202205 AWS Black Belt Online Seminar Amazon VPC IP Address Manager (IPAM)
202205 AWS Black Belt Online Seminar Amazon VPC IP Address Manager (IPAM)202205 AWS Black Belt Online Seminar Amazon VPC IP Address Manager (IPAM)
202205 AWS Black Belt Online Seminar Amazon VPC IP Address Manager (IPAM)
 
202205 AWS Black Belt Online Seminar Amazon FSx for OpenZFS
202205 AWS Black Belt Online Seminar Amazon FSx for OpenZFS202205 AWS Black Belt Online Seminar Amazon FSx for OpenZFS
202205 AWS Black Belt Online Seminar Amazon FSx for OpenZFS
 
202204 AWS Black Belt Online Seminar AWS IoT Device Defender
202204 AWS Black Belt Online Seminar AWS IoT Device Defender202204 AWS Black Belt Online Seminar AWS IoT Device Defender
202204 AWS Black Belt Online Seminar AWS IoT Device Defender
 
Infrastructure as Code (IaC) 談義 2022
Infrastructure as Code (IaC) 談義 2022Infrastructure as Code (IaC) 談義 2022
Infrastructure as Code (IaC) 談義 2022
 
202204 AWS Black Belt Online Seminar Amazon Connect を活用したオンコール対応の実現
202204 AWS Black Belt Online Seminar Amazon Connect を活用したオンコール対応の実現202204 AWS Black Belt Online Seminar Amazon Connect を活用したオンコール対応の実現
202204 AWS Black Belt Online Seminar Amazon Connect を活用したオンコール対応の実現
 
202204 AWS Black Belt Online Seminar Amazon Connect Salesforce連携(第1回 CTI Adap...
202204 AWS Black Belt Online Seminar Amazon Connect Salesforce連携(第1回 CTI Adap...202204 AWS Black Belt Online Seminar Amazon Connect Salesforce連携(第1回 CTI Adap...
202204 AWS Black Belt Online Seminar Amazon Connect Salesforce連携(第1回 CTI Adap...
 
Amazon Game Tech Night #25 ゲーム業界向け機械学習最新状況アップデート
Amazon Game Tech Night #25 ゲーム業界向け機械学習最新状況アップデートAmazon Game Tech Night #25 ゲーム業界向け機械学習最新状況アップデート
Amazon Game Tech Night #25 ゲーム業界向け機械学習最新状況アップデート
 
20220409 AWS BLEA 開発にあたって検討したこと
20220409 AWS BLEA 開発にあたって検討したこと20220409 AWS BLEA 開発にあたって検討したこと
20220409 AWS BLEA 開発にあたって検討したこと
 
202202 AWS Black Belt Online Seminar AWS Managed Rules for AWS WAF の活用
202202 AWS Black Belt Online Seminar AWS Managed Rules for AWS WAF の活用202202 AWS Black Belt Online Seminar AWS Managed Rules for AWS WAF の活用
202202 AWS Black Belt Online Seminar AWS Managed Rules for AWS WAF の活用
 
202203 AWS Black Belt Online Seminar Amazon Connect Tasks.pdf
202203 AWS Black Belt Online Seminar Amazon Connect Tasks.pdf202203 AWS Black Belt Online Seminar Amazon Connect Tasks.pdf
202203 AWS Black Belt Online Seminar Amazon Connect Tasks.pdf
 
Amazon QuickSight の組み込み方法をちょっぴりDD
Amazon QuickSight の組み込み方法をちょっぴりDDAmazon QuickSight の組み込み方法をちょっぴりDD
Amazon QuickSight の組み込み方法をちょっぴりDD
 
マルチテナント化で知っておきたいデータベースのこと
マルチテナント化で知っておきたいデータベースのことマルチテナント化で知っておきたいデータベースのこと
マルチテナント化で知っておきたいデータベースのこと
 
機密データとSaaSは共存しうるのか!?セキュリティー重視のユーザー層を取り込む為のネットワーク通信のアプローチ
機密データとSaaSは共存しうるのか!?セキュリティー重視のユーザー層を取り込む為のネットワーク通信のアプローチ機密データとSaaSは共存しうるのか!?セキュリティー重視のユーザー層を取り込む為のネットワーク通信のアプローチ
機密データとSaaSは共存しうるのか!?セキュリティー重視のユーザー層を取り込む為のネットワーク通信のアプローチ
 
パッケージソフトウェアを簡単にSaaS化!?既存の資産を使ったSaaS化手法のご紹介
パッケージソフトウェアを簡単にSaaS化!?既存の資産を使ったSaaS化手法のご紹介パッケージソフトウェアを簡単にSaaS化!?既存の資産を使ったSaaS化手法のご紹介
パッケージソフトウェアを簡単にSaaS化!?既存の資産を使ったSaaS化手法のご紹介
 
202202 AWS Black Belt Online Seminar Amazon Connect Customer Profiles
202202 AWS Black Belt Online Seminar Amazon Connect Customer Profiles202202 AWS Black Belt Online Seminar Amazon Connect Customer Profiles
202202 AWS Black Belt Online Seminar Amazon Connect Customer Profiles
 
Amazon Game Tech Night #24 KPIダッシュボードを最速で用意するために
Amazon Game Tech Night #24 KPIダッシュボードを最速で用意するためにAmazon Game Tech Night #24 KPIダッシュボードを最速で用意するために
Amazon Game Tech Night #24 KPIダッシュボードを最速で用意するために
 
202202 AWS Black Belt Online Seminar AWS SaaS Boost で始めるSaaS開発⼊⾨
202202 AWS Black Belt Online Seminar AWS SaaS Boost で始めるSaaS開発⼊⾨202202 AWS Black Belt Online Seminar AWS SaaS Boost で始めるSaaS開発⼊⾨
202202 AWS Black Belt Online Seminar AWS SaaS Boost で始めるSaaS開発⼊⾨
 
[20220126] JAWS-UG 2022初頭までに葬ったAWSアンチパターン大紹介
[20220126] JAWS-UG 2022初頭までに葬ったAWSアンチパターン大紹介[20220126] JAWS-UG 2022初頭までに葬ったAWSアンチパターン大紹介
[20220126] JAWS-UG 2022初頭までに葬ったAWSアンチパターン大紹介
 
202111 AWS Black Belt Online Seminar AWSで構築するSmart Mirrorのご紹介
202111 AWS Black Belt Online Seminar AWSで構築するSmart Mirrorのご紹介202111 AWS Black Belt Online Seminar AWSで構築するSmart Mirrorのご紹介
202111 AWS Black Belt Online Seminar AWSで構築するSmart Mirrorのご紹介
 
202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...
202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...
202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...
 

Recently uploaded

Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Nikki Chapple
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructureitnewsafrica
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Nikki Chapple
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Karmanjay Verma
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentMahmoud Rabie
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialJoão Esperancinha
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxfnnc6jmgwh
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesManik S Magar
 

Recently uploaded (20)

Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career Development
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
How Tech Giants Cut Corners to Harvest Data for A.I.
How Tech Giants Cut Corners to Harvest Data for A.I.How Tech Giants Cut Corners to Harvest Data for A.I.
How Tech Giants Cut Corners to Harvest Data for A.I.
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorial
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
 

AWS Black Belt Online Seminar 2017 Amazon EMR

  • 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. w , +1(++( + r4JF 5YNP 5RYa B YV R FRZV N_s 4ZNg 8@E
  • 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • z • w × w w w × • 7MI w 7 Rk _ CH& 7 Rk _ 9I
  • 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. v • – ()/ )) ) n x v v k –4JF Uaa 2 Nd &NZNg &P Z# vk • – x zj 4JF zu j4JF v k • – v k x z n j v k • 7MI U Vd _ e WWVc SZ_UZ_X acZTV bf eVd( 7MI acZTZ_X Zd afS ZT j RgRZ RS V R_U Zd dfS VTe e TYR_XV Z_ RTT cUR_TV hZeY eYV 7MI 9fde Vc 7XcVV V_e RgRZ RS V Re Yeea4))Rhd(R Rk _(T )RXcVV V_e)( 7_j acZTZ_X Z_W c ReZ _ Z_T fUVU Z_ eYZd U Tf V_e Zd ac gZUVU _ j Rd R_ VdeZ ReV W fdRXV TYRcXVd W c 7MI dVcgZTVd SRdVU _ TVceRZ_ Z_W c ReZ _ eYRe j f YRgV ac gZUVU( C _eY j TYRcXVd hZ SV SRdVU _ j fc RTefR fdV W 7MI dVcgZTVd& R_U Rj gRcj Wc eYV VdeZ ReVd ac gZUVU(
  • 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4TR QN • • 7 Rk _ CH • 7 Rk _ CH w , +0)+ $ • 7 Rk _ CH .(i • w 7 Rk _ CH •
  • 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. –~
  • 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E ❏ , 3 w • CRaHVUfTV p + ❏ >RU a × • FZX& >ZgV& >8RdV& FcVde & >fV& >RU a ,(i& IaRc ,(i& Z_ ❏ 7MI w • Ia e& 7C& :j_R :8& LF9& 9 fUMReTY& 9B & AZ_VdZd& ACI → – @N ERQbPR – vm Storage S3 (EMRFS), HDFS YARN Cluster Resource Management Batch MapReduce Interactive Tez In Memory Spark, Flink Applications Hive, Pig, Spark SQL/Streaming/ML, Mahout, Sqoop HBase/Phoenix Presto Hue (SQL Interface/Metastore Management) Zeppelin (Interactive Notebook) Ganglia (Monitoring) HiveServer2/Spark Thriftserver (JDBC/ODBC) Amazon EMR service
  • 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. n S3 EMR EMR Redshift Machine Learning Collect Process Analyze Store Data Collection and Storage Data Processing Event Processing Data Analysis Data Answers Athena Quicksight Kinesis DynamoDB AWS IoT AWS Lambda KCL Apps Glue
  • 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ❏ w ) • CRdeVc)9 cV)JRd _ UV • CH I • 8 edecRa 7TeZ _& IeVad n ❏ w • >RU a& O7HD • >ZgV& FZX& >8RdV& IaRc & Z_ & FcVde & VeT( 4ZNg 8@E
  • 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E n
  • 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Task Node Task Instance Group 4ZNg 8@E n security group security group Master Node Master Instance Group Amazon S3 Amazon DynamoDB Amazon Kinesis Core Node Core Instance Group HDFS HDFS HDFS HDFS Task Node Task Instance Group w >: I 7MI w AWS Glue Data Catalog
  • 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E @N aR_ aN PR :_ b ❏ CRdeVc D UV + • RZ gVc ❏ w • DR VD UV HVd fcTVCR_RXVc • 9 cV D UV JRd D UV Master Node Master Instance Group Hadoop1: JobTracker Hadoop2: ResourceManager HDFS: NameNode Hive: HiveServer2, MetaStore Presto: Coordinator
  • 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E 6 _R aN PR :_ b ❏ + 9 cV D UV ❏ w • D UVCR_RXVc • :ReRD UV w >: I • 8Ip EA HDFS Core Node Core Instance Group Hadoop1: TaskTracker Hadoop2: NodeManager HDFS: DataNode Presto: Worker
  • 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E 6 _R aN PR :_ b ❏ 9 cV D UV • >: I • 9FK)H7C ❏ >: I p • .(+( HDFS Core Node Core Instance Group Hadoop1: TaskTracker Hadoop2: NodeManager HDFS: DataNode Presto: Worker
  • 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E GN aN PR :_ b ❏ >: I 9 cV ❏ >: I w 9 cV D UV ❏ >: I • >: I w :VT ZddZ _$ Task Node Task Instance Group Hadoop1: TaskTracker Hadoop2: NodeManager Presto: Worker
  • 15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E GN aN PR :_ b ❏ =c fa • Ia e SZU • _deR_TV JjaV ❏ H p Ia e Task Instance Group 2 Task Instance Group 1 c3.xlarge * 2 bid: $0.1 r3.xlarge * 2 bid: $0.5
  • 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. F a aN PR Task Instance GroupCore Instance Group ❏ ❏ ❖ B QRZN Q 9 cV D UV ❖ ” × E_ UV R_U ❖ F a aN PR JRd D UV ❖ E_ UV R_U 3 w 4 .(i RcXV 0 + +, p/ $ 6 _R A QR B QRZN Q# 4 (-+2o+ Yo0 GN A QR F a# 4 (-+2o/Yo0 % ( .o/Yo0 , +1 ++ + $ w qIa e q 5 +3( 2 + 5 + (1. /
  • 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E l
  • 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 8@E9F2 4ZNg F 79F w ❏ w ❏ • w ❏ w • w ❏ w EMR EMR Amazon S3
  • 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 8@E9F ❏ ld-4))n ” ❏ 7 Rk _ I- • 4 w 7 Rk _ = RTZVc ❏ 7 Rk _ I- w s • 7 Rk _ I- w ❏ w
  • 20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 8@E9F 6 V aR a IVRd ❏ 7 Rk _ I- • FKJ HVRU RWeVc hcZeV T _dZdeV_Tj$ • FKJ): B J gV_efR T _dZdeV_Tj$ ❏ CH I 9 _dZdeV_e LZVh • 7 Rk _ :j_R :8 w • BZde 9 _dZdeV_Tj • :V VeV 9 _dZdeV_Tj • HVRU RWeVc hcZeV 9 _dZdeV_Tj • Amazon S3 Amazon DynamoDB CH I w Yeead4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)CR_RXV V_e=fZUV)V c a R_ T _dZdeV_e gZVh(Ye
  • 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E – v ❏ ” • • ❏ w 7 Rk _ I- • w • w t
  • 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E 2 5 a a_N 4PaV ❏ D UV • EA • 8RdY& HfSj& FjeY _& VeT( • 7 Rk _ I- p • 7aRTYV 8ZXe a w w p8 edecRa Yeea4))Rhd(ejaVaRU(T )dR a), +0) 3)SfZ UZ_X R_U UVa jZ_X Tfde Raa ZTReZ _d hZeY RaRTYV 8ZXe a R_U R Rk _ V c(Ye
  • 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E 2 FaR ❏ ) • p • 4 JB >ZgVGB ❏ 7 Rk _ I- KH p p w • I- Rc • IecVR Z_X& >ZgV& FZX& IaRc CH w T R_U cf__Vc( Rc $ • dTcZae cf__Vc( Rc SRdY ❏ IeVa 7fe eVc Z_ReV$
  • 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E y ❏ IeVa • 7 Rk _ CH • w w 9B • 7MI I:A • 7C • ) • ❏ 7MI w • BR SUR • :ReR FZaV Z_V ❏ w Amazon EMR Amazon EMR Step API AWS Data Pipeline Airflow, Luigi, or other schedulers on EC2 AWS Lambda
  • 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E y ❏ w w • CRdeVc D UV ddY p p IIC 7 Rk _ 9, IjdeV d CR_RXVc$ Hf_ T R_U • >ZgVIVcgVc,& daRc dfS Ze • PVaaV Z_& >fV& daRc SdVcgVc =K • BZgj HVde IaRc • E kZV w w Web UIs: Hue SQL editor, Zeppelin notebooks, R Studio, and more! Connect with ODBC / JDBC using HiveServer2/Spark Thriftserver Use Spark Actions in your Apache Oozie workflow to create DAGs of jobs. (start using start-thriftserver.sh) Or, use the native APIs and CLIs for each application
  • 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. C_VcNaR FbO Ra 8@E z ❏ IfS_Ve FcZgReV • CH .(,( ❏ FfS ZT • I-):j_R :8 LF9 V_Ua Z_e • I-):j_R :8 w D7J =M • CH IVcgZTV IVTfcZej =c fa Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +/)+,) Rf_TY R Rk _ V c T fdeVcd Z_ R Rk _ gaT acZgReV dfS_Ved)
  • 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4 NPUR F N_ GRg NQ @N ERQbPR C_R a n z Yeea4))U Td(Rhd(R Rk _(T )V c) ReVde)HV VRdV=fZUV)V c UReR V_TcjaeZ _ aeZ _d(Ye ❏ Re cVde pZ_ ecR_dZe p w ❏ Re cVde • CH I& >: Ip p w × w ❏ Z_ ecR_dZe • JBI ❏ CH × • 9 fUW c ReZ _
  • 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E n (). )( #
  • 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E n (). )( # ❏ 7aRTYV Z_ w ❏ w ❏ w w w ❏ 7aRTYV >8RdV w I- ❏ 7 Rk _ 9 fUMReTY gV_ed ❏ w w ❏ 7C w w ❏ Z- dVcZVd 7 Rk _ 9, Z_deR_TVd w ❏ 7 Rk _ CH w
  • 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4 NPUR 9YV n Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0)++)_ h cf_ cVR eZ V decVR ac TVddZ_X Re dTR V hZeY RaRTYV W Z_ _ R Rk _ V c) ❏ HV VRdV /(+( w ❏ HV VRdV /(3( p7aRTYV Z_ +(-(, • w p • IaRc p pIGB • 7 Rk _ AZ_VdZd IecVR d w • w • ViRTe j _TV × • wp × w
  • 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. n y z Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0)++)R Rk _ V c _ h dfaa ced WZ_V XcRZ_VU RTTVdd T _ec fdZ_X T fdeVc eRXd) ❏ CH w p 7C w – + +( :Vae5: L w p × – + ,( :Vae5: Lp pJVde w pz – , +( :Vae5: L p w – , ,( :VaezLR fV pz ❏ w w 9 fdeVc : – ( :VdTcZSV9 fdeVc pBZde9 fdeVcd
  • 32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. n n n ❏ HV VRdV .(i /(i w ❏ w – HV VRdV /(+( • p p w p w • – HV VRdV /(+( • p w Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0)++)R Rk _ V c _ h dfaa ced Rfe dTR Z_X R_U T _WZXfcRS V dTR V U h_ SVYRgZ c)
  • 33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4 NPUR 5N R n F z ❏ HV VRdV /(,( w – w p>8RdV w I- >: I – w I- w ❏ HV VRdV /(1( – >8RdV I- w ❏ KF – CH w w I- – CH I 9 _dZdeV_e LZVh – M7B Z_ V cj W c McZeV – /=8 c Ie cV – Z_ V cj) _ UZd W c HVRU Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0)++)R Rk _ V c _ h dfaa ced fdZ_X R Rk _ d- Rd R UReR de cV W c RaRTYV YSRdV)
  • 34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 6Y bQJNaPU 8cR a Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0)+,)R Rk _ V c _ h dfaa ced VgV_ed R_U _ eZWZTReZ _d hZeY R Rk _ T fUhReTY VgV_ed) ❏ CH – w • IJ7HJ D=& HKDD D=& M7 J D=& J HC D7J :& J HC D7J :QM J>Q HHEHI – w • Wc H I P D= e HKDD D=z & Wc HKDD D= e H I P D=z – w w w • F D: D=& 7JJ79> :& : J79> :& 7 B : – × • F D: D=& 97D9 BQF D: D=& HKDD D=& 9ECFB J :& 97D9 BB :& 7 B :
  • 35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. n n Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +1) -)R Rk _ V c R__ f_TVd Z_deR_TV W VVed W c R Rk _ VT, da e Z_deR_TVd R_U da e S T d) ❏ p CH w ❏ – w p / r × r – – p+ 0 – C9 × g9FK • g9FK p 9, g9FK p CH O7HD D UVCR_RXVc gT cV – w w • w × p w w
  • 36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +1) -)R Rk _ V c R__ f_TVd Z_deR_TV W VVed W c R Rk _ VT, da e Z_deR_TVd R_U da e S T d) n n +( × ,( -( .( c × /( 8I w 0( 1( g9FK
  • 37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg V be 4@ n n Yeead4))Rhd(R Rk _(T ) a)RS fe Rhd)hYRed _Vh), +1) 1)R Rk _ V c _ h dfaa ced Rf_TYZ_X T fdeVcd hZeY Tfde R Rk _ Z_fi R Zd) ❏ 7 Rk _ BZ_fi 7 Rk _ CRTYZ_V RXV 7C $ 7 Rk _ CH w ❏ 7C w p w 7C ❏ w /(1( 7 Rk _ CH w
  • 38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. V n 4ZNg 86 V aN PR n BPa OR_ ) ()/# Yeead4))Rhd(R Rk _(T ) a)RS fe Rhd)hYRed _Vh), +1)+ )R Rk _ V c _ h dfaa ced Z- Z_deR_TVd) ❏ w 7 Rk _ 9, w Z- ❏ Z- pD _ L ReZ V CV cj iacVdd DLCV$ II: w p E E ❏ w /(3( 7 Rk _ CH w
  • 39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E n Yeead4))Rhd(R Rk _(T ) a)RS fe Rhd)hYRed _Vh), +1)+ )R Rk _ V c _ h dfaa ced aVc dVT _U SZ Z_X) ❏ , +1 + , p 7MI w ❏ w w ❏ w + ❏ p+ + $
  • 40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E ,&e
  • 41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E ERYRN R ()- / # • HV VRdV .( ( m , +/) 1 • HV VRdV .(+( m , +/) 3 • HV VRdV .(,( m , +/)++ • HV VRdV .(-( m , +0) + • HV VRdV .(.( m , +0) - • HV VRdV .(/( m , +0) . • HV VRdV .(0( m , +0) . • HV VRdV .(1(i m , +0) 0 1 • HV VRdV .(2(i m , +0) 2 , +1) , • HV VRdV .(3(+ m , +1) - • HV VRdV .(3(, m , +1) 1 7aRTYV 8ZXe a w w × +, +, w >RU a × Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)HV VRdV=fZUV)V c cV VRdV T a _V_ed(Ye • HV VRdV /( ( m , +0) 2 • HV VRdV /( (- m , +0)+ • HV VRdV /(+( m , +0)++ • HV VRdV /(,(i m , +0)+, • HV VRdV /(-(i m , +1) + , • HV VRdV /(.( m , +1) , • HV VRdV /(/( m , +1) . • HV VRdV /(0( m , +1) 0 • HV VRdV /(1( m , +1) 1 • HV VRdV /(2( m , +1) 2 • HV VRdV /(3( m , +1)+
  • 42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E ERYRN R ❏ w • 7 Rk _ CH p w w ❏ w • p w ❏ w w p • GfZT 9cVReV • pCR_RXV V_e =fZUV HV VRdV =fZUV 7C w w p http://docs.aws.amazon.com/ElasticMapReduce/latest/ReleaseGuide/emr-release-differences.html
  • 43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R2 n ❏ w w • w • 4 • jRc_(cVd fcTV R_RXVc(hVSRaa(RUUcVdd a ce 3 ,0 56 2 22 • )Y V)YRU a)YZgV 56 )fdc) ZS)YZgV& )VeT)YZgV)T _W& )gRc) X)YZgV ❏ w w w • YRU a w ” • jRc_ w pYZgV w
  • 44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R2 n ❏ 9 RddZWZTReZ _ % Fc aVceZVd • 8 edecRa 7TeZ _ • w ” Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)HV VRdV=fZUV)V c T _WZXfcV Raad(Ye e.g. /etc/hadoop/conf/mapred-site.xml
  • 45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R2 n j ❏ w w GfZT 9cVReV • p w p × - ” • w = e RUgR_TVU aeZ _d ❏ w • CR_RXV V_e =fZUV • 7 Rk _ CH w w • Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)CR_RXV V_e=fZUV • HV VRdV =fZUV • w w • Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)HV VRdV=fZUV
  • 46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. n o-&1&(# Yeea4))U Td(Rhd(R Rk _(T ))V c) ReVde)HV VRdV=fZUV)V c cV VRdV T a _V_ed(Ye
  • 47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,&ey -&e ❏ w • IaRc ,(i • FPNYN &)) • >ZgV ,(i • Z_ • BZgj ❏ • NcN 0 • JVk >ZgV& FZX$ ❏ • IeVa ❏ IR_US i • w • IR_US i
  • 48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E 4 YVPNaV S _ -&1&( • >RU a& >ZgV& JVk& IaRc & FcVde & FZX& CRY fe& Ib a& Z_ w w • >8RdV& FY V_Zi =K • >fV& PVaaV Z_ w w • E kZV • =R_X ZR& >9ReR X& P VVaVc& YVcf
  • 49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4 YVPNaV 2 ❏ >RU a • qO7HD w $%>: I w $ p qCRaHVUfTV + q CH O7HD p q ❏ >ZgV • O7HD p IGB q CRaHVUfTV qI- w q ❏ JVk • O7HD :7= $ q >ZgV ” p >ZgV q
  • 50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4 YVPNaV 2 ❏ IaRc • w w qO7HD pCRaHVUfTV qIaRc IaRc IGB >ZgV $ IaRc IecVR Z_X w $& CB ZS $ p q ❏ Z_ • w w q O7HD p w p qIaRc p pIGB q w ViRTe j _TV × q ❏ FcVde • IGB qO7HD ” pIGB q w E D qIGB 7DI IGB q
  • 51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4 YVPNaV 2 ❏ FZX • CRaHVUfTV :IB q q ❏ CRY fe • CRaHVUfTV IaRc q q ❏ Ib a • H:8CI p>: I I- w CRaHVUfTV q
  • 52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4 YVPNaV 2 n n ❏ >8RdV • >: I ALIq w w q ❏ FY V_Zi • >8RdV w IGB w q
  • 53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4 YVPNaV 2 :H n n ❏ >fV • >ZgV ×w p =K q ❏ PVaaV Z_ • =K IaRc w IaRc IGB p p s w q ❏ E kZV • w w w qCRaHVUfTV >ZgV q
  • 54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4 YVPNaV 2 ❏ =R_X ZR • w 9FK&CV cj&DVeh c $ p q ❏ >9ReR X • >ZgV ×w FZX w q ❏ P VVaVc • w w pALI p w w q >8RdV $ q
  • 55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4 YVPNaV 2 ❏ BZgj • IaRc CH w H IJ w p pHF9 w w
  • 56. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. n 4ZNg 8@E
  • 57. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E x n ❏ w w • ” w • HV VRdV BRSV )7C LVcdZ _ w p w ❏ w w • 7aRTYV 8ZXe a w w p8 edecRa • w 7C Yeea4))Rhd(ejaVaRU(T )dR a), +0) 3)SfZ UZ_X R_U UVa jZ_X Tfde Raa ZTReZ _d hZeY RaRTYV 8ZXe a R_U R Rk _ V c(Ye
  • 58. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. n 2 NQ ❏ O7HD 5 OVe 7_ eYVc HVd fcTV DVX eZRe c ❏ >RU a, w • SJcRT Vc ❏ HVd fcTV HVbfVde w w • 7aa ZTReZ _CRdeVc p w HVd fcTVCR_RXVc MapReduce Pig Hive Other Other HDFS Other Map ReducePig Hive YARN HDFS Hadoop V1 Hadoop V2
  • 59. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. K4EA2 ER b_PR@N NTR_ ❏ w w ❏ w w • 9FK& CV cj& VeT( ❏ w ❏ • 7aa ZTReZ _CRdeVc Yeea4))YRU a(RaRTYV( cX)U Td)TfccV_e)YRU a jRc_)YRU a jRc_ dZeV)O7HD(Ye
  • 60. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. K4EA2 A QR@N NTR_ ❏ w w ❏ w w HC ❏ w 9 _eRZ_Vc • 9 _eRZ_Vc Yeea4))YRU a(RaRTYV( cX)U Td)TfccV_e)YRU a jRc_)YRU a jRc_ dZeV)O7HD(Ye
  • 61. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. K4EA2 6 aNV R_ ❏ w w ❏ DC ❏ • :VWRf e9 _eRZ_Vc iVTfe c • • BZ_fi9 _eRZ_Vc iVTfe c • TXc fad • : T Vc9 _eRZ_Vc iVTfe c • : T Vc Yeea4))YRU a(RaRTYV( cX)U Td)TfccV_e)YRU a jRc_)YRU a jRc_ dZeV)O7HD(Ye
  • 62. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. K4EA2 4 YVPNaV @N aR_ ❏ 9 _eRZ_Vc • SJcRT Vc • + 7C ❏ 9 _eRZ_Vc Yeea4))YRU a(RaRTYV( cX)U Td)TfccV_e)YRU a jRc_)YRU a jRc_ dZeV)O7HD(Ye
  • 63. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg 8@E K4EA ❏ CRaHVUfTV& >ZgV& FZX& IaRc _ O7HD • 7 Rk _ CH pO7HD w ❏ × • CH >ZgV IaRc p w
  • 64. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. n 2 VcR ❏ IGB Z V p w w ❏ w w w p ×w • >: I w w • 7 Rk _ CH CH I 7 Rk _ I-$ • p7 Rk _ AZ_VdZd& 7 Rk _ :j_R :8 ❏ • CRaHVUfTV& JVk& IaRc Yeead4))YZgV(RaRTYV( cX)
  • 65. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. VcR Metastore HDFS Cluster Amazon S3 HiveServer2 CREATE TABLE SELECT FROM +4 ×w >: I 7 Rk _ I- w ,4 ×w w -4 ×w .4 ×w w /4 04 w w Amazon DynamoDB Amazon Kinesis
  • 66. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. VcR @RaN a _R ❏ IGB CVeRde cV ❏ CVeRde cV CjIGB • 7 Rk _ CH CRdeVc D UV w • p7 Rk _ H:I ❏ w 7MI = fV :ReR 9ReR X CVeRde cV CVeRde cV Amazon S3 HDFS
  • 67. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. v n BE6 9VYR CN_ bRa ❏ w • w • • EaeZ ZkVU H h 9 f _Rc EH9$ Z V ) FRcbfVe ❏ w • + • + p w • J NJ B & I GK D9 B EH9 w Yeead4)) cT(RaRTYV( cX)U Td)daVT Z_ec (Ye
  • 68. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. n w ❏ • w • w w ❏ w ❏ w p+ w +
  • 69. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. VcR BE6 9VYR CN_ bRa v ❏ ×w ” ❏ CREATE TABLE t ( col1 STRING, … ) STORED AS [ORC/PARQUET]; INSERT INTO t (…); SELECT col1 FROM t;
  • 70. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. VcR 4ZNg V R V n ❏ 7 Rk _ AZ_VdZd IecVR w p>ZgV ×w • 7 Rk _ AZ_VdZd p w w • 4 >ZgV p w ❏ IecVR IYRcU CRaaVc w • 7 Rk _ AZ_VdZd w • 7 Rk _ :j_R :8 $ p ” w IYRcU IYRcU + CRaaVc CRaaVc SELECT FROM 123 id data 1 … 2 … 3 … Amazon Kinesis
  • 71. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. VcR 4ZNg V R V n VcR n 7 Rk _ AZ_VdZd IecVR D # >GB ” q 7 Rk _ AZ_VdZd w
  • 72. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. n 2 F N_ ❏ w w • p • $ ❏ w w p w • IGB& w & & http://spark.apache.org/
  • 73. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. F N_ FD 7NaN Ra 4C F N_ & # ❏ IaRc IGB • >ZgVGB p>ZgV CVeRde cV • :89)E:89 • IGB p :ReRdVe):ReR cR V ❏ :ReRdVe 7F • w • ITR R) RgR • :ReR cR V :ReRdVe Yeead4))UReRScZT d(T )S X), +0) 1)+.)R eR V W eYcVV RaRTYV daRc RaZd cUUd UReRWcR Vd R_U UReRdVed(Ye Yeea4))daRc (RaRTYV( cX)U Td) ReVde)db ac XcR Z_X XfZUV(Ye
  • 74. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ❏ :ZdTcVeZkVU IecVR $ p ❏ HVdZ ZV_e :ZdecZSfeVU :ReRdVe $ w • p ❏ pIaRc ,(, CH HV VRdV /(2( $ pIecfTefcVU IecVR Z_X 7F =7 DStream RDD@T1 RDD@T2 Messages Receiver F N_ Fa_RNZV T i Yeea4))daRc (RaRTYV( cX)U Td) ReVde)decVR Z_X ac XcR Z_X XfZUV(Ye Yeead4))daRc (RaRTYV( cX)U Td) ReVde)decfTefcVU decVR Z_X ac XcR Z_X XfZUV(Ye
  • 75. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4ZNg V R V F N_ Fa_RNZV T 7 Rk _ AZ_VdZd IecVR p7 Rk _ CH IaRc :IecVR • IaRc 7 Rk _ AZ_VdZd • pAZ_VdZd 9 ZV_e BZScRcjzA9B x p • w p http://spark.apache.org/docs/latest/streaming-kinesis-integration.html IYRcU IYRcU + A9B eYcVRU A9B eYcVRUAmazon Kinesis DStream RDD@T1 RDD@T2
  • 76. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. F N_ 7f NZVP ER b_PR 4YY PNaV ❏ iVTfe c H:: • >VRa p ❏ :j_R ZT HVd fcTV 7 TReZ _ 5 iVTfe c • IecVR Z_X 7aa ZTReZ _ w • daRc dYV w • .(.( pdaRc (Uj_R ZT7 TReZ _(V_RS VU a_bR • daRc (dYfWW V(dVcgZTV(V_RS VU a_bR https://spark.apache.org/docs/latest/job-scheduling.html
  • 77. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  • 78. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ❏ w ) n ❏ w 4ZNg 8@E w p + “ u
  • 79. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. p q • 7 Rk _ CH CR_RXV V_e =fZUV Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)CR_RXV V_e=fZUV • 7 Rk _ CH HV VRdV =fZUV Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)HV VRdV=fZUV • 7 Rk _ CH 7F HVWVcV_TV Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde)7F ) • 7 Rk _ CH :VgV aVc =fZUV ,(i R_U -(i $ Yeea4))U Td(Rhd(R Rk _(T ) RdeZTCRaHVUfTV) ReVde):VgV aVc=fZUV) • 7MI 9B V c Yeea4))U Td(Rhd(R Rk _(T )T Z) ReVde)cVWVcV_TV)V c)Z_UVi(Ye
  • 80. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. D 4
  • 81. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. n 7MI w • Yeea4))Rhd(R Rk _(T ) a)Rhd a Z_ec UfTeZ _) 7MI I feZ _d 7cTYZeVTe • p w G 7 • Yeea4))Rhd(ejaVaRU(T )dR a)
  • 82. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GdVaaR_ 9NPRO 4JF x 3Nd PY bQMW p p p w v u – Uaa 2 &SO&ZR )cE0fJZ
  • 83. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4JF jx v 7MI p p ” Yeead4))Rhd(R Rk _(T ) a)T _eRTe fd)Rhd dR Vd) 7MI ”
  • 84. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  • 85. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4 R QVe
  • 86. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4@ E YR z , +/ 0 - p 7C H V 7C H V , p • CH c V • 7 Rk _ CH p7 Rk _ 9, • 9, Z_deR_TV ac WZ V • 7 Rk _ 9, p7 Rk _ I- Yeea4))Rhd(ejaVaRU(T )dR a), +/) 0)V c R_URe cj ZR faUReV(Ye
  • 87. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,&( w w • >RU a ,(0( • >ZgV +( ( • FZX (+.( • CRY fe (+ ( • IaRc +(.(+ 7aRTYV 8ZXe a w w × • p
  • 88. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,&) w • IaRc +(/( • >fV -(1(+ $ p w • FcVde (++3 • PVaaV Z_ (0 I_RadY e$ • E kZV .( (+
  • 89. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,& w • IaRc +(/(, • =R_X ZR -(0( $ • FcVde (+,/ • PVaaV Z_ (/(/ • E kZV .(,(
  • 90. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,& w • >RU a ,(1(+ • IaRc +(0( • =R_X ZR -(1(, • FcVde (+-
  • 91. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,&, w • >9ReR X +( ( $ • CRY fe (++(+ • Ib a +(.(0 $ • FcVde (+-0 • PVaaV Z_ (/(0 IaRc • :j_R ZT R TReZ _ RgR 2 • 7L7Q>EC ”
  • 92. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,&- w • IaRc +(0(+ • >RU a ,(1(, • FcVde (+. CH I • I- II ACI • 7MI ACI 9CA
  • 93. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,&. w • >8RdV +(,( $ m D IGB _ >RU a • P VVaVc -(.(2 $ • FcVde (+.-
  • 94. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,&/&) t zu ,&/&(– w • FY V_Zi .(1( $ m IGB _ >8RdV • JVk (2(- $ m :7= V_XZ_V _ O7HD • >8RdV +(,(+ • CRYRfe (+,( • FcVde (+.1
  • 95. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,&/& w • CRY fe (+,(, • IaRc +(0(, • FcVde (+.2
  • 96. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,&0&( w • >8RdV +(,(, • JVk (2(. • PVaaV Z_ (0(+ • FcVde (+/+
  • 97. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,&0& w • >RU a ,(1(- • FcVde (+/,(-
  • 98. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,&0& w • IaRc +(0(- • FcVde (+/1(+ • P VVaVc -(.(3 • CH :j_R :8 9 __VTe c – .(,( – >ZgV& IaRc
  • 99. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R ,&1&) w • FjeY _ ,(0 w • w >ZgV • > L 3310 • > L + + 0
  • 100. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -&(&( w • >ZgV ,(+( • FcVde (+/ • IaRc ,( ( • FPNYN &)) • >fV -(+ ( • FZX (+0( • JVk (2(. • >8RdV +(,(, • PVaaV Z_ (0(+ • NcN 0 • JVk >ZgV& FZX$ • IeVa IR_US i • w • IR_US i
  • 101. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 85F z j @, 6, n CH 7F 8I • >: I • p • 9 fU c ReZ _ 8I I9+)IJ+ w • >: I C. 9. Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0) ,)R Rk _ V c _ h dfaa ced R Rk _ VSd g f Vd R_U . R_U T. R Rk _ VT, WR Z ZVd)
  • 102. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4 NPUR VcR 4 NPUR F N_ 8@E 7f NZ 75 n BFF :j_R :8 :j_R :8 ×w p CH w E D =Ze>fS • Yeead4))XZeYfS(T )Rhd RSd)V c Uj_R US T __VTe c Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +0) 3)eYV R Rk _ V c Uj_R US T __VTe c W c RaRTYV YZgV R_U RaRTYV daRc Zd _ h aV_ d fcTV)
  • 103. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -&(& w • >RU a ,(1(- • FcVde (+/,(- • IaRc ,( (+
  • 104. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -&)&( w • Z_ +(+(- $ • >8RdV +(,(- • PVaaV Z_ (0(,
  • 105. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -& &( w • IaRc ,( (, >8RdV _ I- • >8RdV w I-
  • 106. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -& &) w • FcVde (+/1(+ • P VVaVc -(.(3 • CH :j_R :8 9 __VTe c – .(,( – >ZgV& IaRc
  • 107. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -& &( w • >ZgV ,(+(+ • >fV -(++( • IaRc ,(+( • E kZV .(-( • Z_ +(+(.
  • 108. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -& &) w • PVaaV Z_ • P FF B D +2/0 • P FF B D +311
  • 109. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -&,&( w • FcVde (+00 • FY V_Zi .(3( • >8RdV +(-( • Z_ +(,( • PVaaV Z_ (1(
  • 110. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. E, n Yeead4))Rhd(R Rk _(T )RS fe Rhd)hYRed _Vh), +1) -)R Rk _ V c cV VRdV / . R_U dfaa ce W c c. Z_deR_TVd _ h RgRZ RS V) H. • × w w
  • 111. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -&-&( w • >fV -(+, • FcVde (+1 • B:7F w • PVaaV Z_ (1(+ • P AVVaVc -(.(+
  • 112. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -&.&( w • >8RdV +(-(+ • IaRc ,(+(+ • Z_ +(,(+ • CRY fe (+-( • IaRc ,(i w FcVde Z_ ecR_dZe
  • 113. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -&/&( w • Z_ +(-( • FY V_Zi .(++( • PVaaV Z_ (1(, w p 7 Rk _ BZ_fi 7C 5N R • w w C_R a • _ UV(ac aVceZVd K4EA • T _eRZ_Vc X. (ac aVceZVd F • IGEEF ,22 w
  • 114. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -&0&( w • 7MI I:A +(++(+0 • Z_ +(-(+ • >ZgV ,(-( • IaRc ,(,( • IecfTefcVU IecVR Z_X =7 7MI = fV :ReR 9ReR X • >ZgV IaRc IGB 7MI = fV :ReR 9ReR X >ZgV w B gVR • EEP ,1.2 w bR • >K /2/3 w 5N R • CN $ >8RdV w w w • w RgR CR_RXV V_e ieV_dZ _d
  • 115. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -&1&( w • 7MI I:A W c RgR +(++(+2- • Z_ +(-(, • >fV .( (+ • >fV D eVS W c IaRc w • FZX (+1( • FcVde (+2. • BZgj (.( Z_TfSReZ_X $ Z- dVcZVd 7 Rk _ 9, Z_deR_TVd w ETe SVc +,& , +1$
  • 116. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ERYRN R -&1&( F N_ • w p p7fe ITR Z_X XcRTVWf • IF7HA ,+.3. w LR RYV • P FF B D ,-11 w ON R • >87I +2/-- bR • >ZgV bfVcj VUZe c 7MI = fV :ReR 9ReR X w •