This document discusses various Amazon Web Services (AWS) products for data science and analytics workflows. It highlights how AWS services like Amazon Redshift, Amazon Kinesis, Amazon EMR, and others can be used together for extracting, transforming, loading (ETL) data, performing analytics, and building data science models at scale. Specific applications and customers like Hearst Publishing that leverage these AWS services for clickstream analytics are also mentioned. The document aims to provide an overview of the AWS big data and analytics portfolio.

1. e 4 M G G TS 3EC
p fl 5IF 5 p
n
n n h
(). , )1

2. • n n
• 5IF “
• i n i n
• 5IF S Z2 TOP SL P P STWbK P
5, e SKW W SGRT 4 ZW 3EC TD m np

3. ”
6TR 8L L4 5IF4
6TR 8L L 5IF n
• 5WLc F
• 5WLc T P T
• 5WLc EPO STQ
• 5WLc 9 E
• 5WLc D TNUFTRS
• 5WLc 5 L

4. z 4 M G Gd z 3ECd

5. 4 M G G d
n i
• i k n
• n
• e g
n “
• n i k n “

6. 4 M G G d
n i
• i k n
• n
• e g
n “
• n i k n “
6TR 8L L

7. 4 M G G
n n n i
n nAI n fl ”
n m n
n n
B P B P n
n n o CF n i n n
o

8. 4 M G G
n n n i
n nAI n fl ”
n m n
n n
B P B P n
n n o CF n i n n
o
-
” “ l
) i) i) i r
)

9. e ~
fl
n r n
” fl
i
“
n
i
i i
n
w
u “ v n

10. e ~
fl
n r n
” fl
i
“
n
i
i i
n
w
u “ v n
)

11. z 4 M G G2
n i
• hyh n
• 2 AP fTa 2 n 3 /2
• k i
• fl
• i

12. 4 M G G w s ff
s “ i
h t
s“~ n i
h fl t

13. z 3EC nd
n p a l –
4 M G G ~ p e o
•
• n o
•
•
• n lF $ EPO STQ $ 9 E ”
K QT

14. 3EC w e u– m

15. 3EC w e u– m

16. 4 M G G TS 3EC
n
• -( n i n i i
n o
• n k
• i n i
• 6TR 8L L 3

17. 4 M G G TS 3EC e

18. 5WLc F $ LNTP
5WLc T P T
5IF = G
5WLc 8b LW 86
5WLc E8F 5 L
5IF ?LWMOL
7? 5ZZ
e e e
n
5WLc
9 E
5WLc
EPO STQ
5WLc LNST P
?PL T R
5WLc
D TNUFTRS
5IF 8L L CTZP T Pe e

19. 5WLc F $ LNTP
5WLc T P T
5IF = G
5WLc 8b LW 86
5WLc E8F 5 L
5IF ?LWMOL
7? 5ZZ
e e e
n
5WLc
9 E
5WLc
EPO STQ
5WLc LNST P
?PL T R
5WLc
D TNUFTRS
5IF 8L L CTZP T Pe e

20. Amazon S3
e o r e
~ b e e ~b
n z n n
11 111111111w
) 6 fl , fl
e
00(0000000
n
n5
n
n6
n n7
HTTP/
HTTPS
e
TD
3EC
e

21. 3RG TS C-
e
B BK WN L
e
5, B C
C T GMK
9G K G
4C
BK WN L
5QT TS
e
9E
C T GMK
9G K G
e
QGW I
D GSWIT K
e
9QGI K
e
G G
KQ SK
C- 3EC e

22. C- e e e g h
e e
e
C- C-
B
BK WN L
5,

23. QGW I
G BK IK
SGRT 4 BK CN L
e oC-~i– a ~
w p g e h
C-

24. QGW I
G BK IK
SGRT 4 BK CN L
e oC-~i– a ~
w p g e h
C-
e

25. QGW I
G BK IK
SGRT 4 BK CN L
e oC-~i– a ~
w p g e h
C-

26. SGRT 4 BK CN L
e oC-~i– a ~
w p g e h
C-

27. i C- o u–

28. 5WLc F $ LNTP
5WLc T P T
5IF = G
5WLc 8b LW 86
5WLc E8F 5 L
5IF ?LWMOL
7? 5ZZ
e e e
n
5WLc
9 E
5WLc
EPO STQ
5WLc LNST P
?PL T R
5WLc
D TNUFTRS
5IF 8L L CTZP T Pe e

29. 3RG TS SKW W 5
n w e r
e e ~ o
Data
Sources
AWS Endpoint App.1
[Aggregate &
De-
Duplicate]
Data
Sources
Data
Sources
C- e
BK WN L
App.2
[Sliding
Window
Analysis]
Availability
Zone
Shard 1
Shard 2
Shard N
Availability
Zone
Availability
Zone
Amazon Kinesis
e
-3F& ,.N%
5
5,

30. 3RG TS SKW W KNTWK
e e 3RG TS C- 3RG TS BK WN L
1 n n F EPO STQ i n
i “
e 1 i n n i i
n .(
1 n n i n

31. 3RG TS SKW W 3SGQ IW
e e CA ~ y

32. n
• n n
•
• n
• EG6 n
• n 2 ,( 6
• n n 2 ). /(( 6
• EG6 n 2 .(( 0(G6
3 BTQQ

33. )( d-
+/
Dynamo
DBS3 Lambda
S3
HBase
EC2
EC2
EC2
...
SQS
SQS
SQS
...
n
EC2
EC2
EC2
...
e e
SKW W

34. -
Dynamo
DB
S3
HBase
EC2
EC2
EC2
...
EC2
EC2
EC2
...
Kinesis
e e
SKW W
n
7?

35. • - n
• )-- n
• ., n
• - ) T P T n
SKW W
x . )3) (

36. • )- y n
•
SKW W ~

37. ip e
0( fl n n n T P T

38. ip e
0( fl n n n T P T

39. 5WLc F $ LNTP
5WLc T P T
5IF = G
5WLc 8b LW 86
5WLc E8F 5 L
5IF ?LWMOL
7? 5ZZ
e e e
n
5WLc
9 E
5WLc
EPO STQ
5WLc LNST P
?PL T R
5WLc
D TNUFTRS
5IF 8L L CTZP T Pe e

40. Amazon Redshift
3RG TS C-
3RG TS BK WN L 4 e
k n i n
k n

41. Amazon Redshift
n
•
• fl
•
n
• k n
• C R PFD? FD?
• 6= n n

42. • n FD?
• E86 n
• n 9G?
• ”i
•
n n $ n
orderid name price
1 Book 100
2 Pen 50
…
n Eraser 70
orderid name price
1 Book 100
2 Pen 50
…
n Eraser 70
gB 4 Ch gBK WN L h

43. BK WN L
l
e
B C
C-
ee
e
3
4
e
e
e
e
e
e
e
e
e
e4 e

44. ?DD
n i

45. ?DD
http://media.amazonwebservices.com/jp/csd20140909/BZ-03.pdf

46. ?DD
http://media.amazonwebservices.com/jp/csd20140909/BZ-03.pdf

47. yn er
8I )
n i
)
n fl n
i
n
i
i ,(

48. 5WLc F $ LNTP
5WLc T P T
5IF = G
5WLc 8b LW 86
5WLc E8F 5 L
5IF ?LWMOL
7? 5ZZ
e e e
n
5WLc
9 E
5WLc
EPO STQ
5WLc LNST P
?PL T R
5WLc
D TNUFTRS
5IF 8L L CTZP T Pe e

49. 3RG TS QGW I G BK IKg Bh
w e :G TT )C G P
e e ~
5IF n
• 6TR 8L L
• n n
n
•
•
• n
•
• i
• FZ
LO Z
5WLc 9 E
)
E= FZ

50. w wa
aws emr create-cluster !
--name bigdata-handson !
--ami-version 3.2.1 !
--applications Name=Hive !
--instance-groups
InstanceGroupType=MASTER,InstanceCount=1,InstanceType=m1.large
InstanceGroupType=CORE,InstanceCount=2,InstanceType=m1.large !
--log-uri s3:/PATH/TO/LOG/ !
--ec2-attributes SubnetId=subnet-a06474e6,KeyName=YOUR_KEY!
--steps Type=HIVE,Name='Hive program’, Args=[-f,s3://PATH/TO/
QUERY.q] !
--auto-terminate!

51. C- e l
• 8:F n F n
• =ACHG BHGCHG 2 p “
hadoop jar YOUR_JAR.jar !
--src s3://YOUR_BUCKET/logs/ !
--dest s3://YOUR_BUCKET/output/!
hadoop jar YOUR_JAR.jar !
--src s3://YOUR_BUCKET/logs/ !
--desct hdfs:///output/ !
• F n F “
• F n n 8:F “

52. Amazon Elastic MapReduce
• LO Z n n
• WLZ PO NP$ ST_P$ ZTR$ PLWT R
” LO Z
• ST_P FD?
EPO STQ fl
iGE5AF:BE H8: H85:
” r
• n
• i
fl
“
Amazon Redshift
• E86
• FD?
• 6= n
• fl n
•
•

53. SQL on Big Data
10 GigE
(HPC)
COPY
UNLOAD
SQL Clients/BI Tools
128GB RAM
16TB disk
16 cores
JDBC/ODBC
128GB RAM
16TB disk
16 coresCompute
Node
128GB RAM
16TB disk
16 coresCompute
Node
128GB RAM
16TB disk
16 coresCompute
Node
Leader
Node
SQL Clients/BI Tools
JDBC/ODBC
Master
Node
Core/Task
Node
Core/Task
Node
Core/Task
Node
READ
WRITE
EMRFSS3 / EMR / DynamoDB / SSH

54. SQL on EMR
Query EngineApplicatio
n
Storage
YARN
Map
Reduce Tez Spark
Hive
Spark
SQL
Presto
JDBC/ODBC
HiveMetastore
HDFS
EMRFS
Hue
Zeppelin
SELECT COUNT(*)

55. e e
T TRS i5WLc F
fl n 5WLc
9 L TN LZEPO NP
C P i
R 86 n
n “
m C=
L L b TN i5WLc EPO STQ
n i
LM PL NSL T ” 6= n

56. e e
T TRS i5WLc F
fl n 5WLc
9 L TN LZEPO NP
C P i
R 86 n
n “
m C=
L L b TN i5WLc EPO STQ
n i
LM PL NSL T ” 6= n

57. 5WLc F $ LNTP
5WLc T P T
5IF = G
5WLc 8b LW 86
5WLc E8F 5 L
5IF ?LWMOL
7? 5ZZ
e e e
n
5WLc
9 E
5WLc
EPO STQ
5WLc LNST P
?PL T R
5WLc
D TNUFTRS
5IF 8L L CTZP T Pe e

58. Amazon QuickSight
n 6= n
5IF n n
5
LZS
FC=79
n n
n
n

61. 4 +)+
F L OL O 9OT T 2
• ) n 1 n ) r
• ( - 6 FC=79 n )( 6 fl
9 P Z T P 9OT T 2
• F L OL O 9OT T i n
n i i58
• , n )0 n )
r
• ( 0 6 FC=79 n )( 6 fl
x n “

62. 5WLc F $ LNTP
5WLc T P T
5IF = G
5WLc 8b LW 86
5WLc E8F 5 L
5IF ?LWMOL
7? 5ZZ
e e e
n
5WLc
9 E
5WLc
EPO STQ
5WLc LNST P
?PL T R
5WLc
D TNUFTRS
5IF 8L L CTZP T Pe e

63. 3RG TS B C
n i i i i
e c e e e
3F G 3F H
e
” e e
(5 )

64. 3RG TS 3 T G
• 5WLc fl E86 F
• n o n
• n n o bFD? - . r
k n
k
.,G6
fl

68. 9 GS

71. e 4 M G G TS 3EC
n
• -( n i n i i
n o
• n k
•
• 6TR 8L L 3

72. Data Science
Amazon Redshift
ETL処理
~ w
K1 SZKS , +/ n
e eAmazon
Kinesis
Amazon
EMR
http://www.slideshare.net/AmazonWebServices/bdt306-how-hearst-publishing-manages-clickstream-analytics-with-aws

73. i o k
tv w