R de Hadoop (Oracle R Advanced Analytics for Hadoopご説明資料）

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
> ore.connect
function (user = "", sid = "", host = "localhost", password = "",
port = 1521, service_name = NULL, conn_string = NULL, all = FALSE,
type = c("ORACLE", "HIVE"), tzone = Sys.getenv("TZ"))
{
type <- match.arg(type)
if (ore.is.connected())
ore.disconnect()
.ore.QueryEnvInit()
switch(type, ORACLE = .ore.oracleQuerySetup(), HIVE = .ore.hiveQuerySetup())
.ore.QueryEnv$connect(user, sid, host, password, port, service_name,
conn_string, tzone)
if (all) {
ore.sync()
ore.attach()
}
invisible()
}
<environment: namespace:OREbase>
R de Hadoop
Hadoop R
~
ehn]->b @ m -@EO
&Nrhr ,J d b<Kk e^, hf'

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 2
Safe Harbor Statement

Big Data Management System
3
SOURCES
DATA RESERVOIR DATA WAREHOUSE
Oracle Database
Oracle Industry
Models
Oracle Advanced Analytics
Oracle Spatial & Graph
Big Data Appliance
Apache
Flume
Oracle
GoldenGate
Oracle Event
Processing
Cloudera Hadoop
Oracle Big Data SQL
Oracle NoSQL
Oracle R Distribution
Oracle Big Data
Spatial and Graph
Oracle Database
In-Memory, Multi-tenant
Oracle Industry Models
Oracle Advanced
Analytics
Oracle Spatial and Graph
Exadata
Oracle
GoldenGate
Oracle Event
Processing
Oracle Data
Integrator
Oracle Big Data
Connectors
Oracle Data
Integrator
B

Oracle R Advanced Analytics for Hadoop
~
4
http://www.wanpug.com/ ammi8--inmbr , hf-

Oracle
• i
– @ m [ l^ @ m [ l^
– D ]hhi D ]hhi
• i
– CA - Dbo^ - >b @ m OMH
– B
• i
– | ~
5
Kk e^ N
Agm^kikbl^
Kk e^ N
=]o g ^] =g ermb l
Bhk D ]hhi

N@>IO
~ ~
i
~ ~
i v
i

: (@DB)
Oracle Advanced Analytics
7
Traditional Analytics
Hours, Days or Weeks
Data Extraction
Data Prep &
Transformation
Data Mining
Model Building
Data Mining
Model “Scoring”
Data Preparation
and
Transformation
Data Import
Source
Data
Dataset
s/ Work
Area
Analytic
al
Process
ing
Process
Output
Target
Secs, Mins or Hours
Model “Scoring”
Embedded Data Prep
Data Preparation
Model Building
Oracle Data Mining
w x
l i m u l m
wD3
w i
w i
w
~ @>
OMH8 ~
~ ~ ~
~ @>
CK SXQ

Oracle’s Big Data Advanced Analytics Solutions
KN ¥KMVO 4K KLK O B
8
Oracle Database Server
with Advanced Analytics Option
R
Hadoop Cluster
Big Data
SQL
Oracle R Enterprise
(ORE)
R Client
SQL Developer
SQL
SQL Client
Oracle ExadataBig Data Appliance
SQL
R
R Client
~ ~
~ ^m
~ ~
~
^m
Oracle R Advanced
Analytics for Hadoop

Big Data
~ ~
Hive HiveR
Java MahoutMahout

MapReduce
~ ~
Hive HiveR
Java MahoutMahout

Spark
~ ~
Hive HiveR
Java MahoutMahout
Scala Phython

MapReduce
~ ~
Hive HiveR
Java MahoutMahout
–
Scala Phython

R
~ ~
R R
RR
R

Oracle R
Oracle R Distribution (ORD)
14
• ¥KMVO B i i
• ¥KMVO B 4S ¥SL_ S X B4
– ¥KMVO B z
– B l m ¥KMVO i v f
– SX_b y 9X OV K R O¥XOV SL¥K¥cv
C 1B3 C_X O¥P ¥WKXMO SL¥K¥c
• B4 y
Oracle R Distribution
http://www.oracle.com/technetwork/database/database-technologies/r/r-distribution/downloads/index.html
You are using Oracle's distribution of R. Please
contact Oracle Support for any problems you
encounter with this distribution.

• 9X OV Egm^e
• B4 –
IGH ~
~ •
– Hbgnq H@YHE>N=NUYL=PD IGH
– Sbg]hpl eb[Kk]>e lHh ]^k,]ee fdeYkm,]ee L=PD
– @> ~ &Hbgnq-Sbg]hpl' IGH KN= HAYDKIA-eb[
• Cc 2VK K KMU - – &KN@'
Intel Math Kernel Library (MKL)
ORD MKL
15
R> Sys.BlasLapack()
$vendor
[1] "Intel Math Kernel Library (Intel MKL)"
$nthreads
[1] -1
R> Sys.BlasLapack()
$vendor
[1] "R internal BLAS and LAPACK"
$nthreads
[1] 1
KN==D
IGH ~

Vhk e^<[b ] m ebm^ uW" B
Kk e^ @blmkb[nmbhg h_ N o^klbhg 1,0,. &++' ++ Bnee h_ Eg k^]b^gml
hirkb am & ' Pa^ N Bhng] mbhg _hk Om mblmb e hfinmbg
Le m_hkf8 q64Y42+ngdghpg+ebgnq+ gn &42+[bm'
N
–
%eb ^gl^&'% %eb ^g ^&'%
N
% hgmkb[nmhkl&'%
N N ~
% bm mbhg&'%
%]^fh&'% –
%aêi&'%
%aêi,lm km&'% DPIH
%j&'% N
Uhn k^ nlbg Kk e^%l ]blmkb[nmbhg h_ N, Le^ l^ hgm m
Kk e^ Oniihkm _hk gr ikh[e^fl rhn ^g hngm^k pbma mabl ]blmkb[nmbhg,
Orl,>e lH i d&'
"o^g]hk
V/W Egmê I ma G^kgê Hb[k kr &Egmê IGH'
"gmak^ ]l
V/W +/

Oracle R Technologies ( Blog)
• Kk e^ N
– ~
~
17
https://blogs.oracle.com/R/

~
18

Oracle R Advanced Analytics for Hadoop (ORAAH)
R Hadoop
B KN
– ¥KMVO 2SQ 4K K 3 XXOM ¥
• ¥KMVO B 1N KXMON 1XKVc SM P ¥ KN
( B 46C i
) B S O i
B KN ~
•
• I iN^]n ^-Oi kd D ]hhi –
B K BON_MO
, B Kk e^ @ m [ l^ KN i
- B C K¥U VSL
19

Hadoop Cluster
with Oracle R Advanced Analytics for Hadoop
R
R Client
HQL , Data Prep, Joins and View creation
HQL
R
( 46C i
B u
) Bu S O i
Bu KN
j K BON_MO#C K¥U v k
MLP Neural Nets*, GLM*, LM, PCA,
k-Means, NMF, LMF (*Spark base)
K BON_MO B
Open-source R packages via Map-Reduce
, ¥KMVO 42 KN
i Bu
20

Hadoop : Map-Reduce , Spark-based
Logistic Regression ORAAH
Regression
* Linear Regression
Classification
MLP Neural Networks ORAAH
Attribute Importance
* Principal Components Analysis
Clustering
Hierarchical k-Means - MR
Feature Extraction
Non-negative Matrix Factorization MapReduce
Collaborative Filtering (LMF) MapReduce
21
Basic Statistics
Correlation/Covariance Map-Reduce
GLM ORAAH Map-Reduce
* Random Forests
* Decision Trees
* Hierarchical k-Means
Principal Components Analysis Map-Reduce
* Ridge Regression
* Support Vector Machines
* Support Vector Machines * Random Forest
* LASSO

i
22

• KN==D g i
• B i N
– NOmn]bh •
– N
•
– D@BO + D ]hhi eb^gm
– Dbo^ + F@>
– Oi kd + Oi kd eb^gm
– Kk e^ @ m [ l^ + Ojhhi* KHD* K E
23

R (ORAAH 2.5.1 )
B11 B i
– Kk e^ N @blmkb[nmbhg • ~
– KN D ~
24
B KMUKQO
B3 B11 i
B5LK O B LK O B5 ¥W uV u
B5M WW X B5 B11
B5 K B K B5 ¥O P¥KWO i
B3 M ¥O KN # K BON_MO B11 v f RKN ¥_X# ¥MR UOc KV
1CC ¥KMVO Ih]^kg =iieb^] Om mblmb l pbma O* 2ma A]bmbhg ~
¥:K K ¥KMVO & Kk e^ ~ ' N F o ~ ~
&KPJ ~ lniihkmbg ~ '
nVSL¥K¥c B3 v i
n i v vp

:ORAAH (2.6)
26
ORCH
OREbase
OREstats ORCHcoreORCHstats
OREcommon methods utils stats MASS rJava RJDBC
ORCHtestkit

R Client
Oracle Database Server Hadoop Cluster
R
(ORD/OSS R) Java VMDB Client
Oracle
R Packages
+rJava,RJDBC..
R
(ORD/OSS R)
Oracle
R Enterprise
Package R
(ORD/OSS R)
Oracle
R Packages
Hadoop
JDBC
Spark Client
Oracle RDBMS
Java VM
Hadoop
HDFS
Hive
Spark
n B4 ¥KMVO B 4S ¥SL_ S X

• KN
• 42
– i i vp
ORAAH
R
¥KMVO B 5X O¥ ¥S O B5 n ¥O M XXOM hk^ - @> ~
¥KMVO KNO¥ P ¥ KN ¥MR M XXOM a]_l,qq* hk a 46Cu 42 ~ ~
C ¥MR M XXOM a]_l,qq* hk a @>-D@BO ~
28
R
46C a]_l,qq * hk a
S O ¥O M XXOM hk^ -
C K¥U K¥U M XXOM li kd,qq *hk a,fe
HDFS
Spark Hive
R
¥KMVO B 5X O ¥S O Kk e^ =]o g ^]
=g ermb l @> & '
,
Kk e^ Hh ]^k _hk D ]hhi D ]hhi
Kk e^ @> ~
KN==D

ore.frame
• S O 42 i ¥O P¥KWO
– ~ ~ DERA ~
– ] m ,_k f^
• ¥O P¥KWO ] m ,_k f^ ln[ e ll
29
B5 ( ( i i 1 ¥KMVO B 5X O¥ ¥S O i B t B
R 0 N M ¥KMVO M W MN 5,/(,(J'( N M ( ( O,..)' K K P R W 39 6:116
B5 ( , E O¥ 7_SNO B O¥K ¥ KXN 6_XM S X C_ ¥ ON Lc ¥KMVO B 5X O¥ ¥S O
ammil8--]h l,hk e^, hf- ]-A45600Y./-KNAQC-CQE@+ 1B.76B.+AA7B+23 >+>7B3+ 6@2=B741@7.,amf!KNAQC344
Client R
ore.frame DBSQL
> class(TITANIC01)
[1] "ore.frame"
attr(,"package")
[1] "OREbase"
> colnames(TITANIC01)
[1] "Class" "Sex" "Age" "Survived" "Freq"
> mean(TITANIC01$Freq)
[1] 68.78125
B5 i CA
y r r

Oracle
•
30
> ore.create(iris,table="iris_hive2")
Warning message:
In ore.create(iris, table = "iris_hive2") :
column names modified by "ore.make.names" function
> colnames(iris_hive2)
[1] "sepal_length" "sepal_width" "petal_length" "petal_width" "species"
> class(iris_hive2$species)
[1] "ore.factor"
attr(,"package")
[1] "OREbase"
¥O MRK¥KM O¥ & '
¥O NK O
¥O NK O SWO
¥O NSPP SWO
¥O PKM ¥ ~
¥O P¥KWO ~ ~
¥O SX OQO¥
¥O VS
¥O V QSMKV
¥O WK ¥Sb
¥O X_WO¥SM
¥O LTOM
¥O OM ¥
> test.mat <- matrix(1:16,nrow=4,ncol=4)
> test.mat
[,1] [,2] [,3] [,4]
[1,] 1 5 9 13
[2,] 2 6 10 14
[3,] 3 7 11 15
[4,] 4 8 12 16
> test.mat.hive <- ore.push(test.mat,table="hive_mat")
> test.mat.hive
[,1] [,2] [,3] [,4]
[1,] 1 5 9 13
[2,] 2 6 10 14
[3,] 3 7 11 15
[4,] 4 8 12 16
> class(test.mat.hive)
[1] "ore.vecmatrix"
attr(,"package")
[1] "OREbase"
hive
Matrix

~
31

ORCH
• B11 ~ B3 ~
– KN D ~ Kk e^ N =]o g ^] =g ermb l _hk D ]hhi&KN==D'
• @^[n ~
– –
32
library(ORCH)>
n B B3 i i
KN v
> library(ORCH)
OREbase
OREcommon
: ‘OREbase’
The following objects are masked from ‘package:base’:
cbind, data.frame, eval, interaction, order, paste, pmax, pmin,
rbind, table
OREstats
MASS
ORCHcore
rJava
Oracle R Connector for Hadoop 2.5.1 (rev. 307)
Info: using native C base64 encoding implementation
Info: Hadoop distribution is Cloudera's CDH v5.5.1
Info: using auto-detected ORCH HAL v4.2
Info: HDFS workdir is set to "/user/oracle"
Warning: mapReduce checks are skipped due to "ORCH_MAPRED_CHECK"=FALSE
Warning: HDFS checks are skipped due to "ORCH_HDFS_CHECK"=FALSE
Info: Hadoop 2.6.0-cdh5.5.1 is up
Info: Sqoop 1.4.6-cdh5.5.1 is up
Info: OLH 3.5.0 is up
Info: Hive 1.1.0-cdh5.5.1-standalone is up
Info: loaded ORCH core Java library "orch-core-2.5.1-mr2.jar"
ORCHstats
orch.dbg.on("all")
orch.debug(T)
>
>

KN
¥KMVO 42
HIVE
HDFS
Spark
~
D ]hhi | ~ N
– dfs.id
/user/hoge/
RS O
RDD

ORE/ORCH
34
KN
¥KMVO 42
HIVE
HDFS
Spark
( Д ≡ Д )
spark.connect()
&hk^, mm a,,'
ore.connect() by ORE
orch.connect() by Sqoop
orch.connect() by OLH
ore.connect()

• I iN^]n ^ N
• KN==D
• Oi kd Ieb[
35
• D@BO
• Dbo^
• Oi kd
• ~

• I iN^]n ^ N
• KN==D
• Oi kd Ieb[
36
• D@BO
• Dbo^
• Oi kd
• ~

HDFS
37

HDFS
38
KN
¥KMVO 42
HIVE
HDFS
Spark
x <- hdfs.attach("/user/oracle/")
& y ' z
/user/hoge/.
..

HDFS
39
KN
¥KMVO 42
HIVE
HDFS
Spark
*)
/user/hoge/.
..

ORE/ORCH
40
KN
¥KMVO 42
HIVE
HDFS
Spark
hdfs.ls()
& y ' z
/user/hoge/.
..

ORE/ORCH
41
KN
¥KMVO 42
HIVE
HDFS
Spark
& y ' z
/user/hoge/.
..
~
~ ~
D@BO ~
D@BO N –

: HDFS )
RNP cXM B3
RNP K KMR 46C i i B11 f
RNP NOVSW 46C UOc i # KV_O i e
RNP UOc O 46C UOc i e NOPK_V I
RNP KV_O O 46C KV_O i e NOPK_V I
RNP MKMRO B3 X PP
RNP aN 46C w
RNP MaN 46C w
RNP MN 46C f
RNP WUNS¥ 46C
RNP ¥WNS¥ 464C
RNP ¥ 46C i
RNP O ¥ 46C i
42

:HDFS ( , )
RNP V i 46C f
RNP WO K 46C B3 i e
RNP ObS v 46C z f
RNP SN 46C B3 NP SN f
RNP M 46C i f
RNP W 46C f
RNP ¥W 46Cu f
RNP SdO 46C f
RNP K¥ 46C f
RNP XM V 46C
RNP X¥ a 46C
RNP NO M¥SLO 46C y B i f
RNP NSW 46C NSW
43

: HDFS ( , )
i RNP VO OV 46C i VO OV
RNP KW VO 46C i
RNP MVOKX_ 9X _ 46C i 1 i
RNP ROKN 46C u e i f
RNP KSV 46C u e i f
i i RNP _ B B 46C i
RNP QO 46C i B i
RNP B4K K B i B4K K 46C
RNP P¥ WB4K K B4K K 46C B
RNP _ V KN B i p 46C i
RNP N aXV KN 46C i B i i
44

> hdfs.mkdir("oratest")
[1] "/user/oracle/oratest"
attr(,"orch.isdir")
[1] TRUE
attr(,"orch.dfs.path")
[1] TRUE
> hdfs.cd("oratest")
[1] "/user/oracle/oratest"
attr(,"orch.isdir")
[1] TRUE
[1] TRUE
> hdfs.ls()
NULL
> hdfs.put(iris,dfs.name="iris.hdfs")
[1] "/user/oracle/oratest/iris.hdfs"
attr(,"orch.dfs.id")
[1] TRUE
> hdfs.ls()
[1] "iris.hdfs"
> hdfs.head("iris.hdfs",5)
[1] "5.1,3.5,1.4,0.2,setosa" "4.9,3,1.4,0.2,setosa" "4.7,3.2,1.3,0.2,setosa"
[4] "4.6,3.1,1.5,0.2,setosa" "5,3.6,1.4,0.2,setosa"
• hk m^lm
• hk m^lm
•
• bkbl bkbl,a]_l D@BO ~
•
• bkbl,a]_l 3

HDFS
46C D ]hhi
–
–Oi kd Dbo^ ~ –
– ~
– I iN^]n ^ I ahnm N
~ ~ •
46

Hive
47

Hive
48
KN
¥KMVO 42
HIVE
HDFS
Spark
ore.connect(
host="localhost",
port=10000, user="hive",
schema="default",
type="HIVE")
& y ' z

Hive
49
KN
¥KMVO 42
HIVE
HDFS
Sparkcars
*)

Hive
ore.create
50
KN
¥KMVO 42
HIVE
HDFS
Sparkkl
ore.create(
cars,
table="hive_cars")
& y ' z

Hive
:ore.create ore.push
51
KN
¥KMVO 42
HIVE
HDFS
Sparkkl
RS OJMK¥
hk^, k^ m^
abo^Y kl Dbo^
abo^Y kl Dbo^ hk^,_k f^ N
hk^,inla hk^,_k f^
hive_cars

ORCH
52
KN
¥KMVO 42
HIVE
HDFS
Sparkkl
RS OJMK¥
> ore.connect(type="HIVE")
> ore.create(cars, table="hive_cars")
> summary(hive_cars)
speed dist
Min. : 4.00 Min. : 2.00
1st Qu.:11.38 1st Qu.: 24.25
Median :14.67 Median : 35.00
Mean :15.40 Mean : 42.98
3rd Qu.:18.83 3rd Qu.: 55.50
Max. :25.00 Max. :120.00
> class(hive_cars)
[1] "ore.frame"
attr(,"package")
[1] "OREbase"
hive_cars
n B3 B5 ¥O P¥KWO s
t

) Hive R
¥O MRK¥KM O¥
g a k* mhehp^k* mhnii^k* l^_he]* ln[* ln[lmk*
ln[lmkbg
¥O P¥KWO
lahp* mm a* V* "* "9+* VV* VV9+* a^ ]* m be* e^g ma*
gkhp* g he* JNKS* J KH* ]bf* g f^l* g f^l9+*
heg f^l* heg f^l9+* l,eblm* ngeblm* lnff kr* k[bg]*
[bg]* ] m ,_k f^* l,] m ,_k f^* l,^go* ^o e* )* +*
(* X* ##* #-#* -* hfi k^* Hh b * * qhk* bl,g *
bl,_bgbm^* bl,bg_bgbm^* bl,g g* [l* lb g* ljkm* ^bebg * _ehhk*
mkng * eh * eh /.* eh 0* eh /i* eh [* hl* lbg* m g*
^qi* ^qif/* hl* lbg* m g* khng]* Onff kr*
khpOnfl* heOnfl* khpI^ gl* heI^ gl* ngbjn^* [r*
f^k ^
KNAOm ml8 _bo^gnf* k^ m^* jn gmbe^* l]* o k&hger
_hk o^ mhkl'* f^]b g* EMN
¥O OM ¥
lahp* e^g ma* * bl,o^ mhk* l,o^ mhk* l, a k m^k* l,gnf^kb *
l,bgm^ ^k* l,eh b e* V * V9+ * E* hfi k^* hk^,k^ h]^* bl,g *
#bg# * ngbjn^* lhkm* m [e^* i lm^* m iier* [r* a^ ]* m be
¥O V QSMKV
9* * ::* 9:* :* * qhk* b_^el^* g]* hk
¥O X_WLO¥
)* +* (* X* ##* #-#* -* bl,_bgbm^* bl,bg_bgbm^* bl,g g* [l* lb g*
ljkm* ^bebg * _ehhk* mkng * eh * eh /.* eh 0* eh /i* eh [* hl* lbg
m g* ^qi* ^qif/* hl* lbg* m g* s ilf ee* khng]* Onff kr*
lnff kr* f^ g
i e
hk^, k^ m^* hk^,]khi* hk^,inla* hk^,inee* hk^, ^m
e
bl,hk^,_k f^* bl,hk^,o^ mhk* bl,hk^,eh b e* bl,hk^,bgm^ ^k*
bl,hk^,gnf^kb * bl,hk^, a k m^k* bl,hk^* l,hk^,_k f^*
l,hk^,o^ mhk* l,hk^,eh b e* l,hk^,bgm^ ^k* l,hk^,gnf^kb *
l,hk^, a k m^k* l,hk^

> ore.connect(host="localhost",port=10000,user="hive",schema="default",type="HIVE")
> ore.sync()
> ore.attach()
> ore.ls()
[1] "cmnt" "cust" "datx2_hive"
[4] "datx2_tohive" "datx_hive2"
> ore.create(cbind(iris,id=1:150) ,table="iris_hive")
The following object is masked _by_ .GlobalEnv:
iris_hive
Warning message:
In ore.create(cbind(iris, id = 1:150), table = "iris_hive") :
column names modified by "ore.make.names" function
> nrow(iris_hive)
[1] 150
> colnames(iris_hive)
[1] "sepal_length" "sepal_width" "petal_length" "petal_width" "species"
[6] "id"
> row.names(iris_hive) <- iris_hive$id
> summary(iris_hive$sepal_length)
Min. 1st Qu. Median Mean 3rd Qu. Max.
4.300 5.100 5.800 5.843 6.400 7.900
> iris_hive$newcol <- 150:1
> head(iris_hive,1)
sepal_length sepal_width petal_length petal_width species id newcol
1 5.1 3.5 1.4 0.2 setosa 1 150
• Dbo^
• ~
• ~
• Dbo^
• E@ khp,g f^l
• l^i eYe^g ma lnffkr
• bkblYabo^ ] m ,_k f^ g^p he
• bkbl E@ bkblYabo^
~
• gkhp
• heg f^l

Access previously created HIVE tables
> ore.sync(table=“tab_input")
> ore.sync(table=“tab_input2")
List the HIVE tables
> ore.ls()
HIVE
Oracle R Advanced Analytics
for Hadoop Client Packages
HIVE
Transparency Engine
4
/user/oracle/tab_input
HDFS Storage
HDFS Storage
3
HIVE
Thrift
Server
1
HQL
Metastore
Metastore
Metastore
Metastore
2
[1] “tab_input" “tab_input2"
Check variable names on the files
> names(tab_input)
[1] “target" "v1" "v2" "v3" "v4" "v5" "v6" "v7" "v8"
[10] "v9" "v10" "v11" "v12" "v13" "v14" "v15" "v16" "v17"
[19] "v18" "v19" "v20" "v21"
> names(tab_input2)
[1] “target" "v1" "v2" "v3" "v4" "v5" "v6" "v7" "v8"
[10] "v9" "v10" "v11" "v12" "v13" "v14" "v15" "v16" "v17"
[19] "v18" "v19" "v20" "v21"
Oracle Distribution of R version 3.2.0 (--) -- "Full of Ingredients"

Join the two tables by one common variable
> joined <- merge(tab_input,tab_input2,by="value")
JOIN
HIVE
HIVE
Transparency Engine
The new table is temporary, so it’s not pushed to HIVE
> ore.ls()
[1] “tab_input" “tab_input2"
But, it’s part of the local R objects
> ls()
[1] "joined"
> names(joined)
[1] "value" "v1.x" "v2.x" "v3.x" "v4.x" "v5.x" "v6.x" "v7.x" "v8.x" "v9.x"
[11] "v10.x" "v11.x" "v12.x" "v13.x" "v14.x" "v15.x" "v16.x" "v17.x" "v18.x" "v19.x"
[21] "v20.x" "v21.x" "v1.y" "v2.y" "v3.y" "v4.y" "v5.y" "v6.y" "v7.y" "v8.y"
[31] "v9.y" "v10.y" "v11.y" "v12.y" "v13.y" "v14.y" "v15.y" "v16.y" "v17.y" "v18.y"
[41] "v19.y" "v20.y" "v21.y"
4
/user/oracle/tab_input
HDFS Storage
HDFS Storage
3
HIVE
Thrift
Server
1HQL
Metastore
Metastore
Metastore
Metastore
2
56

Hive
S O ~ ] m ,_k f^ I mkbq
– NK K P¥KWO
&N ] m ,_k f^ Dbo^ – '
57

Spark
58

Spark
59
KN
¥KMVO 42
HIVE
HDFS
Sparkspark.connect(
master="yarn-client",
memory="4G",
dfs.namenode="bigdatalite.localdomain"
)
& y ' z
/user/hoge/k
yp.hdfs

Spark
60
KN
¥KMVO 42
HIVE
HDFS
Spark
& y '
/user/hoge/k
yp.hdfs

Spark
61
KN
¥KMVO 42
HIVE
HDFS
Sparkiris
( Д ) !!
/user/hoge/k
yp.hdfs
kyp.rdd <- hdfs.toRDD("kyp.hdfs")

Spark
62
KN
¥KMVO 42
HIVE
HDFS
Sparkkyp.rdd
kyp.rdd
(p ω p) m m *
/user/hoge/k
yp.hdfs

Spark
63
KN
¥KMVO 42
HIVE
HDFS
Sparkkyp.rdd
kyp.rdd
/user/hoge/k
yp.hdfs

Spark
64
KN
¥KMVO 42
HIVE
HDFS
Sparkiris.rdd
iris.rdd
/user/hoge/i
ris_hdfs
C K¥U B44 y s
p f
Σ(ll ll) !!

Spark
65
KN
¥KMVO 42
HIVE
HDFS
Sparkiris.rdd
iris.rdd
/user/hoge/i
ris_hdfs
C K¥U B44 y s
p f
z C K¥U VSL
¥KMVO f
Σ(ll ll) !!

Spark
66
KN
¥KMVO 42
HIVE
HDFS
Sparkiris
( Д ) !!
/user/hoge/k
yp_hdfs
kyp.fit <- orch.glm2(
Kyphosis ~ Age + Number + Start,
dfs.dat = kyp.hdfs)

> spark.connect(master="yarn-client",memory="1G",dfs.namenode="bigdatalite.localdomain")
> spark.connected()
[1] TRUE
> spark.session()
Reference class object of class ".orch.SparkSession"
Field ".jRef":
[1] "Java-Object{com.oracle.orch.spark.Session@4a901445}"
Field ".jSig":
[1] "Lcom/oracle/orch/spark/Session;"
Field ".stop":
[1] TRUE
Field ".error":
[1] FALSE
Field ".active":
[1] TRUE
> kyp.hdfs <- hdfs.put(kyphosis)
> kyp.rdd <- hdfs.toRDD(kyp.hdfs)
> kyp.rdd
[1] "/tmp/orch66e040bd0859"
[1] TRUE
attr(,"orch.spark.rdd")
[1] "Java-Object{com.oracle.orch.spark.TextRDD@784d9bc}"
> kyp.fit <- orch.glm2(Kyphosis ~ Age + Number + Start, dfs.dat = kyp.hdfs)
• Oi kd
•
•
• driahlbl a]_l ~ ]_l,b]
dri,a]_l
• dri,a]_l Oi kd ~ N@@
dri,k]]
• dri,k]]
• hk a, ef0 ef & N@@
'

Spark
Oi kd D@BO ~
~
– N
– N
68

HDFS -> Oracle DB
70
KN
¥KMVO 42
HIVE
HDFS
Spark
& y ' z
orch.connect(
"oreuser","ora12c"
,"shiva", port=1521,
driver ="olh")
/user/hoge/c
ars.dfs

HDFS -> Oracle DB
71
KN
¥KMVO 42
HIVE
HDFS
Spark
( ω )
/user/hoge/c
ars.dfs
¥KMVO KNO¥
P ¥ KN

HDFS -> Oracle DB
72
KN
¥KMVO 42
HIVE
HDFS
Spark
& y ' z
/user/hoge/c
ars.dfs
¥KMVO KNO¥
P ¥ KN
hdfs.pull(
hdfs.get("cars.dfs"),
db.name="CARS_OLH") )

HDFS -> Oracle DB
73
KN
¥KMVO 42
HIVE
HDFS
Spark
/user/hoge/c
ars.dfs
¥KMVO KNO¥
P ¥ KN
31BCJ

> orch.connect("oreuser","ora12c","shiva",port=1521, driver ="olh")
Connecting ORCH to RDBMS via [olh]
Host: shiva
Port: 1521
SID: ora12c
User: oreuser
Enter password for [oreuser]: ********
Connected to database "shiva:ora12c".
> hdfs.pull(cars.dfs,db.name="CARS_OLH")
> orch.connect("oreuser","ora12c","shiva",port=1521, driver ="sqoop")
Connecting ORCH to RDBMS via [sqoop]
Host: shiva
Port: 1521
SID: ora12c
User: oreuser
Enter password for [oreuser]:******
Connected to database "shiva:ora12c".
> cars.dfs2 <- hdfs.push("CARS_TABLE")
>
> hdfs.toHive(cars.dfs2,table="cars_hive")
• @> KHD
• a]_l,inee D@BO ~
~ ~
• Ojhhi @>
• a]_l,inla =NOYP=>HA D@BO
~
• D@BO kl,]_l0 klYabo^
DERA ~

:
75
i i RNP _ N N D@BO ~
RNP QO D@BO ~ N ~
RNP B4K K N ~ N@ m D@BO
RNP P¥ WB4K K N@ m D@BO N
RNP _ V KN N ~ D@BO ~
RNP N aXV KN D@BO ~ N ~ ~
RNP _ R ~ ~ ~ D@BO ~
RNP _VV D@BO ~ ~ ~ ~
RNP S O D@BO Dbo^ ~
RNP P¥ W S O Dbo^ D@BO ~
RNP B44 D@BO Oi kd N@@ ~ ~

HDFS
76
KN
Oracle
Database
hdfs.pull
hdfs.push
hdfs.toHive
hdfs.fromHive
R Client
R Client
Local Disk
hdfs.downloadhdfs.upload
hdfs.toRDD
hdfs.toRData
hdfs.put
hdfs.get
hdfs.fromRData

46C
• B
• i
• S O
• C K¥U
• ¥KMVO 42
~ B –
77

• I iN^]n ^ N
• KN==D
• Oi kd Ieb[
78
• D@BO
• Dbo^
• Oi kd
• ~

MapReduce by R
79

Map-Reduce by R
SFO
ontime <- ore.pull(ONTIME_S[ONTIME_S$YEAR==2000,])
ontime.dfs <- hdfs.put(ontime, key='UNIQUECARRIER' )
res <- NULL
res <- hadoop.run(
ontime.dfs,
mapper = function(key, ontime) {
if (ontime$DEST == 'SFO') {
keyval(key, ontime)
}
},
reducer = function(key, vals) {
sumAD <- 0; count <- 0
for (x in vals) {
if (!is.na(x$DISTANCE)) {
sumAD <- sumAD + x$DISTANCE; count <- count + 1
}
}
if (count > 0) { res <- sumAD / count }
else {res <- 0 }
keyval(key, res)
}
)
hdfs.get(res)
key val1
1 AA 1361.4643
2 AS 515.8000
3 CO 2507.2857
4 DL 1601.6154
5 HP 549.4286
6 NW 2009.7273
7 TW 1906.0000
8 UA 1134.0821
9 US 2387.5000
10 WN 541.1538

Hadoop(Map-Reduce)
R MapReduce
RKN ObOM D ]hhi ~ ~ N
~ D@BO ~
RKN T L D ]hhi –
RKN ¥_X D ]hhi ~ ~ N ~
D@BO a ]hhi,kng ~ D@BO ~
¥MR N¥c¥_X ~ D ]hhi ~ N ~
¥MR Ob ¥ ~ ~ ~ N N D ]hhi –
I iN^]n ^ –
¥MR UOc KV I iN^]n ^ ~
¥MR KMU ~ ~ ~ / ~N
¥MR _X KMU hk a,i d N
¥MR M¥OK O K¥ KL ~ Dbo^ KN D I iN^]n ^ ~ ~ –
¥MR OW K R ~
81

SUb 5' WSUb(_dc"XaXb Th6 H_TRXTb #
aTb 5' C AA
SUb(aTb 5' WPS _(ad "
SUb
P__Ta 6 Ud RcX " Th eP b# i
TheP " Th eP b#
k
aTSdRTa 6 Ud RcX " Th eP b# i
SPc 5' S (RP "aQX S(SPcP(UaP T eP b#
aRW(S Ve"R P Tb"SPc##
S 6 "ETcP (AT VcW l HT_P (AT VcW ETcP (LXScW SPcP6SPc#
U P T 5' _PbcT" UXc' Th (_ V bT_6 #
_ V"U P T#
_Pa" Ua f6R", ,# RTg6*(0 Pa6R"0 0 0 .# Tg6*(2#
_ c" S XS( 6+ RTg(RP_cX 6*(2 fWXRW6+4.#
STe( UU"#
WSUb(USXa 5' )dbTa)_ VUX Tb
WSUb(U P T 5' _PbcT"WSUb(USXa ) U P T bT_6 #
bhbcT "_PbcT" WPS _ Ub 'R _h=a A RP U P T WSUb(USXa##
_aTS 5' _aTSXRc" S SPc#
TheP "C AA aRW(_PR "_aTS WSUb(U P T##
k
#
Oracle RAAH
Client Packages
Map/Reduce Call
res <- hdfs.get(dfs.res)
finalres = list()
for (i in 1:nrow(res))
{ finalres[[i]] <-
orch.unpack(res[i,])}
/user/oracle/iris
Mapper(s)
Reducer(s)
R Result Object
Stored in HDFS
2
5
lm
Map-Reduce R
1
4
YARN: Hadoop
Map Reduce Job
82

hadoop.run
q
• D ]hhi Omk^ fbg
83
sh -c /usr/lib/hadoop/bin/hadoop jar /usr/lib/hadoop-mapreduce/hadoop-streaming.jar
-libjars /u01/app/oracle/product/12.1.0.2/dbhome_1/R/library/ORCHcore/java/orch-core-2.5.1-mr2.jar
-D stream.io.identifier.resolver.class=com.oracle.orch.streaming.io.NoSplitIdentifierResolver
-D mapred.job.name=ORCH_20160414091432-1
-input /tmp/orch66c568b00bbe
-output /tmp/orch66c58105bce
-mapper "/usr/bin/env R --vanilla --quiet --slave --no-save --no-restore -e
¥"source('orch66c57801bd90.R')¥" --args -m"
-reducer "/usr/bin/env R --vanilla --quiet --slave --no-save --no-restore -e
¥"source('orch66c57801bd90.R')¥" --args -r"
-file /tmp/orch66c57801bd90.R
-file /u01/app/oracle/product/12.1.0.2/dbhome_1/R/library/ORCHcore/libs/ORCHcore.so
-cmdenv R_HOME= -cmdenv R_PROFILE_USER= -cmdenv HADOOP_HOME= 2>&1; echo "?$?"
Hadoop Streaming

Map Reduce by R
K BON_MO B we
KN i z v
w
B11 i i
w
84

ORAAH
85

ORAAH
• KN i
– I i+N^]n ^&I ahnm' *Oi kd
– ~ D@BO Dbo^
– Oi kd I iN^]n ^
• Oi kd Oi kd •
a ]hhi,kng
86

• K BON_MO
– &hk a, ho* hk a, hk'
– &hk a,ikbg hfi* hk a,ik^]b m'
– G+f^ gl &hk a,df^ gl*
hk a,ik^]b m'
– &hk a,ef* hk a,ik^]b m'
– &hk a, ef'
– &Hhp N gd I mkbq
B mhkbs mbhg'
&hk a,ef_'
– &hk a,gf_'
• C K¥U
– ~ ~ &hk a,g^nk e'
– &hk a, ef0'
– C K¥U VSL
• &hk a,fe,]m'
• &hk a,fe,e^ g^k'
• H llh &hk a,fe,e llh'
• Nb] ^ &hk a,fe,kb] ^'
• &hk a,fe,k g]hf,_hk^lm'
• ORI&hk a,fe,lof'
• df^ gl&hk a,fe,df^ gl'
• &hk a,fe,eh blmb '
• &hk a,fe,i '
87
C K¥U
2.6
NEW

Hadoop
orch.princomp
88
> USARRESTS <- hdfs.put(USArrests)
> arrestsModel <- orch.princomp(USARRESTS, cor = TRUE)
> arrestsModel
Call:
princomp(cor = cor, covmat = list(cov = cov, center = center,
n.obs = n.obs))
Standard deviations:
Comp.1 Comp.2 Comp.3 Comp.4
1.5748783 0.9948694 0.5971291 0.4164494
4 variables and 50 observations.
> res <- orch.predict(arrestsModel, USARRESTS)
> hdfs.head(res,3)
[1] "-0.985565884503144,1.13339237770997,0.444268787550733,0.156267144919712"
[2] "-1.95013775033503,1.07321325616849,-2.04000333289159,-0.438583439947189"
[3] "-1.76316353972298,-0.745956780637291,-0.0547808243262842,-0.834652924308098"
• a]_l,inm QO=kk^lml ~
D@BO –
• hk a,ikbg hfi
D ]hhi
•
• ik^]b m
• 1

Mappers
Mappers
Mappers
Mappers
Invoke ORAAH custom parallel distributed model (Linear Regression)
HDFS Hadoop
7 cX T 5' WSUb(PccPRW" )dbTa) aPR T) cX TOb #
7 O S 5' aRW( "8 <A8N l HI8C:< <E <A8N
SUb(SPc6 cX T BP__Tab 6 .
TSdRTab 6 ,#
Machine Learning
algorithms module
7 bd Pah" O S#
:P 4
aT( "U a d P 6 8 <A8N l HI8C:< <E <A8N SPcP 6 DCI B<OH#
TbXSdP b4
BX +F BTSXP -F BPg
'+.0,(./ '0(31 '+(-0 /(*1 3,/(*2
: TUUXRXT cb4
<bcX PcT HcS( <aa a c eP dT Ea"7jcj#
" cTaRT_c# ,(,/.T'*+ /(+31T'*, .(--0 +(./T'*/
HI8C:< '+(,+2T'*- /(2*-T'*/ ',*(313 5 ,T'+0
<E <A8N 3(0,/T'*+ +(+/+T'*- 2-0(,23 5 ,T'+0
'''
HXV XU( R STb4 * n o *(**+ n o *(*+ n o *(*/ n(o *(+ n o +
TbXSdP bcP SPaS Taa a4 +.(1- ,+/+.. STVaTTb U UaTTS
".12/ QbTaePcX b ST TcTS SdT c XbbX V Tbb#
Bd cX_ T 'b dPaTS4 *(10.1 8SYdbcTS 'b dPaTS4 *(10.1
='bcPcXbcXR4 -(.31T */ , P S ,+/+.. = _'eP dT4 5 ,(,T'+0
2
/user/oracle/ontime_s
YARN: Hadoop
Map Reduce Job
1
4
Custom Java
Algorithm ReducersCustom Java
Algorithm Reducers
3
89

R Hadoop
• N D ]hhi ~
ik^]b m –
– ikbg hfi* df^ gl
90
> irisModel <- princomp(~ Sepal.Length + Sepal.Width + Petal.Length + Petal.Width,
data = iris)
> iris.hdfs.put <- hdfs.put(iris)
> iris.hdfs.pred <- orch.predict(irisModel, iris.hdfs.put)
> hdfs.head(iris.hdfs.pred,5)
[1] "-2.68412562596953,-0.319397246585102,0.0279148275894149,0.0022624370713179,setosa"
[2] "-2.71414168729432,0.17700122506478,0.210464272378243,0.0990265503235873,setosa"
[3] "-2.88899056905929,0.144949426085557,-0.0179002563208903,0.019968389709029,setosa"
[4] "-2.74534285564141,0.318298979251915,-0.0315593736056814,-0.0755758166136827,setosa"
[5] "-2.72871653655453,-0.326754512934921,-0.0900792405512,-0.0612585925856928,setosa"
• N ikbg hfi
• &D@BO ~ –'
• N ikbg hfi D@BO ~ ik^]b m

&LMF by mahout als)
• z
91
> u <- sample(1:100, 300, replace=TRUE)
> i <- sample(1:10, 300, replace=TRUE)
> ui <- unique(cbind(u,i))
> r <- sample(1:5, nrow(ui), replace=TRUE)
> input <- cbind(ui,r)
> inputFile <- ORCHcore:::.orch.tmpfile()
> write.table(input, file=inputFile, sep=",", col.names=FALSE, row.names=FALSE)
> fit <- orch.lmf(inputFile, method="mahout-als", rank=3, iterations=5)
> fit
Input HDFS Directory : /tmp/orch6f9c5c0fbbf7
Model HDFS Directory : /user/oracle/orch6f9c7790aff3
> class(fit)
[1] "orch.mahout.lmf.als" "orch.lmf"
> orch.recommend(fit, n=2, maxRating=5)
[1] "/user/oracle/orch6f9c2ddd6da4"
[1] TRUE
• ~
•
• pkbm^,m [e^ ~
• hk a,ef_ I ahnm-=HO HIB
~
• hk a,k^ hffg^] ~

( )
• Number of obs 155671
Number of columns 46 columns
Missing values yes
# hidden Elapsed time Elapsed time
neurons (sec) nnet (sec) orch.neural
10 934.176 44.181
20 1861.812 44.969
30 2634.434 35.196
40 3674.379 39.217
50 4400.551 49.527
• Hardware spec: Single BDA node
MemTotal: 49GB
CPUs: 24 (3058MHz each)
• XXO
–
• ¥MR &Kk e^ N =]o g ^] =ge rmb l _hk D ]hhi'
– i v( ,72 h i
e ¥MR XO_¥KV i
– ~ I iN^]n ^
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
10 20 30 40 50
()
Hidden Layer Node
nnet
orch.neural
N
Oracle R Advanced
Analytics for Hadoop
Spark

Oracle R Advanced Analytics for Hadoop vs. Rhadoop (RMR)
G0 - X NO 241 H ) # (- M ¥O # . 72 B1 b - i
('' 72 46C )'' i
93
https://blogs.oracle.com/R/entry/oraah_enabling_high_performance_r

~ D@BO ~ ,
ORAAH’s Spark-based GLM vs. Spark MLlib GLM
14.5x
13.9x
5x
Oracle Advantage
Oracle Advantage
Oracle Advantage
Oi kd Ieb[ 42C>
KN==D CHI 0C>

v 623 p^b aml
ORAAH’s Map Reduce GLM vs ORAAH’s Spark-based GLM
34 , '#
>@= T1+0 & 02 hk^l * 74 C> N=I' q 4 ~
C K¥U ( ) '
>@= T1+0 &02 hk^l * 02 C> N=I ' q 4 ~
7 C K¥U v

ORAAH
e Bu
KN i w
• pq vqq
– q z q
– O_¥KV O 7 C K¥U vp vt
97

Spark Mlib
98

ORAAH Spark Mlib
ORAAH 2.6
C K¥U VSL
o B i
N
99
orch.ml.kmeans(formula = ~ Number + Age, data = data)
predict(model, newdata = data, supplemental = c("Kyphosis", "Age"))

> spark.connect( master="yarn-client", memory="1G",dfs.namenode="bigdatalite" )
> kyp.mod <- orch.ml.kmeans(formula = ~ Number + Age, data =kyp.dfs)
OBX Model Matrix: created MLlib Vector RDD (81 rows) 0.014 sec
OBX Machine Learning: MLlib K-means elapsed time 4.284 sec
> kyp.mod
$formula
[1] "~Number + Age"
$predictMetadata
[1] "Java-Object{com.oracle.obx.csv.CSVPredictMetadata@5cf1bbd3}"
$predictor
[1] "Java-Object{com.oracle.obx.ml.MLKMeans@204a02a4}"
$technique
[1] "MLlib K-means"
attr(,"class")
[1] "orch.ml.kmeans"
> kyp.pred <- predict(kyp.mod, newdata=kyp.dfs, supplemental = c("Kyphosis", "Age"))
OBX Model Matrix: created predict RDD (81 rows) 0.030 sec
> hdfs.write(kyp.pred,outPath="kyp_pred_km")
> hdfs.attach("kyp_pred_km")
Info: sampling data types of hdfs:"/user/oracle/kyp_pred_km"
Data class : data.frame
Column types: character, numeric, integer
Column count: 3
Has key : no
Pristine : yes
Key delim :
Value delim : ,
[1] "/user/oracle/kyp_pred_km"
[1] TRUE
• li kd, hgg^ m
• hk a,fe,df^ gl
•
• ik^]b m
• Oi kd D@BO pkbm^
• D@BO KN D ~

Invoke ORAAH custom interface to Spark Mllib algorithms within R
Lasso by Spark MLlib (orch.ml.lasso)
7 : TRcb c H_Pa
7 b_Pa (R TRc" hPa 'R XT c T ah6 ,.V #
7 8ccPRWTb cWT > =H UX T U a dbT fXcWX
7 :P dbT TXcWTa > =H X _dc SPcP a > K< cPQ Tb
7 SPcP 5' WSUb(PccPRW" )dbTa) aPR T) h_W bXb #
9dX SX V P A8HHD ST fXcW H_Pa BA XQ Ua P T X T U R ST X
7 ST 5' aRW( ( Pbb "U a d P 6 h_W bXb l Cd QTa 8VT SPcP 6 SPcP#
D9M B ST BPcaXg4 _a RTbbTS + UPRc a ePaXPQ Tb *(+// bTR
D9M B ST BPcaXg4 RaTPcTS BA XQ APQT TSE X c "2+ a fb# *(*+/ bTR
D9M BPRWX T ATPa X V4 BA XQ APbb T P_bTS cX T -(/2, bTR
HR aX V P A8HHD ST fXcW H_Pa B XQ Ua P T X T U R ST X
7 _aTS 5' _aTSXRc" ST TfSPcP 6 SPcP bd__ T T cP 6 R" h_W bXb 8VT ##
D9M B ST BPcaXg4 RaTPcTS _aTSXRc "2+ a fb# *(**1 bTR
Oracle Distribution of R version 3.2.0 (--) -- "Full of Ingredients" YARN: Apache
Spark Job
1
4
2
Spark MLlib Algorithm
distributed in-Memory ComputationSpark MLlib Algorithm
distributed in-Memory Computation
/user/oracle/kyphosis
3
Spark-Based Machine
Learning algorithms
module
101

Spark Mlib
Spark Mlib
¥MR WV N Oi kd Ieb[
¥MR WV VSXOK¥ Oi kd Ieb[
¥MR WV VK Oi kd Ieb[ H llh
¥MR WV ¥SNQO Oi kd Ieb[
¥MR WV V QS SM Oi kd Ieb[
¥MR WV MK Oi kd Ieb[
¥MR WV ¥KXN W P ¥O Oi kd Ieb[
¥MR WV W Oi kd Ieb[ ~
¥MR WV UWOKX Oi kd Ieb[ G+I^ gl
¥MR W NOV WK ¥Sb Oi kd fh]ê,f mkbq
¥MR V KN W NOV D@BO Oi kd Oi kd Ieb[ ~
¥MR K O W NOV Oi kd D@BO Oi kd Ieb[
¥MR ¥O K¥O W NOV WK ¥Sb Oi kd fh]ê,f kmbq
RNP a¥S O Oi kd fh]ê,f mkbq D@BO –
Oi kd ¥MR ¥O K¥O D@BO Oi kd ~
¥MR _X ¥O K¥O Oi kd fh]ê,f mkbq
102

ORAAH
C K¥U VSL Bu
s r
• ) -
103
C K¥U VSL
C K¥U VSL
C K¥U VSL K
C K¥U VSL
C K¥U VSL
C K¥U VSL
C K¥U VSL
C K¥U VSL i
C K¥U VSL OKX

B11
KN B s
S O B s
C K¥U B s
KN i w
104

B11
KN B s
S O B s
C K¥U B s
KN i w
105

ORCH/ORE
108

HDFS (ORCH )
RNP cXM B3
RNP K KMR 46C u B3 i
RNP NOVSW 46C UOc i # KV_O i e
RNP UOc O 46C UOc i e NOPK_V I
RNP KV_O O 46C KV_O i e NOPK_V I
RNP MKMRO B3 X PP
RNP aN 46C w
RNP MaN 46C w
RNP MN 46C f
RNP WUNS¥ 46C
RNP ¥WNS¥ 464C
RNP ¥ 46C i
RNP O ¥ 46C i
109

HDFS ( , )
RNP V i 46C f
RNP WO K 46C B3 i e
RNP ObS v 46C z f
RNP SN 46C B3 NP SN f
RNP M 46C i f
RNP W 46C f
RNP ¥W 46Cu f
RNP SdO 46C f
RNP K¥ 46C f
RNP XM V 46C
RNP X¥ a 46C
RNP NO M¥SLO 46C y B i f
RNP NSW 46C NSW
110

HDFS ( , )
RNP KW VO 46C i
RNP KSV 46C u e i f
i i RNP _ B B 46C i
RNP QO 46C i B i
111

Hadoop(MapReduce)
R MapReduce
RKN ObOM D ]hhi ~ ~ N ~
D@BO ~
RKN T L D ]hhi –
RKN ¥_X D ]hhi ~ ~ N ~
D@BO a ]hhi,kng ~ D@BO ~
¥MR N¥c¥_X ~ D ]hhi ~ N ~
¥MR Ob ¥ ~ ~ ~ N N D ]hhi –
I iN^]n ^ –
¥MR KMU ~ ~ ~ / ~N
¥MR _X KMU hk a,i d N
¥MR M¥OK O K¥ KL ~ Dbo^ KN D I iN^]n ^ ~ ~ –
¥MR OW K R ~
112

HDFS ( )
RNP KW VO 46C i
RNP KSV 46C u e i f
113
i i RNP _ B B 46C i
RNP QO 46C i B i
RNP _ R i i i 46C i
RNP _VV 46C i i i i
RNP S O 46C S O i
RNP P¥ W S O S O 46C i
RNP B44 46C C K¥U B44 i i

ORAAH + ORE /Hive
Hive Oracle DB
6_XM S X 4O M¥S S X
a]_l,_khfDbo^ KN D Dbo^ D@BO
a]_l,mhDbo^ D@BO hk^,_k f^ Dbo^
hk^, k^ m^ ] m ,_k f^ hk^,_k f^ ~ ~
hk^,]khi ~ ~ ~
hk^, ^m hk^,_k f^
hk^,inee Dbo^ N ~ ~
hk^,inla N Dbo^ ~ ~
hk^,k^ h]^ hk^,o^ mhk
114

Hadoop(MapReduce)
k^ m^ ~ • ~
_bo^gnf ~ Pnd^r 3 & '
EMN
f^]b g
jn gmbe^
l]
o k /
115

6_XM S X 4O M¥S S X
¥MR M ¥ L^ klhg
¥MR M
¥MR QO HVO OV fh]ê,f mkbq qeô – ~ , ^mTeôêl
¥MR QVW D@BO ~
¥MR QVW)
¥MR UWOKX D@BO ~ G
¥MR VW
m ee+ g]+ldbggr MN &POMN' Kk e^ N Agm^kikbl^ hk^,ef
~
¥MR VWP
cêer_bla I ahnm em^kg mbg e^ lm ljn k^l pbma p^b am^] k^ ne kbs mbhg &=HO+SN'
¥MR XO_¥KV ~ ~ ~ ~
¥MR XWP
cêer_bla
~ N JIB ~ ~ –
¥MR XWP 6KVQ N JIB ~ ~ ~ ~
¥MR ¥SXM W
¥MR ¥OM WWOXN hk a,f ahnm,ef_, le – ~ ~ g
¥MR KW VO
¥MR MKVO ~
116

R
117
K ¥O ~N KNA
K ¥O MRK¥KM O¥ ~N KNA
K ¥O NK O ~N KNA
K ¥O NK O SWO ~N KNA
K ¥O NSPP SWO ~N KNA
K ¥O PKM ¥ ~N KNA
K ¥O P¥KWO ~N KNA ~
K ¥O SX OQO¥ ~N KNA
K ¥O VS ~N KNA
K ¥O V QSMKV ~N KNA
K ¥O WK ¥Sb ~N KNA
K ¥O X_WO¥SM ~N KNA
K ¥O LTOM ~N KNA
K ¥O OM ¥ ~N KNA

R
Hive
118
S ¥O Kk e^ N Agm^kikbl^
S ¥O MRK¥KM O¥
S ¥O NK O
S ¥O NK O SWO
S ¥O NSPP SWO
S ¥O PKM ¥
S ¥O P¥KWO ~
S ¥O SX OQO¥
S ¥O VS
S ¥O V QSMKV
S ¥O WK ¥Sb
S ¥O X_WO¥SM
S ¥O LTOM
S ¥O OM ¥

hk a,][ ,e lm^kk ~ ~
hk a,][ ,h__ ~
hk a,][ ,hg ~ N KN
¥KMVO B 1N KXMON 1XKVc SM P ¥ KN
hk a,][ ,hnminm
hk a,o^klbhg KN D ~ ~
hk a,]^[n I iN^]n ^ N N

HIVE/R
9F5 B
bgm bgm^ ^k
mbgrbgm bgm^ ^k
lf eebgm bgm^ ^k
[b bgm bgm^ ^k
]hn[e^ gnf^kb
_eh m gnf^kb
]^ bf e gnf^kb
lmkbg a k m^k
a k a k m^k
o k a k a k m^k
[hhe^ g eh b e
] m^ @ m^
mbf^lm fi LKOET m
ohb] Akkhk
120

Spark Mlib
Spark Mlib
¥MR WV N Oi kd Ieb[
¥MR WV VSXOK¥ Oi kd Ieb[
¥MR WV VK Oi kd Ieb[ H llh
¥MR WV ¥SNQO Oi kd Ieb[
¥MR WV V QS SM Oi kd Ieb[
¥MR WV MK Oi kd Ieb[
¥MR WV ¥KXN W P ¥O Oi kd Ieb[
¥MR WV W Oi kd Ieb[ ~
¥MR WV UWOKX Oi kd Ieb[ G+I^ gl
¥MR W NOV WK ¥Sb Oi kd fh]ê,f mkbq
¥MR V KN W NOV D@BO Oi kd Oi kd Ieb[ ~
¥MR K O W NOV Oi kd D@BO Oi kd Ieb[
¥MR ¥O K¥O W NOV WK ¥Sb Oi kd fh]ê,f kmbq
RNP a¥S O Oi kd fh]ê,f mkbq D@BO –
Oi kd ¥MR ¥O K¥O D@BO Oi kd ~
¥MR _X ¥O K¥O Oi kd fh]ê,f mkbq
121

R de Hadoop (Oracle R Advanced Analytics for Hadoopご説明資料）

Recommended

Recommended

More Related Content

What's hot

What's hot (10)

Viewers also liked

Viewers also liked (20)

Similar to R de Hadoop (Oracle R Advanced Analytics for Hadoopご説明資料）

Similar to R de Hadoop (Oracle R Advanced Analytics for Hadoopご説明資料） (20)

More from オラクルエンジニア通信

More from オラクルエンジニア通信 (20)

Recently uploaded

Recently uploaded (20)

R de Hadoop (Oracle R Advanced Analytics for Hadoopご説明資料）