Introduccion a las caracteristicas de Microsoft R Server

1. Microsoft R Server
Ing. Eduardo Castro, PhD
ecastrom@gmail.com
Data Science Specialization
Microsoft Data Platform MVP
PASS Regional Mentor
PASS Board Advisor

2. Organiza

3. Patrocinadores del SQL Saturday

4. Fuentes consultadas
 Esta presentación include slides tomados de las
siguientes fuentes:
 Revolution R Enterprise. Hong Ooi.
 Data Science with Azure Machine Learning, SQL
Server and R. Lukawiecki
 Tutoriales y Demostraciones
 https://msdn.microsoft.com/en-
us/library/mt590536.aspx
7

5. La ciencia de datos
El método científico de
razonamiento aplicado
de decisiones basadas
en datos
Hipótesis,
experimentos, hechos,
lógico razonamiento
+ Ingeniería de datos.

6. Data wrangling
(munging),
retrieval + storage
Data
mining &
machine
learning
Statistics
Big data

7. ¿Cómo?
Datos
Modelos
Necesidad
de
negocios

8. El aprendizaje automático ≣ ciencia de
datos
explora
datos
encuentra
patrones
Predecir
(scoring)

10. Herramientas
Chart from "2014 Data Science Salary Survey" (ISBN 978-1-491-
91842-5)
© 2015 O'Reilly Media, used with permission. Arrows mine.
For more info, and great titles on data science, visit oreilly.com

11. Metodología sugerida
SSAS
Data
Mining
R Azure ML
Fácil, visual,
intuitiva, Excel,
simplemente
funciona
Estadísticas
descriptivas,
“sentir” sus
datos, más
algoritmos
Los algoritmos
avanzados, el
auto-tuning,
servicios web,
nube!

12. Otras herramientas de las ciencias de datos
de Microsoft
Hadoop en la nube
+ Storm (análisis en tiempo real)
+HBase (NoSQL)
+Mahoot (ML!)
Streaming Data procedentes de la nube
Basado en HDInsight/ Hadoop
Power BI: Power Query, Power View, and
Dashboards
Excel
Azure Data Factory (ETL in the cloud)
Analytics Platform System (SQL Server on
steroids + Hadoop + hardware)

13. ¿Qué es R?
 Lenguaje interpretado, pobre IDE
 5000+ paquetes de software estadístico
 Mejor IDE: RStudio
 http://www.rstudio.com/
 Rattle y OnePageR hace que sea aún más fácil
 Código abierto, libre, multiplataforma
 R Core: la versión más pura: http://cran.r-project.org/
 Revolution Analytics: paralelismo y Rendimiento:
http://www.revolutionanalytics.com/
 Azure ML: built-in

14. Limitaciones del open source R
R necesita datos en memoria R solo tiene un hilo
de ejecución
R require habilidades
especializadas para
crear cluster
R Open es soportado por
la comunidad
Revolution R Enterprise brinda una solución a
esto!

15. Usuarios de Revolution Analytics

16. Revolution roadmap con Microsoft
 Continua el soporte para estas plataformas
 Windows
 Linux
 Hadoop
 Teradata
 Integración con nuevas plataformas
 Azure Marketplace
 Azure ML
 Azure HDInsight
 Sql Server 2016
 Azure SQL
 Frontend tooling/BI integration

17. Revolution R vs open source R
 NO tiene límites de RAM
• Open source R llena la
memoria y falla
• RRE escala lineamiente
aunque sobrepase el límite de
RAM
 Algoritmos más rápidos
• RRE optimizado para gran
cantidad de datos
File Name
Compressed
File Size
(MB) No. Rows
Open
Source R
(secs)
Revolution R
(secs)
Tiny 0.3 1,235 0.00 0.05
V. Small 0.4 12,353 0.21 0.05
Small 1.3 123,534 0.03 0.03
Medium 10.7 1,235,349 1.94 0.08
Large 104.5 12,353,496 60.69 0.42
Big (full) 12,960.0
123,534,96
9 Memory! 4.89
V.Big 25,919.7
247,069,93
8 Memory! 9.49
Huge 51,840.2
494,139,87
6 Memory! 18.92
 Public US Flight Data
 Linear Regression sobre el campo Arrival Delay
 Ejecución en 4 core laptop, 16GB RAM and 500GB
SSD

18. Revolution R vs SAS
• Pruebas realizadas por consultores
independientes – 5 x 4 core maquinas
ejecutando sobre CentOS
• SAS 9.4: Base SAS, SAS/STAT, Grid Mgr
• Revolution R Enterprise ScaleR, con IBM
Platform LSF, Platform MPI Release 9
• Data set: 591 columnas y 5,000,000 filas

19. Cientificos de datos
interactuar directamente con
datos
Incorporado a SQL
Server
Desarrollador de datos /
DBA
Manejo de datos y
analíticas en el mismo motor
Incorporando el análisis avanzado
Dentro de la base de datos de análisis
Ejemplo de soluciones
• La detección del fraude
• Pronósticodeventas
• la eficiencia del inventario
• Mantenimiento predictivo
datos relacional
Biblioteca
analítica
T-SQL Interface
Extensibilid
ad
?
R
Integración
R
010010
100100
010101
Microsoft Azure
Machine Learning Marketplace
R nuevas
secuencias de
comandos
010010
100100
010101
010010
100100
010101
010010
100100
010101
010010
100100
010101
010010
100100
010101

20. Integración con R Scripts
Fuente: https://visualstudiomagazine.com/articles/2015/05/04/sql-server-2016-preview.aspx

21. Revolution R vs SAS
• Pruebas realizadas por empresa
independiente – 5 x 4 core machines
ejecutando CentOS
• SAS 9.4: Base SAS, SAS/STAT, Grid Mgr
• Revolution R Enterprise ScaleR, IBM
Platform LSF, Platform MPI Release 9
• Data set: 591 columnas con 5,000,000 filas

22. Revo product suite
• Distribución gratis y open source R
• Mejorado y distribuido por Revolution Analytics
Revolution R Open
• Seguridad, Escalable una Distribución de R con
soporte
• Incluye componentes propietarios creados por
Revolution Analytics
Revolution R Enterprise

23. Revolution R Enterprise (RRE)
 Distribución Open Source de R:
• Conectivida con objetos big-data
• Big-data advanced analytics
• Soporte multiplataforma
• Análisis Predictivo In-Hadoop in-Teradata
• Soporte para ambientes de desarrollo y
producción
• Servicios de soporte técnico y entrenamiento
DistributedR
DeployR DevelopR
ScaleR
ConnectR

24. La Plataforma RRE
ConnectR
• Contiene High-speed &
direct connectors
Available for:
• High-performance XDF
• Formato de archivos SAS,
SPSS, delimited & fixed
format text
• Hadoop HDFS (texto &
XDF)
• Teradata Database & Aster
• EDWs and ADWs
• ODBC
ScaleR
• Incluye características
Ready-to-Use high-
performance para
big data big analytics
• Procesamiento analítico
Fully-parallelized
• Estadística descriptive &
pruebas estadísticas
• Incluye funciones
adicionales de análisis
predictivo
• Herramientas para distribuir
R algorithms entre nodos
• Soporte para Wide data –
miles de variables
DistributedR
• Framework de computación
distribuidad
• Portabilidad multiplataforma
Disponible en:
• Windows Servers
• Red Hat and SuSE Linux Servers
• Teradata Database
• Cloudera Hadoop
• Hortonworks Hadoop
• MapR Hadoop
R+CRAN
• Open source R interpreter
• R 3.2.2
• Gran cantidad de algoritmos
gratuitos
• Algoritmos utilizados por RevoR
• Embeddable R scripts
• 100% Compatible con R scripts,
funciones
y paquetes
RevoR
• Intérprete de R con mejora
de desempeño
• Basado en el open source
R
• Agrega high-performance
math library acelerar las
funciones de algebra lineal

25. Integración de R dentro de SQL Server 2016
exec sp_configure 'external scripts enabled', 1; reconfigure;
 "C:Program filesRRORRO-3.2.2-for-RRE-7.5.0R-
3.2.2libraryRevoScaleRrxLibsx64registerRext.exe" /install

26. Integración de R dentro de SQL Server 2016
USE <target database name>
GO
CREATE LOGIN [<login name>] WITH PASSWORD= '<password>',
CHECK_EXPIRATION=OFF, CHECK_POLICY=OFF;
CREATE USER [<user name>] FOR LOGIN [<login name>] WITH
DEFAULT_SCHEMA=[db_datareader]
ALTER ROLE [db_datareader] ADD MEMBER [<user name>]

27. Integración de R dentro de SQL Server 2016
USE [master]
GO
CREATE USER [<user name>] FOR LOGIN [<login name>] WITH
DEFAULT_SCHEMA=[db_rrerole]
ALTER ROLE [db_rrerole] ADD MEMBER [<user name>]

28.  Demostración
 Instalación de R Server e Integración con
SQL Server 2016

29. Posibles herramientas cliente

30. RRE: escalar a grandes volúmenes de
datos
 “Fragmentación“ de datos
alivia los límites de memoria
 Volumen limitado sólo por la
capacidad de
almacenamiento
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .
21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4
21.0 6 160.0 110 3.90 2.875 17.02 0 1 4 4
22.8 4 108.0 93 3.85 2.320 18.61 1 1 4 1
21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1
18.7 8 360.0 175 3.15 3.440 17.02 0 0 3 2
18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1
14.3 8 360.0 245 3.21 3.570 15.84 0 0 3 4
24.4 4 146.7 62 3.69 3.190 20.00 1 0 4 2
22.8 4 140.8 95 3.92 3.150 22.90 1 0 4 2
19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4
17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4
16.4 8 275.8 180 3.07 4.070 17.40 0 0 3 3
17.3 8 275.8 180 3.07 3.730 17.60 0 0 3 3
15.2 8 275.8 180 3.07 3.780 18.00 0 0 3 3
10.4 8 472.0 205 2.93 5.250 17.98 0 0 3 4
. . .
mpg cyl disp hp drat wt qsec vs am gear carb

31. RRE: escalar a grandes volúmenes de
datos
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .
21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4
21.0 6 160.0 110 3.90 2.875 17.02 0 1 4 4
22.8 4 108.0 93 3.85 2.320 18.61 1 1 4 1
21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1
18.7 8 360.0 175 3.15 3.440 17.02 0 0 3 2
18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1
14.3 8 360.0 245 3.21 3.570 15.84 0 0 3 4
24.4 4 146.7 62 3.69 3.190 20.00 1 0 4 2
22.8 4 140.8 95 3.92 3.150 22.90 1 0 4 2
19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4
17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4
16.4 8 275.8 180 3.07 4.070 17.40 0 0 3 3
17.3 8 275.8 180 3.07 3.730 17.60 0 0 3 3
15.2 8 275.8 180 3.07 3.780 18.00 0 0 3 3
10.4 8 472.0 205 2.93 5.250 17.98 0 0 3 4
. . .
mpg cyl disp hp drat wt qsec vs am gear carb
En un archivo de xdf (local)
21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4
21.0 6 160.0 110 3.90 2.875 17.02 0 1 4 4
22.8 4 108.0 93 3.85 2.320 18.61 1 1 4 1
21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1
18.7 8 360.0 175 3.15 3.440 17.02 0 0 3 2
18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1
14.3 8 360.0 245 3.21 3.570 15.84 0 0 3 4
24.4 4 146.7 62 3.69 3.190 20.00 1 0 4 2
22.8 4 140.8 95 3.92 3.150 22.90 1 0 4 2
19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4
17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4
16.4 8 275.8 180 3.07 4.070 17.40 0 0 3 3
17.3 8 275.8 180 3.07 3.730 17.60 0 0 3 3
15.2 8 275.8 180 3.07 3.780 18.00 0 0 3 3
10.4 8 472.0 205 2.93 5.250 17.98 0 0 3 4
. . .
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .
mpg cyl disp hp drat wt qsec vs am gear carb

32. RRE: escalar a grandes volúmenes de
datos
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .
21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4
21.0 6 160.0 110 3.90 2.875 17.02 0 1 4 4
22.8 4 108.0 93 3.85 2.320 18.61 1 1 4 1
21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1
18.7 8 360.0 175 3.15 3.440 17.02 0 0 3 2
18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1
14.3 8 360.0 245 3.21 3.570 15.84 0 0 3 4
24.4 4 146.7 62 3.69 3.190 20.00 1 0 4 2
22.8 4 140.8 95 3.92 3.150 22.90 1 0 4 2
19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4
17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4
16.4 8 275.8 180 3.07 4.070 17.40 0 0 3 3
17.3 8 275.8 180 3.07 3.730 17.60 0 0 3 3
15.2 8 275.8 180 3.07 3.780 18.00 0 0 3 3
10.4 8 472.0 205 2.93 5.250 17.98 0 0 3 4
. . .
mpg cyl disp hp drat wt qsec vs am gear carb Teradata
VAMPs
Teradata
Databas
e
ODB
C
Revolution R
Enterprise
Data
Segments
Database Nodes
Hybrid Storage
Parse
Engine
External
Stored
Procedure
Table
Operator
Table
Operator
Table
Operator
Table
Operator
Desktops &
Servers
21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4
21.0 6 160.0 110 3.90 2.875 17.02 0 1 4 4
22.8 4 108.0 93 3.85 2.320 18.61 1 1 4 1
21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1
18.7 8 360.0 175 3.15 3.440 17.02 0 0 3 2
18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1
14.3 8 360.0 245 3.21 3.570 15.84 0 0 3 4
24.4 4 146.7 62 3.69 3.190 20.00 1 0 4 2
22.8 4 140.8 95 3.92 3.150 22.90 1 0 4 2
19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4
17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4
16.4 8 275.8 180 3.07 4.070 17.40 0 0 3 3
17.3 8 275.8 180 3.07 3.730 17.60 0 0 3 3
15.2 8 275.8 180 3.07 3.780 18.00 0 0 3 3
10.4 8 472.0 205 2.93 5.250 17.98 0 0 3 4
. . .
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .

33. RRE: escalar a grandes volúmenes de
datos
Slave
node
Master node
Hadoop
Slave
node
Slave
node
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .
21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4
21.0 6 160.0 110 3.90 2.875 17.02 0 1 4 4
22.8 4 108.0 93 3.85 2.320 18.61 1 1 4 1
21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1
18.7 8 360.0 175 3.15 3.440 17.02 0 0 3 2
18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1
14.3 8 360.0 245 3.21 3.570 15.84 0 0 3 4
24.4 4 146.7 62 3.69 3.190 20.00 1 0 4 2
22.8 4 140.8 95 3.92 3.150 22.90 1 0 4 2
19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4
17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4
16.4 8 275.8 180 3.07 4.070 17.40 0 0 3 3
17.3 8 275.8 180 3.07 3.730 17.60 0 0 3 3
15.2 8 275.8 180 3.07 3.780 18.00 0 0 3 3
10.4 8 472.0 205 2.93 5.250 17.98 0 0 3 4
. . .
mpg cyl disp hp drat wt qsec vs am gear carb
21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4
21.0 6 160.0 110 3.90 2.875 17.02 0 1 4 4
22.8 4 108.0 93 3.85 2.320 18.61 1 1 4 1
21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1
18.7 8 360.0 175 3.15 3.440 17.02 0 0 3 2
18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1
14.3 8 360.0 245 3.21 3.570 15.84 0 0 3 4
24.4 4 146.7 62 3.69 3.190 20.00 1 0 4 2
22.8 4 140.8 95 3.92 3.150 22.90 1 0 4 2
19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4
17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4
16.4 8 275.8 180 3.07 4.070 17.40 0 0 3 3
17.3 8 275.8 180 3.07 3.730 17.60 0 0 3 3
15.2 8 275.8 180 3.07 3.780 18.00 0 0 3 3
10.4 8 472.0 205 2.93 5.250 17.98 0 0 3 4
. . .
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .
10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
. . .

34. RRE: cómputo distribuido
 Ningún movimiento de datos
 Establecer el contexto de cálculo determina donde
se realiza la transformación
VAMPs
Teradata
Databas
e
ODB
C
Revolution R
Enterprise
Data
Segments
Database Nodes
Hybrid Storage
Parse
Engine
External
Stored
Procedure
Table
Operator
Table
Operator
Table
Operator
Table
Operator
Desktops &
Servers

35. Contexto de cómputo local
### LOCAL COMPUTE CONTEXT ###
rxSetComputeContext("local")
### CREATE DIRECTORY AND FILE OBJECTS ###
AirlineDatabase <-file.path("datasets","AirlineDemoSmall")
AirlineDataSet <- RxXdfData(file.path(AirlineDatabase,"AirlineDemoSmall.xdf"))
### ANALYTICAL PROCESSING ###
### Statistical Summary of the data
rxSummary(~ArrDelay+DayOfWeek, data= AirlineDataSet, reportProgress=1)
### CrossTab the data
rxCrossTabs(ArrDelay ~ DayOfWeek, data= AirlineDataSet, means=T)
### Linear Model and plot
arrLateLinMod <- rxLinMod(ArrDelay ~ DayOfWeek + 0 , data = AirlineDataSet)
plot(arrLateLinMod$coefficients)

36. Remote Compute: Teradata
### SETUP TERADATA ENVIRONMENT VARIABLES ###
dbConnStr <- "Driver=Teradata; Server=dbHostName; Database=RevoDb; Uid=xxxx; pwd=xxxx"
myTeradataCC <- RxInTeradata(connectionString = dbConnStr, shareDir = "/tmp",
remoteShareDir = "/tmp/revoJobs", revoPath = "/usr/lib64/Revo-7.0/R-3.0.2/lib64/R")
### TERADATA COMPUTE CONTEXT ###
rxSetComputeContext(myTeradataCC)
### CREATE TERADATA DATA SOURCE ###
AirlineDemoQuery <- "SELECT * FROM AirlineDemoSmall;"
AirlineDataSet <- RxTeradata(connectionString = dbConnStr, sqlQuery = AirlineDemoQuery)
### ANALYTICAL PROCESSING ###
### Statistical Summary of the data
rxSummary(~ArrDelay+DayOfWeek, data= AirlineDataSet, reportProgress=1)
### CrossTab the data
rxCrossTabs(ArrDelay ~ DayOfWeek, data= AirlineDataSet, means=T)
### Linear Model and plot
arrLateLinMod <- rxLinMod(ArrDelay ~ DayOfWeek + 0 , data = AirlineDataSet)
plot(arrLateLinMod$coefficients)

37. Remote compute: Hadoop
### SETUP HADOOP ENVIRONMENT VARIABLES ###
myNameNode <- "master"
myUser <- "root"
myPort <- 8020
myHadoopCluster <- RxHadoopMR(sshUsername = myUser, sshHostname = myNameNode, port = myPort)
### HADOOP COMPUTE CONTEXT USING HDFS ###
rxSetComputeContext(myHadoopCluster)
### CREATE HDFS, DIRECTORY AND FILE OBJECTS ###
hdfsFS <- RxHdfsFileSystem(hostName=myNameNode, port=myPort)
AirlineDatabase <-file.path("datasets","AirlineDemoSmall")
AirlineDataSet <- RxXdfData(file.path(AirlineDatabase,"AirlineDemoSmall.xdf"), fileSystem = hdfsFS)
### ANALYTICAL PROCESSING ###
### Statistical Summary of the data
rxSummary(~ArrDelay+DayOfWeek, data= AirlineDataSet, reportProgress=1)
### CrossTab the data
rxCrossTabs(ArrDelay ~ DayOfWeek, data= AirlineDataSet, means=T)
### Linear Model and plot
arrLateLinMod <- rxLinMod(ArrDelay ~ DayOfWeek + 0 , data = AirlineDataSet)
plot(arrLateLinMod$coefficients)

38. Contexto remoto: SQL Server *
### SETUP SQL SERVER ENVIRONMENT VARIABLES ###
dbConnStr <- "Driver=SQL Server; Server=dbHostName; Database=RevoDb; Uid=xxxx; pwd=xxxx"
mySqlServerCC <- RxInSqlServer(connectionString = dbConnStr, consoleOutput = TRUE)
### SQL SERVER COMPUTE CONTEXT ###
rxSetComputeContext(mySqlServerCC)
### CREATE SQL SERVER DATA SOURCE ###
AirlineDemoQuery <- "SELECT * FROM AirlineDemoSmall;"
AirlineDataSet <- RxSqlServer(connectionString = dbConnStr, sqlQuery = AirlineDemoQuery)
### ANALYTICAL PROCESSING ###
### Statistical Summary of the data
rxSummary(~ArrDelay+DayOfWeek, data= AirlineDataSet, reportProgress=1)
### CrossTab the data
rxCrossTabs(ArrDelay ~ DayOfWeek, data= AirlineDataSet, means=T)
### Linear Model and plot
arrLateLinMod <- rxLinMod(ArrDelay ~ DayOfWeek + 0 , data = AirlineDataSet)
plot(arrLateLinMod$coefficients) * In 2016

39. ScaleR funciones y algoritmos
Data step
Data import – delimited, fixed, SAS, SPSS, ODBC
Variable creation & transformation
Recode variables
Factor variables
Missing value handling
Sort, merge, split
Aggregate by category (means, sums)
Descriptive statistics
Min / Max, Mean, Median (approx.)
Quantiles (approx.)
Standard deviation
Variance
Correlation
Covariance
Sum of squares (cross product matrix for set
variables)
Pairwise cross tabs
Risk ratio & odds ratio
Crosstabulation of data (standard tables & long
form)
Marginal summaries of crosstabulations
Statistical tests
Chi square test
Kendall rank correlation
Fisher’s exact test
Student’s t-test
Sampling
Subsample (observations & variables)
Random sampling
Predictive models
Sum of squares (cross product matrix for set
variables)
Multiple linear regression
Generalized linear models (GLM) exponential
family distributions: binomial, Gaussian, inverse
Gaussian, Poisson, Tweedie. Standard link
functions: cauchit, identity, log, logit, probit.
User defined distributions & link functions.
Covariance & correlation matrices
Logistic regression
Classification & regression trees
Predictions/scoring for models
Residuals for all models
Variable selection
Stepwise regression
Simulation
Simulation (eg Monte Carlo)
Parallel random number generation
Cluster analysis
K-means clustering
Classification
Decision forests (random forests)
Decision trees
Gradient boosted decision trees
Naïve Bayes
Combination
PEMA API
rxDataStep
rxExec

40. DeployR
 Marco de I como un servicio para
aplicaciones de BI / web

41. Arquitectura de DeployR

42. 42 |
Nombre
expositor
email blog
twitter
PREGUNTAS Y RESPUESTAS