Introduction to R for Big Data Analysis

Introduction to R
For Big Data Analysis
Wednesday, October 13, 2015
6:00pm – 6:45 pm
Raastech, Inc.
2201 Cooperative Way, Suite 600
Herndon, VA 20171
+1-703-884-2223
info@raastech.com

© Raastech, Inc. 2015 | All rights reserved. Slide 2 of 51@Raastech
About Me
 Harold Dost III @hdost
 7+ years of Oracle Middleware experience
 OCE (SOA Foundation Practitioner)
 Oracle ACE Associate
 From Michigan
 blog.raastech.com

About Raastech
 Small systems integrator founded in 2009
 Headquartered in the Washington DC area
 Specializes in Oracle Fusion Middleware
 Oracle Platinum Partner – 1 in 3,000 worldwide
 Oracle SOA Specialized – 1 in 1,500 worldwide
 Oracle ACE – 2 of 500 worldwide
 100% of consultants are Oracle certified
 100% of consultants present at major Oracle conferences
 100% of consultants have published books, whitepapers, or articles

Outline
1. Getting Started
 Installing R
 Installing Tools
 Getting Data
2. Understanding R
 Data Types
 Functions
 Data Import Mechanisms

Outline (Cont.)
3. Manipulating Data (Large Data Sets)
 Deriving Simple Statistics
 Graphing
4. Demo
5. Incorporating into an Enterprise
 Using Enterprise Data Sources
 Running R in your environment.
 Familiarize with Oracle's R offerings

Know CRAN
 Comprehensive
 R
 Archive
 Network

Installing R
 Windows
 Mac
 Linux

Installing R
 Windows https://cran.r-project.org/bin/windows/
 Mac https://cran.r-project.org/bin/macosx/
 Linux https://cran.r-project.org/bin/linux/

Development Tools
 Rstudio - http://www.rstudio.com/products/rstudio/
 Open Source Edition
 Commercial License - $995
 Eclipse
 Sublime, TextPad, Other Simple Text Editors,…

Installing Packages
 Anything From CRAN
 Anywhere
install.packages(c(“first”, “second”))
> sudo R CMD INSTALL package-version.tar.gz

Data Types
 Vectors
 Matrices
 Arrays
 Data Frames
 Lists
 Factors

Special Values
 Infinity, Positive and Negative: Inf and –Inf
 Not A Number: NaN
 Not Available: NA
 Complex Numbers, 1+9i

Use Case for Infinities
 Finding Maximums and Minimums
 Placeholder values when others won’t work

Not a Number (NaN)
 In means something went wrong somewhere
 A missing argument
 Invalid number
 Check for with is.nan(x) to prevent leaking
 Don’t use “==“ to find NaN, it will only give more NaN

Assigning NaN
> a = NaN
> a
[1] NaN

Adding NaN
Adding NaN
> b = 1
> c = a + b
> c
[1] NaN
When adding a number to NaN “Not a Number” you will get
NaN.

Comparing NaN to Regular Number
> d = b == c
> d
[1] NA
When comparing a number to NaN “Not a Number” you will
get NA.

Comparing NaN to NaN
> e = c == a
> e
[1] NA
When comparing NaN “Not a Number” to NaN you will get
NA.

Detecting NaN
> a
[1] NaN
> is.nan(a)
[1] TRUE
> is.na(a)
[1] TRUE
Since NaN aren’t proper numbers, special functions must be
used to detect them. They are the result of math gone wrong.

Detecting NA
> e = c == a
> e
[1] NA
> is.nan(e)
[1] FALSE
> is.na(e)
[1] TRUE
Just as with NaN special functions must be used, but NA
generally indicates that there is missing information

Operators
 Assignment ( ->, <-)
 Addition (+)
 Subtraction (–)
 Division (/)
 Multiplication (*)
 Exponent (^)
 Parentheses ( (, ) )

Math Functions
 max()
 min()
 log()
 sqrt()

Deriving Simple Statistics
 Minimum
 Maximum
 Median
 Arithmetic Mean
 Function estimation
 Linear
 Log
 Exponential
 R-Values
 Standard Deviation

How to define your own functions
firstfunction <- function(arg1, arg2, ... ){
statements
return(someoutput)
}

Twitter Example
 First Install the Package
install.packages("twitteR”)

Twitter Example
 Authenticate
consumer = "CONSUMER KEY"
secret = "SECRET KEY"
setup_twitter_oauth(consumer,secret)

Twitter Example
 Get Trend Locations
 The resulting WOEID (Where on Earth ID) can be
chosen
availableTrendLocations()

Twitter Example
 Get Trends
trends = getTrends(SOMEWOEID)

Twitter Example
 Retrieve Tweets
tweets <- searchTwitter(trends[XX,XX],n=1500)
tweetdf <- do.call("rbind",lapply(tweets,as.data.frame))

Twitter Example
 Filter
 complete.cases is used to check for NA and NaN
numbers
tweetdf <- tweetdf[complete.cases(tweetdf[,15]),]
tweetdf <- tweetdf[tweetdf[,15] != 0,]

Twitter Example
 Simplify the dataframe
simpledf <- tweetdf[c("screenName","longitude","latitude")]

Twitter Example
 Create Matrix from Dataframe
tweetMatrix <- data.matrix(simpledf[2:3],rownames.force = FALSE)

Twitter Example
 Plot the Latitude and Longitude
plot(tweetMatrix)

Graphing
 Image
 Contour
 Box Chart

K-Means
 Essentially a search algorithm
 Divides a dataset into k-clusters

Time Series
 Stock Quotes
 Infection Incidents
 Gas Prices
 Audio
 Etc.
Source: http://www.loc.gov/pictures/resource/hec.23488/

Time Series Analysis
 Regression
 Forecasting
 Time Frequency (FFTs)
Source: http://groups.csail.mit.edu/netmit/sFFT/algorithm.html

Using Enterprise Data Sources
 Database
 Streams
 Files
 Etc.

Oracle R Distribution
 Available on Oracle Public Yum
 Enhanced dynamic Library loading
 Enterprise Support Available
 Oracle Advanced Analytics
 Oracle Linux
 Oracle Big Data Appliance
 http://www.oracle.com/technetwork/database/database-technologies/r/r-
distribution/overview/index.html

Oracle R Enterprise
 Component of the Oracle Advanced
Analytics Option on Oracle Database EE
 Allows use of R in the database without SQL
 Save R Objects in the database
 Easily Integrate with OBIEE
 http://www.oracle.com/technetwork/database/database-
technologies/r/r-enterprise/overview/index.html

Oracle R Advanced Analytics for Hadoop
 Component of the Oracle Big
Data Software Connectors Suite,
an option for the BDA
 Provides abstraction from HiveQL
through R just as in Oracle R
Enterprise does for SQL
 http://www.oracle.com/technetwork/database/
database-technologies/bdc/r-advanalytics-for-
hadoop/overview/index.html

ROracle
 Open Source Package
 Maintained by Oracle
 Uses OCI Interface to interact with databases
 http://www.oracle.com/technetwork/database/database-technologies/r/r-
technologies/overview/index.html

Contact Information
 Harold Dost III
 Principal Consultant
 @hdost
 harold.dost@raastech.com

Resources
 https://en.wikibooks.org/wiki/Statistical_Analysis:_an_Introduction_using_R/R_basics
 http://www.r-project.org/
 https://docs.oracle.com/cd/E57012_01/doc.141/e56973/toc.htm
 http://cran.r-project.org/web/packages/akmeans/index.html
 http://cran.r-project.org/web/packages/twitteR/index.html
 http://en.wikipedia.org/wiki/K-means_clustering
 http://www.rdatamining.com/examples/kmeans-clustering
 http://blog.revolutionanalytics.com/2009/02/how-to-choose-a-random-number-in-r.html
 https://www.packtpub.com/books/content/text-mining-r-part-2
 http://www.eia.gov/totalenergy/data/monthly/index.cfm#consumption

Introduction to R for Big Data Analysis

Recommended

Recommended

More Related Content

Similar to Introduction to R for Big Data Analysis

Similar to Introduction to R for Big Data Analysis (20)

More from Revelation Technologies

More from Revelation Technologies (20)

Recently uploaded

Recently uploaded (20)

Introduction to R for Big Data Analysis