Financial Data Access with SQL, Excel & VBA
Upcoming SlideShare
Loading in...5
×
 

Financial Data Access with SQL, Excel & VBA

on

  • 1,292 views

 

Statistics

Views

Total Views
1,292
Views on SlideShare
1,147
Embed Views
145

Actions

Likes
1
Downloads
12
Comments
0

1 Embed 145

http://sajumavi.wordpress.com 145

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Financial Data Access with SQL, Excel & VBA Financial Data Access with SQL, Excel & VBA Presentation Transcript

  • Computational Finance and Risk Management Financial Data Access with SQL, Excel & VBA Guy Yollin Instructor, Applied Mathematics University of Washington Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 1 / 59
  • Outline 1 What is RExcel? 2 Installation and configuration 3 Getting help with RExcel 4 Getting started with RExcel 5 The RCommander 6 Basic data transfer 7 Scratchpad mode 8 Scratchpad example Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 2 / 59
  • Lecture references T. Baier, E. Neuwirth Using R from within Excel. 2007. T. Baier, E. Neuwirth Creating and Deploying an Application with (R)Excel and R. The R Journal Vol. 3/2, December 2011. Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 3 / 59
  • Computational Finance and Risk Management Let’s not kid ourselves: The most widely used piece of software for statistics is Excel. Brian D. Ripley Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 4 / 59
  • Outline 1 What is RExcel? 2 Installation and configuration 3 Getting help with RExcel 4 Getting started with RExcel 5 The RCommander 6 Basic data transfer 7 Scratchpad mode 8 Scratchpad example Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 5 / 59
  • RExcel - an interface between R and Excel RExcel is an interface program that uses R as an add-in to Excel Transfers data between R and Excel, using Excel as a data editor for data to be analyzed with R Uses Excel as a container and simple code editor for R commands, and running R commands from Excel Runs R commands from Excel VBA macros allowing users to create Excel based applications and Excel add-ins using R Uses R functions in Excel worksheets functions, integrating R functionality into the automatic recalculation engine of Excel Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 6 / 59
  • RExcel modes of operation RExcel supports three modes of operation: Scratchpad and data transfer mode Menus control data transfer from R to Excel and back; commands can be executed immediately, either from Excel cells or from R command line Macro mode Macros, invisible to the user, control data transfer and R command execution Spreadsheet mode Formulas in Excel cells control data transfer and command execution, automatic recalculation is controlled by Excel Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 7 / 59
  • RExcel components statconnDCOM A DCOM† server which allows to seamlessly integrate R or Scilab into other applications. rcom An R package allowing to have R and another program running at the same time and let them communicate. rscproxy An R package which is required for rcom and for the statconn (D)COM Server. RExcel An Excel addin using statconn (D)COM or rcom to allow Excel to call R from within Excel. RAndFriends An installation wrapper containing installation programs for R (including many packages), statconnDCOM, RExcel, and the example files for the book R through Excel. † http://en.wikipedia.org/wiki/Component_Object_Model Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 8 / 59
  • Outline 1 What is RExcel? 2 Installation and configuration 3 Getting help with RExcel 4 Getting started with RExcel 5 The RCommander 6 Basic data transfer 7 Scratchpad mode 8 Scratchpad example Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 9 / 59
  • RAndFriends Download the latest version of RAndFriends from the statconn website. Download includes: Latest version of R (2.15.1) Package rscproxy 2.0-5 Package rcom 2.2-5 To install: Run the executable and accept default installation options http://rcom.univie.ac.at/download.html#RAndFriends Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 10 / 59
  • Excel add-ins and developer tabs Add-Ins Tab Contains RExcel add-in menu Developer Tab Contains access to macro creation and VBA development If the Add-Ins and/or Developer tab is not visible: Go to File → Options → Customize Ribbon. Activate the Add-Ins and Developer checkbox under "Main Tabs". Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 11 / 59
  • Establishing a reference to RExcelVBAlib To use RExcel VBA procedures, a reference must be established to the RExcelVBAlib object library From the VBE in Excel: Select the "Tools" tab Select "References" Find the object library labeled RExcelVBAlib and activate the checkbox Click "OK" and exit the VBA editor Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 12 / 59
  • Outline 1 What is RExcel? 2 Installation and configuration 3 Getting help with RExcel 4 Getting started with RExcel 5 The RCommander 6 Basic data transfer 7 Scratchpad mode 8 Scratchpad example Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 13 / 59
  • Getting help To access the RExcel help click the RExcel add-in menu and select "RExcel Help" This help option opens "Using R from within Excel" in a web browser Contents "Using R from within Excel" is a quick reference guide For additional information, see "R Through Excel" Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 14 / 59
  • RExcel tutorial video "Combining R and Excel" tutorial video T. Baier, E. Neuwirth Provides RExcel walkthrough commentary Walkthrough material is available through the RExcel add-in menu http://rcom.univie.ac.at/RExcelDemo/ It is highly recommended that students view this material before getting started with RExcel. Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 15 / 59
  • Walkthroughs RExcel comes with walkthrough demo worksheets. To start working on the demo material click the RExcel add-in and select "Demo Worksheets" Demo worksheets include: Data transfer Writing macros Worksheet functions Graphics with sliders Interactive graphics Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 16 / 59
  • Outline 1 What is RExcel? 2 Installation and configuration 3 Getting help with RExcel 4 Getting started with RExcel 5 The RCommander 6 Basic data transfer 7 Scratchpad mode 8 Scratchpad example Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 17 / 59
  • Starting RExcel To start RExcel: Navigate to the add-ins tab Click on the RExcel add-in to bring up the RExcel add-in menu Click "Start R" The RExcel splash screen should appear briefly Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 18 / 59
  • The RExcel add-in menu Once the R Server is started, the RExcel add-in menu will appear as follows† : † If foreground server is started, option will read Disconnect R rather then Close R Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 19 / 59
  • RExcel add-in menu commands RExcel add-in commands: Start R Starting RExcel will begin a new R session. Close R Run Code Closing RExcel will end your R session. Executes valid R code from a cell or a range of cells. Get R Value Puts the value of an R expression into the active cell or range of cells. Put R Var Puts a valid selection of cells into an array or dataframe object. Get R output Retrieves R output from the most recent command. Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 20 / 59
  • RExcel add-in menu commands cont. Set R working dir Allows the user to set a new working directory. Load R file Loads an .R file into the current R session from the working directory. Copy Code Copies selected R code and inputs the selection into a VBA procedure. Debug R Displays the debug window in which all calls to R are displayed before execution. Error Log Displays an additional window with error messages. Options Set R server Displays an options menu. Allows the user to switch between a foreground server or background server. Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 21 / 59
  • RExcel add-in menu commands cont. RExcel Help R Help RCommander Demo Worksheets RthroughExcel Worksheets Mark Calc cells About RExcel Guy Yollin (Copyright © 2012) Opens "Using R from within Excel" in a browser window. Opens the R help documentation. Opens the RCommander with Excel or separate menu options. Opens the Demo walkthroughs in a new Excel spreadsheet. Opens RExcel projects with advance graphics and analysis. Marks cells containing calculated results. Opens the "About RExcel" menu. Data Access with SQL, Excel & VBA RExcel 22 / 59
  • The RExcel RMB menu Once the R Server is started, there will be RExcel specific right-mouse-button options: Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 23 / 59
  • The RExcel RMB menu cont. The RExcel RMB menu adds the following options: Insert Current R Plot Inserts the current R Plot into a given range. Name Range Create a named range for easier selection. Prettyformat Numbers Aligns decimal points in a selected range. Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 24 / 59
  • Setting the R Server type Setting the R Server Before starting RExcel, the "Set R server" option is available. There are two methods by which to run RExcel: Background server Foreground server Starting RExcel initiates a number of different processes at startup. These processes are explained at greater length in the Startup section of "Using R from within Excel". Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 25 / 59
  • Setting the R Server type Background Server The background server is totally hidden from the user, all interaction with R has to be done in Excel. Foreground Server The foreground server allows direct access to the R GUI command line while working in Excel. See Using R from with Excel, T. Baier and E. Neuwirth Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 26 / 59
  • Background server To initiate the background server: Open the RExcel add-in menu Select Set R server Under server type, click "Background" Note No visible R GUI All R commands must be sent from Excel Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 27 / 59
  • Foreground server To initiate the foreground server: Open the RExcel add-in menu Select Set R server Under server type, click "Foreground" Note R GUI is visible R commands can be typed directly into the command line or sent from Excel Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 28 / 59
  • Background vs. foreground Background advantages R operates in the background, invisible to front-end users Background disadvantages R operates in the background, invisible to front-end users Foreground advantages For those familiar with R, a foreground server provides a more convenient method by which to pass R commands A foreground server is simply supplemental, no RExcel functionality is lost by running a foreground server Foreground disadvantages Do not close the R GUI before disconnecting from the DCOM server. This may cause system instability. If R is upgraded but RExcel is not reinstalled, RExcel will not work with the foreground server Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 29 / 59
  • Outline 1 What is RExcel? 2 Installation and configuration 3 Getting help with RExcel 4 Getting started with RExcel 5 The RCommander 6 Basic data transfer 7 Scratchpad mode 8 Scratchpad example Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 30 / 59
  • RCommander R Commander is a GUI for the R† Developed by John Fox at McMaster University Licensed under GNU GPL Along with plug-ins, arguably the most fully featured GUI for R along the lines of S-PLUS † http://socserv.mcmaster.ca/jfox/Misc/Rcmdr/ Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 31 / 59
  • The RCommander Window The RCommander windows can be thought of as a re-skinned version of the R GUI. RCommander contains three distinct sub-windows: Script Window Type or open R scripts Output Window Displays output from submitted R commands Messages Displays error messages and general information Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 32 / 59
  • The RCommander functions and methods RCommander functionality: File Save and/or open scripts, set a new working directory Edit Copy, paste, cut, undo, redo, etc. Data Import data from various sources and manage datasets Statistics Get statistics regarding the active dataset Graph Plot the active dataset Models Create statistical models based upon the active dataset Distributions Plot distributions and compute relevant statistical values Tools Load packages/plug-ins and access RCommander options Help Access R and RCommander Help Depending upon the nature of the active dataset, some options may not be available. Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 33 / 59
  • Starting the RCommmander RExcel provides menu options, via the RCommander, for some data transfer, analysis and visualization. To start the RCommander: Start RExcel Open the RExcel add-in menu Click "RCommander" Select either "with Excel menus" or "with separate menus" Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 34 / 59
  • Activating a dataset In order create models and distributions using the RCommander, a dataset must be defined: To activate a dataset click "<No active dataset>" located below the tab bar Notice that the user must define at least one dataset in the current R session to activate a dataset (to be explained in the following sections) Select a dataset from the dropdown menu and click "OK" Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 35 / 59
  • Outline 1 What is RExcel? 2 Installation and configuration 3 Getting help with RExcel 4 Getting started with RExcel 5 The RCommander 6 Basic data transfer 7 Scratchpad mode 8 Scratchpad example Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 36 / 59
  • Arrays and dataframes RExcel can handle two classes of data: Array RExcel uses the term array to mean R arrays, matrices, and vectors. Basic R data types where all elements are of the same class. Dataframe A dataframe is a two-dimensional matrix-like structure where each column can be of different classes. Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 37 / 59
  • Using RExcel to create a data.frame in R There are various methods by which RExcel can create a data.frame in R. RExcel add-in menu options RMB menu options RCommander menu options R commands typed directly into the R GUI or the RCommander script window RExcel worksheet functions RExcel VBA procedures Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 38 / 59
  • Sending a data.frame from Excel via RExcel add-in menu For beginning users, the most convenient method is to use the RMB menu options. Start RExcel Highlight a selection of data Right-click the data selection Choose "Put R DataFrame" Create a dataframe name or accept default name Note RExcel assumes that the data selection contains column headers By default, RExcel names the dataframe after the spreadsheet title. This can be changed in the RExcel add-in menu under "Options". Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 39 / 59
  • Importing data via RCommander A dataframe can also be created through the RCommander menu options: Start RExcel and open the RCommander Open the "Data" tab Import data Note The RCommander accepts various data forms: .txt, .dat, .csv, .xls, .xlsx, etc. The RCommander can also load data from any attached R package To attach additional packages go to the "Tools" tab in the RCommander and select "Load package(s)" Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 40 / 59
  • Importing data via RCommander cont. RCommander will create R code based upon user input from the data import wizard. Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 41 / 59
  • Retrieving a data.frame from R There are a few methods by which to retrieve a dataframe from R. RExcel add-in menu options RMB menu options RExcel VBA procedures Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 42 / 59
  • Retrieving a data.frame from R via RExcel RMB RMB menu options Insure RExcel is running Right click the cell in which you want the dataframe to begin Choose "Get R DataFrame" Click "Get from R" Select a dataframe from the dropdown Note If you ended the connection to the DCOM server, previously created dataframes will have been deleted Sequential numbering will be included unless the "with rownames" checkbox is off Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 43 / 59
  • Outline 1 What is RExcel? 2 Installation and configuration 3 Getting help with RExcel 4 Getting started with RExcel 5 The RCommander 6 Basic data transfer 7 Scratchpad mode 8 Scratchpad example Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 44 / 59
  • RExcel as code container and editor RExcel can run valid R code from a cell or selection of cells. Sending and receiving information between R and Excel is accessible through the RExcel add-in menu and the RMB menu. Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 45 / 59
  • The "Run code" option By interfacing R and Excel through RExcel, R code can be typed directly into a cell or a range of cells and be sent to R: There is no need to enclose the R expression in quotations Code can be run by accessing the RExcel add-in menu or the RMB menu If there is a coding issue, an pop-up will relay error messages directly from R Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 46 / 59
  • Special RExcel commands RExcel implements a few special commands that allow the automation of data exchanage without VBA. They can be run through the "Run Code" option. Command Description #!rput variable range store the value (contents) of a range in an R variable #!rputdataframe variable range store the value of a range in an R data frame #!rputpivottable variable range store the value of a range in an R variable #!rget r-expression range store the value of the R expression in the range #!rgetdataframe r-expression range store the data frame of the R expression in the range #!insertcurrentrplot cell-address insert the active plot into the worksheet Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 47 / 59
  • Outline 1 What is RExcel? 2 Installation and configuration 3 Getting help with RExcel 4 Getting started with RExcel 5 The RCommander 6 Basic data transfer 7 Scratchpad mode 8 Scratchpad example Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 48 / 59
  • The quantmod package The quantmod package for R is designed to assist the quantitative trader in the development, testing, and deployment of statistically based trading models. Key functions: getSymbols load or download price data Yahoo Finance / Google Finance FRED Oanda csv, RData MySQL, SQLite chartSeries charting tool to create standard financial charts Author: Jeffrey Ryan Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 49 / 59
  • The getSymbols function The getSymbols function loads (downloads) historic price data R Code: The getSymbols function > library(quantmod) > args(getSymbols) function (Symbols = NULL, env = .GlobalEnv, reload.Symbols = FALSE, verbose = FALSE, warnings = TRUE, src = "yahoo", symbol.lookup = TRUE, auto.assign = TRUE, ...) NULL Main arguments: Symbols symbols to be loaded src source of the data (Yahoo, Google, FRED, etc.) Return value: a time series object Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 50 / 59
  • Federal reserve economic data The function getSymbols can also be used to access data from the Federal Reserve Economic Data (FRED) database http://research.stlouisfed.org/fred2/ Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 51 / 59
  • Sending command to R with Run Code Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 52 / 59
  • Ready to Run code Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 53 / 59
  • Data and plot retrieved from R Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 54 / 59
  • Data and plot retrieved from R Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 55 / 59
  • Sending data.frame to R via special command Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 56 / 59
  • Sending data.frame to R via special command Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 57 / 59
  • Sending data.frame to R via special command Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 58 / 59
  • Computational Finance and Risk Management http://depts.washington.edu/compfin Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 59 / 59