Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

KDB+/R Integration

399 views

Published on

Overview of some kdb+ and R integration methods.

Published in: Software
  • Be the first to comment

  • Be the first to like this

KDB+/R Integration

  1. 1. KDB+ AND R INTEGRATIONRORY WINSTON
  2. 2. KDB+ AND R INTEGRATION AGENDA ▸ Why integrate R/kdb+ ▸ Basic R integration via CSV import/export ▸ R integration via qserver ▸ Integrating R math functionality into kdb+ ▸ Generating kdb+ data with R
  3. 3. KDB+ AND R INTEGRATION WHAT IS R? ▸ DSL For Statistical Techniques and Data Analysis ▸ Integrated programming and EDA (Exploratory Data Analysis) environment ▸ Open-Source ▸ Thousands of contributed packages
  4. 4. KDB+ AND R INTEGRATION WHY USE R? ▸ Comprehensive Environment ▸ Tooling ▸ Command-Line ▸ IDE ▸ Web ▸ Thousands of Packages ▸ Rich Reporting/Graphing/Analysis Capabilities
  5. 5. KDB+ AND R INTEGRATION PART ONE: CSV IMPORT/EXPORT ▸ CSV import/export ▸ In kdb+: ▸ save `:table.csv ▸ In R: ▸ read.csv(‘table.csv’, header=TRUE)
  6. 6. EXAMPLE #1 CSV IMPORT/EXPORT
  7. 7. KDB+ AND R INTEGRATION CSV IMPORT / EXPORT ▸ Simplest approach ▸ Interchange with any other system ▸ No type information ▸ Manual overhead ▸ Can be a slow/laborious process
  8. 8. KDB+ AND R INTEGRATION PART TWO: THE QSERVER LIBRARY ▸ Available from ▸ http://code.kx.com/wsvn/code/cookbook_code/r/ ▸ Consists of: ▸ A native library containing kdb+ connectivity ▸ An R wrapper ▸ Binaries for Windows/Linux/OSX
  9. 9. KDB+ AND R INTEGRATION THE QSERVER LIBRARY qserver kdb+layer Rlayer
  10. 10. KDB+ AND R INTEGRATION BASIC QSERVER OPERATIONS ▸ Connect to a kdb+ instance: ▸ open_connection( host, port, credentials ) ▸ Returns a handle ▸ Execute a query: ▸ execute( handle, query ) ▸ Close connection: ▸ close_connection( handle )
  11. 11. EXAMPLE #2 QSERVER
  12. 12. KDB+ AND R INTEGRATION QSERVER INTEGRATION ▸ Most Efficient ▸ Can Manage Multiple Connections ▸ Can Be Wrapped Into An R Library ▸ Easy to Extend With Custom R Code
  13. 13. KDB+ AND R INTEGRATION PART THREE: LEVERAGING THE R MATH LIBRARY ▸ R Comes With A Standalone Math Library ▸ Contains A Large Range Of Base Statistical/Math Routines ▸ E.g. Base Probability Functions/RNG ▸ Can Be Packaged Into a Library And Reused in kdb+ ▸ I have packed a library and a set of routines on: ▸ https://github.com/rwinston/kdb-rmathlib
  14. 14. KDB+ AND R INTEGRATION PART THREE: LEVERAGING THE R MATH LIBRARY ▸ Example In R: ▸ rnorm(1000,5,2) ▸ Generates 1000 random variables with a mean of 5 and standard deviation of 2 ▸ In q, using the rmathlib integration: ▸ norm[1000;5;2] ▸ R’s math library contains hundreds of useful base functions
  15. 15. EXAMPLE #3 RMATHLIB
  16. 16. KDB+ AND R INTEGRATION USING R TO GENERATE Q CODE ▸ Generate q code from R ▸ Similar in principle to CSV export ▸ See ▸ http://www.theresearchkitchen.com/archives/776
  17. 17. EXAMPLE #4 Q CODE GENERATION
  18. 18. THANKS

×