The document discusses using R and Python together in polyglot programs. It describes how R and Python code and libraries can be accessed natively from each other using the rpy2 module, which allows R code and objects to be used as regular Python functions, modules, and data types. This allows teams with different skills to collaborate effectively by bridging the languages.
COMPUTER LANGUAGES AND THERE DIFFERENCE Pavan Kalyan
In this ppt you will understand the difference among languages and You will know what is necessary for a language to become best in the present software filed
COMPUTER LANGUAGES AND THERE DIFFERENCE Pavan Kalyan
In this ppt you will understand the difference among languages and You will know what is necessary for a language to become best in the present software filed
Hard to figure out which one is better between Julia & Python? Here's a detailed comparison between #JuliaLang vs #Python
#programming #Coding #Software #developer #technology #tech
R programming advantages and disadvantagesPrwaTech
Prwatech is the leading Training institute for R Programming training in Bangalore Offering Best R Programming training courses with our Qualified Industry Certified Experts. Our R Programming training institute in bangalore was specially designed for those who are keen to learn the R Programming course from Scratch to Advanced level.
Python is a deciphered, elevated level, universally useful programming language. Made by Guido van Rossum and first discharged in 1991, Python's plan reasoning underscores code comprehensibility with its striking utilization of critical blank area. Python is powerfully composed and trash gathered.
Language standards: Interpreted language, Dejango
Language architects: Guido van Rossum
From Programming to Modeling And Back AgainMarkus Voelter
Is programming = modeling? Are there differences, conceptual and tool-wise? Should there be differences? What if we programmed the way we model? Or vice versa? In this slidedeck I explore this question and introduce interesting developments in the space of projectional editing and modern parser technology. This leads to the concept of modular programming languages and a new way of looking at programming. I will demonstrate the idea with tools that are available today, for example TMF Xtext, JetBrains MPS and Intentional’s Domain Workbench.
JetBrains MPS: Projectional Editing in Domain-Specific LanguagesOscar Rodriguez
During this webinar, Václav Pech introduced the extent and potential the projectional editing technology has; and how our clients are using it in their domain-specific languages. He also touched on some philosophical ideas, suggesting that perhaps, code should not necessarily be text.
Webinar: https://www.youtube.com/watch?v=bw8X3wkBUr8&feature=youtu.be
65 - An Empirical Simulation-based Study of Real-Time Speech Translation for ...ESEM 2014
Context: Real-time speech translation technology is today available but still lacks a complete understanding of how such technology may affect communication in global software projects. Goal: To investigate the adoption of combining speech recognition and machine translation in order to overcome language barriers among stakeholders who are remotely negotiating software requirements.
Method: We performed an empirical simulation-based study including: Google Web Speech API and Google Translate service, two groups of four subjects, speaking Italian and Brazilian Portuguese, and a test set of 60 technical and non-technical utterances.
Results: Our findings revealed that, overall: (i) a satisfactory accuracy in terms of speech recognition was achieved, although significantly affected by speaker and utterance differences; (ii) adequate translations tend to follow accurate transcripts, meaning that speech recognition is the most critical part for speech translation technology.
Conclusions: Results provide a positive albeit initial evidence towards the possibility to use speech translation technologies to help globally distributed team members to communicate in their native languages.
Hard to figure out which one is better between Julia & Python? Here's a detailed comparison between #JuliaLang vs #Python
#programming #Coding #Software #developer #technology #tech
R programming advantages and disadvantagesPrwaTech
Prwatech is the leading Training institute for R Programming training in Bangalore Offering Best R Programming training courses with our Qualified Industry Certified Experts. Our R Programming training institute in bangalore was specially designed for those who are keen to learn the R Programming course from Scratch to Advanced level.
Python is a deciphered, elevated level, universally useful programming language. Made by Guido van Rossum and first discharged in 1991, Python's plan reasoning underscores code comprehensibility with its striking utilization of critical blank area. Python is powerfully composed and trash gathered.
Language standards: Interpreted language, Dejango
Language architects: Guido van Rossum
From Programming to Modeling And Back AgainMarkus Voelter
Is programming = modeling? Are there differences, conceptual and tool-wise? Should there be differences? What if we programmed the way we model? Or vice versa? In this slidedeck I explore this question and introduce interesting developments in the space of projectional editing and modern parser technology. This leads to the concept of modular programming languages and a new way of looking at programming. I will demonstrate the idea with tools that are available today, for example TMF Xtext, JetBrains MPS and Intentional’s Domain Workbench.
JetBrains MPS: Projectional Editing in Domain-Specific LanguagesOscar Rodriguez
During this webinar, Václav Pech introduced the extent and potential the projectional editing technology has; and how our clients are using it in their domain-specific languages. He also touched on some philosophical ideas, suggesting that perhaps, code should not necessarily be text.
Webinar: https://www.youtube.com/watch?v=bw8X3wkBUr8&feature=youtu.be
65 - An Empirical Simulation-based Study of Real-Time Speech Translation for ...ESEM 2014
Context: Real-time speech translation technology is today available but still lacks a complete understanding of how such technology may affect communication in global software projects. Goal: To investigate the adoption of combining speech recognition and machine translation in order to overcome language barriers among stakeholders who are remotely negotiating software requirements.
Method: We performed an empirical simulation-based study including: Google Web Speech API and Google Translate service, two groups of four subjects, speaking Italian and Brazilian Portuguese, and a test set of 60 technical and non-technical utterances.
Results: Our findings revealed that, overall: (i) a satisfactory accuracy in terms of speech recognition was achieved, although significantly affected by speaker and utterance differences; (ii) adequate translations tend to follow accurate transcripts, meaning that speech recognition is the most critical part for speech translation technology.
Conclusions: Results provide a positive albeit initial evidence towards the possibility to use speech translation technologies to help globally distributed team members to communicate in their native languages.
Certainly! Here's a detailed 3000-word description of Python:
# Python: A Comprehensive Overview
Python is a high-level, versatile, and dynamically-typed programming language known for its simplicity and readability. Created by Guido van Rossum in the late 1980s, Python has since become one of the most popular programming languages worldwide. In this comprehensive overview, we will delve into the key aspects of Python, from its history and design philosophy to its syntax, libraries, and real-world applications.
## **History and Evolution of Python**
Python's history dates back to December 1989 when Guido van Rossum, a Dutch programmer, began working on it as a side project during his Christmas holidays. His aim was to create a language that emphasized code readability and allowed developers to express their ideas in fewer lines of code compared to other languages like C++ or Perl.
The first official Python release, Python 0.9.0, was released in February 1991. Python's name was inspired by Guido's love for the British comedy group Monty Python. Despite its humorous origins, Python quickly gained popularity in the software development community.
Python's major versions include Python 1.0 (1994), Python 2.0 (2000), Python 3.0 (2008), and the subsequent 3.x releases. The transition from Python 2 to Python 3 was a significant milestone in Python's history, as it involved breaking compatibility with Python 2 to introduce improvements and address some language inconsistencies. Python 2 reached its end of life on January 1, 2020, and Python 3 is now the standard and recommended version for new projects.
## **Design Philosophy: The Zen of Python**
Python's success can be attributed, in part, to its clear and guiding design principles, often referred to as "The Zen of Python" or "PEP 20" (Python Enhancement Proposal 20). These principles encapsulate the language's philosophy and provide a framework for writing clean, readable, and maintainable code. Some notable principles from "The Zen of Python" include:
- **Readability Counts:** Code should be easy to read and understand. Python's syntax enforces this with its use of indentation for block structure.
- **Simple is Better Than Complex:** Python encourages simplicity in both code design and implementation. It favors straightforward solutions over convoluted ones.
- **Explicit is Better Than Implicit:** Code should be explicit and not rely on hidden or magical behavior. This principle promotes code clarity and predictability.
- **There Should Be One-- and Preferably Only One --Obvious Way to Do It:** Python aims to provide a single, clear way to perform a specific task to reduce confusion and make code more consistent.
- **Errors Should Never Pass Silently:** Python encourages robust error handling and reporting to help developers identify and fix issues promptly.
## **Python Syntax and Language Features**
Python's syntax is known for its simplicity and readability. Here are some key languag
When it comes to backend development, there are several languages to choose from. Go and Python is two of the most popular programming languages for backend development.
Both languages have their advantages and disadvantages, making it difficult to decide which one best language for backend development. Both languages offer a range of features and benefits that can make them a great option for backend development.
Python is a versatile and widely-used high-level programming language known for its simplicity, readability, and extensive library support. Created by Guido van Rossum and first released in 1991, Python has since gained immense popularity across various domains, including web development, data science, scientific computing, artificial intelligence, and more. In this comprehensive description, we'll delve into Python's history, features, applications, and its vibrant community, highlighting why it continues to be a preferred choice for developers worldwide.
Table of Contents
Introduction to Python
Python's History and Evolution
Python's Key Features
3.1. Readability and Simplicity
3.2. High-level Language
3.3. Interpreted and Dynamic
3.4. Cross-platform Compatibility
3.5. Rich Standard Library
3.6. Community Support
Python's Application Domains
4.1. Web Development
4.2. Data Science and Machine Learning
4.3. Scientific Computing
4.4. Automation and Scripting
4.5. Game Development
4.6. Desktop Applications
Python Development Environments
5.1. IDLE
5.2. PyCharm
5.3. Jupyter Notebook
5.4. Visual Studio Code
Getting Started with Python
6.1. Installing Python
6.2. Your First Python Program
Python Syntax and Basic Concepts
7.1. Variables and Data Types
7.2. Conditional Statements
7.3. Loops
7.4. Functions
7.5. Exception Handling
Working with Python Libraries
8.1. NumPy
8.2. Pandas
8.3. Matplotlib
8.4. Scikit-Learn
Python and Web Development
9.1. Frameworks (Django, Flask)
9.2. Front-end Integration (HTML/CSS)
9.3. Database Interaction (SQL, NoSQL)
Python in Data Science
10.1. Data Analysis with Pandas
10.2. Data Visualization with Matplotlib and Seaborn
10.3. Machine Learning with Scikit-Learn
10.4. Deep Learning with TensorFlow and PyTorch
Scientific Computing with Python
11.1. Scientific Libraries (SciPy, SymPy)
11.2. Plotting and Visualization (Matplotlib)
Automation and Scripting
12.1. Automating Tasks
12.2. Scripting for System Administration
Game Development with Python
13.1. Pygame
13.2. Unity and Unreal Engine Integration
Desktop Applications with Python
14.1. Tkinter
14.2. PyQt
Python's Ecosystem and Package Management
Python Best Practices
16.1. Code Readability (PEP 8)
16.2. Documentation and Comments
16.3. Testing (Unit Testing, pytest)
16.4. Version Control (Git)
Python's Future and Trends
Conclusion
1. Introduction to Python
Python is a general-purpose, high-level programming language that was designed with a focus on code readability and simplicity. It uses an elegant and straightforward syntax that makes it easy for developers to express their ideas effectively, reducing the cost of program maintenance. Python's philosophy emphasizes the importance of code clarity and readability, which is encapsulated in the Zen of Python (PEP 20).
The language has gained immense popularity due to its versatility and a rich ecosystem of libraries and frameworks. Python is renowned for its vibrant community and extensive documentation, making it in p
R Vs Python – The most trending debate of aspiring Data Scientistsabhishekdf3
Now, it’s the time for a battle of two most demanding programming languages that is R vs Python. We will go deep in understanding the differences between the two languages. And, I assure you that you will not have any confusion left after completing this article i.e. R vs Python – the most trending debate of aspiring data scientists.
Learn more at :- https://data-flair.training/
How community software supports language documentation and data analysisPeter Bouda
Field linguists have increasingly adopted the latest technologies and tools for language documentation. Their needs have led to remarkable developments in software and archiving, exemplified by work at the MPI in Nijmegen, which leads the innovation cycles that take place in the digital working environments of field linguists. The next step in research is now the analysis and theoretical exploitation of the huge amount of data that has been collected in numerous language documentation projects that use these environments. This research will also rely on computer-based strategies, as data is instantly available in digital formats.
In this talk I will introduce some of the lesser known tools and software packages for annotation and analysis tasks. Some of these tools were created within DOBES projects and/or as community projects by small teams; they can be combined with well-known tools like ELAN or Toolbox to give researchers access to their data. I will focus on how a combination of simple, special purpose tools makes researchers more productive and how existing software libraries allow scientific projects to create their own, task-specific software tools that they can tailor to their own needs.
3. Only one language ?
R and Python
Disclaimer
• This is not about:
• The comparative merits of scripting languages
4. Only one language ?
R and Python
Disclaimer
• This is not about:
• The comparative merits of scripting languages
• Only Python & R
5. Only one language ?
R and Python
Disclaimer
• This is not about:
• The comparative merits of scripting languages
• Only Python & R
• This is about:
6. Only one language ?
R and Python
Disclaimer
• This is not about:
• The comparative merits of scripting languages
• Only Python & R
• This is about:
• Accessing natively libraries implemented in a different language
7. Only one language ?
R and Python
Disclaimer
• This is not about:
• The comparative merits of scripting languages
• Only Python & R
• This is about:
• Accessing natively libraries implemented in a different language
• Bridging people and skill sets through a glue language
8. Only one language ?
R and Python
Disclaimer
• This is not about:
• The comparative merits of scripting languages
• Only Python & R
• This is about:
• Accessing natively libraries implemented in a different language
• Bridging people and skill sets through a glue language
• Going from ideas to prototypes faster
9. Only one language ?
R and Python
Disclaimer
• This is not about:
• The comparative merits of scripting languages
• Only Python & R
• This is about:
• Accessing natively libraries implemented in a different language
• Bridging people and skill sets through a glue language
• Going from ideas to prototypes faster
• Putting some production into research
10. Only one language ?
R and Python
Disclaimer
• This is not about:
• The comparative merits of scripting languages
• Only Python & R
• This is about:
• Accessing natively libraries implemented in a different language
• Bridging people and skill sets through a glue language
• Going from ideas to prototypes faster
• Putting some production into research
• Bringing research to production
11. Only one language ?
R and Python
Preamble
Scenario
• data people: Statisticians, data analysts
12. Only one language ?
R and Python
Preamble
Scenario
• data people: Statisticians, data analysts
• Data people have a method M
13. Only one language ?
R and Python
Preamble
Scenario
• data people: Statisticians, data analysts
• Data people have a method M
• Data people want to work on something new
14. Only one language ?
R and Python
Preamble
Scenario
• data people: Statisticians, data analysts
• Data people have a method M
• Data people want to work on something new
• Management wants an application for method M
15. Only one language ?
R and Python
Preamble
Scenario
• data people: Statisticians, data analysts
• Data people have a method M
• Data people want to work on something new
• Management wants an application for method M
• Management wants an application that uses method M
16. Only one language ?
Polyglot programs
R and Python
1 Only one language ?
Polyglot programs
2 R and Python
Mapping types
Functions
Evaluation and memory
Building an application
17. Only one language ?
Polyglot programs
R and Python
Uniformity and coding standards
18. Only one language ?
Polyglot programs
R and Python
Uniformity and coding standards
19. Only one language ?
Polyglot programs
R and Python
Uniformity and coding standards
21. Only one language ?
Polyglot programs
R and Python
Get the job done
Soloist
22. Only one language ?
Polyglot programs
R and Python
Get the job done
Soloist
• Multi-talented individual
• Documentation ?
23. Only one language ?
Polyglot programs
R and Python
Get the job done
Teamwork
Soloist
• Multi-talented individual
• Documentation ?
24. Only one language ?
Polyglot programs
R and Python
Get the job done
Teamwork
Soloist
• Paired-programming
• Multi-talented individual
• Use the same tools ?
• Documentation ?
• Overlapping skills ?
25. Only one language ?
Polyglot programs
R and Python
Monolithic development
26. Only one language ?
Polyglot programs
R and Python
Monolithic development
• Centralized
• Top-down
• Lot of planning
• Long development, mostly only usable when complete
• Stand in time
27. Only one language ?
Polyglot programs
R and Python
Maintainability
Why use one unique language ?
• A legitimate managerial concern
• In places Java Certifications replaced general programming
degrees
• Could good programmers matter more than the language ?
• Back to finding a needle in a haystack
28. Only one language ?
Polyglot programs
R and Python
Modularity at the heart of UNIX philosophy.
|><
• No branching logic, unless going for shell scripts.
• Shell script no often thought after for applications
• The birth of scripting languages (Sed, Awk, Perl, . . . )
29. Only one language ?
Polyglot programs
R and Python
• Projects are cross-fields, cross-specialization
• Cost of specification - design - implementation too high
• Especially when the lifespan of the application is too short (or the
user base too small).
30. Only one language ?
Polyglot programs
R and Python
Example from video games
• Engines (generally in C++)
• Scripting language for the ‘story’ and content
• Python
• Lua
• Proprietary, others, . . .
• Large projects (with a lot of money at stake)
• Diverse competences (3D engine = story logic)
• When speed of development is more important than speed of
execution
This can apply to other industries
• Pipelines in visual effects
• Bioinformatics
31. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
1 Only one language ?
Polyglot programs
2 R and Python
Mapping types
Functions
Evaluation and memory
Building an application
32. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R
• Language for statistics, data analysis, and data visualization
• Unmatched1 number of libraries for anything having to do with
data
• Specialized set of libraries for bioinformatics (Bioconductor)
1
Almost certainly
33. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Python
• All-purpose scripting language
• Unmatched2 number of libraries for about anything
• Specialized sets of libraries for bioinformatics (Biopython, and a
myriad smaller projects)
2
May be
34. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Python (continued)
• Machine learning R does not have: PyBrain
• Visualization tools R does not have: Mayavis, Blender
35. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Python is popular in Bioinformatics / DNA sequencing.
• Galaxy pipeline/server framework is in Python
• Some of the internal tools for the SOLiD are written in Python
• Ion Torrent Server is a Django server
• Oxford Nanopore control system is a server running Python
36. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Why use anything else than R ?
• Build an application
• Work with very large data
• ‘Just because it can be done in R doesn’t mean you should do it’3
3
John Dennison, R Meetup presentation
37. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R embedded in Python
38. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
rpy2
• Feels like a regular Python library
• Embeds an R process
• Can be thought of as a stateful library
39. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Two main parts:
• Low-level interface
• High-level interface
40. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Low-level interface
• Close to R’s C-API
• Let you do anything safe4 from that API
• Expose R data structures as Python builtin structures
4
or so is the intent
41. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Types
R rpy2 Python
numeric FloatSexpVector float
integer IntSexpVector int
char StrSexpVector str
logical BoolSexpVector bool
complex ComplexSexpVector complex
list ListSexpVector list
environment SexpEnvironment dict
function SexpClosure function
S4 SexpS4 object
SexpLang object
SexpExtPtr object
42. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Vectors and arrays
• C-like: Contiguous blocks of memory
• R objects exposed to Python as sequences or C-like arrays, with
or without copy
43. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R
v <- seq(1, 10)
v[1] # select the first element
w <- v + 1 # add 1 to all elts
44. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R
v <- seq(1, 10)
v[1] # select the first element
w <- v + 1 # add 1 to all elts
rpy2.rinterface
import rpy2.rinterface as ri; ri.initr()
v = ri.IntSexpVector(range(1, 11))
v[0] # select the first element
w = ri.IntSexpVector([x+1 for x in v])
45. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R
v <- seq(1, 10)
v[1] # select the first element
w <- v + 1 # add 1 to all elts
rpy2.rinterface
import rpy2.rinterface as ri; ri.initr()
v = ri.IntSexpVector(range(1, 11))
v[0] # select the first element
w = ri.IntSexpVector([x+1 for x in v])
rpy2.robjects
import rpy2.robjects as ro
v = ro.IntVector(range(1, 11))
v[0] # select the first element
w = v.ro + 1
46. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Missing values
NaN:
numeric data type value representing an undefined or
unrepresentable value, especially in floating-point
calculations.
• Also used for missing values.
• Is a standard.
NA:
• Used for missing values by R.
• Not a standard.
• Pitfall when passing data to C without copy/checks
• Applies to any C libraries (includes rpy2)
47. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Functions
R functions can be called as if they were Python functions
import rpy2.robjects as ro
f = ro.r("function(x, y) { 2 * (x + y) }")
f(1, 2)
• conversion on-the-fly
• translated signatures (dot-to-underscore)
48. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Packages and modules
R
Namespaces attached to the search path
> searchpaths()
[1] ".GlobalEnv"
[2] "/usr/local/packages/R/2.15/lib/R/library/stats"
[3] "/usr/local/packages/R/2.15/lib/R/library/graphics"
[4] "/usr/local/packages/R/2.15/lib/R/library/grDevices"
[5] "/usr/local/packages/R/2.15/lib/R/library/utils"
[6] "/usr/local/packages/R/2.15/lib/R/library/datasets"
[7] "/usr/local/packages/R/2.15/lib/R/library/methods"
[8] "Autoloads"
[9] "/usr/local/packages/R/2.15/lib/R/library/base"
Python
Python modules as namespaces
import os
os.path.basename(’/path/to/a/file’)
49. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R packages (almost) as Python modules
from rpy2.robjects.packages import importr
stats = importr(’stats’)
# PCA !
pc = stats.prcomp(m)
50. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R scripts as modules !
from rpy2.robjects.packages import SignatureTranslatedAnonymousPackage
# R code in a file as a package
with open(’rflib.R’) as f:
code = ’’.join(f.readlines())
rf = SignatureTranslatedAnonymousPackage(code, "rf")
imp = rf.get_importance(dataf, response)
51. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R environments
• Associate symbols to objects
• Exposed as Python dictionaries (key - value)
R
env <- new.env()
assign(’x’, 123, envir = env)
y <- 456
Python
import rpy2.robjects as ro
env = ro.Environment()
env[’x’] = 123
ro.globalenv[’y’] = 456
52. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R and callback functions
Common R idiom
# m: matrix of numerical values
f <- function(x) sum(x[x > 0])
res <- apply(m, 1, f)
How to do that with rpy2 ?
53. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R and callback functions
Common R idiom
# m: matrix of numerical values
f <- function(x) sum(x[x > 0])
res <- apply(m, 1, f)
How to do that with rpy2 ?
import rpy2.interactive as r
import rpy2.rinterface as ri
r_code = """
function(x)
sum(x[x > 0])
"""
tmp = ri.parse(r_code)
eval = r.packages.base.eval
r_func = eval(tmp)
r.base.apply(m, 1, r_func)
54. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R and callback functions
Common R idiom
# m: matrix of numerical values
f <- function(x) sum(x[x > 0])
res <- apply(m, 1, f)
How to do that with rpy2 ?
import rpy2.interactive as r
import rpy2.rinterface as ri
def tmp(x):
gnr = elt for elt in x
if elt > 0
return sum(gnr)
r_func = ri.rternalize(tmp)
r.base.apply(m, 1, r_func)
55. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Evaluation strategies
R
• Pass-by-value / Call-by-value
• Modifying an object locally is always safe
• Unncessary copies
Python
• Pass-by-reference
• Explicit request if copy
Rpy2 exposes R as if it was pass-by-reference
56. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Python
from rpy2.robjects.vectors import IntVector
def f(x):
x[0] = 123
v = ro.IntVector(range(1, 11))
f(v)
R
f <- function(x) {
x[0] = 123
return(x)
}
v = seq(1, 11)
v = f(v)
57. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Memory management and garbage collection
58. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Memory management and garbage collection
R
• Tracing GC (check for reachability)
• R PreciousList
59. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Memory management and garbage collection
R
• Tracing GC (check for reachability)
• R PreciousList
Python
• Reference counting
• Tracing GC
60. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Memory management and garbage collection
R
• Tracing GC (check for reachability)
• R PreciousList
Python
• Reference counting
• Tracing GC
• Bridge different memory models
• Intermediate reference counting of R objects exposed
• That part could become very generic.
61. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R objects exposed to R
import rpy2.rinterface as ri
ri.initr()
baseenv = ri.baseenv
letters = baseenv.get(’letters’)
62. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R objects exposed to R (not so simple)
import rpy2.rinterface as ri
ri.initr()
base = ri.baseenv
letters = base[’letters’]
63. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
>>> letters = base[’letters’]
>>> letters.rid # varies
123456
>>> letters.__sexp_refcount__
1
>>> letters2 = base[’letters’]
>>> letters2.__sexp_refcount__
2
>>> letters.__sexp_refcount__
2
>>> letters_2.rid # same R ID
123456
64. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Exceptions
RRuntimeError: error while evaluating R code
KeyError: symbol not found in an environment
ValueError: invalid value passed to an rpy2 function
65. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Performances
function(x) {
total = 0;
for (elt in x) {
total <- total + elt
}
}
Function Sequence Speedup
R 1.00
R compiled 6.52
R builtin 329.29
pure python FloatVector 0.51
builtin python FloatVector 0.54
pure python SexpVector 7.45
builtin python SexpVector 20.92
builtin python array.array 53.62
builtin python list 90.47
R through rpy2 can be faster than R
66. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Let’s build a web application
• Why do that ?
• Allow access to computing ressources
• Use the UI of the browser
• Good example
• Micro web framework: Flask
67. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Hello world with Flask
from flask import Flask
app = Flask(__name__)
@app.route(’/’)
def hello_world():
return ’Hello World!’
if __name__ == ’__main__’:
app.run()
python hello.py
68. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Importance of variables with random forest
1 Data in a CSV file
2 Use R to compute a random forest and compute importance of
variables
3 Make a pretty plot with ggplot2
69. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
70. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
## data
dataf <- read.csv("/some/data/file.csv")
response <- ’var_name’
## importance of variables
library(randomForest)
get_importance <- function(dataf, response) {
fmla <- formula(paste(response, ’˜ .’))
dataf_rf <- randomForest(fmla, data = dataf,
keep.forest = FALSE,
importance = TRUE)
imp <- importance(dataf_rf, type = 1)
imp <- as.data.frame(imp[order(imp[,1]), , drop=FALSE]) 10
return(imp)
%IncMSE
} 12.5
%IncMSE
10.0
7.5
imp <- get_importance(dataf, response) 5.0
5
## plot
library(ggplot2)
get_plot <- function(imp) {
rn <- rownames(imp)
0
rn <- factor(rn, levels=rn, ordered=TRUE)
gear qsec carb drat am vs cyl wt hp disp
imp <- cbind(as.data.frame(imp), rn = rn) Variables
p = ggplot(imp) +
geom_bar(aes(y = ‘%IncMSE‘,
x = rn,
fill = ‘%IncMSE‘)) +
scale_x_discrete("Variables")
return(p)
}
p <- get_plot(imp)
print(p)
71. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R library
1 get_dataframe <- function(filename) {
2 return(read.csv(filename))
3 }
4
5 ## importance of variables
6 library(randomForest)
7 get_importance <- function(dataf, response) {
8 fmla <- formula(paste(response, ’˜ .’))
9 dataf_rf <- randomForest(fmla, data = dataf,
10 keep.forest = FALSE,
11 importance = TRUE)
12 imp <- importance(dataf_rf, type = 1)
13 imp <- as.data.frame(imp[order(imp[,1]), , drop=FALSE])
14 return(imp)
15 }
72. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
R library
17 ## plot
18 library(ggplot2)
19 get_plot <- function(imp) {
20 rn <- rownames(imp)
21 rn <- factor(rn, levels=rn, ordered=TRUE)
22 imp <- cbind(as.data.frame(imp), rn = rn)
23 p = ggplot(imp) +
24 geom_bar(aes(y = ‘%IncMSE‘,
25 x = rn,
26 fill = ‘%IncMSE‘)) +
27 scale_x_discrete("Variables")
28 return(p)
29 }
30
31 make_PNGplot <- function(imp, dir) {
32 filename <- tempfile(tmpdir = dir, fileext = ’.png’)
33 p <- get_plot(imp)
34 png(filename)
35 print(p)
36 dev.off()
37 return(basename(filename))
38 }
73. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Python application
1 import os
2 from flask import Flask, render_template, flash
3 from flask import url_for, send_from_directory
4 from flask import request
5 from werkzeug import secure_filename
6 from rpy2.robjects.packages import SignatureTranslatedAnonymousPackage
7
8 UPLOAD_FOLDER = ’/tmp’
9
10 # R code as a package
11 with open(’rflib.R’) as f:
12 code = ’’.join(f.readlines())
13 rf = SignatureTranslatedAnonymousPackage(code, "rf")
74. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
15 # create application
16 app = Flask(__name__)
17 app.secret_key = ’change this !!!’
18 app.config[’UPLOAD_FOLDER’] = UPLOAD_FOLDER
19
20 # serve files
21 @app.route(’/files/<filename>’)
22 def files(filename):
23 return send_from_directory(UPLOAD_FOLDER,
24 filename)
75. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
15 # create application
16 app = Flask(__name__)
17 app.secret_key = ’change this !!!’
18 app.config[’UPLOAD_FOLDER’] = UPLOAD_FOLDER
19
20 # serve files
21 @app.route(’/files/<filename>’)
22 def files(filename):
23 return send_from_directory(UPLOAD_FOLDER,
24 filename)
25
26 def plot(dataf, response):
27 # compute importance of variables
28 imp = rf.get_importance(dataf, response)
29 # plot into a file
30 plot_fn = rf.make_PNGplot(imp, UPLOAD_FOLDER)[0]
31 return url_for(’files’, filename = plot_fn)
76. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
33 # main function
34 @app.route(’/’, methods=[’GET’, ’POST’])
35 def index():
36 plot_url = None
37 # test if data posted
38 if request.method == ’POST’:
39 f = request.files[’data’]
40 response = request.form[’response’]
41 # test is file ’data’ uploaded
42 if f:
43 # save the uploaded file
44 filename = secure_filename(f.filename)
45 f.save(os.path.join(app.config[’UPLOAD_FOLDER’], filename))
46 # get R data.frame from the file
47 dataf = rf.get_dataframe(f.filename)
48 # check if response variable is present
49 if response in dataf.names:
50 plot_url = plot(dataf, response)
51 else:
52 flash(’No such response variable’, category = ’error’)
53 else:
54 flash(’Invalid file extension’, category = ’error’)
55 #
56 return render_template(’index.html’, plot_url = plot_url)
77. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Showtime. . .
78. Mapping types
Only one language ? Functions
R and Python Evaluation and memory
Building an application
Next steps
• Generic library to bridge R to anything with a C API
• Julia and R (hopefully end of 2012)