- Andre Pemmelaar has experience in quantitative finance and uses Julia for statistical arbitrage and algorithmic trading.
- He discusses his journey to adopting Julia including initial skepticism due to immaturity, but finding the language easy to use for reinforcement learning and order book simulation projects.
- Pemmelaar provides tips for introducing Julia to others within an organization, including documenting examples to overcome lack of documentation, choosing a standardized environment to ease troubleshooting, and showing success stories to gain adoption.
4. My road to using
• Started reading about it in late 2012 ~ early 2013
• Wrote to John Myles White in Spring 2013
John Myles White 3.20.2013 at 9:38 am | Permalink
Hi Andre,
!
In the abstract, I think Julia is the ideal language for doing both prototype modeling and transition to
production.
!
But Julia is still very immature as a language, so I would not recommend it being used in production
for another year or so. In addition, if you’re looking for an existing toolbox of models, R is the way to
go. Even Python has still not caught up with R in this regard.
@QuantixResearch
• Decided it was too early -> kept following, but didn’t use
5. My road to using
• Revisited ~ early 2014
• Began trying some simple projects
• Reinforcement Learning using tictactoe.jl
• Found the code very easy to follow
• Started using the DataFrame.jl
• Found it to be very stable and close enough to Panda (python)
• Started writing first serious attempt at something important in May 2014
• Orderbook simulation frame work
• Joined new company 3 months ago - using Julia almost exclusively for 3
month on real world problems in Finance
@QuantixResearch
6. My Moment
Realized I could…
• Remain mostly functional in my approach to programming (but not 100%)
• Use fast for loops wherever appropriate (used in a lot of time series simulations)
• Easily code linear algebra, matrix calculations for machine learning, etc.(native in
Julia)
• Do it all it parallel (note: Julia’s parallel not yet 100% there yet)
• All of the above can be done in Python (Sci-kit, Numpy, etc) but often faster and
with slightly less code in Julia
@QuantixResearch
8. Some background
on my company
• One of Japan’s largest financial front-office system solution providers
• Started off in derivative valuation and derivative OMS systems
• Now offers an entire suite of products aimed at Japanese mega banks, and
2nd-tier financial organization
• About 600 employees (about 60%~70 are technical)
• Primarily production language is company isJava, with some done in C++,
or C
• Quantitative analysis is done in Java (heavy duty large data set analysis) or
R for smaller datasets) with a few using Python users
• Most quants are focused on Risk or Valuation, but a smaller team (mine)
makes use of predictive analytics, statistics, and ML to enhance various
algorithms
@QuantixResearch
9. Nothing sells like success
• It helps to have a successful example to sell it internally
• In my case, during my first week I found some R code that was
used every night (had lots of loops = ripe for porting to Julia)
• Re-wrote in Julia ->
• R took about 15:46m
• Java about +/- 20s
• Julia about 4.3 secs
• Note: Better Java programmer recently bested Julia version (3.9
secs)
11. Making the first
experience easier
• Set the expectation correctly
• Documentation is sparse.
• The stuff that is out there may not be current
• Julia is fast, but can lose a lot of speed if coded improperly
@QuantixResearch
Poor Performance
Better Performance
12. Roadblocks
I asked Julia colleagues, “What are/were the 3
biggest hurdles”
#1 Lack of documentation
#2 Lack of current documentation
#3 Package breaking/incompatibility on update
No one said bugs in base code, or lack of
some critical feature. Everyone wants correct,
examples of “here’s how you do this”
to initial adoption
14. DIY Documentation
• Julia base documentation is good
• The package’s docs vary greatly
• The one great example is Gadfly
• Code, output, & explanation
• Not so great doc ex: DataFrame
• No longer current
• Many common tasks missing
• Create you own documentation
• The single most difficult part of
learning Julia is the lack of
current correct examples
• IJulia is fantastic for creating
these!
• My Advice
• Initially target early users
cases
• DIY Document anything
people are struggling with
@QuantixResearch
15. Decide on the
environment/tools
IJulia
LightTables + Jewel
@QuantixResearch
16. Decide on the
environment & tools
• Julia is still new enough that small upgrades can
break critical packages
• As the initial “Julia person” in your organization
you will often be called on to solve various
problems
• Solving new users problems is much easier if they
are using the same tools and packages. Don’t
underestimate this!
• At the beginning sharing exactly the same
environment will make things smoother
• Recommend one person download the
installers
• Create an thorough install read me file
Our stack:
• Julia 3.1
• IJulia
• Light Tables
@QuantixResearch
17. How did we do?
• 6 people set out to learn Julia
• 4 of them are now using it everyday
• 1 is using it occasional along with Perl
• 1 gave up
• Why did that one give up?
• He as serious Java skills and good R
• Started with Julia Studio (bad 1 st
experience)
• Didn’t know about Light tables
• Is physically separated from the rest
of us and thus didn’t get initial support
to get through the initial low
productivity period
@QuantixResearch
18. Julia: Real example
Rejection Order Algorithm
• The model:
• Determine if a order to lift a quote (execute against someones else's quote)
in an OTC markets will be rejected
• Background: OTC market are “over the counter” and depending on the
rules, the quoter can reject your order if it suits them
• Julia tools used:
• DataFrame.jl, StatsBase.jl, DecisionTrees.jl, SVM.jl
• Classification problem: 0 not rejected, 1 rejected
• Still on-going project: current best is about 0.54 Kappa
19. Julia: Real example
Rejection order algorithm con’t
Very unbalanced classes (0.1% are rejected)
• Regime shift means it needs to be somewhat
adaptive
• Required us to change some of the libraries
• One of Julia’s great strength’s is that you can easily
changed the libraries to suit you needs
20. What makes Julia great?
• Speed? Julia is quite good, but Java can be as fast or faster. C
++ and C are faster
• Time to get a model out? Largely dependent on your
knowledge of the tools you are using
• Parallelization? Not really. Still kinda raw. Memory usage can be
a bit of an issue.
• Safer code via Functional approach? No. One can code
functionally but doesn’t enforce it
• Easy to code and to access/read/understand others code? Yes
21. What makes Julia great?
√ Java
Δ Python (Cython,etc)
Δ R (vectorized)
When coded well, it
Clear, concise
code that can easily
be changed
is very fast
Great ability to mix
loop based &
matrix/vector
operations
Δ Java (not really)
√ Python
Δ Java (not concise) Δ R (only vectorized)
√ Python
Δ R (only R code.
not C or C++)