SlideShare a Scribd company logo
1 of 22
Download to read offline
|> inside the 
corporation 
Andre Pemmelaar @QuantixResearch
About Me 
Andre Pemmelaar 
• 5-yrs Matsushita Financial System Solutions (Panasonic) 
• 12 Buy-Side Finance 
• 7-yrs Japanese Gov’t Bond Options Market Maker (HNL) 
• 5-yrs Statistical Arbitrage (Global Equities) 
• Low latency & Quantitative Algorithm 
• Primarily use mixture of basic statistics and machine 
learning 
• R,Python,Java, F# …. and of course JULIA! 
• Prefer function programming approach (F#, Scala, Haskell) 
@QuantixResearch
My road to
My road to using 
• Started reading about it in late 2012 ~ early 2013 
• Wrote to John Myles White in Spring 2013 
John Myles White 3.20.2013 at 9:38 am | Permalink 
Hi Andre, 
! 
In the abstract, I think Julia is the ideal language for doing both prototype modeling and transition to 
production. 
! 
But Julia is still very immature as a language, so I would not recommend it being used in production 
for another year or so. In addition, if you’re looking for an existing toolbox of models, R is the way to 
go. Even Python has still not caught up with R in this regard. 
@QuantixResearch 
• Decided it was too early -> kept following, but didn’t use
My road to using 
• Revisited ~ early 2014 
• Began trying some simple projects 
• Reinforcement Learning using tictactoe.jl 
• Found the code very easy to follow 
• Started using the DataFrame.jl 
• Found it to be very stable and close enough to Panda (python) 
• Started writing first serious attempt at something important in May 2014 
• Orderbook simulation frame work 
• Joined new company 3 months ago - using Julia almost exclusively for 3 
month on real world problems in Finance 
@QuantixResearch
My Moment 
Realized I could… 
• Remain mostly functional in my approach to programming (but not 100%) 
• Use fast for loops wherever appropriate (used in a lot of time series simulations) 
• Easily code linear algebra, matrix calculations for machine learning, etc.(native in 
Julia) 
• Do it all it parallel (note: Julia’s parallel not yet 100% there yet) 
• All of the above can be done in Python (Sci-kit, Numpy, etc) but often faster and 
with slightly less code in Julia 
@QuantixResearch
carefully 
insert here
Some background 
on my company 
• One of Japan’s largest financial front-office system solution providers 
• Started off in derivative valuation and derivative OMS systems 
• Now offers an entire suite of products aimed at Japanese mega banks, and 
2nd-tier financial organization 
• About 600 employees (about 60%~70 are technical) 
• Primarily production language is company isJava, with some done in C++, 
or C 
• Quantitative analysis is done in Java (heavy duty large data set analysis) or 
R for smaller datasets) with a few using Python users 
• Most quants are focused on Risk or Valuation, but a smaller team (mine) 
makes use of predictive analytics, statistics, and ML to enhance various 
algorithms 
@QuantixResearch
Nothing sells like success 
• It helps to have a successful example to sell it internally 
• In my case, during my first week I found some R code that was 
used every night (had lots of loops = ripe for porting to Julia) 
• Re-wrote in Julia -> 
• R took about 15:46m 
• Java about +/- 20s 
• Julia about 4.3 secs 
• Note: Better Java programmer recently bested Julia version (3.9 
secs)
On boarding new users
Making the first 
experience easier 
• Set the expectation correctly 
• Documentation is sparse. 
• The stuff that is out there may not be current 
• Julia is fast, but can lose a lot of speed if coded improperly 
@QuantixResearch 
Poor Performance 
Better Performance
Roadblocks 
I asked Julia colleagues, “What are/were the 3 
biggest hurdles” 
#1 Lack of documentation 
#2 Lack of current documentation 
#3 Package breaking/incompatibility on update 
No one said bugs in base code, or lack of 
some critical feature. Everyone wants correct, 
examples of “here’s how you do this” 
to initial adoption
Roadblocks 
Really just two problems 
1.Documentation 
2.Update Chaos 
to initial adoption
DIY Documentation 
• Julia base documentation is good 
• The package’s docs vary greatly 
• The one great example is Gadfly 
• Code, output, & explanation 
• Not so great doc ex: DataFrame 
• No longer current 
• Many common tasks missing 
• Create you own documentation 
• The single most difficult part of 
learning Julia is the lack of 
current correct examples 
• IJulia is fantastic for creating 
these! 
• My Advice 
• Initially target early users 
cases 
• DIY Document anything 
people are struggling with 
@QuantixResearch
Decide on the 
environment/tools 
IJulia 
LightTables + Jewel 
@QuantixResearch
Decide on the 
environment & tools 
• Julia is still new enough that small upgrades can 
break critical packages 
• As the initial “Julia person” in your organization 
you will often be called on to solve various 
problems 
• Solving new users problems is much easier if they 
are using the same tools and packages. Don’t 
underestimate this! 
• At the beginning sharing exactly the same 
environment will make things smoother 
• Recommend one person download the 
installers 
• Create an thorough install read me file 
Our stack: 
• Julia 3.1 
• IJulia 
• Light Tables 
@QuantixResearch
How did we do? 
• 6 people set out to learn Julia 
• 4 of them are now using it everyday 
• 1 is using it occasional along with Perl 
• 1 gave up 
• Why did that one give up? 
• He as serious Java skills and good R 
• Started with Julia Studio (bad 1 st 
experience) 
• Didn’t know about Light tables 
• Is physically separated from the rest 
of us and thus didn’t get initial support 
to get through the initial low 
productivity period 
@QuantixResearch
Julia: Real example 
Rejection Order Algorithm 
• The model: 
• Determine if a order to lift a quote (execute against someones else's quote) 
in an OTC markets will be rejected 
• Background: OTC market are “over the counter” and depending on the 
rules, the quoter can reject your order if it suits them 
• Julia tools used: 
• DataFrame.jl, StatsBase.jl, DecisionTrees.jl, SVM.jl 
• Classification problem: 0 not rejected, 1 rejected 
• Still on-going project: current best is about 0.54 Kappa
Julia: Real example 
Rejection order algorithm con’t 
Very unbalanced classes (0.1% are rejected) 
• Regime shift means it needs to be somewhat 
adaptive 
• Required us to change some of the libraries 
• One of Julia’s great strength’s is that you can easily 
changed the libraries to suit you needs
What makes Julia great? 
• Speed? Julia is quite good, but Java can be as fast or faster. C 
++ and C are faster 
• Time to get a model out? Largely dependent on your 
knowledge of the tools you are using 
• Parallelization? Not really. Still kinda raw. Memory usage can be 
a bit of an issue. 
• Safer code via Functional approach? No. One can code 
functionally but doesn’t enforce it 
• Easy to code and to access/read/understand others code? Yes
What makes Julia great? 
√ Java 
Δ Python (Cython,etc) 
Δ R (vectorized) 
When coded well, it 
Clear, concise 
code that can easily 
be changed 
is very fast 
Great ability to mix 
loop based & 
matrix/vector 
operations 
Δ Java (not really) 
√ Python 
Δ Java (not concise) Δ R (only vectorized) 
√ Python 
Δ R (only R code. 
not C or C++)
Thank You!

More Related Content

What's hot

NLP Transfer learning platform
NLP Transfer learning platformNLP Transfer learning platform
NLP Transfer learning platformmanusuryavansh
 
Swift: A parallel scripting for applications at the petascale and beyond.
Swift: A parallel scripting for applications at the petascale and beyond.Swift: A parallel scripting for applications at the petascale and beyond.
Swift: A parallel scripting for applications at the petascale and beyond.Nagasuri Bala Venkateswarlu
 
Ryan Curtin, Principal Research Scientist, Symantec at MLconf ATL 2016
Ryan Curtin, Principal Research Scientist, Symantec at MLconf ATL 2016Ryan Curtin, Principal Research Scientist, Symantec at MLconf ATL 2016
Ryan Curtin, Principal Research Scientist, Symantec at MLconf ATL 2016MLconf
 
Deep learning for NLP and Transformer
 Deep learning for NLP  and Transformer Deep learning for NLP  and Transformer
Deep learning for NLP and TransformerArvind Devaraj
 
Coding For Cores - C# Way
Coding For Cores - C# WayCoding For Cores - C# Way
Coding For Cores - C# WayBishnu Rawal
 
NLP using transformers
NLP using transformers NLP using transformers
NLP using transformers Arvind Devaraj
 
Feature Engineering for NLP
Feature Engineering for NLPFeature Engineering for NLP
Feature Engineering for NLPBill Liu
 
A Folksonomy of styles, aka: other stylists also said and Subjective Influenc...
A Folksonomy of styles, aka: other stylists also said and Subjective Influenc...A Folksonomy of styles, aka: other stylists also said and Subjective Influenc...
A Folksonomy of styles, aka: other stylists also said and Subjective Influenc...Natalia Díaz Rodríguez
 
Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016
Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016
Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016MLconf
 
Deep learning on Hadoop/Spark -NextML
Deep learning on Hadoop/Spark -NextMLDeep learning on Hadoop/Spark -NextML
Deep learning on Hadoop/Spark -NextMLAdam Gibson
 
DAWN and Scientific Workflows
DAWN and Scientific WorkflowsDAWN and Scientific Workflows
DAWN and Scientific WorkflowsMatthew Gerring
 
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog DetectorDeep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog DetectorRoelof Pieters
 
[Impl] neural machine translation
[Impl] neural machine translation[Impl] neural machine translation
[Impl] neural machine translationJaeHo Jang
 
Keras Tutorial For Beginners | Creating Deep Learning Models Using Keras In P...
Keras Tutorial For Beginners | Creating Deep Learning Models Using Keras In P...Keras Tutorial For Beginners | Creating Deep Learning Models Using Keras In P...
Keras Tutorial For Beginners | Creating Deep Learning Models Using Keras In P...Edureka!
 
Are High Level Programming Languages for Multicore and Safety Critical Conver...
Are High Level Programming Languages for Multicore and Safety Critical Conver...Are High Level Programming Languages for Multicore and Safety Critical Conver...
Are High Level Programming Languages for Multicore and Safety Critical Conver...InfinIT - Innovationsnetværket for it
 
Introduction to Keras
Introduction to KerasIntroduction to Keras
Introduction to KerasJohn Ramey
 
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017StampedeCon
 

What's hot (20)

NLP Transfer learning platform
NLP Transfer learning platformNLP Transfer learning platform
NLP Transfer learning platform
 
Swift: A parallel scripting for applications at the petascale and beyond.
Swift: A parallel scripting for applications at the petascale and beyond.Swift: A parallel scripting for applications at the petascale and beyond.
Swift: A parallel scripting for applications at the petascale and beyond.
 
Ryan Curtin, Principal Research Scientist, Symantec at MLconf ATL 2016
Ryan Curtin, Principal Research Scientist, Symantec at MLconf ATL 2016Ryan Curtin, Principal Research Scientist, Symantec at MLconf ATL 2016
Ryan Curtin, Principal Research Scientist, Symantec at MLconf ATL 2016
 
Deep learning for NLP and Transformer
 Deep learning for NLP  and Transformer Deep learning for NLP  and Transformer
Deep learning for NLP and Transformer
 
Coding For Cores - C# Way
Coding For Cores - C# WayCoding For Cores - C# Way
Coding For Cores - C# Way
 
NLP using transformers
NLP using transformers NLP using transformers
NLP using transformers
 
Feature Engineering for NLP
Feature Engineering for NLPFeature Engineering for NLP
Feature Engineering for NLP
 
A Folksonomy of styles, aka: other stylists also said and Subjective Influenc...
A Folksonomy of styles, aka: other stylists also said and Subjective Influenc...A Folksonomy of styles, aka: other stylists also said and Subjective Influenc...
A Folksonomy of styles, aka: other stylists also said and Subjective Influenc...
 
Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016
Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016
Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016
 
Deep learning on Hadoop/Spark -NextML
Deep learning on Hadoop/Spark -NextMLDeep learning on Hadoop/Spark -NextML
Deep learning on Hadoop/Spark -NextML
 
Introduction to Transformer Model
Introduction to Transformer ModelIntroduction to Transformer Model
Introduction to Transformer Model
 
DAWN and Scientific Workflows
DAWN and Scientific WorkflowsDAWN and Scientific Workflows
DAWN and Scientific Workflows
 
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog DetectorDeep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
 
[Impl] neural machine translation
[Impl] neural machine translation[Impl] neural machine translation
[Impl] neural machine translation
 
Tensorflow Ecosystem
Tensorflow EcosystemTensorflow Ecosystem
Tensorflow Ecosystem
 
Keras Tutorial For Beginners | Creating Deep Learning Models Using Keras In P...
Keras Tutorial For Beginners | Creating Deep Learning Models Using Keras In P...Keras Tutorial For Beginners | Creating Deep Learning Models Using Keras In P...
Keras Tutorial For Beginners | Creating Deep Learning Models Using Keras In P...
 
Are High Level Programming Languages for Multicore and Safety Critical Conver...
Are High Level Programming Languages for Multicore and Safety Critical Conver...Are High Level Programming Languages for Multicore and Safety Critical Conver...
Are High Level Programming Languages for Multicore and Safety Critical Conver...
 
Introduction to Keras
Introduction to KerasIntroduction to Keras
Introduction to Keras
 
Practical Deep Learning
Practical Deep LearningPractical Deep Learning
Practical Deep Learning
 
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
 

Viewers also liked

Data analysis with R and Julia
Data analysis with R and JuliaData analysis with R and Julia
Data analysis with R and JuliaMark Tabladillo
 
High performance computing language,julia
High performance computing language,juliaHigh performance computing language,julia
High performance computing language,juliaAnusha sweety
 
XML Tools for Perl
XML Tools for PerlXML Tools for Perl
XML Tools for PerlGeir Aalberg
 
GoOpen/Nordic Perl Workshop 2009
GoOpen/Nordic Perl Workshop 2009GoOpen/Nordic Perl Workshop 2009
GoOpen/Nordic Perl Workshop 2009Geir Aalberg
 
julia-Latest Programming language
julia-Latest Programming  languagejulia-Latest Programming  language
julia-Latest Programming languageNithya Prakasan
 
Julia meetup bangalore
Julia meetup bangaloreJulia meetup bangalore
Julia meetup bangaloreKrishna Kalyan
 
All About that Bayes: Probability, Statistics, and the Quest to Quantify Unce...
All About that Bayes: Probability, Statistics, and the Quest to Quantify Unce...All About that Bayes: Probability, Statistics, and the Quest to Quantify Unce...
All About that Bayes: Probability, Statistics, and the Quest to Quantify Unce...Lawrence Livermore National Laboratory
 
Portunus webinar: Floating port technologies could revolutionize container co...
Portunus webinar: Floating port technologies could revolutionize container co...Portunus webinar: Floating port technologies could revolutionize container co...
Portunus webinar: Floating port technologies could revolutionize container co...Lawrence Livermore National Laboratory
 
XML's validation - XML Schema
XML's validation - XML SchemaXML's validation - XML Schema
XML's validation - XML Schemavidede_group
 
Julia - Easier, Better, Faster, Stronger
Julia - Easier, Better, Faster, StrongerJulia - Easier, Better, Faster, Stronger
Julia - Easier, Better, Faster, StrongerKenta Sato
 
Introduction to Julia for bioinformacis
Introduction to Julia for bioinformacisIntroduction to Julia for bioinformacis
Introduction to Julia for bioinformacisKenta Sato
 
Apache Spark超入門 (Hadoop / Spark Conference Japan 2016 講演資料)
Apache Spark超入門 (Hadoop / Spark Conference Japan 2016 講演資料)Apache Spark超入門 (Hadoop / Spark Conference Japan 2016 講演資料)
Apache Spark超入門 (Hadoop / Spark Conference Japan 2016 講演資料)NTT DATA OSS Professional Services
 
ちょっと理解に自信がないな という皆さまに贈るHadoop/Sparkのキホン (IBM Datapalooza Tokyo 2016講演資料)
ちょっと理解に自信がないなという皆さまに贈るHadoop/Sparkのキホン (IBM Datapalooza Tokyo 2016講演資料)ちょっと理解に自信がないなという皆さまに贈るHadoop/Sparkのキホン (IBM Datapalooza Tokyo 2016講演資料)
ちょっと理解に自信がないな という皆さまに贈るHadoop/Sparkのキホン (IBM Datapalooza Tokyo 2016講演資料)hamaken
 
Advanced Production Control Using Julia & IMPL
Advanced Production Control Using Julia & IMPLAdvanced Production Control Using Julia & IMPL
Advanced Production Control Using Julia & IMPLAlkis Vazacopoulos
 
Apache Sparkに手を出してヤケドしないための基本 ~「Apache Spark入門より」~ (デブサミ 2016 講演資料)
Apache Sparkに手を出してヤケドしないための基本 ~「Apache Spark入門より」~ (デブサミ 2016 講演資料)Apache Sparkに手を出してヤケドしないための基本 ~「Apache Spark入門より」~ (デブサミ 2016 講演資料)
Apache Sparkに手を出してヤケドしないための基本 ~「Apache Spark入門より」~ (デブサミ 2016 講演資料)NTT DATA OSS Professional Services
 
Julia Computing - an alternative to Hadoop
Julia Computing - an alternative to HadoopJulia Computing - an alternative to Hadoop
Julia Computing - an alternative to HadoopShaurya Shekhar
 

Viewers also liked (19)

Data analysis with R and Julia
Data analysis with R and JuliaData analysis with R and Julia
Data analysis with R and Julia
 
High performance computing language,julia
High performance computing language,juliaHigh performance computing language,julia
High performance computing language,julia
 
XML Tools for Perl
XML Tools for PerlXML Tools for Perl
XML Tools for Perl
 
GoOpen/Nordic Perl Workshop 2009
GoOpen/Nordic Perl Workshop 2009GoOpen/Nordic Perl Workshop 2009
GoOpen/Nordic Perl Workshop 2009
 
Everything wrong with statistics (and how to fix it)
Everything wrong with statistics (and how to fix it)Everything wrong with statistics (and how to fix it)
Everything wrong with statistics (and how to fix it)
 
julia-Latest Programming language
julia-Latest Programming  languagejulia-Latest Programming  language
julia-Latest Programming language
 
Julia meetup bangalore
Julia meetup bangaloreJulia meetup bangalore
Julia meetup bangalore
 
All About that Bayes: Probability, Statistics, and the Quest to Quantify Unce...
All About that Bayes: Probability, Statistics, and the Quest to Quantify Unce...All About that Bayes: Probability, Statistics, and the Quest to Quantify Unce...
All About that Bayes: Probability, Statistics, and the Quest to Quantify Unce...
 
Portunus webinar: Floating port technologies could revolutionize container co...
Portunus webinar: Floating port technologies could revolutionize container co...Portunus webinar: Floating port technologies could revolutionize container co...
Portunus webinar: Floating port technologies could revolutionize container co...
 
XML's validation - XML Schema
XML's validation - XML SchemaXML's validation - XML Schema
XML's validation - XML Schema
 
Introduction to Julia Language
Introduction to Julia LanguageIntroduction to Julia Language
Introduction to Julia Language
 
Julia - Easier, Better, Faster, Stronger
Julia - Easier, Better, Faster, StrongerJulia - Easier, Better, Faster, Stronger
Julia - Easier, Better, Faster, Stronger
 
Introduction to Julia for bioinformacis
Introduction to Julia for bioinformacisIntroduction to Julia for bioinformacis
Introduction to Julia for bioinformacis
 
Spark MLlibではじめるスケーラブルな機械学習
Spark MLlibではじめるスケーラブルな機械学習Spark MLlibではじめるスケーラブルな機械学習
Spark MLlibではじめるスケーラブルな機械学習
 
Apache Spark超入門 (Hadoop / Spark Conference Japan 2016 講演資料)
Apache Spark超入門 (Hadoop / Spark Conference Japan 2016 講演資料)Apache Spark超入門 (Hadoop / Spark Conference Japan 2016 講演資料)
Apache Spark超入門 (Hadoop / Spark Conference Japan 2016 講演資料)
 
ちょっと理解に自信がないな という皆さまに贈るHadoop/Sparkのキホン (IBM Datapalooza Tokyo 2016講演資料)
ちょっと理解に自信がないなという皆さまに贈るHadoop/Sparkのキホン (IBM Datapalooza Tokyo 2016講演資料)ちょっと理解に自信がないなという皆さまに贈るHadoop/Sparkのキホン (IBM Datapalooza Tokyo 2016講演資料)
ちょっと理解に自信がないな という皆さまに贈るHadoop/Sparkのキホン (IBM Datapalooza Tokyo 2016講演資料)
 
Advanced Production Control Using Julia & IMPL
Advanced Production Control Using Julia & IMPLAdvanced Production Control Using Julia & IMPL
Advanced Production Control Using Julia & IMPL
 
Apache Sparkに手を出してヤケドしないための基本 ~「Apache Spark入門より」~ (デブサミ 2016 講演資料)
Apache Sparkに手を出してヤケドしないための基本 ~「Apache Spark入門より」~ (デブサミ 2016 講演資料)Apache Sparkに手を出してヤケドしないための基本 ~「Apache Spark入門より」~ (デブサミ 2016 講演資料)
Apache Sparkに手を出してヤケドしないための基本 ~「Apache Spark入門より」~ (デブサミ 2016 講演資料)
 
Julia Computing - an alternative to Hadoop
Julia Computing - an alternative to HadoopJulia Computing - an alternative to Hadoop
Julia Computing - an alternative to Hadoop
 

Similar to Inside the Corporation: How Julia is Used in a Japanese Financial Firm

Artur Suchwalko “What are common mistakes in Data Science projects and how to...
Artur Suchwalko “What are common mistakes in Data Science projects and how to...Artur Suchwalko “What are common mistakes in Data Science projects and how to...
Artur Suchwalko “What are common mistakes in Data Science projects and how to...Lviv Startup Club
 
Lessons learned after 190M lessons served
Lessons learned after 190M lessons servedLessons learned after 190M lessons served
Lessons learned after 190M lessons servedRicardo Bánffy
 
2CPP01 - Intro to Module
2CPP01 - Intro to Module2CPP01 - Intro to Module
2CPP01 - Intro to ModuleMichael Heron
 
Agile Software Development
Agile Software DevelopmentAgile Software Development
Agile Software DevelopmentAhmet Bulut
 
Random thoughts and dev practices / advices to build a great product
Random thoughts and dev practices / advices to build a great productRandom thoughts and dev practices / advices to build a great product
Random thoughts and dev practices / advices to build a great productGuillaume POTIER
 
Shipping code is not the problem, deciding what to ship it is!
Shipping code is not the problem, deciding what to ship it is!Shipping code is not the problem, deciding what to ship it is!
Shipping code is not the problem, deciding what to ship it is!Mauro Servienti
 
Untangling - fall2017 - week 7
Untangling - fall2017 - week 7Untangling - fall2017 - week 7
Untangling - fall2017 - week 7Derek Jacoby
 
Maintaining Large Scale Julia Ecosystems
Maintaining Large Scale Julia EcosystemsMaintaining Large Scale Julia Ecosystems
Maintaining Large Scale Julia EcosystemsChris Rackauckas
 
Student Survey Nov 2022
Student Survey Nov 2022Student Survey Nov 2022
Student Survey Nov 2022Peter Windle
 
Better User Onboarding (for web & mobile products)
Better User Onboarding (for web & mobile products)Better User Onboarding (for web & mobile products)
Better User Onboarding (for web & mobile products)Pulkit Agrawal
 
Data science unit 1 By: Professor Lili Saghafi
Data science unit 1 By: Professor Lili Saghafi Data science unit 1 By: Professor Lili Saghafi
Data science unit 1 By: Professor Lili Saghafi Professor Lili Saghafi
 
Android Developer Skills, Techniques, and Patterns
Android Developer Skills, Techniques, and PatternsAndroid Developer Skills, Techniques, and Patterns
Android Developer Skills, Techniques, and Patternsgdgut
 
It's XP Stupid (2019)
It's XP Stupid (2019)It's XP Stupid (2019)
It's XP Stupid (2019)Mike Harris
 
Deep learning in production with the best
Deep learning in production   with the bestDeep learning in production   with the best
Deep learning in production with the bestAdam Gibson
 
JIRA 101 - Over(our)head No Longer!
JIRA 101 - Over(our)head No Longer!JIRA 101 - Over(our)head No Longer!
JIRA 101 - Over(our)head No Longer!Frank Caron
 
"Startups, comment gérer une équipe de développeurs" par Laurent Cerveau
"Startups, comment gérer une équipe de développeurs" par Laurent Cerveau"Startups, comment gérer une équipe de développeurs" par Laurent Cerveau
"Startups, comment gérer une équipe de développeurs" par Laurent CerveauTheFamily
 

Similar to Inside the Corporation: How Julia is Used in a Japanese Financial Firm (20)

Artur Suchwalko “What are common mistakes in Data Science projects and how to...
Artur Suchwalko “What are common mistakes in Data Science projects and how to...Artur Suchwalko “What are common mistakes in Data Science projects and how to...
Artur Suchwalko “What are common mistakes in Data Science projects and how to...
 
Lessons learned after 190M lessons served
Lessons learned after 190M lessons servedLessons learned after 190M lessons served
Lessons learned after 190M lessons served
 
Specification by example
Specification by exampleSpecification by example
Specification by example
 
2CPP01 - Intro to Module
2CPP01 - Intro to Module2CPP01 - Intro to Module
2CPP01 - Intro to Module
 
Agile Software Development
Agile Software DevelopmentAgile Software Development
Agile Software Development
 
Random thoughts and dev practices / advices to build a great product
Random thoughts and dev practices / advices to build a great productRandom thoughts and dev practices / advices to build a great product
Random thoughts and dev practices / advices to build a great product
 
Shipping code is not the problem, deciding what to ship it is!
Shipping code is not the problem, deciding what to ship it is!Shipping code is not the problem, deciding what to ship it is!
Shipping code is not the problem, deciding what to ship it is!
 
Untangling - fall2017 - week 7
Untangling - fall2017 - week 7Untangling - fall2017 - week 7
Untangling - fall2017 - week 7
 
Software development fundamentals
Software development fundamentalsSoftware development fundamentals
Software development fundamentals
 
Maintaining Large Scale Julia Ecosystems
Maintaining Large Scale Julia EcosystemsMaintaining Large Scale Julia Ecosystems
Maintaining Large Scale Julia Ecosystems
 
Student Survey Nov 2022
Student Survey Nov 2022Student Survey Nov 2022
Student Survey Nov 2022
 
Better User Onboarding (for web & mobile products)
Better User Onboarding (for web & mobile products)Better User Onboarding (for web & mobile products)
Better User Onboarding (for web & mobile products)
 
Data science unit 1 By: Professor Lili Saghafi
Data science unit 1 By: Professor Lili Saghafi Data science unit 1 By: Professor Lili Saghafi
Data science unit 1 By: Professor Lili Saghafi
 
Android Developer Skills, Techniques, and Patterns
Android Developer Skills, Techniques, and PatternsAndroid Developer Skills, Techniques, and Patterns
Android Developer Skills, Techniques, and Patterns
 
It's XP Stupid (2019)
It's XP Stupid (2019)It's XP Stupid (2019)
It's XP Stupid (2019)
 
Deep learning in production with the best
Deep learning in production   with the bestDeep learning in production   with the best
Deep learning in production with the best
 
JIRA 101 - Over(our)head No Longer!
JIRA 101 - Over(our)head No Longer!JIRA 101 - Over(our)head No Longer!
JIRA 101 - Over(our)head No Longer!
 
It's XP, Stupid
It's XP, StupidIt's XP, Stupid
It's XP, Stupid
 
Afganistan Culture Shock
Afganistan Culture ShockAfganistan Culture Shock
Afganistan Culture Shock
 
"Startups, comment gérer une équipe de développeurs" par Laurent Cerveau
"Startups, comment gérer une équipe de développeurs" par Laurent Cerveau"Startups, comment gérer une équipe de développeurs" par Laurent Cerveau
"Startups, comment gérer une équipe de développeurs" par Laurent Cerveau
 

Recently uploaded

Low Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service Bhilai
Low Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service BhilaiLow Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service Bhilai
Low Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service BhilaiSuhani Kapoor
 
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130Suhani Kapoor
 
VidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptxVidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptxolyaivanovalion
 
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM  TRACKING WITH GOOGLE ANALYTICS.pptxEMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM  TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptxthyngster
 
Week-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interactionWeek-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interactionfulawalesam
 
RA-11058_IRR-COMPRESS Do 198 series of 1998
RA-11058_IRR-COMPRESS Do 198 series of 1998RA-11058_IRR-COMPRESS Do 198 series of 1998
RA-11058_IRR-COMPRESS Do 198 series of 1998YohFuh
 
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Callshivangimorya083
 
Introduction-to-Machine-Learning (1).pptx
Introduction-to-Machine-Learning (1).pptxIntroduction-to-Machine-Learning (1).pptx
Introduction-to-Machine-Learning (1).pptxfirstjob4
 
Midocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFxMidocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFxolyaivanovalion
 
Dubai Call Girls Wifey O52&786472 Call Girls Dubai
Dubai Call Girls Wifey O52&786472 Call Girls DubaiDubai Call Girls Wifey O52&786472 Call Girls Dubai
Dubai Call Girls Wifey O52&786472 Call Girls Dubaihf8803863
 
04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationships04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationshipsccctableauusergroup
 
Generative AI on Enterprise Cloud with NiFi and Milvus
Generative AI on Enterprise Cloud with NiFi and MilvusGenerative AI on Enterprise Cloud with NiFi and Milvus
Generative AI on Enterprise Cloud with NiFi and MilvusTimothy Spann
 
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...Suhani Kapoor
 
Industrialised data - the key to AI success.pdf
Industrialised data - the key to AI success.pdfIndustrialised data - the key to AI success.pdf
Industrialised data - the key to AI success.pdfLars Albertsson
 
BigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptxBigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptxolyaivanovalion
 
Halmar dropshipping via API with DroFx
Halmar  dropshipping  via API with DroFxHalmar  dropshipping  via API with DroFx
Halmar dropshipping via API with DroFxolyaivanovalion
 
Beautiful Sapna Vip Call Girls Hauz Khas 9711199012 Call /Whatsapps
Beautiful Sapna Vip  Call Girls Hauz Khas 9711199012 Call /WhatsappsBeautiful Sapna Vip  Call Girls Hauz Khas 9711199012 Call /Whatsapps
Beautiful Sapna Vip Call Girls Hauz Khas 9711199012 Call /Whatsappssapnasaifi408
 
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfSchema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfLars Albertsson
 
April 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's AnalysisApril 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's Analysismanisha194592
 
(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service
(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service
(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Serviceranjana rawat
 

Recently uploaded (20)

Low Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service Bhilai
Low Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service BhilaiLow Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service Bhilai
Low Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service Bhilai
 
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
 
VidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptxVidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptx
 
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM  TRACKING WITH GOOGLE ANALYTICS.pptxEMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM  TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
 
Week-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interactionWeek-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interaction
 
RA-11058_IRR-COMPRESS Do 198 series of 1998
RA-11058_IRR-COMPRESS Do 198 series of 1998RA-11058_IRR-COMPRESS Do 198 series of 1998
RA-11058_IRR-COMPRESS Do 198 series of 1998
 
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
 
Introduction-to-Machine-Learning (1).pptx
Introduction-to-Machine-Learning (1).pptxIntroduction-to-Machine-Learning (1).pptx
Introduction-to-Machine-Learning (1).pptx
 
Midocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFxMidocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFx
 
Dubai Call Girls Wifey O52&786472 Call Girls Dubai
Dubai Call Girls Wifey O52&786472 Call Girls DubaiDubai Call Girls Wifey O52&786472 Call Girls Dubai
Dubai Call Girls Wifey O52&786472 Call Girls Dubai
 
04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationships04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationships
 
Generative AI on Enterprise Cloud with NiFi and Milvus
Generative AI on Enterprise Cloud with NiFi and MilvusGenerative AI on Enterprise Cloud with NiFi and Milvus
Generative AI on Enterprise Cloud with NiFi and Milvus
 
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
 
Industrialised data - the key to AI success.pdf
Industrialised data - the key to AI success.pdfIndustrialised data - the key to AI success.pdf
Industrialised data - the key to AI success.pdf
 
BigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptxBigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptx
 
Halmar dropshipping via API with DroFx
Halmar  dropshipping  via API with DroFxHalmar  dropshipping  via API with DroFx
Halmar dropshipping via API with DroFx
 
Beautiful Sapna Vip Call Girls Hauz Khas 9711199012 Call /Whatsapps
Beautiful Sapna Vip  Call Girls Hauz Khas 9711199012 Call /WhatsappsBeautiful Sapna Vip  Call Girls Hauz Khas 9711199012 Call /Whatsapps
Beautiful Sapna Vip Call Girls Hauz Khas 9711199012 Call /Whatsapps
 
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfSchema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdf
 
April 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's AnalysisApril 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's Analysis
 
(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service
(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service
(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service
 

Inside the Corporation: How Julia is Used in a Japanese Financial Firm

  • 1. |> inside the corporation Andre Pemmelaar @QuantixResearch
  • 2. About Me Andre Pemmelaar • 5-yrs Matsushita Financial System Solutions (Panasonic) • 12 Buy-Side Finance • 7-yrs Japanese Gov’t Bond Options Market Maker (HNL) • 5-yrs Statistical Arbitrage (Global Equities) • Low latency & Quantitative Algorithm • Primarily use mixture of basic statistics and machine learning • R,Python,Java, F# …. and of course JULIA! • Prefer function programming approach (F#, Scala, Haskell) @QuantixResearch
  • 4. My road to using • Started reading about it in late 2012 ~ early 2013 • Wrote to John Myles White in Spring 2013 John Myles White 3.20.2013 at 9:38 am | Permalink Hi Andre, ! In the abstract, I think Julia is the ideal language for doing both prototype modeling and transition to production. ! But Julia is still very immature as a language, so I would not recommend it being used in production for another year or so. In addition, if you’re looking for an existing toolbox of models, R is the way to go. Even Python has still not caught up with R in this regard. @QuantixResearch • Decided it was too early -> kept following, but didn’t use
  • 5. My road to using • Revisited ~ early 2014 • Began trying some simple projects • Reinforcement Learning using tictactoe.jl • Found the code very easy to follow • Started using the DataFrame.jl • Found it to be very stable and close enough to Panda (python) • Started writing first serious attempt at something important in May 2014 • Orderbook simulation frame work • Joined new company 3 months ago - using Julia almost exclusively for 3 month on real world problems in Finance @QuantixResearch
  • 6. My Moment Realized I could… • Remain mostly functional in my approach to programming (but not 100%) • Use fast for loops wherever appropriate (used in a lot of time series simulations) • Easily code linear algebra, matrix calculations for machine learning, etc.(native in Julia) • Do it all it parallel (note: Julia’s parallel not yet 100% there yet) • All of the above can be done in Python (Sci-kit, Numpy, etc) but often faster and with slightly less code in Julia @QuantixResearch
  • 8. Some background on my company • One of Japan’s largest financial front-office system solution providers • Started off in derivative valuation and derivative OMS systems • Now offers an entire suite of products aimed at Japanese mega banks, and 2nd-tier financial organization • About 600 employees (about 60%~70 are technical) • Primarily production language is company isJava, with some done in C++, or C • Quantitative analysis is done in Java (heavy duty large data set analysis) or R for smaller datasets) with a few using Python users • Most quants are focused on Risk or Valuation, but a smaller team (mine) makes use of predictive analytics, statistics, and ML to enhance various algorithms @QuantixResearch
  • 9. Nothing sells like success • It helps to have a successful example to sell it internally • In my case, during my first week I found some R code that was used every night (had lots of loops = ripe for porting to Julia) • Re-wrote in Julia -> • R took about 15:46m • Java about +/- 20s • Julia about 4.3 secs • Note: Better Java programmer recently bested Julia version (3.9 secs)
  • 11. Making the first experience easier • Set the expectation correctly • Documentation is sparse. • The stuff that is out there may not be current • Julia is fast, but can lose a lot of speed if coded improperly @QuantixResearch Poor Performance Better Performance
  • 12. Roadblocks I asked Julia colleagues, “What are/were the 3 biggest hurdles” #1 Lack of documentation #2 Lack of current documentation #3 Package breaking/incompatibility on update No one said bugs in base code, or lack of some critical feature. Everyone wants correct, examples of “here’s how you do this” to initial adoption
  • 13. Roadblocks Really just two problems 1.Documentation 2.Update Chaos to initial adoption
  • 14. DIY Documentation • Julia base documentation is good • The package’s docs vary greatly • The one great example is Gadfly • Code, output, & explanation • Not so great doc ex: DataFrame • No longer current • Many common tasks missing • Create you own documentation • The single most difficult part of learning Julia is the lack of current correct examples • IJulia is fantastic for creating these! • My Advice • Initially target early users cases • DIY Document anything people are struggling with @QuantixResearch
  • 15. Decide on the environment/tools IJulia LightTables + Jewel @QuantixResearch
  • 16. Decide on the environment & tools • Julia is still new enough that small upgrades can break critical packages • As the initial “Julia person” in your organization you will often be called on to solve various problems • Solving new users problems is much easier if they are using the same tools and packages. Don’t underestimate this! • At the beginning sharing exactly the same environment will make things smoother • Recommend one person download the installers • Create an thorough install read me file Our stack: • Julia 3.1 • IJulia • Light Tables @QuantixResearch
  • 17. How did we do? • 6 people set out to learn Julia • 4 of them are now using it everyday • 1 is using it occasional along with Perl • 1 gave up • Why did that one give up? • He as serious Java skills and good R • Started with Julia Studio (bad 1 st experience) • Didn’t know about Light tables • Is physically separated from the rest of us and thus didn’t get initial support to get through the initial low productivity period @QuantixResearch
  • 18. Julia: Real example Rejection Order Algorithm • The model: • Determine if a order to lift a quote (execute against someones else's quote) in an OTC markets will be rejected • Background: OTC market are “over the counter” and depending on the rules, the quoter can reject your order if it suits them • Julia tools used: • DataFrame.jl, StatsBase.jl, DecisionTrees.jl, SVM.jl • Classification problem: 0 not rejected, 1 rejected • Still on-going project: current best is about 0.54 Kappa
  • 19. Julia: Real example Rejection order algorithm con’t Very unbalanced classes (0.1% are rejected) • Regime shift means it needs to be somewhat adaptive • Required us to change some of the libraries • One of Julia’s great strength’s is that you can easily changed the libraries to suit you needs
  • 20. What makes Julia great? • Speed? Julia is quite good, but Java can be as fast or faster. C ++ and C are faster • Time to get a model out? Largely dependent on your knowledge of the tools you are using • Parallelization? Not really. Still kinda raw. Memory usage can be a bit of an issue. • Safer code via Functional approach? No. One can code functionally but doesn’t enforce it • Easy to code and to access/read/understand others code? Yes
  • 21. What makes Julia great? √ Java Δ Python (Cython,etc) Δ R (vectorized) When coded well, it Clear, concise code that can easily be changed is very fast Great ability to mix loop based & matrix/vector operations Δ Java (not really) √ Python Δ Java (not concise) Δ R (only vectorized) √ Python Δ R (only R code. not C or C++)