Arrhenius.jl is a differentiable combustion simulation package that: 1) Uses stochastic gradient descent and efficient computation of parameter gradients to optimize complex kinetic models with hundreds or thousands of parameters using large datasets. 2) Has been used for applications like mechanism reduction, uncertainty quantification, inverse modeling, and model discovery. 3) Is open source and aims to enable fast incorporation of new physics through differentiable programming.

1. Arrhenius.jl: A Differentiable Combustion Simulation Package
Weiqi Ji, Sili Deng
Department of Mechanical Engineering
Massachusetts Institute of Technology
May 25, 2021
1
12th U.S. National Combustion Meeting, Texas A&M University, Virtual
Paper ID: 2E06

2. 2
❑ Complex kinetic models are not only expensive to
simulate but also challenging to develop
▪ More rate parameters to determine
❑ Experimental challenges
▪ Need larger datasets
▪ More important species to measure
▪ More experimental conditions to explore
❑ Computational challenges
▪ More parameters to optimize
▪ Evaluation is more expensive
Challenges for Developing Complex Kinetic Models
Adapted from Lu & Law, PECS, 2009
High-dimensional nonlinear optimization

3. 3
❑ Evolutionary algorithms are usually employed for data-driven combustion modeling.
❑ Cost scales with number of parameters and number of data samples, not suitable for complex kinetic
models and large datasets.
Conventional Optimization without Differential Programming
Genetic
algorithms
Evolutionary
programming
Evolution
strategies
Particle
Swarm
Evolutionary algorithms
Simple models
< 100 parameters
Small datasets
Pros: easy implementation, multi-procs parallelization
Arrhenius.jl aims at optimizing complex models with hundreds and thousands of parameters,
utilizing large datasets from high throughput measurements.

4. 4
❑ Efficient and robust optimization using
Stochastic Gradient Descent (SGD).
❑ Efficient and easy computation of gradients of
simulation results to model parameters.
Optimization with Differential Programming
Arrhenius.jl generalizes differential programing from deep learning to kinetic modeling.
LeCun et al., Nature 521 (2015) 436:444.
Differential programing refers to the optimization techniques for training deep neural networks.

5. 5
▪ Datasets: Species profiles during propane
pyrolysis in shock tubes @Stanford.
▪ Optimization targets: 110 reactions related to
propane pyrolysis in USC Mech II (three
Arrhenius parameters for each reaction.
▪ Challenges: Highest dimension ever achieved
without parameter selection.
▪ Cost: Less than 100 lines of code; 4 hours on
a common PC (without HPC).
Example: Optimization of 330 Kinetic Parameters
Collaboration with Hanson’s group @Stanford

6. 6
Framework of Arrhenius.jl
Kinetic
database
Thermodynamic
database
Transport
database
Mechanism
interpreter
Reaction
source term
Mixture properties
Applications
e.g., auto-ignition, jet stirred reactor, one-dimensional flames
Experimental data
solving
evaluation
differentiation
Neural Network
sub-models
Learning algorithms
Key Features
▪ Differentiable CHEMKIN
▪ Open source
▪ GPU support
▪ Augmented neural network

7. 7
Differentiable Combustion Code
Differentiate computational blocks
Differentiate final solutions
𝜕F
𝜕𝜃 model parameters (e.g., A, b, Ea)
reaction source term, convection and diffusion terms
𝜕G
𝜕𝜃
species profiles, ignition delay, flame speed, extinction strain rate
model parameters (e.g., A, b, Ea), boundary/initial conditions, neural network submodels
convection diffusion reaction

8. 8
Applications of Arrhenius.jl
Mechanism
Reduction
Uncertainty
Quantification
Inverse
Modeling
Model
Discovery
Data
Assimilation
Presented today
Preliminary results

9. 9
❑ Conventional mechanism reduction keeps rate constants unchanged while removing pathways.
❑ Deep reduction: first over-reduce and then re-train to balance size and accuracy.
Mechanism Reduction
Over-reduced
mechanism
Re-train with
IDT data
Validate with
IDT / SL
P: 1 - 60 atm
T: 1100 - 2000 K
Phi: 0.5 - 1.8
Stochastic/mini-batch
Early-stopping
Weight decay
Fix key SL sensitive reactions
Compared to others
▪ Better fitness
▪ Better generalization
▪ IDT: Ignition Delay Time
▪ SL: Laminar Flame Speed
▪ Grad of IDT computed via SensBVP + auto-diff
SensBVP: Gururajan, and Egolfopoulos, CNF, 2019

10. 10
Results: Deep Mechanism Reduction
▪ Performance: Deeply reduced models
work well for both IDT and SL.
▪ Advantage: Arrhenius.jl can optimize
hundreds of parameters, key to the
success of deep reduction.
▪ Potential: Further reduction is possible
for specific targets.
▪ Cost: Differential programing 1 hour on
a PC vs Evolution algorithm on an HPC
cluster

11. 11
▪ Goal: Dimension reduction via PCA in gradient space
▪ Challenge: 4846 is the highest dimension ever
reached in combustion uncertainty quantification
▪ Insight: Observed 1D subspace for large HC,
suggesting low-cost UQ studies
▪ Cost: (GRI3.0) Arrhenius.jl ~ one minute on a laptop
vs Finite difference ~ one hour
Results: Uncertainty Quantification via Active Subspace
Active variable: project x into leading eigenvectors.
Arrhenius.jl tackles curse of dimensionality via efficiently identifying low-dimensional active subspaces.

12. 12
Results: Autonomous Model Discovery
❑ Goal: Learn pyrolysis steps in HyChem without expert knowledges.
❑ Challenge: Augment NN into governing equations and optimize hundreds of NN weights.
CRNN (Chemical Reaction Neural Network):
An interpretable NN model with mass action
law and Arrhenius law encoded into structure.
Augment neural network sub model for autonomous scientific discovery.
Ji & Deng, JPC A, 2021
Talk: 3B01 @ Fire Research

13. 13
Ongoing: Data Assimilation and Inverse Modeling
❑ Assimilate propane pyrolysis data from shock
tube, involving 330 parameters.
❑ Inference of unknown upstream with sparse
and noise measurements at downstream.
Collaboration with Hanson’s group @Stanford Submitted to ASME IMECE
Downstream
measurements
Upstream
parameterized
by neural
network
Inference

14. 14
Open Source: Code, Discussions, Documents
❑ Code hosted on Github ❑ Online document
https://deng-mit.github.io/Arrhenius.jl/dev/
https://github.com/DENG-MIT/Arrhenius.jl

15. 15
❑ Arrhenius.jl enables easy access to differentiable programing for optimizing complex combustion
kinetic models.
❑ Arrhenius.jl shows benefits of differential programing in many computational tasks.
Summary
Mechanism
Reduction
Uncertainty
Quantification
Inverse
Modeling
Model
Discovery
Data
Assimilation
▪ Deep reduction ▪ Active subspaces ▪ CRNN-HyChem
▪ Shock tube measurements ▪ Neural differential equations

16. 16
Future Work
Functionality/Scalability
Multiphases/Multiphysics
▪ Solver for 1D flame (currently calling Cantera), extinction
▪ GPU support, matrix operation, batch-operation
▪ 2D/3D flame solver and gradient computing
▪ Scientific machine learning
▪ Super-critical, real gas states
▪ Condensed phases: soot, flame synthesis, energetic materials, fire
▪ Electrochemistry, catalyst, plasma
▪ Optics: differentiable ray-tracing, laser spectroscopy
We welcome collaborators,
contributors and users join the
journey in all means
Arrhenius.jl enables fast incorporation of new physics via its readability, efficiency, multi-dispatch.

17. Publications
Applications are also open-source
https://github.com/DENG-MIT
▪ DeepReduction
▪ ArrheniusActiveSubspace
▪ Arrhenius_Flame_1D
▪ NN-ShockTube
▪ …

18. 18
❑ Developers
▪ @jiweiqi (MIT)
▪ @TJP-Karpowski (TU Darmstadt)
▪ @SuXY15 (Tsinghua)
▪ @RSuryaNarayan (NIT, Tiruchirappalli)
❑ Lead Developer of SciML
▪ @ChrisRackauckas
▪ @YingboMa
❑ Lead Developer of RMS.jl
▪ @mjohnson541 (MIT)
❑ Cantera’s open-source community
❑ Vyaas Gururajan (ANL): SensBVP
❑ Ji-Woong Park (ANL): deep reduction
Acknowledgements

19. Backup Slides

20. 20
SensBVP for Ignition Delay Time
𝑑𝑦
𝑑𝑡′ = 𝜏𝑓 𝑦, 𝜏𝑡′ , 𝑡 = 𝜏𝑡′
B.C. 𝑡′
= 0, y = 𝑦0
𝑡′ = 1, y = 𝑦𝜏 (T = Tign)
▪ Initial Value Problem
(IVP)
▪ Boundary Value Problem
(BVP)
𝐹(𝛼) = 0
𝜕𝐹
𝜕𝑦
𝜕𝑦
𝜕𝛼
+
𝜕𝐹
𝜕𝛼
= 0
▪ SensBVP
𝑑𝑦
𝑑𝑡
= 𝑓 𝑦, 𝑡
Auto-Differentiation
𝜕𝐹
𝜕𝑦
,
𝜕𝐹
𝜕𝛼
Gururajan, Vyaas, and Fokion N. Egolfopoulos. "Direct sensitivity analysis for ignition delay
times." Combustion and Flame 209 (2019): 478-480.