Learn how to implement linear and logistic regression in Rust from scratch. Discover machine learning libraries you can use to implement many other statistical learning algorithms in pure Rust.

1. Machine Learning in Rust, Logistic and
Linear Regression From Scratch
Vlad Orlov
volodymyr.orlov@gmail.com

2. Agenda
How can I build my own
Regressor?
Classifier?
Overview of Rust creates: nalgebra, ndarray, argmin ,
smartcore

3. Disclamers
In this presentation we won’t talk about:
Why Rust is great for scientific computing and ML
Deep Learning
Large scale ML
GPU, FPGA and other accelerators

4. ML Refresher: Statistical Learning
When we have
Quantitative response Y
M different predictors: X1, X2, . . . , Xm
Relationship between Y and X
can be written as: Y = f(X) + ϵ

5. ML Refresher: X and Y
* https://archive.ics.uci.edu/ml/machine-learning-databases/housing/

6. ML Refresher: Problem Categories
Do we have Y?
Yes! We can do supervised learning
No, unsupervised learning
Y is quantitative or qualitative?
quantitative, we can build a regressor*
qualitative, we can build a classifier
* Classifier is also possible if there is a way to covert numerical response to a category

7. ML Refresher: Prerequisites
Linear algebra Basic statistics
Probability theory Optimization

8. ML Refresher: Linear Models
y1
y2
. . .
yn
=
β0 + β1x11 + β2x12
β0 + β1x21 + β2x22
. . .
β0 + β1xn1 + β2xn2
+
ϵ1
ϵ2
. . .
ϵn
Y = Xβ + ϵ
When we have 2 predictors, x1 and x2:
In vector form:

9. Linear Algebra: nalgebra/ndarray
1 https://nalgebra.org/
2 https://github.com/rust-ndarray
Slicing, decomposition, solver for system of
linear equations
Only 1D vectors and 2D matrices
Can do decomposition and solve equations
in pure Rust
Slicing, decomposition, solver for system of
linear equations, stats functions
Supports arbitrary number of dimensions
You will need BLAS/LAPACK backend to solve
linear equations and matrix decomposition
nalgebra1
ndarray2

10. All Together: Smartcore*
* http://smartcorelib.org/
22 algorithms
Supervised & unsupervised learning
Matrix decomposition + solvers
Model selection & evaluation
Simple API & user manual with lots of examples
Model serialization/deserialization
Supports Rust vectors, ndarray, nalgebra, etc.

11. Smartcore, our Mission
Our target: production facing applications + embedded devices + WebAssembly.
Straightforward API with superior documentation is our commitment
Cutting-edge implementations
Minimum dependencies
Great selection of algorithms
Model selection and evaluation toolset
Join our community!

12. Logistic Regression
* https://en.wikipedia.org/wiki/Sigmoid_function
Y = σ(Xβ)
σ(x) =
1
1 + e−x
Sigmoid function

13. Optimization: argmin{}*
* http://argmin-rs.org/
Pure rust implementation of convex and nonconvex
optimization methods
Gradient-fee and gradient methods
Supports native vectors and ndarray
Logging and checkpoints

14. Q&A
My email: volodymyr.orlov@gmail.com
Check out my blog: https://vlad-orlov.com/blog/
This presentation: https://github.com/VolodymyrOrlov/rust4ml