Rust & Python for Python WA meetup

1. Rust
and
Python
John Vandenberg @jayvdb

2. 10/05/23
2
RustPython
●
https://rustpython.github.io/
●
Started May 2018
●
Version 0.3 released September 2023
●
Includes standard library written in Rust
●
pip works
●
RustPython can be compiled to WebAssembly, allowing running Python
code in the web browser
●
Has an experimental JIT compiler to compile python to native code

3. 10/05/23
3
RustPython
> rustup run stable cargo install --path . --features ssl,bz2,jit
Installing rustpython v0.3.0 (/home/jayvdb/rust/RustPython)
Updating crates.io index
Updating git repository `https://github.com/RustPython/Parser.git`
Updating git repository `https://github.com/youknowone/unicode_names2
Updating git repository `https://github.com/RustPython/__doc__`
Compiling openssl-sys v0.9.93
Compiling bzip2-sys v0.1.11+1.0.8
Compiling openssl v0.10.57
Compiling foreign-types-shared v0.1.1
Compiling openssl-macros v0.1.1
Compiling rustpython-stdlib v0.3.0 (/home/jayvdb/rust/RustPython/stdli
Compiling foreign-types v0.3.2
Compiling bzip2 v0.4.4
Compiling rustpython v0.3.0 (/home/jayvdb/rust/RustPython)
Finished release [optimized] target(s) in 33.37s
Replacing /home/jayvdb/.cargo/bin/rustpython

4. 10/05/23
4
RustPython
> rustpython --help
RustPython 0.3.0
RustPython Team
Rust implementation of the Python language
USAGE:
rustpython [OPTIONS] [-c CMD | -m MODULE | FILE] [PYARGS]...
FLAGS:
-b issue warnings about using bytes where strings are u
-d Debug the parser.
-B don't write .pyc files on import
-h, --help Prints help information
-E Ignore environment variables PYTHON* such as PYTHONP
-i Inspect interactively after running the script.
-I isolate Python from the user's environment (implies
-S don't imply 'import site' on initialization
-s don't add user site directory to sys.path.

5. 10/05/23
5
RustPython
> rustpython -m ensurepip
Defaulting to user installation because normal site-packages is not write
Looking in links: /tmp/tmp6zbvlck5
Processing /tmp/tmp6zbvlck5/setuptools-65.5.0-py3-none-any.whl
Processing /tmp/tmp6zbvlck5/pip-22.3.1-py3-none-any.whl
Installing collected packages: setuptools, pip
Successfully installed pip-22.3.1 setuptools-65.5.0

6. 10/05/23
6
RustPython
> rustpython -m pip install --upgrade pip
Defaulting to user installation because normal site-packages is not write
Requirement already satisfied: pip in /home/jayvdb/.local/lib/rustpython3
Collecting pip
Downloading pip-23.2.1-py3-none-any.whl (2.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/2.1 MB ? eta -:--:--<bo
AttributeError("ConsoleThreadLocals has no attribute 'buffer'")
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 1.5 MB/s eta 0:0
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 22.3.1
Uninstalling pip-22.3.1:
Successfully uninstalled pip-22.3.1
Successfully installed pip-23.2.1
WARNING: There was an error checking the latest version of pip.

7. 10/05/23
7
RustPython
> rustpython -m pip install cython
Defaulting to user installation because normal site-packages is not write
Collecting cython
Obtaining dependency information for cython from https://files.pythonho
Using cached Cython-3.0.2-py2.py3-none-any.whl.metadata (3.1 kB)
Using cached Cython-3.0.2-py2.py3-none-any.whl (1.2 MB)
Installing collected packages: cython
Successfully installed cython-3.0.2
> rustpython -m cython --help
usage: cython.py [-h] [-V] [-l] [-I INCLUDE_PATH] [-o OUTPUT_FILE] [-t] [
[--annotate-fullc] [--annotate-coverage ANNOTATE_COVERAG
[-Wextra] [-X NAME=VALUE,...] [-E NAME=VALUE,...] [--mod
[sources ...]
Cython (https://cython.org/) is a compiler for code written in the Cython
positional arguments:

8. 10/05/23
8
RustPython
> time rustpython -m cython pylib/Lib/glob.py -o glob.c.rustpython
/home/jayvdb/.local/lib/rustpython3.11/site-packages/Cython/Compiler/Main
File: /home/jayvdb/rust/RustPython/pylib/Lib/glob.py
tree = Parsing.p_module(s, pxd, full_module_name)
real 0m3.522s
user 0m3.427s
sys 0m0.094s
> time python3 -m cython pylib/Lib/glob.py -o glob.c.cpython
/usr/lib64/python3.11/site-packages/Cython/Compiler/Main.py:384: FutureWa
/home/jayvdb/rust/RustPython/pylib/Lib/glob.py
tree = Parsing.p_module(s, pxd, full_module_name)
real 0m0.406s
user 0m0.336s
sys 0m0.062s

9. 10/05/23
9
RustPython
> rustpython -m pip install django
Defaulting to user installation because normal site-packages is not write
Collecting django
Obtaining dependency information for django from https://files.pythonho
Downloading Django-4.2.5-py3-none-any.whl.metadata (4.1 kB)
Collecting asgiref<4,>=3.6.0 (from django)
Obtaining dependency information for asgiref<4,>=3.6.0 from https://fil
Downloading asgiref-3.7.2-py3-none-any.whl.metadata (9.2 kB)
Collecting sqlparse>=0.3.1 (from django)
Downloading sqlparse-0.4.4-py3-none-any.whl (41 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.2/41.2 kB 1.1 MB/s eta 0
Downloading Django-4.2.5-py3-none-any.whl (8.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/8.0 MB ? eta -:--:--<boun
AttributeError("ConsoleThreadLocals has no attribute 'buffer'")
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.0/8.0 MB 1.4 MB/s eta 0:00:
Downloading asgiref-3.7.2-py3-none-any.whl (24 kB)
Installing collected packages: sqlparse, asgiref, django
Successfully installed asgiref-3.7.2 django-4.2.5 sqlparse-0.4.4

10. 10/05/23
10
RustPython
>> rustpython -m django --help
Traceback (most recent call last):
...
File "/home/jayvdb/.local/lib/rustpython3.11/site-packages/django/apps/
from .config import AppConfig
File "/home/jayvdb/.local/lib/rustpython3.11/site-packages/django/apps/
def ready(self):
File "/home/jayvdb/.local/lib/rustpython3.11/site-packages/django/apps/
return settings.DEFAULT_AUTO_FIELD
File "/home/jayvdb/.local/lib/rustpython3.11/site-packages/django/utils
from django.utils.deprecation import RemovedInDjango50Warning
File "/home/jayvdb/.local/lib/rustpython3.11/site-packages/django/utils
from asgiref.sync import iscoroutinefunction, markcoroutinefunction,
File "/home/jayvdb/.local/lib/rustpython3.11/site-packages/asgiref/sync
class SyncToAsync(Generic[_P, _R]):
File "/home/jayvdb/.local/lib/rustpython3.11/site-packages/asgiref/sync
contextvars.ContextVar("thread_sensitive_context")
RuntimeError: Expected payload 'ContextVar' but 'ContextVar' found

11. 10/05/23
11
RustPython
●
https://rustpython.github.io/pages/regression-tests-results.html
●
_ctypes missing: pull 2364
– Flask, asyncpg, psycopg2, jinja2, and many more
●
Compilation variables issue 4984
●
Stdlib not all updated to Cpython 3.11 issue 4564
●
_multibytecodec missing
– charset_normalizer, requests, etc
– big5, big5hkscs, cp932, cp949, cp950, euc_jis_2004, euc_jisx0213, euc_jp, euc_kr,
gb18030, gb2312, gbk, hz, iso2022_jp, iso2022_jp_1, iso2022_jp_2,
iso2022_jp_2004, iso2022_jp_3, iso2022_jp_ext, iso2022_kr, johab, shift_jis,
shift_jis_2004, shift_jisx0213

12. 10/05/23
12
Python packages
Rank
(downloads)
Name
14 pyyaml YAML processing Cython
18 numpy Scientific computing C
25 protobuf Google's data interchange format C
27 pandas Data analysis / manipulation Cython
33 markupsafe HTML/XML markup C
38 cffi Foreign Function Interface C
57 grpcio Core validation logic for pydantic Cython
58 pyarrow Data interchange & in-memory processing C++
60 wrapt Decorators, wrappers & monkey patching C
62 lxml XML toolkit Cython
66 psutil Process utils C
69 frozenlist Frozen list Cython

13. 10/05/23
13
Python packages
Rank
(downloads)
Name
74 greenlet Lightweight in-process concurrency C/C++
76 yarl Yet another URL library Cython
77 multidict Container of many values per key C
78 scipy Mathematics, science, and engineering Fortran/C
80 pillow Python Imaging Library C
112 pynacl Networking and cryptography C
110 scikit-learn Machine learning Cython
120 matplotlib Plotting C++
122 cython Python to C compiler Cython
125 regex Regex C
131 msgpack MessagePack serializer C++
137 pycryptodome Cryptography C

14. 10/05/23
14
Python packages
Rank
(downloads)
Name
138 pyrsistent Lightweight in-process concurrency C/C++
150 snowflake-connector-python Snowflake Connector C++
154 google-crc32c CRC32C hashing C
170 kiwisolver Cassowary constraint solving C++
201 pyodbc ODBC databases C++
202 ruamel-yaml-clib YAML processing C
220 pymongo Mongo driver C
242 contourpy Contours of 2D quadrilateral grids C++
260 rapidfuzz String matching library C++
265 lazy-object-proxy Lazy object proxy C
267 tensorflow Machine learning C++
271 transformers Machine learning C++/CUDA

15. 10/05/23
15
Python packages
Rank
(downloads)
Name
273 simplejson JSON library C
274 h5py HDF5 binary data format C/Cython
284 websockets WebSocket implemention C
286 debugpy Debug Adapter Protocol C++
267 tensorflow
...
Machine learning C++
399 xgboost Gradient boosting (ML) C++ (Boost)

16. 10/05/23
16
Python packages
Rank
(downloads)
Name
15 cryptography Cryptographic primitives and recipes
99 rpds-py Binding to Rust crate rpds for persistent data structures
(dependency of jsonschema)
114 bcrypt Password hashing
243 pydantic-core Core validation logic for pydantic
318 orjson Fast, correct Python JSON library supporting dataclasses,
datetimes, and numpy
476 libcst Concrete syntax tree parser and serializer for Python
823 ruff Python linter
2,791 rustworkx Replacement for networkx (#195) graph library

17. 10/05/23
17
Python packaging
●
https://github.com/PyO3 (Pythonium Trioxide)
●
setuptools-rust
– 50,000 downloads per day
– https://github.com/PyO3/setuptools-rust
●
maturin
– 20,000 downloads per day
– https://github.com/PyO3/maturin

18. 10/05/23
18
Python packaging
●
Cryptography – Using setuptools-rust

19. 10/05/23
19
Python packaging

20. 10/05/23
20
Python packaging

21. 10/05/23
21
Python packaging

22. 10/05/23
22
Python packaging

23. 10/05/23
23
Python packaging

24. 10/05/23
24
Python packaging

25. 10/05/23
25
Python packaging

26. 10/05/23
26
Python packaging
...
#[pyo3::prelude::pyfunction]
fn openssl_version() -> i64 {
openssl::version::number()
}
#[pyo3::prelude::pyfunction]
fn is_fips_enabled() -> bool {
cryptography_openssl::fips::is_enabled()
}
...

27. 10/05/23
27
result
from result import Result, Ok, Err
def divide(a: int, b: int) -> Result[int, str]:
if b == 0:
return Err("Cannot divide by zero")
return Ok(a // b)
values = [(10, 0), (10, 5)]
for a, b in values:
divide_result = divide(a, b)
match divide_result:
case Ok(value):
print(f"{a} // {b} == {value}")
case Err(e):
print(e)
https://github.com/rustedpy/result
4k downloads/day

28. 10/05/23
28
PyOxidizer
●
https://github.com/indygreg/PyOxidizer
●
Combines CPython runtime, Python code and dependencies into a single
executable
●
Supports Windows, Linux and Mac executable formats
●
Also creates installers for Windows, Linux and Mac
●
Pyembed crate controls an embedded CPython inside Rust

29. 10/05/23
29
Python tools
●
Package managers like pipenv, poetry, etc
– https://github.com/mitsuhiko/rye
●
Also includes Python runtime management, like pyenv
– https://github.com/cnpryer/huak
●
Incomplete and not as actively developed
●
Vulnerability scanner
– https://github.com/aswinnnn/pyscan
●
Static code analysis
– Ruff
– https://github.com/mtshiba/pylyzer

30. 10/05/23
30
py2many
●
https://github.com/py2many/py2many
●
Transpiles subset of Python to:
– C++, Dart, Go, Julia, Kotlin, Nim, Rust, V
●
Supports CLI sys.argv and sys.stdout