The document outlines the integration of Rust and Python to create a FUSE-based toy filesystem utilizing the PyO3 library. It discusses the benefits of Rust for system programming and Python for rapid prototyping while explaining the project structure for developing a Python package that includes Rust code. Additionally, it highlights the challenges of data type conversion between the two languages and provides examples of method definitions and interactions using PyO3.

1. Rust + Python
Lessons from building a FUSE toy filesystem
2018/9/10
@GregWeng

2. Rust + Python
Rust: system programming language
Easy to adapt C and keep code safe
Primitive type, low-level structure
Python: fast prototyping and easy to extend
Easy to adapt C as native module
GC, dynamic types

3. Rust + Python
Rust: system programming language
Easy to adapt C and keep code safe
Primitive type, low-level structure
Python: fast prototyping and easy to extend
Easy to adapt C as native module
GC, dynamic types
Rust module
Python library Python programPython module
...
System API

4. RestFS
Rust adapts FUSE API
PyO3 builds this Rust as native python module
Python extend the RestFS.rs to adapt real site
By urllib to define what file operations to what
site operations (ex: GET -> create new file)
RestFS.rs
YourSite.py Python programPython module
...
FUSE API
PyO3
urllib
https://github.com/snowmantw/restfs

5. Not really makes sense
File system is persistent storage, while RESTful API is designed to be stateless
GET /item -> File makes sense doesn't mean GET /items -> Directory makes sense
(response of an HTTP request has no definition of it is a single resource or a list)

6. Not really makes sense, but...
Everything is a file! cat "hello" > pastebin.com/hello looks so right (isn't it?)
C is a sharp knife with greasy handle that may cut your hand easily
Python is good but it lives too high to let us know the details (sure that's why it is so nice)
Therefore:
Find a topic related to system programming (so file system)
that I can have a look at Python native details (so PyO3)
and practice Rust for fun ("摻在一起做成撒尿牛丸就好啦 ")

7. PyO3

8. PyO3
Necessary for every method imported and used from the
Python side (ex: as class or module method)

9. PyO3
Need to get Python instance for almost all PyO3 APIs
(In Rust you borrow it from the top-level you acquire it)

10. PyO3
You have corresponding Python data structures in Rust
by PyO3, but you need to do conversion frequently

11. PyO3
Primitive types can be used easily (but things like Map aren't)

12. Project structure

13. Project structure
Help to export your Rust native
module as a Python package

14. Project structure
Where the Rust module is
(/lib/src/lib.rs)

15. Project structure
Your Rust project is actually inside a Python project
The `setup.py` will help to build it (or just using pyo3-pack)

16. Project structure
From the top of the project to run a Python file to import
and use the Rust package.module

17. Project structure
Note the setup.py will
install the package as a
site package in your
environment (ex: pyenv)

18. Python callable Rust class/method
Where the Rust module is
(/lib/src/lib.rs)

19. Python callable Rust class/method
To define a Python class in Rust:
#pyclass - define a Python class in Rust (struct)
#pymethods - define class methods
#new - define __new__ (NOT __init__) for Python
__new__:
receive a new PyRawObject (borrowed)
init it with attributes
(using PyRawObject init method)

20. Python callable Rust class/method
In Python, pass argument as init arguments

21. Python callable Rust class/method
Output: PyResult<> with Rust types as type variables
Input: primitive or PyO3 wrapped types as arguments
Input
PyO3/Primitive
Output
PyResult<Rust types>

22. ToPyObject and FromPyObject
Option<T>
&'a/&'a mut
(),(A,)/(A,B)/(A...I)
bool, f32, f64, i8, u8...u64
&'a str, String, &'a String
HashMap<K,V,H>
BTreeMap<K,V>
Vec<T>
ToPyObject/IntoPyObject
(implemented traits)
PyObject
py.None/PyObject (delegated)
PyObject (delegated)
py.None, PyTuple...
PyBool, PyFloat, PyLong...
PyString
PyDict
PyList
https://docs.rs/pyo3/0.2.1/pyo3/trait.IntoPyObject.html

23. ToPyObject and FromPyObject
"PyObject":
py.None, PyTuple...
PyBool, PyFloat, PyLong...
PyString
PyDict
PyList
...
FromPyObject<'a> PyResult<...>
(implemented traits)
PyResult<$rust_type>
where $rust_type is:
bool, f32, f64, i8, u8...u64
Vec<T>
std::mem::transmute
(macro + re-interpreting)
https://docs.rs/pyo3/0.2.1/pyo3/trait.FromPyObject.html

24. Conversion everywhere...
Input
PyO3/Primitive
Output
PyResult<Rust types>
fn rust_py_method_A fn rust_py_method_B
Input
PyO3/Primitive
Output
PyResult<Rust types>
fn rust_py_method_C....
Input
PyO3/Primitive
Output
PyResult<Rust types>
(Python) (Python)
(Python)

25. A "Better" pattern
Input
PyO3/Primitive
Output
PyResult<Rust types>
fn rust_py_method_A
(Python)
fn rust_method_A
fn rust_method_B
fn rust_method_C
(Python)
Avoid type conversion overhead
(But this encourages to have a gigantic
entry-only method from Rust)