2. Table of content
What we will talk about today
• Why Python is awesome
• Python applications
• Speeding python using Rust library
• Examples – Sentry, Csv CLI application
• What is Rust?
• Why Rust?
• How Rust looks? –syntax, similarities
• Rust trifecta – Speed, Concurrency,
Safety
• Rust tooling
• Rust Pipeline
• Who is using Rust?
• Rust Meetup
• Q & A
3. What my talk is not
Introduction
• It’s not about which language is
great/ better
• Why a certain language is not
good
• Comparing languages
• Promoting a language
4. What my talk is …
Introduction
• Leverage on different technologies
• Python and Rust power
• Introduce Rust programming
language
7. Why Python is awesome …
Why python
• Readability - resembles the English language
• Libraries - Python has been around for over 20 years
• Community - has user groups everywhere
• Dynamically typed language – no types declaring variable
• Simple syntax – less debugging
• One liner – elegant 1-liner solution i.e. x, y = y, x
• Intuitive data structures – List, tuples, dictionaries, mixed
data types
12. What is Rust?
• Systems programming language
• Trifecta – Speed, concurrency and Safety
• Rust is considered a C-like language with more features – advantage
13. History
• Graydon Hoare dev 2006
• Mozilla starts sponsoring Rust in 2009
• Version 1.0.0 (2015-)
• > 19,567 crates – libraries & 584,987,337 Downloads
• >2,211 contributors on Github. Compiler
• Big areas: game dev, operating systems, web development, block chain
15. About csvterm cli project
Csvterm cli project
• Count the number of rows in the CSV
data
• Show the last N rows of CSV data.
• Search CSV data using regex
• Sort CSV data
Commands
sort
search
count
head
headers
23. Extending Python with Rust
Why python
Useful libraries:
• Cffi
• Milksnake
Other libs
• Click
• terminaltables
Procedure
1. build the Rust code into libs (crate)
2. build a new crate that exposes a nice C-ABI
3. use cbindgen to generate C headers
4. use Milksnake to automatically generate low-level
bindings from this crate and headers
5. generate high-level Python wrappers around the low-
level bindings
24. Extending Python with Rust
Why python
Procedure
1. build the Rust code into libs (crate)
2. build a new crate that exposes a nice C-ABI
3. Use cbindgen to generate C headers
4. use Milksnake to automatically generate low-level
bindings from this crate and headers
5. generate high-level Python wrappers around the low-
level bindings
cargo new crate_name --lib
cargo new crate_name --bin
25. Extending Python with Rust
Why python
Procedure
1. build the Rust code into libs
(crate)
2. build a new crate that exposes
a nice C-ABI
3. Use cbindgen to generate C
headers
4. use Milksnake to
automatically generate low-
level bindings from this crate
and headers
5. generate high-level Python
wrappers around the low-
level bindings
27. Extending Python with Rust
Why python
Procedure
1. build the Rust code into libs (crate)
2. build a new crate that exposes a nice C-ABI
3. Use cbindgen to generate C headers
4. use Milksnake to automatically generate low-level
bindings from this crate and headers
5. generate high-level Python wrappers around the low-
level bindings
28. Extending Python with Rust
Why python
Procedure
1. build the Rust code into libs (crate)
2. build a new crate that exposes a nice C-ABI
3. Use cbindgen to generate C headers
4. use Milksnake to automatically generate low-level
bindings from this crate and headers
5. generate high-level Python wrappers around the low-
level bindings
29. Extending Python with Rust
Why python
Procedure
1. build the Rust code into libs
(crate)
2. build a new crate that
exposes a nice C-ABI
3. Use cbindgen to generate C
headers
4. use Milksnake to
automatically generate low-
level bindings from this crate
and headers
5. generate high-level Python
wrappers around the low-
level bindings
35. Why Rust?
Speed
Rust has great functionality – raw binary data
Reliability – rare to break, once it runs
Control
limited resources
Concurrency
Type inference - let b= 5u8; let a=5;
High level abstraction – minimizes code
39. • Run piece of code
concurrently
• leverages ownership
safety
• message-passing
between threads
• One data owner at a
time
• more than one pointer
to the same memory
no changing of the
data
• Uses Ownership and
borrowing concept
• No Garbage Collection
• LLVM - compiler
infrastructure
• Zero Cost
Abstractions
• Minimal Runtime – No
GC, can compile
without stdlib
Speed, Safety and Concurrency
SUB-HEADING
How could we leverage on different technologies to solve our issues and achieve solutions in optimal way
Web and Internet Development – django, flask, pyramid, bottle
Scientific and Numeric – SciPy, Pandas- Data science
Education – Deep Learning- Natural Language Processing - like detect spam emails, write poetry, spin articles, and group together similar words.
Scripting & Analysis – Machine learning
Desktop GUIs – pyqt, Tk
Software Development – build control
Business Applications – ERPs
Rust is a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety. Avoiding data races.
Rust is a systems programming language focused on three goals: safety, speed, and concurrency.
Segfaults are caused by a program trying to read or write an illegal memory location.
A data race occurs when: two or more threads in a single process access the same memory location concurrently, and. at least one of the accesses is for writing, and. the threads are not using any exclusive locks to control their accesses to that memory.
Abstract Machine Model : C
Data types : C, SML, OCaml, Lisp, Limbo
Optional Bindings : Swift
Hygienic Macros : Scheme
Functional Programming : Haskell, OCaml, F#
Attributes : ECMA-335
Memory Model and Memory Management : C++, ML Kit, Cyclone
Type Classes : Haskell
Crate : Assembly in the ECMA-335 CLI model
Channels and Concurrency : Newsqueak, Alef, Limbo
Message passing and Thread failure : Erlang
C Foreign Function Interface - provides a simple to use mechanism for interfacing with C from both CPython and PyPy.
It supports two modes: an inline ABI compatibility mode
an inline ABI compatibility mode (example provided below)
which allows you to dynamically load and run functions from executable modules (essentially exposing the same functionality as LoadLibrary or dlopen)
---------------------------------
ctypes is the de facto library for interfacing with C/C++ from CPython, and it provides not only full access to the native C interface of most major operating systems (e.g., kernel32 on Windows, or libc on *nix), but also provides support for loading and interfacing with dynamic libraries, such as DLLs or shared objects at runtime.
What is ABI:application binary interface (ABI) is an interface between two binary program modules; often, one of these modules is a library or operating system facility, and the other is a program that is being run by a user.
This is using ctypes
Load dynamic library in rust
Generally import the function as it is pass in the arguments/ params
Does computation inside rust and give the results
These are some useful libraries
Milksnake is an extension for setuptools that allows you to distribute dynamic linked libraries in Python wheels in the most portable way imaginable.
=========================
Dylib – dynamic library
What is ABI:application binary interface (ABI) is an interface between two binary program modules; often, one of these modules is a library or operating system facility, and the other is a program that is being run by a user.
Lib creates a crate (reusable library)
Bin creates a binary project
Sometimes, your Rust code may need to interact with code written in another language. Because that is the way we interact with the python through c.
To do this, Rust has a keyword, extern, that facilitates creating and using a Foreign Function Interface (FFI).
No_mangle is used in manglingmangling is the encoding of function and variable names into unique names so that linkers can separate common names in the language.
Linkers - a program used with a compiler or assembler to provide links to the libraries needed for an executable program
What is ABI:application binary interface (ABI) is an interface between two binary program modules; often, one of these modules is a library or operating system facility, and the other is a program that is being run by a user.
Cargo new –bin <name-project>
Cargo new –bin <name-project>
Cargo new –bin <name-project>
Sentry - error tracking that helps developers monitor and fix crashes in real time.
cut down the processing time from >20 seconds to <0.5 sec. This meant that even ignoring any optimizations, just replacing Python with Rust could relieve our chokepoint
CPUs efficiently processing, Their worst source map times reduced to a tenth of their original time.
Actual graph of CPU processing
CPUs efficiently processing
- The average processing time reduced to ~400ms.
Rust has great functionality which we don’t find in python – raw binary data
If you needed to control of your machine and make most of your resource in the past you had one language C/ C++.
Great power comes with great responsibility. Any small mistake can have catastrophic consequences or do the unexpected.
The best scenario being a crash or worse open you to great security vulnerability.
This is why systems programming has come to have horror of being a super hero.
- Rust focuses on minimal core and extended by powerful libraries
- Making it possible to incorporate other libraries - Great package management – crates.io
- Open governance model - everyone can contribute to code
High level abstraction – minimizes code, if you were to write a code to remove spaces
Type inference - if you declare a variable as Unsigned 8 bit integer and you have pushed it to a vector which you didn’t define it will automatically give the vector unsigned 8bit integer datatype
Reliability – rare to break – rust code is statically checked at compile time and also compile enforces the 2 concepts used in rust called ownership and borrowing.
Interoperability with other languages using a Foreign Function Interface (FFI)
- Simple main function
Nightly – unstable features
6 weeks release cycle : only stable features
A Request for Comments (RFC) is a formal document from the Internet Engineering Task Force ( IETF ) that is the result of committee drafting and subsequent review by interested parties.
RFC becomes the standard and no further comments or changes are permitted.
- I have referenced him since he is one python builder and currently building sentry- He created the python micro framework and most importantly he found rust efficient and useful in his work
- He blended python and rust in an amazing way by also creating and contributing to the milksnake library
Rust has great functionality which we don’t find in python – raw binary data
If you needed to control of your machine and make most of your resource in the past you had one language C/ C++.
Great power comes with great responsibility. Any small mistake can have catastrophic consequences or do the unexpected.
The best scenario being a crash or worse open you to great security vulnerability.
This is why systems programming has come to have horror of being a super hero.
- Rust focuses on minimal core and extended by powerful libraries
- Making it possible to incorporate other libraries - Great package management – crates.io
- Open governance model - everyone can contribute to code
High level abstraction – minimizes code, if you were to write a code to remove spaces
Type inference - if you declare a variable as Unsigned 8 bit integer and you have pushed it to a vector which you didn’t define it will automatically give the vector unsigned 8bit integer datatype
Reliability – rare to break – rust code is statically checked at compile time and also compile enforces the 2 concepts used in rust called ownership and borrowing.