SlideShare a Scribd company logo
Henry Schreiner • May 17, 2024 • PyCon US
Modern binary build systems
About me
@henryiii
2
Bec
a
me involved in p
a
ck
a
ging vi
a
Scikit-HEP
W
a
nted to ship bin
a
ry extensions
(boost-histogr
a
m,
a
wkw
a
rd, iminuit)
Joined forces with cibuildwheel
Currently m
a
int
a
in 34+ p
a
ck
a
ges
21 in top 8,000 on PyPI
pybind11 (python_example, cmake_example, scikit_build_example) •
cibuildwheel • build • pipx • pyproject-metadata •
scikit-build (core, cmake, ninja, moderncmakedomain, example-projects) •
nox • validate-pyproject(-schema-store) • pytest GHA annotate-failures •
Plumbum •
fl
ake8-errmsg • check-sdist •
boost-histogram • Hist • UHI • Vector • GooFit • Particle • DecayLanguage •
uproot-browser • Conda-Forge ROOT •
Scienti
fi
c-Python/cookie • repo-review • meson-python •
POVM • hypernewsviewer •
CLI11 • beautifulhugo • Jekyll-Indico
https://iscinumpy.dev
(Slides will be posted here l
a
ter)
Bene
f
its?
3
Fast
Wrap existing libraries
Cross-language
Ship existing CLI tools on PyPI
Can release GIL
Of the top 8,000 p
a
ck
a
ges on PyPI:
6,433 ship
a
single wheel
896 ship multiple wheels (prob
a
bly bin
a
ries!)
671 don't ship
a
wheel (unknown)
Over 10%!
charset-normalizer 367 M
pyyaml 279 M
numpy 246 M
cryptography 238 M
c
f
207 M
pandas 195 M
protobuf 174 M
markupsafe 151 M
wrapt 120 M
pyarrow 110 M
sqlalchemy 104 M
aiohttp 102 M
scipy 101 M
multidict 97 M
psutil 96 M
yarl 95 M
frozenlist 90 M
pillow 89 M
grpcio 88 M
greenlet 88 M
pydantic-core 83 M
Pure Python packages
4
Build con
fi
guration One wheel, one sdist
Binding API Build con
fi
guration Many wheels
pybind11 (C++11)
n
a
nobind (C++17)
Cython (C, C++)
SWIG (C, C++)
PyO3 (Rust)
N
a
tive (C)
cibuildwheel
m
a
turin-
a
ction (Rust)
Scikit-build-core
meson-python
m
a
turin
Binary packages
h
a
tchling
setuptools
poetry-core
build
hynek/build-
a
nd-inspect-python-p
a
ck
a
ge
f
lit-core
pdm-b
a
ckend
...
Making the wheels
cibuildwheel on GitHub Actions
5
on: [push, pull_request]
jobs:
build_wheels:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-13, macos-14]
steps:
- uses: actions/checkout@v4
- uses: pypa/cibuildwheel@v2.17.0
- uses: actions/upload-artifact@v4
with:
name: wheels-${{ strategy.job-index }}
path: ./wheelhouse/*.whl
[tool.cibuildwheel]
test-extras = "test"
test-command = "pytest {project}/tests"
cibuildwheel
Local runs
6
pipx run cibuildwheel --only pp310-manylinux_x86_64
cibuildwheel
New features
• CPython 3.13b1 (opt-in)
• Inherit for overrides (except con
f
ig-settings for now)
• Musllinux 1.2 def
a
ult
• Support Apple Silicon runners on GHA
• Addition
a
l
f
l
a
gs to build frontend
• Better reproducible build support
• E
a
sier loc
a
l runs
7
WIP:
Fix con
f
ig-settings inherit override bug
Free-thre
a
ded support (in m
a
nylinux now!)
Investig
a
ting uv support
Before PEP 517 (2017)
Setuptools / distutils
8
from setuptools import Extension, setup
setup(
name = "mylib",
version = "1.0.0",
ext_modules = [
Extension(
name = "mylib.foo",
sources = ["foo.c"],
),
]
)
Simple! Unless you need…
Third p
a
rty dependencies
Compiler speci
f
ic
f
l
a
gs, like C++ version
P
a
r
a
llel (
f
ile) compiles
C
a
ching / sm
a
rt recompiles
bdist_wheel customiz
a
tions (no public API)
Fortr
a
n
Other compilers
IDE support
Tooling integr
a
tion (debuggers, etc)
Cross-compil
a
tion
Setuptools isn’t trying to
a
dd fe
a
tures to support this
So everyone h
a
s to reinvent the wheel
NumPy h
a
d 13,000 LoC for building!
(MANIFEST.in omitted)
Before PEP 517 (2017)
Extending setuptools / distutils
9
Ex
a
mples of setuptools-b
a
sed builders
Scikit-build (cl
a
ssic)
setuptools-rust
pybind11’s setup_helper
cython (integr
a
ted)
Distutils -> setuptools h
a
s l
a
yered complexity
h
a
rd to debug
e
a
sy to bre
a
k on upd
a
te
h
a
rd to extend
Modern build system
Hooks for build backends
10
Simple st
a
nd
a
rdized API
Supports inst
a
lling build requirements (dyn
a
mic
a
lly too)
B
a
ckend is responsible for b
a
sic
a
lly everything
Edit
a
ble inst
a
lls
a
dded l
a
ter
Build backend hooks
11
Dyn
a
mic
a
lly request
dependencies per st
a
ge
get_requires_for_build_sdist(…)
get_requires_for_build_wheel(…)
get_requires_for_build_editable(…)
build_sdist(…)
build_wheel(…)
build_editable(…)
prepare_metadata_for_build_wheel()
Given source
a
nd con
f
ig,
m
a
ke *.t
a
r.gz
f
ile
And .whl
f
ile
And “edit
a
ble” vi
a
.whl
f
ile
Option
a
l get met
a
d
a
t
a
without build
Rise of PEP 517 build backends
The new era of Python packaging
12
Now we h
a
ve bin
a
ry ones too!
enscons (2017)
m
a
turin (2019)
meson-python (2021)
scikit-build-core (2022)
Pure-Python b
a
ckends were
f
irst:
f
lit-core (2017*)
poetry-core (2018*)
pdm-b
a
ckend (2020*)
h
a
tchling (2022)
setuptools (2022)
Speci
a
l mention: enscons w
a
s the
f
irst PEP 517 build b
a
ckend for bin
a
ries!
(We’ll focus on the other three tod
a
y)
History of meson-python
13
SciPy st
a
rted the project in prep
a
r
a
tion of Python 3.12
Other m
a
jor Scienti
f
ic Python libr
a
ries joined
NumPy went from 13K to 2K LoC for building
(Not counting forking meson (100K)
a
nd vendoring it)
Meson-Python
Meson
14
example-project
├── example.cpp
├── pyproject.toml
└── meson.build
[build-system]
requires = ["meson-python", "pybind11"]
build-backend = "mesonpy"
[project]
name = "example"
version = "0.0.1"
#include <pybind11/pybind11.h>
namespace py = pybind11;
float square(float x) { return x * x; }
PYBIND11_MODULE(example, m) {
m.def("square", &square);
}
project(
'example',
'cpp',
default_options: [
'cpp_std=c++11',
],
)
py = import('python').find_installation(pure: false)
pybind11_dep = dependency('pybind11')
py.extension_module(
'_core',
'example.cpp',
install: true,
dependencies : [pybind11_dep],
)
pipx run build --installer=uv
(Also requires git set up)
Meson-Python
Features
15
C
a
n skip PEP 621
a
nd use Meson con
f
ig for quick projects
Sever
a
l options c
a
n be speci
f
ied in pyproject.toml or vi
a
con
f
ig-settings
Developed rebuild support for edit
a
ble inst
a
lls
Ninj
a
p
a
ck
a
ge
a
dded only if missing
Scikit-build
A family of tools
• Scikit-build-core
• Scikit-build (cl
a
ssic)
• Ninj
a
Python Distribution (pip inst
a
ll cm
a
ke)
• CM
a
ke Python Distribution (pip inst
a
ll ninj
a
with jobserver support)
• ModernCM
a
keDom
a
in (Sphinx plugin)
• Dyn
a
mic-Met
a
d
a
t
a
(WIP project for gener
a
l met
a
d
a
t
a
plugins)
16
Scikit-build-core funded vi
a
NSF gr
a
nt OAC 2209877
Scikit-build-core
CMake
17
example-project
├── example.cpp
├── pyproject.toml
└── CMakeLists.txt
[build-system]
requires = ["scikit-build-core", "pybind11"]
build-backend = "scikit_build_core.build"
[project]
name = "example"
version = "0.0.1"
cmake_minimum_required(VERSION 3.15…3.29)
project(example LANGUAGES CXX)
set(PYBIND11_NEWPYTHON ON)
find_package(pybind11 CONFIG REQUIRED)
pybind11_add_module(example example.cpp)
install(TARGETS example LIBRARY DESTINATION .)
pipx run build --installer=uv
#include <pybind11/pybind11.h>
namespace py = pybind11;
float square(float x) { return x * x; }
PYBIND11_MODULE(example, m) {
m.def("square", &square);
}
Scikit-build-core
Features
• Autom
a
tic inclusion of cm
a
ke/ninj
a
a
s needed
• Limited API / St
a
ble ABI
a
nd pythonless t
a
gs supported vi
a
con
f
ig option
• Support for writing out to extr
a
wheel folders (scripts, he
a
ders, d
a
t
a
, met
a
d
a
t
a
)
• Dedic
a
ted entrypoints for p
a
ck
a
ges to host CM
a
ke code
• Experiment
a
l edit
a
ble mode support, with option
a
l experiment
a
l
a
uto rebuilds on import
• Free-thre
a
ded Python support
18
Scikit-build-core con
f
ig system
A peek into the internals
19
D
a
t
a
cl
a
ss-b
a
sed con
f
ig system
Gener
a
tes JSONSchem
a
Gener
a
tes section in README using cog
Supports pyproject.toml, con
f
ig-settings,
a
nd environment v
a
ri
a
bles
@dataclasses.dataclass
class WheelSettings:
packages: Optional[List[str]] = None
"""
A list of packages to auto-copy into the wheel. If this is not set, it will
default to the first of ``src/<package>``, ``python/<package>``, or
``<package>`` if they exist. The prefix(s) will be stripped from the
package name inside the wheel.
"""
py_api: str = ""
"""
The Python tags. The default (empty string) will use the default Python
version. You can also set this to "cp37" to enable the CPython 3.7+ Stable
ABI / Limited API (only on CPython and if the version is sufficient,
otherwise this has no effect). Or you can set it to "py3" or "py2.py3" to
ignore Python ABI compatibility. The ABI tag is inferred from this tag.
"""
expand_macos_universal_tags: bool = False
"""
Fill out extra tags that are not required. This adds "x86_64" and "arm64"
to the list of platforms when "universal2" is used, which helps older
Pip's (before 21.0.1) find the correct wheel.
"""
Scikit-build-core advanced con
f
ig
Over 40 options
20
[tool.scikit-build]
minimum-version = "0.2"
cmake.verbose = true
logging.level = "INFO"
build-dir = "build/{wheel_tag}"
sdist.include = ["src/some_generated_file.txt"]
sdist.reproducible = false
wheel.expand-macos-universal-tags = true
install.components = ["python"]
[tool.scikit-build.cmake.define]
SOME_DEFINE = {env="SOME_DEFINE", default="EMPTY"}
Scikit-build-core
Dynamic metadata
21
[tool.scikit-build.metadata.version]
provider = "scikit_build_core.metadata.regex"
input = "src/mypackage/__init__.py"
[[tool.scikit-build.generate]]
location = "install"
path = "mypackage/_version.py"
template = '''
version = "${version}"
'''
Join us on scikit-build/dyn
a
mic-met
a
d
a
t
a
if you
a
re
interested in gener
a
lizing for multiple b
a
ckends!
Scikit-build-core
Overrides
22
[[tool.scikit-build.overrides]]
if.platform-system = "darwin"
cmake.version = ">=3.18"
Designed
a
fter cibuildwheel's overrides, inspired by mypy
M
a
tch top to bottom
C
a
n use if.
a
ny
C
a
n inherit with "
a
ppend", "prepend", or "none"
T
a
kes regex, speci
f
ier set, or bool
[[tool.scikit-build.overrides]]
if.state = "editable"
build-dir = "build"
[[tool.scikit-build.overrides]]
if.any.env.SET_FOO = true
if.any.env.FOO_SET = true
inherit.cmake.define = "append"
cmake.define.FOO = "1"
Nanobind example
Using scikit-build-core
23
# CMakeLists.txt
cmake_minimum_required(VERSION 3.15...3.26)
project(nanobind_example LANGUAGES CXX)
find_package(Python 3.8
REQUIRED COMPONENTS Interpreter Development.Module
OPTIONAL_COMPONENTS Development.SABIModule)
find_package(nanobind CONFIG REQUIRED)
nanobind_add_module(
nanobind_example_ext
STABLE_ABI
NB_STATIC
src/nanobind_example_ext.cpp
)
install(TARGETS nanobind_example_ext LIBRARY DESTINATION nanobind_example)
# pyproject.toml
[build-system]
requires = [
"scikit-build-core",
"nanobind",
]
build-backend = "scikit_build_core.build"
[tool.scikit-build]
wheel.py-api = "cp312"
[tool.cibuildwheel.macos.environment]
MACOSX_DEPLOYMENT_TARGET = "10.14"
[project]
name = "some-package"
version = "1.0.0"
Scikit-build-core
Some projects
24
R
a
pids.
a
i
Added to
f
ile gener
a
tor
a
nd
deployed
a
cross
a
ll projects
[tool.scikit-build]
wheel.packages = ["zmq"]
wheel.license-files = ["licenses/LICENSE*"]
cmake.version = ">=3.15"
# only build/install the pyzmq component
cmake.targets = ["pyzmq"]
install.components = ["pyzmq"]
PyZMQ
Added to
f
ile gener
a
tor
a
nd
deployed
a
cross
a
ll projects
[tool.scikit-build]
minimum-version = "0.8"
build-dir = "build/{wheel_tag}"
cmake.version = "" # We are cmake, so don't request cmake
ninja.make-fallback = false
wheel.py-api = "py3"
wheel.expand-macos-universal-tags = true
wheel.install-dir = "cmake/data"
[[tool.scikit-build.generate]]
path = "cmake/_version.py"
template = '''
version = "${version}"
'''
cm
a
ke (python distributions)
Fun met
a
problem: we
a
re cm
a
ke!
Bonus: Scikit-build-core plugins
Scikit-build-core is also useful for creating plugins!
25
[build-system]
requires = ["hatchling", "scikit-build-core~=0.9.0"]
build-backend = "hatchling.build"
[project]
name = "hatchling_example"
version = "0.1.0"
[tool.hatch.build.targets.wheel.hooks.scikit-build]
experimental = true
This will
a
dd
a
CM
a
ke extension to
a
h
a
tchling project!
PyPI stats
Just download every pyproject.toml on PyPI
26
https://github.com/henryiii/pyst
a
ts & https://hugovk.github.io/top-pypi-p
a
ck
a
ges
m
a
turin: 1,744
scikit_build_core.build: 222
mesonpy: 123
maturin
#81 pydantic-core 82,784,026
#108 rpds-py 61,448,636
#250 pendulum 26,584,509
#298 tokenizers 22,436,866
#330 orjson 19,225,277
#402 safetensors 15,136,797
#511 ruff 9,718,295
#663 cramjam 6,773,950
#664 watchfiles 6,741,022
#732 nh3 6,193,025
#790 polars 5,731,171
#810 uv 5,443,201
#875 dbt-extractor 4,830,288
#1124 jellyfish 2,835,548
#1319 deltalake 2,017,365
#1375 py-spy 1,827,491
#1766 tlparse 1,126,435
#2001 pywinpty 871,147
#2158 y-py 750,010
#2269 clarabel 686,786
#2338 kornia-rs 648,550
#2901 regress 403,177
#3165 rtoml 327,378
#3309 tsdownsample 302,822
#3348 grimp 294,794
#3386 chia-rs 287,786
#3525 openlineage-sql 269,605
#3694 h3ronpy 253,144
#3737 cmsis-pack-manager 247,677
#3989 blake3 218,936
#4168 pyxirr 200,494
#4449 sqlglotrs 175,146
#4481 akinator-py 173,092
#4565 hf-transfer 165,738
#5479 clvm-tools-rs 125,016
#5699 css-inline 114,355
#5947 mitmproxy-rs 104,179
#6269 py-sr25519-bindings 91,533
#6558 python-calamine 82,461
#6626 qh3 80,379
#6674 mitmproxy-wireguard 79,065
#6712 deptry 78,113
#6907 polars-lts-cpu 73,655
#7019 minify-html 70,966
#7039 solders 70,434
#7042 topgrade 70,373
#7240 typos 66,022
#7313 tzfpy 64,947
#7512 netifaces2 61,198
#7516 rjieba 61,141
#7566 lintrunner 60,240
#7647 py-bip39-bindings 58,624
#7698 py-ed25519-zebra-bindings 57,611
#7752 uuid-utils 56,790
scikit-build-core
#229 pyzmq 30,543,660
#645 lightgbm 7,046,707
#709 cmake 6,369,474
#1771 phik 1,118,403
#2090 clang-format 797,795
#3005 awkward-cpp 369,071
#3544 llama-cpp-python 267,061
#4047 openexr 213,581
#4636 coreforecast 160,430
#4711 sparse-dot-topn 154,874
#5685 laszip 114,988
#6299 iminuit 90,796
#6594 spglib 81,422
meson-python
#18 numpy 245,757,478
#26 pandas 195,143,812
#61 scipy 100,730,066
#107 scikit-learn 62,470,305
#112 matplotlib 59,187,475
#208 contourpy 33,391,835
#437 scikit-image 13,129,903
#475 pywavelets 11,126,522
#1781 pygobject 1,104,329
#1875 meson-python 988,201
#1924 scs 934,563
#4505 dftd3 170,689
#5410 dbus-python 129,002
(Doesn't include RAPIDS.
a
i p
a
ck
a
ges like cudf)
PyPI Stats
Looking at every pyproject.toml
27
tool.*:
poetry: 52183
black: 24730
pytest: 18229
isort: 18111
setuptools: 15210
mypy: 13030
ruff: 12213
coverage: 10749
setuptools_scm: 7006
hatch: 6900
pylint: 4261
pyright: 3635
flit: 3039
flake8: 2216
pdm: 1995
tox: 1577
semantic_release: 1553
cibuildwheel: 1474
codespell: 1431
maturin: 1207
poetry-dynamic-versioning: 1150
C
a
n look
a
t the most
popul
a
r tool sections:
This helped bl
a
ck
check the imp
a
ct
of
a
con
f
ig bug!
tool.black.line-length contents:
120: 5755
88: 3578
79: 2876
100: 2714
80: 834
...
C
a
n check to see wh
a
t
our users
a
re using
a
nd
setting!
*:
build-system: 136738
tool: 100937
project: 51268
options: 305
tools: 279
metadata: 202
coverage: 148
mypy: 127
requires: 119
flake8: 119
virtualenvs: 90
build-backend: 81
pytest: 76
dependencies: 67
bdist_wheel: 40
build: 36
pypi: 34
build_system: 32
dev-dependencies: 30
package: 30
bumpver: 28
manageprojects: 25
too: 25
...
Or typos:
Nothing p
a
st
here is v
a
lid!
(Use v
a
lid
a
te-pyproject)!
PyPI Stats
Scikit-build-core usage
28 tool.scikit-build.*.*:
metadata.version: 83
build-dir: 78
wheel.packages: 76
sdist.include: 69
minimum-version: 68
sdist.exclude: 60
cmake.define: 57
wheel.expand-macos-universal-tags: 54
wheel.py-api: 53
cmake.verbose: 47
cmake.minimum-version: 44
cmake.build-type: 43
cmake.args: 32
wheel.install-dir: 32
cmake.version: 30
logging.level: 29
wheel.license-files: 20
install.components: 20
cmake.targets: 17
experimental: 15
ninja.version: 14
ninja.make-fallback: 14
ninja.minimum-version: 13
cmake.source-dir: 12
sdist.cmake: 12
sdist.reproducible: 9
install.strip: 9
generate: 9
wheel.exclude: 6
editable.mode: 5
strict-config: 5
metadata.readme: 5
editable.verbose: 4
editable.rebuild: 4
wheel.platlib: 3
backport.find-python: 3
wheel.cmake: 2
wheel.build-tag: 2
metadata.scripts: 2
overrides: 2
metadata.optional-dependencies: 1
tool.scikit-build.wheel.py-api contents:
{}: 146661
'cp312': 30
'py3': 10
'py2.py3': 7
'cp310': 1
'py37': 1
'cp37': 1
tool.scikit-build.minimum-version contents:
{}: 146646
'0.4': 18
'0.8': 11
'0.5': 9
'0.5.1': 6
'0.8.1': 5
'0.9': 4
'0.3': 2
'0.5.0': 2
'0.6.1': 2
'0.7': 2
'0.2.1': 2
'0.2': 1
'0.6': 1
'0.8.2': 1
'0.4.4': 1
'0.3.0': 1
Maturin
Cargo (Rust)
29
A Rust-b
a
sed build system for C
a
rgo
example-project
├── src/lib.rs
├── Cargo.toml
└── pyproject.toml
[package]
name = "package"
version = "0.1.0"
edition = "2021"
[lib]
name = "example"
crate-type = ["cdylib"]
[dependencies.pyo3]
version = "0.21.1"
features = [
"extension-module",
"abi3-py38",
"experimental-declarative-modules",
]
use pyo3::prelude::*;
#[pymodule]
mod example {
use super::*;
#[pyfunction]
fn add(x: i64, y: i64) -> i64 {
x + y
}
}
[build-system]
requires = ["maturin~=1.5"]
build-backend = "maturin"
[project]
name = "example"
dynamic = ["version"]
pipx run build —installer=uv
maturin build --release
Maturin
Features
30
Written in Rust
(c
a
n build
a
p
a
ck
a
ge without running Python!)
Limited API doesn't even require Python
a
t
a
ll!
C
a
rgo's build-from-source p
a
ck
a
ging is perfect
for cross-compil
a
tion
a
nd m
a
king wheels!
CLI c
a
n cre
a
te
a
new project.
Why Rust?
Rust/C
a
rgo m
a
kes using
libr
a
ries
a
s e
a
sy
a
s Python!
Cross-compiles
a
re gre
a
t
(c
a
n
a
void most m
a
nylinux issues)
Tooling is f
a
nt
a
stic
(linter, form
a
tter, IDEs, etc)
PyO3 bindings
a
re gre
a
t
Single bin
a
ry
a
ppro
a
ch
works well for wheels
(And m
a
ybe some memory s
a
fety stu
ff
)
Maturin-Action
GitHub Action for Building
31
Gre
a
t
a
t cross-compiling with extensive support
(f
a
st)
Builds non-Python Rust p
a
rts once for
a
ll versions
Every wheel supported except musl s390x wheels
(some pl
a
tforms like 32-bit musl c
a
n't be done with
cibuildwheel since rust doesn't build c
a
rgo for them)
...
Compiling lexical-sort v0.3.1
Compiling taplo v0.13.0
Compiling pep508_rs v0.6.0
Compiling pyproject-fmt-rust v1.1.1 (/home/runner/...
Finished release [optimized] target(s) in 13.53s
📦 Including files matching "rust-toolchain.toml"
📦 Built wheel for abi3 Python ≥ 3.8 to dist/...
⚠ Warning: PyPy does not yet support abi3 so the build...
Compiling pyo3-build-config v0.21.2
Compiling pyo3-macros-backend v0.21.2
Compiling pyo3-ffi v0.21.2
Compiling pyo3 v0.21.2
Compiling pyo3-macros v0.21.2
Compiling pyproject-fmt-rust v1.1.1 (/home/runner/...
Finished release [optimized] target(s) in 4.25s
📦 Including files matching "rust-toolchain.toml"
📦 Built wheel for PyPy 3.8 to dist/pyproject_fmt_rust-...
Compiling pyo3-build-config v0.21.2
Compiling pyo3-macros-backend v0.21.2
Compiling pyo3-ffi v0.21.2
Compiling pyo3 v0.21.2
Compiling pyo3-macros v0.21.2
Compiling pyproject-fmt-rust v1.1.1 (/home/runner/...
Finished release [optimized] target(s) in 4.24s
...
Resouces
Scienti
fi
c-Python
Development Guide
32
Useful for
a
ll projects
(not just scienti
f
ic ones!)
Up-to-d
a
te guides
Integr
a
ted templ
a
te rendered
in some pl
a
ces vi
a
cog
Versions upd
a
ted by CI
Integr
a
ted WASM repo-review
Resources
Scienti
fi
c-python/cookie
33
11 b
a
ckends
Cookiecutter & copier supported
In sync with the dev guide
Gener
a
tion tested by nox
🎤 The name of your project
myprog
🎤 The name of your (GitHub?) org
henryiii
🎤 The url to your GitHub or GitLab repository
https://github.com/henryiii/myprog
🎤 Your name
Henry Schreiner
🎤 Your email
a@b.com
🎤 A short description of your project
A great package.
🎤 Select a license
BSD
🎤 Choose a build backend
Scikit-build-core - Compiled C++ (recommended)
🎤 Use version control for versioning
No
Copying from template version 2024.4.23
create .
create CMakeLists.txt
create .pre-commit-config.yaml
create pyproject.toml
create tests
create tests/test_package.py
create tests/test_compiled.py
create .git_archival.txt
create LICENSE
create docs
create docs/conf.py
create docs/index.md
create README.md
create .gitignore
create .github
create .github/workflows
create .github/workflows/cd.yml
create .github/workflows/ci.yml
create .github/CONTRIBUTING.md
create .github/dependabot.yml
create .gitattributes
create .copier-answers.yml
create noxfile.py
create .readthedocs.yaml
create src
create src/main.cpp
create src/myprog
create src/myprog/__init__.py
create src/myprog/_core.pyi
create src/myprog/py.typed
Resources
Repo-review
34
Python 3.10+ fr
a
mework
sp-repo-review: checks from guide
Supports WebAssembly
a
nd CLI
(
a
nd pre-commit, etc)
A checker for your checker con
f
ig!
Checks linked to integr
a
ted t
a
gs in guide
V
a
lid
a
te-pyproject
a
lso integr
a
ted
Summary
So much has happened in the last two years!
• Three f
a
nt
a
stic build-b
a
ckends
a
re now
a
v
a
il
a
ble for bin
a
ry projects
• Gre
a
t resources
a
v
a
il
a
ble for developing p
a
ck
a
ges
• You c
a
n ship C/C++/Rust code for
a
ll m
a
jor pl
a
tforms even for
a
sm
a
ll project
35
Credits and thanks
36
scikit-build te
a
m
Je
a
n-Christophe Fillion-Robin (Kitw
a
re)
M
a
tt McCormick (Kitw
a
re)
Cristi
a
n Le
meson-python te
a
m
R
a
lf Gommers
D
a
niele Nicolodi
Henry Schreiner
Thom
a
s Li
Filipe L
a
íns (Emeritus)
cibuildwheel te
a
m
Joe Rickerby
Y
a
nnick J
a
doul
M
a
tthieu D
a
rbois
Grzegorz Bokot
a
And th
a
nks to the m
a
turin developers for th
a
t f
a
nt
a
stic tool!
Thanks to my other team members!

More Related Content

Similar to Modern binary build systems - PyCon 2024

Digital RSE: automated code quality checks - RSE group meeting
Digital RSE: automated code quality checks - RSE group meetingDigital RSE: automated code quality checks - RSE group meeting
Digital RSE: automated code quality checks - RSE group meeting
Henry Schreiner
 
Introduction to cython
Introduction to cythonIntroduction to cython
Introduction to cython
John(Qiang) Zhang
 
PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat
PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat
PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat
Pôle Systematic Paris-Region
 
Princeton RSE Peer network first meeting
Princeton RSE Peer network first meetingPrinceton RSE Peer network first meeting
Princeton RSE Peer network first meeting
Henry Schreiner
 
Pythonpresent
PythonpresentPythonpresent
Pythonpresent
Chui-Wen Chiu
 
Cluj.py Meetup: Extending Python in C
Cluj.py Meetup: Extending Python in CCluj.py Meetup: Extending Python in C
Cluj.py Meetup: Extending Python in C
Steffen Wenz
 
PyCon Estonia 2019
PyCon Estonia 2019PyCon Estonia 2019
PyCon Estonia 2019
Travis Oliphant
 
나도 할 수 있다 오픈소스
나도 할 수 있다 오픈소스나도 할 수 있다 오픈소스
나도 할 수 있다 오픈소스
효준 강
 
PyCon 2013 : Scripting to PyPi to GitHub and More
PyCon 2013 : Scripting to PyPi to GitHub and MorePyCon 2013 : Scripting to PyPi to GitHub and More
PyCon 2013 : Scripting to PyPi to GitHub and More
Matt Harrison
 
Pybind11 - SciPy 2021
Pybind11 - SciPy 2021Pybind11 - SciPy 2021
Pybind11 - SciPy 2021
Henry Schreiner
 
First python project
First python projectFirst python project
First python project
Neetu Jain
 
OpenStack API's and WSGI
OpenStack API's and WSGIOpenStack API's and WSGI
OpenStack API's and WSGI
Mike Pittaro
 
Heroku pycon
Heroku pyconHeroku pycon
Heroku pycon
Sabatino Severino
 
Getting started with open cv in raspberry pi
Getting started with open cv in raspberry piGetting started with open cv in raspberry pi
Getting started with open cv in raspberry pi
Jayaprakash Nagaruru
 
Ekon 25 Python4Delphi_MX475
Ekon 25 Python4Delphi_MX475Ekon 25 Python4Delphi_MX475
Ekon 25 Python4Delphi_MX475
Max Kleiner
 
Python on pi
Python on piPython on pi
Python on pi
swee meng ng
 
Software development practices in python
Software development practices in pythonSoftware development practices in python
Software development practices in python
Jimmy Lai
 
Viktor Tsykunov "Microsoft AI platform for every Developer"
Viktor Tsykunov "Microsoft AI platform for every Developer"Viktor Tsykunov "Microsoft AI platform for every Developer"
Viktor Tsykunov "Microsoft AI platform for every Developer"
Lviv Startup Club
 
R sharing 101
R sharing 101R sharing 101
R sharing 101
Omnia Safaan
 
[GSoC 2017] gopy: Updating gopy to support Python3 and PyPy
[GSoC 2017] gopy: Updating gopy to support Python3 and PyPy[GSoC 2017] gopy: Updating gopy to support Python3 and PyPy
[GSoC 2017] gopy: Updating gopy to support Python3 and PyPy
Dong-hee Na
 

Similar to Modern binary build systems - PyCon 2024 (20)

Digital RSE: automated code quality checks - RSE group meeting
Digital RSE: automated code quality checks - RSE group meetingDigital RSE: automated code quality checks - RSE group meeting
Digital RSE: automated code quality checks - RSE group meeting
 
Introduction to cython
Introduction to cythonIntroduction to cython
Introduction to cython
 
PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat
PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat
PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat
 
Princeton RSE Peer network first meeting
Princeton RSE Peer network first meetingPrinceton RSE Peer network first meeting
Princeton RSE Peer network first meeting
 
Pythonpresent
PythonpresentPythonpresent
Pythonpresent
 
Cluj.py Meetup: Extending Python in C
Cluj.py Meetup: Extending Python in CCluj.py Meetup: Extending Python in C
Cluj.py Meetup: Extending Python in C
 
PyCon Estonia 2019
PyCon Estonia 2019PyCon Estonia 2019
PyCon Estonia 2019
 
나도 할 수 있다 오픈소스
나도 할 수 있다 오픈소스나도 할 수 있다 오픈소스
나도 할 수 있다 오픈소스
 
PyCon 2013 : Scripting to PyPi to GitHub and More
PyCon 2013 : Scripting to PyPi to GitHub and MorePyCon 2013 : Scripting to PyPi to GitHub and More
PyCon 2013 : Scripting to PyPi to GitHub and More
 
Pybind11 - SciPy 2021
Pybind11 - SciPy 2021Pybind11 - SciPy 2021
Pybind11 - SciPy 2021
 
First python project
First python projectFirst python project
First python project
 
OpenStack API's and WSGI
OpenStack API's and WSGIOpenStack API's and WSGI
OpenStack API's and WSGI
 
Heroku pycon
Heroku pyconHeroku pycon
Heroku pycon
 
Getting started with open cv in raspberry pi
Getting started with open cv in raspberry piGetting started with open cv in raspberry pi
Getting started with open cv in raspberry pi
 
Ekon 25 Python4Delphi_MX475
Ekon 25 Python4Delphi_MX475Ekon 25 Python4Delphi_MX475
Ekon 25 Python4Delphi_MX475
 
Python on pi
Python on piPython on pi
Python on pi
 
Software development practices in python
Software development practices in pythonSoftware development practices in python
Software development practices in python
 
Viktor Tsykunov "Microsoft AI platform for every Developer"
Viktor Tsykunov "Microsoft AI platform for every Developer"Viktor Tsykunov "Microsoft AI platform for every Developer"
Viktor Tsykunov "Microsoft AI platform for every Developer"
 
R sharing 101
R sharing 101R sharing 101
R sharing 101
 
[GSoC 2017] gopy: Updating gopy to support Python3 and PyPy
[GSoC 2017] gopy: Updating gopy to support Python3 and PyPy[GSoC 2017] gopy: Updating gopy to support Python3 and PyPy
[GSoC 2017] gopy: Updating gopy to support Python3 and PyPy
 

More from Henry Schreiner

Software Quality Assurance Tooling - Wintersession 2024
Software Quality Assurance Tooling - Wintersession 2024Software Quality Assurance Tooling - Wintersession 2024
Software Quality Assurance Tooling - Wintersession 2024
Henry Schreiner
 
What's new in Python 3.11
What's new in Python 3.11What's new in Python 3.11
What's new in Python 3.11
Henry Schreiner
 
Everything you didn't know you needed
Everything you didn't know you neededEverything you didn't know you needed
Everything you didn't know you needed
Henry Schreiner
 
PyCon 2022 -Scikit-HEP Developer Pages: Guidelines for modern packaging
PyCon 2022 -Scikit-HEP Developer Pages: Guidelines for modern packagingPyCon 2022 -Scikit-HEP Developer Pages: Guidelines for modern packaging
PyCon 2022 -Scikit-HEP Developer Pages: Guidelines for modern packaging
Henry Schreiner
 
boost-histogram / Hist: PyHEP Topical meeting
boost-histogram / Hist: PyHEP Topical meetingboost-histogram / Hist: PyHEP Topical meeting
boost-histogram / Hist: PyHEP Topical meeting
Henry Schreiner
 
CMake best practices
CMake best practicesCMake best practices
CMake best practices
Henry Schreiner
 
RDM 2020: Python, Numpy, and Pandas
RDM 2020: Python, Numpy, and PandasRDM 2020: Python, Numpy, and Pandas
RDM 2020: Python, Numpy, and Pandas
Henry Schreiner
 
HOW 2019: Machine Learning for the Primary Vertex Reconstruction
HOW 2019: Machine Learning for the Primary Vertex ReconstructionHOW 2019: Machine Learning for the Primary Vertex Reconstruction
HOW 2019: Machine Learning for the Primary Vertex Reconstruction
Henry Schreiner
 
HOW 2019: A complete reproducible ROOT environment in under 5 minutes
HOW 2019: A complete reproducible ROOT environment in under 5 minutesHOW 2019: A complete reproducible ROOT environment in under 5 minutes
HOW 2019: A complete reproducible ROOT environment in under 5 minutes
Henry Schreiner
 
ACAT 2019: A hybrid deep learning approach to vertexing
ACAT 2019: A hybrid deep learning approach to vertexingACAT 2019: A hybrid deep learning approach to vertexing
ACAT 2019: A hybrid deep learning approach to vertexing
Henry Schreiner
 
2019 CtD: A hybrid deep learning approach to vertexing
2019 CtD: A hybrid deep learning approach to vertexing2019 CtD: A hybrid deep learning approach to vertexing
2019 CtD: A hybrid deep learning approach to vertexing
Henry Schreiner
 
2019 IRIS-HEP AS workshop: Boost-histogram and hist
2019 IRIS-HEP AS workshop: Boost-histogram and hist2019 IRIS-HEP AS workshop: Boost-histogram and hist
2019 IRIS-HEP AS workshop: Boost-histogram and hist
Henry Schreiner
 
IRIS-HEP: Boost-histogram and Hist
IRIS-HEP: Boost-histogram and HistIRIS-HEP: Boost-histogram and Hist
IRIS-HEP: Boost-histogram and Hist
Henry Schreiner
 
2019 IRIS-HEP AS workshop: Particles and decays
2019 IRIS-HEP AS workshop: Particles and decays2019 IRIS-HEP AS workshop: Particles and decays
2019 IRIS-HEP AS workshop: Particles and decays
Henry Schreiner
 
IRIS-HEP Retreat: Boost-Histogram Roadmap
IRIS-HEP Retreat: Boost-Histogram RoadmapIRIS-HEP Retreat: Boost-Histogram Roadmap
IRIS-HEP Retreat: Boost-Histogram Roadmap
Henry Schreiner
 
PyHEP 2019: Python 3.8
PyHEP 2019: Python 3.8PyHEP 2019: Python 3.8
PyHEP 2019: Python 3.8
Henry Schreiner
 
PyHEP 2019: Python Histogramming Packages
PyHEP 2019: Python Histogramming PackagesPyHEP 2019: Python Histogramming Packages
PyHEP 2019: Python Histogramming Packages
Henry Schreiner
 
2019 IML workshop: A hybrid deep learning approach to vertexing
2019 IML workshop: A hybrid deep learning approach to vertexing2019 IML workshop: A hybrid deep learning approach to vertexing
2019 IML workshop: A hybrid deep learning approach to vertexing
Henry Schreiner
 
CHEP 2019: Recent developments in histogram libraries
CHEP 2019: Recent developments in histogram librariesCHEP 2019: Recent developments in histogram libraries
CHEP 2019: Recent developments in histogram libraries
Henry Schreiner
 
DIANA: Recent developments in GooFit
DIANA: Recent developments in GooFitDIANA: Recent developments in GooFit
DIANA: Recent developments in GooFit
Henry Schreiner
 

More from Henry Schreiner (20)

Software Quality Assurance Tooling - Wintersession 2024
Software Quality Assurance Tooling - Wintersession 2024Software Quality Assurance Tooling - Wintersession 2024
Software Quality Assurance Tooling - Wintersession 2024
 
What's new in Python 3.11
What's new in Python 3.11What's new in Python 3.11
What's new in Python 3.11
 
Everything you didn't know you needed
Everything you didn't know you neededEverything you didn't know you needed
Everything you didn't know you needed
 
PyCon 2022 -Scikit-HEP Developer Pages: Guidelines for modern packaging
PyCon 2022 -Scikit-HEP Developer Pages: Guidelines for modern packagingPyCon 2022 -Scikit-HEP Developer Pages: Guidelines for modern packaging
PyCon 2022 -Scikit-HEP Developer Pages: Guidelines for modern packaging
 
boost-histogram / Hist: PyHEP Topical meeting
boost-histogram / Hist: PyHEP Topical meetingboost-histogram / Hist: PyHEP Topical meeting
boost-histogram / Hist: PyHEP Topical meeting
 
CMake best practices
CMake best practicesCMake best practices
CMake best practices
 
RDM 2020: Python, Numpy, and Pandas
RDM 2020: Python, Numpy, and PandasRDM 2020: Python, Numpy, and Pandas
RDM 2020: Python, Numpy, and Pandas
 
HOW 2019: Machine Learning for the Primary Vertex Reconstruction
HOW 2019: Machine Learning for the Primary Vertex ReconstructionHOW 2019: Machine Learning for the Primary Vertex Reconstruction
HOW 2019: Machine Learning for the Primary Vertex Reconstruction
 
HOW 2019: A complete reproducible ROOT environment in under 5 minutes
HOW 2019: A complete reproducible ROOT environment in under 5 minutesHOW 2019: A complete reproducible ROOT environment in under 5 minutes
HOW 2019: A complete reproducible ROOT environment in under 5 minutes
 
ACAT 2019: A hybrid deep learning approach to vertexing
ACAT 2019: A hybrid deep learning approach to vertexingACAT 2019: A hybrid deep learning approach to vertexing
ACAT 2019: A hybrid deep learning approach to vertexing
 
2019 CtD: A hybrid deep learning approach to vertexing
2019 CtD: A hybrid deep learning approach to vertexing2019 CtD: A hybrid deep learning approach to vertexing
2019 CtD: A hybrid deep learning approach to vertexing
 
2019 IRIS-HEP AS workshop: Boost-histogram and hist
2019 IRIS-HEP AS workshop: Boost-histogram and hist2019 IRIS-HEP AS workshop: Boost-histogram and hist
2019 IRIS-HEP AS workshop: Boost-histogram and hist
 
IRIS-HEP: Boost-histogram and Hist
IRIS-HEP: Boost-histogram and HistIRIS-HEP: Boost-histogram and Hist
IRIS-HEP: Boost-histogram and Hist
 
2019 IRIS-HEP AS workshop: Particles and decays
2019 IRIS-HEP AS workshop: Particles and decays2019 IRIS-HEP AS workshop: Particles and decays
2019 IRIS-HEP AS workshop: Particles and decays
 
IRIS-HEP Retreat: Boost-Histogram Roadmap
IRIS-HEP Retreat: Boost-Histogram RoadmapIRIS-HEP Retreat: Boost-Histogram Roadmap
IRIS-HEP Retreat: Boost-Histogram Roadmap
 
PyHEP 2019: Python 3.8
PyHEP 2019: Python 3.8PyHEP 2019: Python 3.8
PyHEP 2019: Python 3.8
 
PyHEP 2019: Python Histogramming Packages
PyHEP 2019: Python Histogramming PackagesPyHEP 2019: Python Histogramming Packages
PyHEP 2019: Python Histogramming Packages
 
2019 IML workshop: A hybrid deep learning approach to vertexing
2019 IML workshop: A hybrid deep learning approach to vertexing2019 IML workshop: A hybrid deep learning approach to vertexing
2019 IML workshop: A hybrid deep learning approach to vertexing
 
CHEP 2019: Recent developments in histogram libraries
CHEP 2019: Recent developments in histogram librariesCHEP 2019: Recent developments in histogram libraries
CHEP 2019: Recent developments in histogram libraries
 
DIANA: Recent developments in GooFit
DIANA: Recent developments in GooFitDIANA: Recent developments in GooFit
DIANA: Recent developments in GooFit
 

Recently uploaded

E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
Hornet Dynamics
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
Peter Muessig
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
rodomar2
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata
 
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s EcosystemUI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
Peter Muessig
 
What is Master Data Management by PiLog Group
What is Master Data Management by PiLog GroupWhat is Master Data Management by PiLog Group
What is Master Data Management by PiLog Group
aymanquadri279
 
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Crescat
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
Remote DBA Services
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptxLORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
lorraineandreiamcidl
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
Green Software Development
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
ICS
 
Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
brainerhub1
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Łukasz Chruściel
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
Aftab Hussain
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Grant Fritchey
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
Rakesh Kumar R
 
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j
 
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdfVitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
SOCRadar
 

Recently uploaded (20)

E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
 
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s EcosystemUI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
 
What is Master Data Management by PiLog Group
What is Master Data Management by PiLog GroupWhat is Master Data Management by PiLog Group
What is Master Data Management by PiLog Group
 
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptxLORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
 
Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
 
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
 
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdfVitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdf
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
 

Modern binary build systems - PyCon 2024

  • 1. Henry Schreiner • May 17, 2024 • PyCon US Modern binary build systems
  • 2. About me @henryiii 2 Bec a me involved in p a ck a ging vi a Scikit-HEP W a nted to ship bin a ry extensions (boost-histogr a m, a wkw a rd, iminuit) Joined forces with cibuildwheel Currently m a int a in 34+ p a ck a ges 21 in top 8,000 on PyPI pybind11 (python_example, cmake_example, scikit_build_example) • cibuildwheel • build • pipx • pyproject-metadata • scikit-build (core, cmake, ninja, moderncmakedomain, example-projects) • nox • validate-pyproject(-schema-store) • pytest GHA annotate-failures • Plumbum • fl ake8-errmsg • check-sdist • boost-histogram • Hist • UHI • Vector • GooFit • Particle • DecayLanguage • uproot-browser • Conda-Forge ROOT • Scienti fi c-Python/cookie • repo-review • meson-python • POVM • hypernewsviewer • CLI11 • beautifulhugo • Jekyll-Indico https://iscinumpy.dev (Slides will be posted here l a ter)
  • 3. Bene f its? 3 Fast Wrap existing libraries Cross-language Ship existing CLI tools on PyPI Can release GIL Of the top 8,000 p a ck a ges on PyPI: 6,433 ship a single wheel 896 ship multiple wheels (prob a bly bin a ries!) 671 don't ship a wheel (unknown) Over 10%! charset-normalizer 367 M pyyaml 279 M numpy 246 M cryptography 238 M c f 207 M pandas 195 M protobuf 174 M markupsafe 151 M wrapt 120 M pyarrow 110 M sqlalchemy 104 M aiohttp 102 M scipy 101 M multidict 97 M psutil 96 M yarl 95 M frozenlist 90 M pillow 89 M grpcio 88 M greenlet 88 M pydantic-core 83 M
  • 4. Pure Python packages 4 Build con fi guration One wheel, one sdist Binding API Build con fi guration Many wheels pybind11 (C++11) n a nobind (C++17) Cython (C, C++) SWIG (C, C++) PyO3 (Rust) N a tive (C) cibuildwheel m a turin- a ction (Rust) Scikit-build-core meson-python m a turin Binary packages h a tchling setuptools poetry-core build hynek/build- a nd-inspect-python-p a ck a ge f lit-core pdm-b a ckend ...
  • 5. Making the wheels cibuildwheel on GitHub Actions 5 on: [push, pull_request] jobs: build_wheels: runs-on: ${{ matrix.os }} strategy: matrix: os: [ubuntu-latest, windows-latest, macos-13, macos-14] steps: - uses: actions/checkout@v4 - uses: pypa/cibuildwheel@v2.17.0 - uses: actions/upload-artifact@v4 with: name: wheels-${{ strategy.job-index }} path: ./wheelhouse/*.whl [tool.cibuildwheel] test-extras = "test" test-command = "pytest {project}/tests"
  • 6. cibuildwheel Local runs 6 pipx run cibuildwheel --only pp310-manylinux_x86_64
  • 7. cibuildwheel New features • CPython 3.13b1 (opt-in) • Inherit for overrides (except con f ig-settings for now) • Musllinux 1.2 def a ult • Support Apple Silicon runners on GHA • Addition a l f l a gs to build frontend • Better reproducible build support • E a sier loc a l runs 7 WIP: Fix con f ig-settings inherit override bug Free-thre a ded support (in m a nylinux now!) Investig a ting uv support
  • 8. Before PEP 517 (2017) Setuptools / distutils 8 from setuptools import Extension, setup setup( name = "mylib", version = "1.0.0", ext_modules = [ Extension( name = "mylib.foo", sources = ["foo.c"], ), ] ) Simple! Unless you need… Third p a rty dependencies Compiler speci f ic f l a gs, like C++ version P a r a llel ( f ile) compiles C a ching / sm a rt recompiles bdist_wheel customiz a tions (no public API) Fortr a n Other compilers IDE support Tooling integr a tion (debuggers, etc) Cross-compil a tion Setuptools isn’t trying to a dd fe a tures to support this So everyone h a s to reinvent the wheel NumPy h a d 13,000 LoC for building! (MANIFEST.in omitted)
  • 9. Before PEP 517 (2017) Extending setuptools / distutils 9 Ex a mples of setuptools-b a sed builders Scikit-build (cl a ssic) setuptools-rust pybind11’s setup_helper cython (integr a ted) Distutils -> setuptools h a s l a yered complexity h a rd to debug e a sy to bre a k on upd a te h a rd to extend
  • 10. Modern build system Hooks for build backends 10 Simple st a nd a rdized API Supports inst a lling build requirements (dyn a mic a lly too) B a ckend is responsible for b a sic a lly everything Edit a ble inst a lls a dded l a ter
  • 11. Build backend hooks 11 Dyn a mic a lly request dependencies per st a ge get_requires_for_build_sdist(…) get_requires_for_build_wheel(…) get_requires_for_build_editable(…) build_sdist(…) build_wheel(…) build_editable(…) prepare_metadata_for_build_wheel() Given source a nd con f ig, m a ke *.t a r.gz f ile And .whl f ile And “edit a ble” vi a .whl f ile Option a l get met a d a t a without build
  • 12. Rise of PEP 517 build backends The new era of Python packaging 12 Now we h a ve bin a ry ones too! enscons (2017) m a turin (2019) meson-python (2021) scikit-build-core (2022) Pure-Python b a ckends were f irst: f lit-core (2017*) poetry-core (2018*) pdm-b a ckend (2020*) h a tchling (2022) setuptools (2022) Speci a l mention: enscons w a s the f irst PEP 517 build b a ckend for bin a ries! (We’ll focus on the other three tod a y)
  • 13. History of meson-python 13 SciPy st a rted the project in prep a r a tion of Python 3.12 Other m a jor Scienti f ic Python libr a ries joined NumPy went from 13K to 2K LoC for building (Not counting forking meson (100K) a nd vendoring it)
  • 14. Meson-Python Meson 14 example-project ├── example.cpp ├── pyproject.toml └── meson.build [build-system] requires = ["meson-python", "pybind11"] build-backend = "mesonpy" [project] name = "example" version = "0.0.1" #include <pybind11/pybind11.h> namespace py = pybind11; float square(float x) { return x * x; } PYBIND11_MODULE(example, m) { m.def("square", &square); } project( 'example', 'cpp', default_options: [ 'cpp_std=c++11', ], ) py = import('python').find_installation(pure: false) pybind11_dep = dependency('pybind11') py.extension_module( '_core', 'example.cpp', install: true, dependencies : [pybind11_dep], ) pipx run build --installer=uv (Also requires git set up)
  • 15. Meson-Python Features 15 C a n skip PEP 621 a nd use Meson con f ig for quick projects Sever a l options c a n be speci f ied in pyproject.toml or vi a con f ig-settings Developed rebuild support for edit a ble inst a lls Ninj a p a ck a ge a dded only if missing
  • 16. Scikit-build A family of tools • Scikit-build-core • Scikit-build (cl a ssic) • Ninj a Python Distribution (pip inst a ll cm a ke) • CM a ke Python Distribution (pip inst a ll ninj a with jobserver support) • ModernCM a keDom a in (Sphinx plugin) • Dyn a mic-Met a d a t a (WIP project for gener a l met a d a t a plugins) 16 Scikit-build-core funded vi a NSF gr a nt OAC 2209877
  • 17. Scikit-build-core CMake 17 example-project ├── example.cpp ├── pyproject.toml └── CMakeLists.txt [build-system] requires = ["scikit-build-core", "pybind11"] build-backend = "scikit_build_core.build" [project] name = "example" version = "0.0.1" cmake_minimum_required(VERSION 3.15…3.29) project(example LANGUAGES CXX) set(PYBIND11_NEWPYTHON ON) find_package(pybind11 CONFIG REQUIRED) pybind11_add_module(example example.cpp) install(TARGETS example LIBRARY DESTINATION .) pipx run build --installer=uv #include <pybind11/pybind11.h> namespace py = pybind11; float square(float x) { return x * x; } PYBIND11_MODULE(example, m) { m.def("square", &square); }
  • 18. Scikit-build-core Features • Autom a tic inclusion of cm a ke/ninj a a s needed • Limited API / St a ble ABI a nd pythonless t a gs supported vi a con f ig option • Support for writing out to extr a wheel folders (scripts, he a ders, d a t a , met a d a t a ) • Dedic a ted entrypoints for p a ck a ges to host CM a ke code • Experiment a l edit a ble mode support, with option a l experiment a l a uto rebuilds on import • Free-thre a ded Python support 18
  • 19. Scikit-build-core con f ig system A peek into the internals 19 D a t a cl a ss-b a sed con f ig system Gener a tes JSONSchem a Gener a tes section in README using cog Supports pyproject.toml, con f ig-settings, a nd environment v a ri a bles @dataclasses.dataclass class WheelSettings: packages: Optional[List[str]] = None """ A list of packages to auto-copy into the wheel. If this is not set, it will default to the first of ``src/<package>``, ``python/<package>``, or ``<package>`` if they exist. The prefix(s) will be stripped from the package name inside the wheel. """ py_api: str = "" """ The Python tags. The default (empty string) will use the default Python version. You can also set this to "cp37" to enable the CPython 3.7+ Stable ABI / Limited API (only on CPython and if the version is sufficient, otherwise this has no effect). Or you can set it to "py3" or "py2.py3" to ignore Python ABI compatibility. The ABI tag is inferred from this tag. """ expand_macos_universal_tags: bool = False """ Fill out extra tags that are not required. This adds "x86_64" and "arm64" to the list of platforms when "universal2" is used, which helps older Pip's (before 21.0.1) find the correct wheel. """
  • 20. Scikit-build-core advanced con f ig Over 40 options 20 [tool.scikit-build] minimum-version = "0.2" cmake.verbose = true logging.level = "INFO" build-dir = "build/{wheel_tag}" sdist.include = ["src/some_generated_file.txt"] sdist.reproducible = false wheel.expand-macos-universal-tags = true install.components = ["python"] [tool.scikit-build.cmake.define] SOME_DEFINE = {env="SOME_DEFINE", default="EMPTY"}
  • 21. Scikit-build-core Dynamic metadata 21 [tool.scikit-build.metadata.version] provider = "scikit_build_core.metadata.regex" input = "src/mypackage/__init__.py" [[tool.scikit-build.generate]] location = "install" path = "mypackage/_version.py" template = ''' version = "${version}" ''' Join us on scikit-build/dyn a mic-met a d a t a if you a re interested in gener a lizing for multiple b a ckends!
  • 22. Scikit-build-core Overrides 22 [[tool.scikit-build.overrides]] if.platform-system = "darwin" cmake.version = ">=3.18" Designed a fter cibuildwheel's overrides, inspired by mypy M a tch top to bottom C a n use if. a ny C a n inherit with " a ppend", "prepend", or "none" T a kes regex, speci f ier set, or bool [[tool.scikit-build.overrides]] if.state = "editable" build-dir = "build" [[tool.scikit-build.overrides]] if.any.env.SET_FOO = true if.any.env.FOO_SET = true inherit.cmake.define = "append" cmake.define.FOO = "1"
  • 23. Nanobind example Using scikit-build-core 23 # CMakeLists.txt cmake_minimum_required(VERSION 3.15...3.26) project(nanobind_example LANGUAGES CXX) find_package(Python 3.8 REQUIRED COMPONENTS Interpreter Development.Module OPTIONAL_COMPONENTS Development.SABIModule) find_package(nanobind CONFIG REQUIRED) nanobind_add_module( nanobind_example_ext STABLE_ABI NB_STATIC src/nanobind_example_ext.cpp ) install(TARGETS nanobind_example_ext LIBRARY DESTINATION nanobind_example) # pyproject.toml [build-system] requires = [ "scikit-build-core", "nanobind", ] build-backend = "scikit_build_core.build" [tool.scikit-build] wheel.py-api = "cp312" [tool.cibuildwheel.macos.environment] MACOSX_DEPLOYMENT_TARGET = "10.14" [project] name = "some-package" version = "1.0.0"
  • 24. Scikit-build-core Some projects 24 R a pids. a i Added to f ile gener a tor a nd deployed a cross a ll projects [tool.scikit-build] wheel.packages = ["zmq"] wheel.license-files = ["licenses/LICENSE*"] cmake.version = ">=3.15" # only build/install the pyzmq component cmake.targets = ["pyzmq"] install.components = ["pyzmq"] PyZMQ Added to f ile gener a tor a nd deployed a cross a ll projects [tool.scikit-build] minimum-version = "0.8" build-dir = "build/{wheel_tag}" cmake.version = "" # We are cmake, so don't request cmake ninja.make-fallback = false wheel.py-api = "py3" wheel.expand-macos-universal-tags = true wheel.install-dir = "cmake/data" [[tool.scikit-build.generate]] path = "cmake/_version.py" template = ''' version = "${version}" ''' cm a ke (python distributions) Fun met a problem: we a re cm a ke!
  • 25. Bonus: Scikit-build-core plugins Scikit-build-core is also useful for creating plugins! 25 [build-system] requires = ["hatchling", "scikit-build-core~=0.9.0"] build-backend = "hatchling.build" [project] name = "hatchling_example" version = "0.1.0" [tool.hatch.build.targets.wheel.hooks.scikit-build] experimental = true This will a dd a CM a ke extension to a h a tchling project!
  • 26. PyPI stats Just download every pyproject.toml on PyPI 26 https://github.com/henryiii/pyst a ts & https://hugovk.github.io/top-pypi-p a ck a ges m a turin: 1,744 scikit_build_core.build: 222 mesonpy: 123 maturin #81 pydantic-core 82,784,026 #108 rpds-py 61,448,636 #250 pendulum 26,584,509 #298 tokenizers 22,436,866 #330 orjson 19,225,277 #402 safetensors 15,136,797 #511 ruff 9,718,295 #663 cramjam 6,773,950 #664 watchfiles 6,741,022 #732 nh3 6,193,025 #790 polars 5,731,171 #810 uv 5,443,201 #875 dbt-extractor 4,830,288 #1124 jellyfish 2,835,548 #1319 deltalake 2,017,365 #1375 py-spy 1,827,491 #1766 tlparse 1,126,435 #2001 pywinpty 871,147 #2158 y-py 750,010 #2269 clarabel 686,786 #2338 kornia-rs 648,550 #2901 regress 403,177 #3165 rtoml 327,378 #3309 tsdownsample 302,822 #3348 grimp 294,794 #3386 chia-rs 287,786 #3525 openlineage-sql 269,605 #3694 h3ronpy 253,144 #3737 cmsis-pack-manager 247,677 #3989 blake3 218,936 #4168 pyxirr 200,494 #4449 sqlglotrs 175,146 #4481 akinator-py 173,092 #4565 hf-transfer 165,738 #5479 clvm-tools-rs 125,016 #5699 css-inline 114,355 #5947 mitmproxy-rs 104,179 #6269 py-sr25519-bindings 91,533 #6558 python-calamine 82,461 #6626 qh3 80,379 #6674 mitmproxy-wireguard 79,065 #6712 deptry 78,113 #6907 polars-lts-cpu 73,655 #7019 minify-html 70,966 #7039 solders 70,434 #7042 topgrade 70,373 #7240 typos 66,022 #7313 tzfpy 64,947 #7512 netifaces2 61,198 #7516 rjieba 61,141 #7566 lintrunner 60,240 #7647 py-bip39-bindings 58,624 #7698 py-ed25519-zebra-bindings 57,611 #7752 uuid-utils 56,790 scikit-build-core #229 pyzmq 30,543,660 #645 lightgbm 7,046,707 #709 cmake 6,369,474 #1771 phik 1,118,403 #2090 clang-format 797,795 #3005 awkward-cpp 369,071 #3544 llama-cpp-python 267,061 #4047 openexr 213,581 #4636 coreforecast 160,430 #4711 sparse-dot-topn 154,874 #5685 laszip 114,988 #6299 iminuit 90,796 #6594 spglib 81,422 meson-python #18 numpy 245,757,478 #26 pandas 195,143,812 #61 scipy 100,730,066 #107 scikit-learn 62,470,305 #112 matplotlib 59,187,475 #208 contourpy 33,391,835 #437 scikit-image 13,129,903 #475 pywavelets 11,126,522 #1781 pygobject 1,104,329 #1875 meson-python 988,201 #1924 scs 934,563 #4505 dftd3 170,689 #5410 dbus-python 129,002 (Doesn't include RAPIDS. a i p a ck a ges like cudf)
  • 27. PyPI Stats Looking at every pyproject.toml 27 tool.*: poetry: 52183 black: 24730 pytest: 18229 isort: 18111 setuptools: 15210 mypy: 13030 ruff: 12213 coverage: 10749 setuptools_scm: 7006 hatch: 6900 pylint: 4261 pyright: 3635 flit: 3039 flake8: 2216 pdm: 1995 tox: 1577 semantic_release: 1553 cibuildwheel: 1474 codespell: 1431 maturin: 1207 poetry-dynamic-versioning: 1150 C a n look a t the most popul a r tool sections: This helped bl a ck check the imp a ct of a con f ig bug! tool.black.line-length contents: 120: 5755 88: 3578 79: 2876 100: 2714 80: 834 ... C a n check to see wh a t our users a re using a nd setting! *: build-system: 136738 tool: 100937 project: 51268 options: 305 tools: 279 metadata: 202 coverage: 148 mypy: 127 requires: 119 flake8: 119 virtualenvs: 90 build-backend: 81 pytest: 76 dependencies: 67 bdist_wheel: 40 build: 36 pypi: 34 build_system: 32 dev-dependencies: 30 package: 30 bumpver: 28 manageprojects: 25 too: 25 ... Or typos: Nothing p a st here is v a lid! (Use v a lid a te-pyproject)!
  • 28. PyPI Stats Scikit-build-core usage 28 tool.scikit-build.*.*: metadata.version: 83 build-dir: 78 wheel.packages: 76 sdist.include: 69 minimum-version: 68 sdist.exclude: 60 cmake.define: 57 wheel.expand-macos-universal-tags: 54 wheel.py-api: 53 cmake.verbose: 47 cmake.minimum-version: 44 cmake.build-type: 43 cmake.args: 32 wheel.install-dir: 32 cmake.version: 30 logging.level: 29 wheel.license-files: 20 install.components: 20 cmake.targets: 17 experimental: 15 ninja.version: 14 ninja.make-fallback: 14 ninja.minimum-version: 13 cmake.source-dir: 12 sdist.cmake: 12 sdist.reproducible: 9 install.strip: 9 generate: 9 wheel.exclude: 6 editable.mode: 5 strict-config: 5 metadata.readme: 5 editable.verbose: 4 editable.rebuild: 4 wheel.platlib: 3 backport.find-python: 3 wheel.cmake: 2 wheel.build-tag: 2 metadata.scripts: 2 overrides: 2 metadata.optional-dependencies: 1 tool.scikit-build.wheel.py-api contents: {}: 146661 'cp312': 30 'py3': 10 'py2.py3': 7 'cp310': 1 'py37': 1 'cp37': 1 tool.scikit-build.minimum-version contents: {}: 146646 '0.4': 18 '0.8': 11 '0.5': 9 '0.5.1': 6 '0.8.1': 5 '0.9': 4 '0.3': 2 '0.5.0': 2 '0.6.1': 2 '0.7': 2 '0.2.1': 2 '0.2': 1 '0.6': 1 '0.8.2': 1 '0.4.4': 1 '0.3.0': 1
  • 29. Maturin Cargo (Rust) 29 A Rust-b a sed build system for C a rgo example-project ├── src/lib.rs ├── Cargo.toml └── pyproject.toml [package] name = "package" version = "0.1.0" edition = "2021" [lib] name = "example" crate-type = ["cdylib"] [dependencies.pyo3] version = "0.21.1" features = [ "extension-module", "abi3-py38", "experimental-declarative-modules", ] use pyo3::prelude::*; #[pymodule] mod example { use super::*; #[pyfunction] fn add(x: i64, y: i64) -> i64 { x + y } } [build-system] requires = ["maturin~=1.5"] build-backend = "maturin" [project] name = "example" dynamic = ["version"] pipx run build —installer=uv maturin build --release
  • 30. Maturin Features 30 Written in Rust (c a n build a p a ck a ge without running Python!) Limited API doesn't even require Python a t a ll! C a rgo's build-from-source p a ck a ging is perfect for cross-compil a tion a nd m a king wheels! CLI c a n cre a te a new project. Why Rust? Rust/C a rgo m a kes using libr a ries a s e a sy a s Python! Cross-compiles a re gre a t (c a n a void most m a nylinux issues) Tooling is f a nt a stic (linter, form a tter, IDEs, etc) PyO3 bindings a re gre a t Single bin a ry a ppro a ch works well for wheels (And m a ybe some memory s a fety stu ff )
  • 31. Maturin-Action GitHub Action for Building 31 Gre a t a t cross-compiling with extensive support (f a st) Builds non-Python Rust p a rts once for a ll versions Every wheel supported except musl s390x wheels (some pl a tforms like 32-bit musl c a n't be done with cibuildwheel since rust doesn't build c a rgo for them) ... Compiling lexical-sort v0.3.1 Compiling taplo v0.13.0 Compiling pep508_rs v0.6.0 Compiling pyproject-fmt-rust v1.1.1 (/home/runner/... Finished release [optimized] target(s) in 13.53s 📦 Including files matching "rust-toolchain.toml" 📦 Built wheel for abi3 Python ≥ 3.8 to dist/... ⚠ Warning: PyPy does not yet support abi3 so the build... Compiling pyo3-build-config v0.21.2 Compiling pyo3-macros-backend v0.21.2 Compiling pyo3-ffi v0.21.2 Compiling pyo3 v0.21.2 Compiling pyo3-macros v0.21.2 Compiling pyproject-fmt-rust v1.1.1 (/home/runner/... Finished release [optimized] target(s) in 4.25s 📦 Including files matching "rust-toolchain.toml" 📦 Built wheel for PyPy 3.8 to dist/pyproject_fmt_rust-... Compiling pyo3-build-config v0.21.2 Compiling pyo3-macros-backend v0.21.2 Compiling pyo3-ffi v0.21.2 Compiling pyo3 v0.21.2 Compiling pyo3-macros v0.21.2 Compiling pyproject-fmt-rust v1.1.1 (/home/runner/... Finished release [optimized] target(s) in 4.24s ...
  • 32. Resouces Scienti fi c-Python Development Guide 32 Useful for a ll projects (not just scienti f ic ones!) Up-to-d a te guides Integr a ted templ a te rendered in some pl a ces vi a cog Versions upd a ted by CI Integr a ted WASM repo-review
  • 33. Resources Scienti fi c-python/cookie 33 11 b a ckends Cookiecutter & copier supported In sync with the dev guide Gener a tion tested by nox 🎤 The name of your project myprog 🎤 The name of your (GitHub?) org henryiii 🎤 The url to your GitHub or GitLab repository https://github.com/henryiii/myprog 🎤 Your name Henry Schreiner 🎤 Your email a@b.com 🎤 A short description of your project A great package. 🎤 Select a license BSD 🎤 Choose a build backend Scikit-build-core - Compiled C++ (recommended) 🎤 Use version control for versioning No Copying from template version 2024.4.23 create . create CMakeLists.txt create .pre-commit-config.yaml create pyproject.toml create tests create tests/test_package.py create tests/test_compiled.py create .git_archival.txt create LICENSE create docs create docs/conf.py create docs/index.md create README.md create .gitignore create .github create .github/workflows create .github/workflows/cd.yml create .github/workflows/ci.yml create .github/CONTRIBUTING.md create .github/dependabot.yml create .gitattributes create .copier-answers.yml create noxfile.py create .readthedocs.yaml create src create src/main.cpp create src/myprog create src/myprog/__init__.py create src/myprog/_core.pyi create src/myprog/py.typed
  • 34. Resources Repo-review 34 Python 3.10+ fr a mework sp-repo-review: checks from guide Supports WebAssembly a nd CLI ( a nd pre-commit, etc) A checker for your checker con f ig! Checks linked to integr a ted t a gs in guide V a lid a te-pyproject a lso integr a ted
  • 35. Summary So much has happened in the last two years! • Three f a nt a stic build-b a ckends a re now a v a il a ble for bin a ry projects • Gre a t resources a v a il a ble for developing p a ck a ges • You c a n ship C/C++/Rust code for a ll m a jor pl a tforms even for a sm a ll project 35
  • 36. Credits and thanks 36 scikit-build te a m Je a n-Christophe Fillion-Robin (Kitw a re) M a tt McCormick (Kitw a re) Cristi a n Le meson-python te a m R a lf Gommers D a niele Nicolodi Henry Schreiner Thom a s Li Filipe L a íns (Emeritus) cibuildwheel te a m Joe Rickerby Y a nnick J a doul M a tthieu D a rbois Grzegorz Bokot a And th a nks to the m a turin developers for th a t f a nt a stic tool! Thanks to my other team members!