Curl with rust
1 like558 views
Daniel Stenberg gave a presentation on using Rust with curl. He discussed how curl has traditionally used C but now supports alternative backends implemented in other languages like Rust. He described challenges in integrating the Hyper, rustls, and quiche Rust crates but curl now supports HTTP/1-2 with Hyper and TLS with rustls in an experimental way. Future work includes improving test coverage when using Rust backends and potentially enabling them by default.
More Related Content
What's hot (20)
Similar to Curl with rust (20)
More from Daniel Stenberg (20)
Recently uploaded (20)
Curl with rust
- 1. @bagder curl with rust Daniel Stenberg – January 27, 2022
- 4. @bagder Agenda curl is C Stable API and ABI Backends Making an HTTP backend … and other backends Challenges Current status Future @bagder
- 7. @bagder When curl started there was no choice C89 Only now alternatives appear for libraries C keeps curl extremely portable C code will remain a build option @bagder
- 8. @bagder Stable API and ABI @bagder
- 9. @bagder Reliable and predictable libcurl is forwards API compatible libcurl does not break the ABI – ever libcurl exists everywhere Compiled code from 2001 still works The armored front we can’t tamper with @bagder
- 11. Vulnerabilities 50% of past curl vulnerabilities are “C mistakes” Memory-safe languages could help Also doing other things to reduce risk The future is more Internet-connected devices @bagder
- 12. @bagder Backends and third parties @bagder
- 14. @bagder Backends Backends are selectable and deselectable Often platform dependent Often use 3rd party libraries … which can differ in features, licensing and maturity Can be done in other programming languages The internal APIs are never exposed externally @bagder
- 15. third party world map February 2022 I/O layer URL parser libidn2 winidn HTTP TLS OpenSSL gskit mbedTLS wolfSSL Schannel Secure Transport GnuTLS NSS boringssl libressl AmiSSL SFTP SCP LDAP WinLDAP OpenLDAP RTMP librtmp Name resolver c-ares compression libz brotli cookies libpsl IMAP SMTP POP3 HTTP/2 nghttp2 authentication winsspi Heimdal MIT kerberos HTTP/3 quiche ngtcp2 HTTP/1 SSH wolfSSH libssh2 libssh @bagder BearSSL nghttp3 zstd FTP Hyper FTPS IMAPS POP3S SMBS SMTPS GOPHERS HTTPS LDAPS RTMPS libgsasl rustls
- 16. @bagder Many different backends IDN Name resolving TLS SSH HTTP/3 HTTP content encoding HTTP @bagder
- 17. @bagder The making of an HTTP backend libcurl API Generic transfer engine HTTP Creating and reusing connections
- 18. @bagder HTTP involves a lot of things Authentication Creating a set of headers Proxy specifics Parsing content headers Parsing transfer headers HTTP/1 and HTTP/2 transmission
- 19. @bagder Hyper is transmission-focused Authentication Creating a set of headers Proxy specifics Parsing content headers Parsing transfer headers HTTP/1 and HTTP/2 transmission
- 20. @bagder Splitting up HTTP Authentication Creating a set of headers Proxy specifics Parsing content headers Parsing transfer headers HTTP/1 and HTTP/2 transmission Parsing transfer headers HTTP/1 and HTTP/2 transmission Hyper Built-in + nghttp2
- 21. @bagder build curl with hyper $ ./configure --with-hyper=... https://github.com/curl/curl/blob/master/docs/HYPER.md
- 22. @bagder HTTP backends libcurl API Generic transfer engine High-level HTTP Creating and reusing connections built-in HTTP/1 nghttp2 HTTP/2 Hyper HTTP/1 + 2
- 23. @bagder libcurl backends application HTTP API libcurl Public API Hyper built-in nghttp2
- 25. @bagder rustls provides a C API in rustls-ffi curl builds with different TLS libraries build curl to use rustls-ffi … profit! another TLS backend
- 27. @bagder quiche provides a C API curl builds with different QUIC libraries build curl to use quiche … profit! another HTTP/3 and QUIC backend
- 28. @bagder libcurl backends libidn2 winidn Hyper built-in threaded c-ares sync quiche ngtcp2 + nghttp3 libssh2 wolfSSH libssh BearSSL Gskit GnuTLS mbedSSL NSS OpenSSL Schannel wolfSSL Secure Transport rustls application HTTP API TLS API IDN API SSH API HTTP/3 API Resolver API Public API libcurl Content encoding API brotli zstd zlib = just one = one or more nghttp2
- 29. @bagder libcurl backends libidn2 winidn Hyper built-in threaded c-ares sync quiche ngtcp2 + nghttp3 libssh2 wolfSSH libssh BearSSL Gskit GnuTLS mbedSSL NSS OpenSSL Schannel wolfSSL Secure Transport rustls application HTTP API TLS API IDN API SSH API HTTP/3 API Resolver API Public API libcurl Content encoding API brotli zstd zlib = just one = one or more nghttp2
- 31. @bagder Using hyper First user of the Hyper C API Headers-only, no C docs yet Splitting HTTP for built-in + Hyper took thinking Keeping behavior with different API paradigms Memory leaks are tricky to hunt down Rust use is mostly seamless (and not the first use) @bagder
- 32. @bagder Using rustls Early user of rustls-ffi Headers-only, no C docs yet Mostly done by others so easier for me! The TLS backend situation was already mature Not yet feature-complete @bagder
- 33. @bagder Using quiche Early user of quiche Headers-only, no C docs yet Messy TLS situation (for QUIC) Not yet feature-complete @bagder
- 34. @bagder Using rust Updating rust and cargo “all the time” Suitable for replacing one-component-at-a-time I am a total rust rookie @bagder
- 36. @bagder Status Using curl and Hyper from main dev branches “experimental” and opt-in HTTP/1 and HTTP/2 HTTP and HTTPS (TLS backend agnostic) HTTP(S) proxies Identical HTTP requests over the wire 98% test case success rate (~800 test cases) @bagder
- 37. @bagder Status now hyper “experimental” and opt-in HTTP/1 and HTTP/2 HTTP and HTTPS (TLS backend agnostic) HTTP(S) proxies Identical HTTP requests over the wire 98% test case success rate (~800 test cases) @bagder rustls “experimental” and opt-in 12 test cases disabled no IP-addresses in cert? quiche “experimental” and opt-in no HTTP/3 tests yet QUIC and HTTP/3 are not “there” yet anyway rust in curl works perfectly well depends entirely on the C APIs … which are under-documented still no stable OOM == panic (will abort) You can help!
- 39. @bagder Coming up Make all test cases succeed Verify libcurl API corner cases Encourage users and developers to use rust backends Provide binary builds Enable by default? More rust components? More backend flavors? @bagder more curl!