Let me tell you about...
March 24, 2022
Daniel Stenberg
@bagder
https://daniel.haxx.se
start
growth
now
future
news
@bagder
@bagder
Just ask!
@bagder
An open source project that
makes a command line tool
and a library for transferring
data using Internet protocols
@bagder
curl.se
Once upon the time...
@bagder
@bagder
@bagder
… while I was
writing this IRC
bot...
Let’s put it online... as open source
@bagder
… became curl 1998
HTTP
Gopher
FTP
@bagder
… and time passed...
@bagder
… and time passed...
@bagder
… and time passed...
@bagder
… and time passed...
@bagder
Features!
DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP,
IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTMPS, RTSP, SCP,
SFTP, SMB, SMBS, SMTP, SMTPS, TELNET and TFTP
TLS certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP
form based upload, HTTP/HTTPS/SOCKS proxy, cookies,
authentication (Basic, Digest, NTLM, Negotiate, Kerberos),
HTTP/2, HTTP/3, alt-svc:, happy eyeballs, file transfer
resume, proxy tunneling, DNS-over-HTTPS, HTTP
compression and much more
@bagder
@bagder
Web sites on the Internet
1996: 250,000
2022: 2,000,000,000
(multiplied 8,000 times)
@bagder
@bagder
curl runs in all your devices
@bagder
transfer protocols: 26
TCP
filesystem
UDP
TLS
SSH
QUIC
HTTP
HTTPS
TFTP
FILE
FTP
IMAP
SMTP
POP3
GOPHER
TELNET
DICT
RTSP
RTMP
SMB
LDAP
SFTP
SCP
FTPS
IMAPS
SMTPS
POP3S
RTMPS
SMBS
LDAPS
@bagder
MQTT
GOPHERS
bindings: 60
application
Falcon
D
C++
Requests
ScriptBasi
c
Ferite
Delphi
Chcurl
curlpp Gambas
Eiffel
BBHTTP
(Cocoa)
curlcpp glib/GTK+
Euphoria
Curlhandl
e
(Cocoa)
go-curl
Object-
Pascal
Lua-cURL
Java
Guile O’Caml
Mono
Julia
Harbour Pascal
.NET
Common
Lisp
Haskell
WWW::Curl
(perl)
node.js
luacurl
perl6-net-
curl
PHP/CURL Rexx
PostgreSQ
L
Ring
pycURL RPG
Tclcurl Q
Visual
Foxpro
Visual
Basic
vXWidgets S-Lang
Xojo
XBLite Smalltalk
SP-
Forth
Scilab
Scheme
curl-
rust
SPL Ada95
Curb
(Ruby)
Clojure R
Kapito
(Erlang)
PureBasic
Net::Curl
(perl)
Nim
@bagder
third party deps: 35
I/O layer
URL parser libidn2
winidn
HTTP
TLS
OpenSSL
Mesalink
gskit
mbedTLS
wolfSSL
Schannel
Secure
Transport
GnuTLS
NSS
boringssl
libressl
AmiSSL
SFTP SCP LDAP
WinLDAP
OpenLDAP
RTMP
librtmp
Name resolver c-ares
compression
libz
brotl
i
cookies
libpsl
IMAP SMTP POP3
HTTP/2
nghttp2
authentication
winsspi Heimdal
MIT
kerberos
HTTP/3
quiche
ngtcp2
HTTP/1
SSH
wolfSSH
libssh2
libssh
@bagder
BearSSL
nghttp3
zstd
FTP
Hyper
FTPS IMAPS POP3S SMBS SMTPS
GOPHERS HTTPS LDAPS RTMPS
libgsasl
rustls
operating systems: 86
@bagder
Syllable OS TPF
Tizen
Symbian Tru64
SunOS tvOS ucLinux
Genode Hurd iOS
Integrity
Illumos
HP-UX
HardenedBS
D
Haiku
z/OS
Nintendo
Switch
NonStop OS
NetWare
MorphOS MPE/iX MS-DOS NCR MP-RAS NetBSD
RISC OS
Redox
ReactOS Sailfish OS SCO Unix Serenity SINIX-Z
Qubes OS
UnixWare WebOS
vxWorks
VMS Windows
UNICOS Windows CE
Wii System
Software
AmigaOS
Blackberry
10
BeOS
Android
Blackberry
Tablet OS
AIX Cell OS
Aros
IRIX LineageOS Mbed Micrium
macOS
Mac OS 9
Linux Lua RTOS
eCOS FreeRTOS
FreeBSD
FreeDOS Fuchsia
DragonFly
BSD
Cygwin
Cisco IOS
OpenBSD OS/2 OS/400
Ultrix
ipadOS
NuttX
Solaris
Xbox
System
ChromeOS
MINIX
Garmin OS
QNX
PlayStation
Portable
Plan 9
OS21
OpenStep Orbis OS
z/TPF z/VM z/VSE
Operating systems known to have run curl
Atari
FreeMiNT
CPU architectures: 22
@bagder
x86
MIPS
ARM
PowerPC
SPARC POWER
m68k
s390 HP-PA
SH4
Nios
RISC-V
OpenRISC
ARC
Cell
Itanium VAX
MicroBlaze
Alpha Xtensa
x86-64
AVR32
CPU architectures known to have run curl
Planets: 2
@bagder
Earth Mars
Planets known to have run curl
>10,000,000,000
installations
@bagder
Hi Daniel,
I’m the marketing director for Dice.com and I wanted
to reach out to you to thank you for spotting our
billboard error on the 101. We are deeply
embarrassed by this mistake to say the least. In a
classic coding scenario, our QA failed us.
Unfortunately for us, we bought this spot long-term
and we are trying to figure out how quickly we can
replace the content.
@bagder
Subject: Multimedya isc-v:85
I have toyota corola with multimedya
system that you have its copyright.
I need a advice to know how to use
the gps.
Master of many things
@bagder
Cisco Small Business Routers, March 2019
@bagder
Why?
@bagder
Why use libcurl?
Stable well-documented API – examples from 2000 still work
Same API everywhere – 86 OSs, 22 CPU archs...
Open source – free and immortal
Rock solid – battle proven, furiously maintained, secure
Customizable – use what you want, disable what you don’t
Internet hardened – speaks protocols the way they should
Multiple protocols – 50% of users use more than two!
Keeps up – TLS 1.3, HTTP/2, HTTP/3, ...
@bagder
Why Open Source?
There was never any alternative to me
Wanted to contribute back
Would never even come close unless
No, I would not be rich otherwise
@bagder
How?
@bagder
0821 0822 0850 0854 0959 0974 1035 1081 1123 1225 1350 1425 1427 1436
1460 1510 1635 1639 1651 1653 1725 1730 1734 1738 1777 1808 1867 1869 1870
1884 1928 1939 1945 1950 1951 1952 1959 1964 2045 2046 2047 2048 2049
2060 2061 2068 2095 2104 2109 2133 2145 2183 2184 2192 2195 2222 2228
2229 2231 2246 2255 2326 2373 2384 2388 2389 2396 2428 2449 2459 2478
2487 2518 2553 2554 2577 2595 2616 2617 2640 2718 2732 2817 2818 2821 2831
2854 2936 2964 2965 3207 3280 3490 3493 3501 3513 3617 3659 3961 3986
4120 4121 4178 4217 4248 4346 4366 4422 4511 4516 4559 4616 4954 4959
5034 5092 5321 5322 5335 5336 5849 5890 6531 6532 6749 7230 7231 7232
7233 7234 7235 7238 7540 7541 7628 7838 8314 8446 8484 8999 9000 9001
9002
143 Relevant RFCs (280,000 lines)
libcurl
@bagder
1,451,035 words
@bagder
Contributors
2,600 in total
50-70 per release
Increasing
Small core team
Volunteers
@bagder
Independent
Everything is
public
@bagder
mailing lists
@bagder
a selected few
have push rights
@bagder
https: github.com/curl
Money flows
@bagder
@bagder
Sponsors
Customers
Official curl sponsors March 2022
@bagder
Secure enough for the billions?
Reviews
(at 111 CVEs and counting)
Code audit
Code style
Fuzzing
Docs
Static code
analyzers
Valgrind and
sanitizers
Many tests
@bagder
CI like crazy
curl CI 2022
>100 build + test “rounds” per commit
Tests code style, indenting etc
Thousands of tests per build
Builds and tests on tens of platforms
25 hours of CI per commit
@bagder
Code and test policies
Fix all warnings (oh well...)
No defects left
Use the most picky compiler options
As many tests as possible
Fix security issues asap
@bagder
curl bug bounty
@bagder
Let's make it personal
This is the lead
developer of this project
@bagder
… and time passed...
@bagder
Security issues
Release management
Web site admin
Mailing list admin
Patch reviewing
User support
Blogging about it
What’s maintaining?
Debugging
Patch merging
Feature development
Write documentation
Event planning
Getting stickers
Doing talks
@bagder
Now
@bagder
I still lead the development
Two hours spare time per day
Every day, every week, every year, since 1998
Part time paid since 2014
Full time since early 2019
Yes, I totally mix and blur spare time and work!
@bagder
@bagder
“The created economic value
cannot be overstated.”
@bagder
@bagder
Where does the road lead from here?
@bagder
Future
@bagder
@bagder
Everything will be networked
If it isn’t powered now, it will be soon
If it is powered, it will be networked
If it is networked, it needs Internet access
If it needs Internet access, curl can help
@bagder
@bagder
sewing machine, microwave, advertisement sign, kitchen scale, electric
razor, kitchen oven, fan, dog collar, bicycle computer, power plug, piano,
child monitor, radiator, vacuum cleaner, escooter, lawn mower, air
conditioner, printer, projector, kettle, bluray player, settop box,
computer monitor, alarm clock, clothes iron, powertool, exercise
equipment, mouse, wifi router, car, headphones, electric toothbrush,
bluetooth speaker, air purifier, doorbell, watch, game console, keychain
fob, flashlight, ebook-tablet, washing machine, stereo equipment,
ebikes, tv, toaster, thermostat, door lock, drilling machine, fridge and
freezer, lamp switch, clothing, bathroom scale, camera, smart phone,
garage door opener, medical device, bus-stop sign, stove, blender, light
bulb, tablet
Everything will be networked
@bagder
@bagder
All network technologies
Bluetooth 5, Wifi 6, 5G/6G, TbE etc drive
more networking
More networking means more curl
@bagder
@bagder
All platforms
curl runs on 80 operating systems,
20 CPU architectures
and 2 planets
… and your next platform
@bagder
@bagder
HTTP APIs everywhere
everyone offers HTTP APIs
curl is the universal API consumer and
debugger
@bagder
@bagder
A defacto standard API
stable and solid API
since twenty years
secure and fast transfers
all the protocols you need
@bagder
@bagder
Unparalleled
Commercial support
Rock solid
Stable API
Open Source
@bagder
@bagder
@bagder
more everywhere
@bagder
stuff
time
Future
No, it truly never gets done
Protocols keep evolving
Open source code survives
No slow-down in sight
You can help!
@bagder
News
@bagder
Done over the last 4 years GOPHERS
HTTP/3
MQTT
zstd
--alt-svc
--hsts
--aws-sigv4
--remove-on-error
--retry-all-errors
--tls13-ciphers
--proxy-tls13-ciphers
--disallow-username-in-url
w: %header{}
%{errormsg}
%{exitcode}
%{header_json}
%{json}
%{method}
%{num_headers}
%{onerror}
%{urlnum}
%{url}
--json
--no-clobber
--parallel
--doh-url
--etag-compare
--fail-with-body
--haproxy-protocol
@bagder
Done over the last 4 years
--alt-svc
--hsts
--aws-sigv4
--remove-on-error
--retry-all-errors
--tls13-ciphers
--proxy-tls13-ciphers
--disallow-username-in-url
w: %header{}
%{errormsg}
%{exitcode}
%{header_json}
%{json}
%{method}
%{num_headers}
%{onerror}
%{urlnum}
%{url}
--json
--no-clobber
--parallel
--doh-url
--etag-compare
--fail-with-body
--haproxy-protocol
@bagder
JSON!
https://everything.curl.dev/
@bagder
You can help!
@bagder
68
Daniel Stenberg
@bagder
https://daniel.haxx.se/
Thank you!
Questions?
@bagder
License
This presentation and its contents are
licensed under the Creative Commons
Attribution 4.0 license:
http://creativecommons.org/licenses/by/4.0/
@bagder

Let me tell you about curl

  • 1.
    Let me tellyou about... March 24, 2022
  • 2.
  • 3.
  • 4.
  • 5.
    An open sourceproject that makes a command line tool and a library for transferring data using Internet protocols @bagder curl.se
  • 6.
    Once upon thetime... @bagder
  • 7.
  • 8.
    @bagder … while Iwas writing this IRC bot...
  • 9.
    Let’s put itonline... as open source @bagder
  • 10.
    … became curl1998 HTTP Gopher FTP @bagder
  • 11.
    … and timepassed... @bagder
  • 12.
    … and timepassed... @bagder
  • 13.
    … and timepassed... @bagder
  • 14.
    … and timepassed... @bagder
  • 15.
    Features! DICT, FILE, FTP,FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET and TFTP TLS certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, HTTP/HTTPS/SOCKS proxy, cookies, authentication (Basic, Digest, NTLM, Negotiate, Kerberos), HTTP/2, HTTP/3, alt-svc:, happy eyeballs, file transfer resume, proxy tunneling, DNS-over-HTTPS, HTTP compression and much more @bagder
  • 16.
  • 17.
    Web sites onthe Internet 1996: 250,000 2022: 2,000,000,000 (multiplied 8,000 times) @bagder
  • 18.
    @bagder curl runs inall your devices
  • 19.
  • 20.
  • 21.
    bindings: 60 application Falcon D C++ Requests ScriptBasi c Ferite Delphi Chcurl curlpp Gambas Eiffel BBHTTP (Cocoa) curlcppglib/GTK+ Euphoria Curlhandl e (Cocoa) go-curl Object- Pascal Lua-cURL Java Guile O’Caml Mono Julia Harbour Pascal .NET Common Lisp Haskell WWW::Curl (perl) node.js luacurl perl6-net- curl PHP/CURL Rexx PostgreSQ L Ring pycURL RPG Tclcurl Q Visual Foxpro Visual Basic vXWidgets S-Lang Xojo XBLite Smalltalk SP- Forth Scilab Scheme curl- rust SPL Ada95 Curb (Ruby) Clojure R Kapito (Erlang) PureBasic Net::Curl (perl) Nim @bagder
  • 22.
    third party deps:35 I/O layer URL parser libidn2 winidn HTTP TLS OpenSSL Mesalink gskit mbedTLS wolfSSL Schannel Secure Transport GnuTLS NSS boringssl libressl AmiSSL SFTP SCP LDAP WinLDAP OpenLDAP RTMP librtmp Name resolver c-ares compression libz brotl i cookies libpsl IMAP SMTP POP3 HTTP/2 nghttp2 authentication winsspi Heimdal MIT kerberos HTTP/3 quiche ngtcp2 HTTP/1 SSH wolfSSH libssh2 libssh @bagder BearSSL nghttp3 zstd FTP Hyper FTPS IMAPS POP3S SMBS SMTPS GOPHERS HTTPS LDAPS RTMPS libgsasl rustls
  • 23.
    operating systems: 86 @bagder SyllableOS TPF Tizen Symbian Tru64 SunOS tvOS ucLinux Genode Hurd iOS Integrity Illumos HP-UX HardenedBS D Haiku z/OS Nintendo Switch NonStop OS NetWare MorphOS MPE/iX MS-DOS NCR MP-RAS NetBSD RISC OS Redox ReactOS Sailfish OS SCO Unix Serenity SINIX-Z Qubes OS UnixWare WebOS vxWorks VMS Windows UNICOS Windows CE Wii System Software AmigaOS Blackberry 10 BeOS Android Blackberry Tablet OS AIX Cell OS Aros IRIX LineageOS Mbed Micrium macOS Mac OS 9 Linux Lua RTOS eCOS FreeRTOS FreeBSD FreeDOS Fuchsia DragonFly BSD Cygwin Cisco IOS OpenBSD OS/2 OS/400 Ultrix ipadOS NuttX Solaris Xbox System ChromeOS MINIX Garmin OS QNX PlayStation Portable Plan 9 OS21 OpenStep Orbis OS z/TPF z/VM z/VSE Operating systems known to have run curl Atari FreeMiNT
  • 24.
    CPU architectures: 22 @bagder x86 MIPS ARM PowerPC SPARCPOWER m68k s390 HP-PA SH4 Nios RISC-V OpenRISC ARC Cell Itanium VAX MicroBlaze Alpha Xtensa x86-64 AVR32 CPU architectures known to have run curl
  • 25.
  • 26.
  • 27.
    Hi Daniel, I’m themarketing director for Dice.com and I wanted to reach out to you to thank you for spotting our billboard error on the 101. We are deeply embarrassed by this mistake to say the least. In a classic coding scenario, our QA failed us. Unfortunately for us, we bought this spot long-term and we are trying to figure out how quickly we can replace the content. @bagder
  • 28.
    Subject: Multimedya isc-v:85 Ihave toyota corola with multimedya system that you have its copyright. I need a advice to know how to use the gps. Master of many things @bagder
  • 29.
    Cisco Small BusinessRouters, March 2019 @bagder
  • 30.
  • 31.
    Why use libcurl? Stablewell-documented API – examples from 2000 still work Same API everywhere – 86 OSs, 22 CPU archs... Open source – free and immortal Rock solid – battle proven, furiously maintained, secure Customizable – use what you want, disable what you don’t Internet hardened – speaks protocols the way they should Multiple protocols – 50% of users use more than two! Keeps up – TLS 1.3, HTTP/2, HTTP/3, ... @bagder
  • 32.
    Why Open Source? Therewas never any alternative to me Wanted to contribute back Would never even come close unless No, I would not be rich otherwise @bagder
  • 33.
  • 34.
    0821 0822 08500854 0959 0974 1035 1081 1123 1225 1350 1425 1427 1436 1460 1510 1635 1639 1651 1653 1725 1730 1734 1738 1777 1808 1867 1869 1870 1884 1928 1939 1945 1950 1951 1952 1959 1964 2045 2046 2047 2048 2049 2060 2061 2068 2095 2104 2109 2133 2145 2183 2184 2192 2195 2222 2228 2229 2231 2246 2255 2326 2373 2384 2388 2389 2396 2428 2449 2459 2478 2487 2518 2553 2554 2577 2595 2616 2617 2640 2718 2732 2817 2818 2821 2831 2854 2936 2964 2965 3207 3280 3490 3493 3501 3513 3617 3659 3961 3986 4120 4121 4178 4217 4248 4346 4366 4422 4511 4516 4559 4616 4954 4959 5034 5092 5321 5322 5335 5336 5849 5890 6531 6532 6749 7230 7231 7232 7233 7234 7235 7238 7540 7541 7628 7838 8314 8446 8484 8999 9000 9001 9002 143 Relevant RFCs (280,000 lines) libcurl @bagder
  • 35.
  • 36.
    Contributors 2,600 in total 50-70per release Increasing Small core team Volunteers @bagder
  • 37.
  • 38.
  • 39.
  • 40.
    a selected few havepush rights @bagder https: github.com/curl
  • 41.
  • 42.
    Official curl sponsorsMarch 2022 @bagder
  • 43.
    Secure enough forthe billions? Reviews (at 111 CVEs and counting) Code audit Code style Fuzzing Docs Static code analyzers Valgrind and sanitizers Many tests @bagder CI like crazy
  • 44.
    curl CI 2022 >100build + test “rounds” per commit Tests code style, indenting etc Thousands of tests per build Builds and tests on tens of platforms 25 hours of CI per commit @bagder
  • 45.
    Code and testpolicies Fix all warnings (oh well...) No defects left Use the most picky compiler options As many tests as possible Fix security issues asap @bagder
  • 46.
  • 47.
    Let's make itpersonal This is the lead developer of this project @bagder
  • 48.
    … and timepassed... @bagder
  • 49.
    Security issues Release management Website admin Mailing list admin Patch reviewing User support Blogging about it What’s maintaining? Debugging Patch merging Feature development Write documentation Event planning Getting stickers Doing talks @bagder
  • 50.
  • 51.
    I still leadthe development Two hours spare time per day Every day, every week, every year, since 1998 Part time paid since 2014 Full time since early 2019 Yes, I totally mix and blur spare time and work! @bagder @bagder
  • 52.
    “The created economicvalue cannot be overstated.” @bagder @bagder
  • 53.
    Where does theroad lead from here? @bagder
  • 54.
  • 55.
    Everything will benetworked If it isn’t powered now, it will be soon If it is powered, it will be networked If it is networked, it needs Internet access If it needs Internet access, curl can help @bagder @bagder
  • 56.
    sewing machine, microwave,advertisement sign, kitchen scale, electric razor, kitchen oven, fan, dog collar, bicycle computer, power plug, piano, child monitor, radiator, vacuum cleaner, escooter, lawn mower, air conditioner, printer, projector, kettle, bluray player, settop box, computer monitor, alarm clock, clothes iron, powertool, exercise equipment, mouse, wifi router, car, headphones, electric toothbrush, bluetooth speaker, air purifier, doorbell, watch, game console, keychain fob, flashlight, ebook-tablet, washing machine, stereo equipment, ebikes, tv, toaster, thermostat, door lock, drilling machine, fridge and freezer, lamp switch, clothing, bathroom scale, camera, smart phone, garage door opener, medical device, bus-stop sign, stove, blender, light bulb, tablet Everything will be networked @bagder @bagder
  • 57.
    All network technologies Bluetooth5, Wifi 6, 5G/6G, TbE etc drive more networking More networking means more curl @bagder @bagder
  • 58.
    All platforms curl runson 80 operating systems, 20 CPU architectures and 2 planets … and your next platform @bagder @bagder
  • 59.
    HTTP APIs everywhere everyoneoffers HTTP APIs curl is the universal API consumer and debugger @bagder @bagder
  • 60.
    A defacto standardAPI stable and solid API since twenty years secure and fast transfers all the protocols you need @bagder @bagder
  • 61.
  • 62.
  • 63.
    Future No, it trulynever gets done Protocols keep evolving Open source code survives No slow-down in sight You can help! @bagder
  • 64.
  • 65.
    Done over thelast 4 years GOPHERS HTTP/3 MQTT zstd --alt-svc --hsts --aws-sigv4 --remove-on-error --retry-all-errors --tls13-ciphers --proxy-tls13-ciphers --disallow-username-in-url w: %header{} %{errormsg} %{exitcode} %{header_json} %{json} %{method} %{num_headers} %{onerror} %{urlnum} %{url} --json --no-clobber --parallel --doh-url --etag-compare --fail-with-body --haproxy-protocol @bagder
  • 66.
    Done over thelast 4 years --alt-svc --hsts --aws-sigv4 --remove-on-error --retry-all-errors --tls13-ciphers --proxy-tls13-ciphers --disallow-username-in-url w: %header{} %{errormsg} %{exitcode} %{header_json} %{json} %{method} %{num_headers} %{onerror} %{urlnum} %{url} --json --no-clobber --parallel --doh-url --etag-compare --fail-with-body --haproxy-protocol @bagder JSON!
  • 67.
  • 68.
  • 69.
  • 70.
    License This presentation andits contents are licensed under the Creative Commons Attribution 4.0 license: http://creativecommons.org/licenses/by/4.0/ @bagder