20202020
DanielStenberg–curlup-May5th
,2020
in curl
Daniel Stenberg
@bagderhttps://daniel.haxx.se
How HTTP/3 differs from 1 and 2How HTTP/3 differs from 1 and 2
Build curl to enable HTTP/3Build curl to enable HTTP/3
The TLS situationThe TLS situation
HTTP/3 with the curl toolHTTP/3 with the curl tool
HTTP/3 with libcurlHTTP/3 with libcurl
Remaining issuesRemaining issues
WhenWhen
@bagder@bagder
This is not my HTTP/3 explanation talk
HTTP/3 for everyone (FOSDEM 2020)
https://daniel.haxx.se/videos/
@bagder@bagder
@bagder@bagder
Improvements in QUICImprovements in QUIC
TCP head of line blockingTCP head of line blocking
Faster handshakesFaster handshakes
Earlier dataEarlier data
More encryption, alwaysMore encryption, always
Future developmentFuture development
@bagder@bagder
QUIC on top of UDP
TCP and UDP remain “the ones”TCP and UDP remain “the ones”
Use UDP instead of IPUse UDP instead of IP
Reliable transport protocol - inReliable transport protocol - in
user-spaceuser-space
A little like TCP + TLSA little like TCP + TLS
@bagder@bagder
Streams!
QUIC provides streamsQUIC provides streams
Many logical flows within a single connectionMany logical flows within a single connection
Similar to HTTP/2 but in the transport layerSimilar to HTTP/2 but in the transport layer
IndependentIndependent streamsstreams
@bagder@bagder
@bagder@bagder
HTTP/3 = HTTP over QUIC
@bagder@bagder
Stacks: old vs new
TCP
TLS 1.2+
HTTP/2
UDP
HTTP/3
QUIC
TLS 1.3
IP
HTTP/1
@bagder@bagder
streams
@bagder@bagder
HTTPS is TCP?
HTTPS:// URLs are everywhereHTTPS:// URLs are everywhere
TCP (and TLS) on TCP port 443TCP (and TLS) on TCP port 443
@bagder@bagder
This service - over there!
The Alt-Svc: response header
Another host, protocol or port number is the
same “origin”
This site also runs on HTTP/3 “over there”, for
the next NNNN seconds
@bagder@bagder
HTTP/3 challenges
3-7% something of all QUIC attempts fail
Clients need “fall back” algorithms
CPU intensive
Unoptimized UDP stacks
“Funny” TLS layer
All QUIC stacks are user-land
No standard QUIC API
Lack of tooling
@bagder@bagder
Ship date
@bagder@bagder
2020?2020?
curl runs in all your devices
@bagder@bagder
Work in progress
Early days – your help is appreciated
HTTP/3 and QUIC support is experimental
Things might will change
Code is in git master and shipped
Only HTTP/3 (not QUIC) is exposed in curl
@bagder@bagder
@bagder@bagder
Build
@bagder@bagder
Build curl
Early support aids the protocol community
Requires 3rd party libraries for low level
Selectable backend, use one out of several choices
Quiche or ngtcp2 for now
- Different TLS requirements
- Very different APIs
@bagder@bagder
@bagder@bagder
Build curl with quiche
https://github.com/cloudflare/quiche
Uses BoringSSL
(Look at docs/HTTP3.md)
You want alt-svc support as well
@bagder@bagder
@bagder@bagder
Build curl with ngtcp2 (and nghttp3)
https://github.com/ngtcp2/ngtcp2
https://github.com/ngtcp2/nghttp3
Uses a patched OpenSSL or GnuTLS from git
(Look at docs/HTTP3.md)
You want alt-svc support as well
@bagder@bagder
@bagder@bagder
Why is it called ngtcp2?
Jana Iyengar (then at Google) did a QUIC presentation in the HTTP
Workshop Stockholm in July 2016
In that presentation, he showed...
@bagder@bagder
The TLS situation
The TLS situation (1/2)
TLS was made for TCP
TLS is sent over TCP as records containing
individual messages
QUIC uses TLS messages
No TLS library support(ed) TLS messages
QUIC also needs additional secrets
@bagder@bagder
The TLS situation (2/2)
@bagder@bagder
Frame 0
Message 0 Message 1
Frame 1
Message 2 Message 3TCPTCP
Message 0 Message 1 Message 2 Message 3
QUICQUIC
@bagder@bagder
BoringSSL
Supports QUIC already
BoringSSL is really hard for non-Google products to use
Quiche uses BoringSSL
… and so does Chrome/Edge and some other
implementations
@bagder@bagder
@bagder@bagder
OpenSSL
No support in any release, nor git
“once the 3.0 release is done, QUIC will become a significant
focus of our effort” + “final [3.0] release in early Q4 2020”
PR #8797 OpenSSL offers an API similar to the BoringSSL one.
https://github.com/openssl/openssl/pull/8797
ngtcp2 uses OpenSSL+8797
… and so does msquic, node.js and more
@bagder@bagder
@bagder@bagder
GnuTLS
Has initial support landed in git
ngtcp2 can be built to use it
@bagder@bagder
@bagder@bagder
Other libraries
The other TLS libraries curl supports lack the necessary APIs
for QUIC
@bagder@bagder
@bagder@bagder
curl uses a single TLS lib
When built to support QUIC, the same library is used for
“normal” TLS.
Use
@bagder@bagder
It looks like HTTP/1
As for HTTP/2, HTTP/3 in curl is made to lookmade to look
like HTTP/1like HTTP/1 when curl shows requests,
headers and similar. For consistency and easy
of use.
@bagder@bagder
Run curl
--http3
Forces curl to try QUIC and HTTP/3 on the given host name
No fallback!
--alt-svc <filename>
Bootstraps into HTTP/3 the “standard way”
Requires alt-svc: response headers (several dev servers don’t do those)
Takes an additional round-trip
Makes the initial request HTTP/1 or HTTP/2 the “usual way”
The alt-svc file format: https://curl.haxx.se/docs/alt-svc.html
@bagder@bagder
Run curl
$ curl –-http3 https://quic.tech:8443/ -v
$ curl –-alt-svc alt.txt https://quic.tech:8443/ -v
$ curl --version
…
Features: alt-svc ... HTTP3 ...
@bagder@bagder
Works now with HTTP/3
@bagder@bagder
HTTP/3 draft-27
Connecting over IPv4 and IPv6 and “Happy eyeballs”
Direct or via alt-svc
Funny host name/DNS tricks like --resolve and friends
HTTP GET and POST requests
HTTP header parsing, adding and removing headers
Cookies, connection caching, connection re-use etc
SSLKEYLOGFILE
HTTP/3 in the curl CI
We verify the HTTP/3 builds in every curl commit
and PR
Both quiche and ngtcp2
Builds only, they don’t run tests yet
@bagder@bagder
libcurllibcurl
@bagder@bagder
curl is powered by libcurl
Previously mentioned features are in libcurl
… with a few more knobs to tweak
@bagder@bagder
“Force” HTTP/3
@bagder@bagder
Alt-Svc style
@bagder@bagder
Check for support
@bagder@bagder
Remaining issues right now
@bagder@bagder
HTTP/3 is still “experimental” in curl
Fallback
(happy
eyeballs?)
https://github.com/curl/curl/wiki/HTTP3
Multiplexing Certificates
Trailers Stability Tests
Push
qlog
@bagder@bagder
Upgrading to h3 - long term
Tricky
HTTP/2 reuses the same
connection - easier
alt-svc needs cache on disk to be
effective
Happy eyeballs...
@bagder@bagder
When
@bagder@bagder
Specifications
Ship curl HTTP/3-enabled?Ship curl HTTP/3-enabled?
Deployed servers
Browser support
libcurl
TLS libraries
QUIC and HTTP/3 libraries
@bagder@bagder
@bagder@bagder
It will take a while
@bagder@bagder
48
You can help!You can help!
@bagder@bagder
December 6, 2019December 6, 2019
@bagder@bagder
HTTP/3 Explained
https://http3-explained.haxx.se
@bagder@bagder
Daniel Stenberg
@bagder
https://daniel.haxx.se/
Thank you!Thank you!
Questions?Questions?

HTTP/3 in curl 2020

  • 1.
  • 2.
  • 3.
    How HTTP/3 differsfrom 1 and 2How HTTP/3 differs from 1 and 2 Build curl to enable HTTP/3Build curl to enable HTTP/3 The TLS situationThe TLS situation HTTP/3 with the curl toolHTTP/3 with the curl tool HTTP/3 with libcurlHTTP/3 with libcurl Remaining issuesRemaining issues WhenWhen @bagder@bagder
  • 4.
    This is notmy HTTP/3 explanation talk HTTP/3 for everyone (FOSDEM 2020) https://daniel.haxx.se/videos/ @bagder@bagder
  • 5.
  • 6.
    Improvements in QUICImprovementsin QUIC TCP head of line blockingTCP head of line blocking Faster handshakesFaster handshakes Earlier dataEarlier data More encryption, alwaysMore encryption, always Future developmentFuture development @bagder@bagder
  • 7.
    QUIC on topof UDP TCP and UDP remain “the ones”TCP and UDP remain “the ones” Use UDP instead of IPUse UDP instead of IP Reliable transport protocol - inReliable transport protocol - in user-spaceuser-space A little like TCP + TLSA little like TCP + TLS @bagder@bagder
  • 8.
    Streams! QUIC provides streamsQUICprovides streams Many logical flows within a single connectionMany logical flows within a single connection Similar to HTTP/2 but in the transport layerSimilar to HTTP/2 but in the transport layer IndependentIndependent streamsstreams @bagder@bagder
  • 9.
    @bagder@bagder HTTP/3 = HTTPover QUIC @bagder@bagder
  • 11.
    Stacks: old vsnew TCP TLS 1.2+ HTTP/2 UDP HTTP/3 QUIC TLS 1.3 IP HTTP/1 @bagder@bagder streams @bagder@bagder
  • 12.
    HTTPS is TCP? HTTPS://URLs are everywhereHTTPS:// URLs are everywhere TCP (and TLS) on TCP port 443TCP (and TLS) on TCP port 443 @bagder@bagder
  • 13.
    This service -over there! The Alt-Svc: response header Another host, protocol or port number is the same “origin” This site also runs on HTTP/3 “over there”, for the next NNNN seconds @bagder@bagder
  • 14.
    HTTP/3 challenges 3-7% somethingof all QUIC attempts fail Clients need “fall back” algorithms CPU intensive Unoptimized UDP stacks “Funny” TLS layer All QUIC stacks are user-land No standard QUIC API Lack of tooling @bagder@bagder
  • 15.
  • 16.
    curl runs inall your devices @bagder@bagder
  • 17.
    Work in progress Earlydays – your help is appreciated HTTP/3 and QUIC support is experimental Things might will change Code is in git master and shipped Only HTTP/3 (not QUIC) is exposed in curl @bagder@bagder
  • 18.
  • 19.
    Build curl Early supportaids the protocol community Requires 3rd party libraries for low level Selectable backend, use one out of several choices Quiche or ngtcp2 for now - Different TLS requirements - Very different APIs @bagder@bagder
  • 20.
    @bagder@bagder Build curl withquiche https://github.com/cloudflare/quiche Uses BoringSSL (Look at docs/HTTP3.md) You want alt-svc support as well @bagder@bagder
  • 21.
    @bagder@bagder Build curl withngtcp2 (and nghttp3) https://github.com/ngtcp2/ngtcp2 https://github.com/ngtcp2/nghttp3 Uses a patched OpenSSL or GnuTLS from git (Look at docs/HTTP3.md) You want alt-svc support as well @bagder@bagder
  • 22.
    @bagder@bagder Why is itcalled ngtcp2? Jana Iyengar (then at Google) did a QUIC presentation in the HTTP Workshop Stockholm in July 2016 In that presentation, he showed...
  • 23.
  • 24.
    The TLS situation(1/2) TLS was made for TCP TLS is sent over TCP as records containing individual messages QUIC uses TLS messages No TLS library support(ed) TLS messages QUIC also needs additional secrets @bagder@bagder
  • 25.
    The TLS situation(2/2) @bagder@bagder Frame 0 Message 0 Message 1 Frame 1 Message 2 Message 3TCPTCP Message 0 Message 1 Message 2 Message 3 QUICQUIC
  • 26.
    @bagder@bagder BoringSSL Supports QUIC already BoringSSLis really hard for non-Google products to use Quiche uses BoringSSL … and so does Chrome/Edge and some other implementations @bagder@bagder
  • 27.
    @bagder@bagder OpenSSL No support inany release, nor git “once the 3.0 release is done, QUIC will become a significant focus of our effort” + “final [3.0] release in early Q4 2020” PR #8797 OpenSSL offers an API similar to the BoringSSL one. https://github.com/openssl/openssl/pull/8797 ngtcp2 uses OpenSSL+8797 … and so does msquic, node.js and more @bagder@bagder
  • 28.
    @bagder@bagder GnuTLS Has initial supportlanded in git ngtcp2 can be built to use it @bagder@bagder
  • 29.
    @bagder@bagder Other libraries The otherTLS libraries curl supports lack the necessary APIs for QUIC @bagder@bagder
  • 30.
    @bagder@bagder curl uses asingle TLS lib When built to support QUIC, the same library is used for “normal” TLS.
  • 31.
  • 32.
    It looks likeHTTP/1 As for HTTP/2, HTTP/3 in curl is made to lookmade to look like HTTP/1like HTTP/1 when curl shows requests, headers and similar. For consistency and easy of use. @bagder@bagder
  • 33.
    Run curl --http3 Forces curlto try QUIC and HTTP/3 on the given host name No fallback! --alt-svc <filename> Bootstraps into HTTP/3 the “standard way” Requires alt-svc: response headers (several dev servers don’t do those) Takes an additional round-trip Makes the initial request HTTP/1 or HTTP/2 the “usual way” The alt-svc file format: https://curl.haxx.se/docs/alt-svc.html @bagder@bagder
  • 34.
    Run curl $ curl–-http3 https://quic.tech:8443/ -v $ curl –-alt-svc alt.txt https://quic.tech:8443/ -v $ curl --version … Features: alt-svc ... HTTP3 ... @bagder@bagder
  • 35.
    Works now withHTTP/3 @bagder@bagder HTTP/3 draft-27 Connecting over IPv4 and IPv6 and “Happy eyeballs” Direct or via alt-svc Funny host name/DNS tricks like --resolve and friends HTTP GET and POST requests HTTP header parsing, adding and removing headers Cookies, connection caching, connection re-use etc SSLKEYLOGFILE
  • 36.
    HTTP/3 in thecurl CI We verify the HTTP/3 builds in every curl commit and PR Both quiche and ngtcp2 Builds only, they don’t run tests yet @bagder@bagder
  • 37.
  • 38.
    curl is poweredby libcurl Previously mentioned features are in libcurl … with a few more knobs to tweak @bagder@bagder
  • 39.
  • 40.
  • 41.
  • 42.
    Remaining issues rightnow @bagder@bagder
  • 43.
    HTTP/3 is still“experimental” in curl Fallback (happy eyeballs?) https://github.com/curl/curl/wiki/HTTP3 Multiplexing Certificates Trailers Stability Tests Push qlog @bagder@bagder
  • 44.
    Upgrading to h3- long term Tricky HTTP/2 reuses the same connection - easier alt-svc needs cache on disk to be effective Happy eyeballs... @bagder@bagder
  • 45.
  • 46.
    Specifications Ship curl HTTP/3-enabled?Shipcurl HTTP/3-enabled? Deployed servers Browser support libcurl TLS libraries QUIC and HTTP/3 libraries @bagder@bagder
  • 47.
    @bagder@bagder It will takea while @bagder@bagder
  • 48.
    48 You can help!Youcan help! @bagder@bagder
  • 49.
    December 6, 2019December6, 2019 @bagder@bagder
  • 50.
  • 51.