HTTP/3 and QUIC: Past, Present, and Future
You may have seen the announcements over the past two weeks -- the IETF QUIC RFCs have been published! That leads to a lot of questions, depending on how closely you've followed this space. You might be wondering what this means to you, or you might think QUIC has been an established thing for years now. And how does HTTP/3 fit into this? Is HTTP/3 another name for QUIC?
QUIC is a new transport for the internet. The use of QUIC enables deployment of years of research developments at once, improves privacy, and offers substantially better performance in challenging network conditions. HTTP/3 is the mapping of HTTP which uses IETF QUIC as a transport.
I'm the editor of HTTP/3 and an active contributor to the IETF QUIC specs. In this post, I'm going to take a look at where IETF QUIC came from, how far it has progressed, and what's left to do.
Origins of QUIC
The IETF published TCP four decades ago. TCP is the layer underneath the vast majority of network connections on the internet. It gets information from one computer to another in the same order, even when the network loses or reorders packets.
As we discussed before, every device on the network can (and does) inspect TCP packets as they flow by. As a result, it's been difficult to successfully deploy changes to TCP. Many innovations for TCP have seen very little use on the live internet because they confuse these "middleboxes." Even when manufacturers have written the software to support a new TCP feature, it can be difficult to get all the old devices upgraded, especially when that can require a full OS upgrade on each device. For example, TCP Fast Open (published in 2014) offers comparable latency benefits to QUIC for connection reestablishment in some cases, when combined with TLS 1.3. Despite a multiyear deployment effort from all major OS vendors, it has yet to see widespread use.
Encryption at the application layer has become routine -- for example, the median Akamai CDN customer serves over 99% of requests via HTTPS. Google observed that moving this encryption further down into the transport would prevent middleboxes from interfering. This would enable them to deploy new transport features more rapidly. At that point, only the endpoints need updates to add any new features; this simplifies the path to deploying new features.
SCTP (the Stream Control Transmission Protocol) was a previous attempt to replace and improve upon TCP. Instead, it demonstrated that it's very difficult to deploy a protocol other than TCP or UDP on the internet. QUIC solves this by re-implementing basic transport services within an encrypted envelope, using UDP to cross the internet. Google first announced the availability of Google QUIC eight years ago, using it between Google Chrome and Google services. This allowed them to make improvements independent of the operating system or the OS update schedule.
Google QUIC provided many performance benefits for Chrome, and other companies began to contribute to this protocol. Akamai deployed Google QUIC in 2016, and by 2018 had nearly as many QUIC endpoints as Google. Since then, we have gradually increased the amount of traffic served over the new protocol.
What's new here?
So if QUIC is eight years old and Akamai's had it for most of that time, what's all the hubbub about now?
Google brought QUIC to the IETF six years ago to begin the standards process. Contributors to the IETF expressed interest in developing a new transport beginning from their experimental protocol. This proprietary version is often called "Google QUIC" or "gQUIC."
The IETF formed a QUIC working group in 2016. Their charter was to take Google's web-specific implementation and adapt it to be a general-purpose transport protocol. The internals of Google QUIC looked very much like HTTP/2, but mixed in innovative ideas in transport and encryption. In creating "IETF QUIC," we've focused on teasing the functionality into separate components.
Google QUIC was implemented as a single reference codebase that exposed an HTTP-like API, employed custom cryptography, and output UDP packets on the bottom. The IETF process evolved that protocol in many ways, including:
- Separating the application-layer protocol from the transport layer. IETF QUIC is a general-purpose transport, comparable to TCP. HTTP/3 is the mapping of HTTP which uses IETF QUIC as a transport.
- Creating an application-independent set of transport primitives that any application can use. IETF working groups are already exploring QUIC as a transport for DNS, SSH, BGP, RTP, and many others.
- Replacing the custom cryptography with TLS 1.3. QUIC Crypto informed the design of TLS 1.3, which became a Proposed Standard in 2018. Some of the ideas first explored by QUIC Crypto are now proposed TLS extensions. Transitioning from custom cryptography to standardized TLS 1.3 helps remove ambiguity around whether using gQUIC is a problem for sites with regulatory and/or compliance requirements.
- Other substantial changes to the protocol. Thousands of issues and contributed specification changes have improved the quality and clarity of the specifications.
There are more than a dozen open-source IETF QUIC implementations at this point. Many of their authors have participated in IETF Hackathons. These events have discovered issues in the specification and helped us clarify it. When two developers produce incompatible code from the same specification, the specification has a problem!
The specifications published in the past two weeks are the heart of the IETF QUIC transport protocol:
RFC 8999 (Version-Independent Properties of QUIC) describes the small part of QUIC that the network can observe. This cannot change between IETF QUIC versions.
RFC 9000 (QUIC: A UDP-Based Multiplexed and Secure Transport) is version 1 of IETF QUIC, the core transport protocol. It describes how clients and servers communicate with each other in this version.
RFC 9001 (Using TLS to Secure QUIC) describes the integration of TLS 1.3 and QUIC version 1. TLS is neither on top of nor below QUIC; the relationship is more complicated. QUIC provides TLS a reliable in-order stream, while TLS provides QUIC encryption keys and handshake validation.
RFC 9002 (QUIC Loss Detection and Congestion Control) gives an example of loss detection and congestion control algorithms for IETF QUIC. As with TCP, there are many strategies an endpoint might use for both loss detection and congestion control, and we expect this will continue to be an area of active experimentation and innovation. This draft provides a starting point for anyone trying to get QUIC off the ground.
With the specification complete, this means the IETF version of the transport is stable and ready for production use. Google has already retired all versions of Google QUIC that don't conform to the invariants in RFC 8999, and will soon retire all remaining versions of Google QUIC.
What's missing?
There are some things you don't see in that list -- work is ongoing in several areas, both in the QUIC WG and across the IETF. Let's do a quick flyby of the things you might have expected to see that are still on the way.
HTTP/3
The above set of RFCs does not include HTTP/3 (or its companion document, QPACK). That might surprise many, since the specification was codeveloped in the QUIC working group and submitted for publication at the same time.
Core HTTP semantics don't change between versions -- the HTTP protocol has essentially the same functionality, whether it's HTTP/1.1, HTTP/2, or HTTP/3. However, the specification of HTTP semantics is currently intertwined with the definition of HTTP/1.1. This is currently found in RFC 7230 and its companion documents. The IETF HTTP working group is extracting these version-independent semantics into two new documents:
HTTP Semantics describes the operation of HTTP across any transport or mapping.
HTTP Caching describes how endpoints can cache and reuse responses across any transport or mapping.
A new trio of specifications setting out the different mappings of HTTP semantics to various transport protocols rely on those two documents:
It's important to note that none of these versions of HTTP deprecates the others, because each is appropriate to different use cases. There are networks that block UDP, meaning that HTTP/3 cannot be used; the spec recommends using a TCP-based version of HTTP on those networks. HTTP/1.1 and HTTP/2 are both still receiving attention from the IETF.
Because all of these new mapping documents rely on the Semantics document, none of them can be published until the Semantics document is complete. That should happen in the next few months. The RFC Editor will publish the updated HTTP/1.1 specification and the HTTP/3 specification at roughly the same time, since they've been co-developed with HTTP Semantics and QUIC respectively. The HTTP working group has adopted the corresponding updated HTTP/2 specification, which is almost complete.
Because the HTTP/3 specification is essentially complete and stable at this point, you can expect to see many providers deploying the "final" version even while we wait for this dependency to land.
Version negotiation
Google QUIC had a mechanism for negotiating between multiple versions of the protocol. During the standardization process, defining the exact security goals of this piece of the protocol became a point of contention.
To unblock the publication of IETF QUIC version 1, it was agreed that the v1 specification would define only how to indicate that version 1 is not supported. Clients that only support version 1 would fail to connect to a server which does not support that version. This is sufficient for HTTP/3, because the methods of discovering QUIC support on an origin can also inform you which version of QUIC the server supports.
The QUIC WG is actively developing the full version negotiation mechanism. It will define how implementations can support multiple versions of IETF QUIC and agree on a version without prior knowledge.
Applicability and manageability
An encrypted transport protocol has implications both for engineers choosing a transport for their protocols and for network operators trying to monitor activity.
Applicability of the QUIC Transport Protocol describes considerations for application protocols that might want to use QUIC as a transport layer. The properties of QUIC are different from those of TCP in some interesting and subtle ways, such as the availability of data before the handshake has completed, or the availability of multiple simultaneous streams. This document discusses the design choices someone implementing a protocol might need to consider when designing a version that uses IETF QUIC as the transport.
Manageability of the QUIC Transport Protocol describes the impact of an encrypted transport layer on network monitoring and management. Many networks assume the ability to inspect TCP's state for indications of network problems (poor performance, high packet loss) or abusive traffic (data exfiltration, attacks). Since QUIC obscures the contents of the packets, this document discusses limitations and alternative strategies for achieving these goals. As another example, this document recommends that networks should either entirely allow or block QUIC connections, as dropping QUIC packets randomly will hurt user performance.
Other application protocols over QUIC
QUIC started as a project to improve web browser performance, and the QUIC working group was chartered to make version 1 contain the features necessary to enable an HTTP/2-equivalent mapping. But IETF QUIC is not only for web browsers. As noted above, the IETF is working on updating several other protocols to use QUIC as a transport. For example, DNS-over-QUIC may be a natural way to secure DNS recursive to authoritative communications. There are also external applications and standards bodies which have already started to build on top of IETF QUIC for their own internal protocols. Notable examples include 3GPP, Windows SMB, and Syncthing.
Load-Balancer integration
With TCP, it was common for load balancers to terminate TCP (and perhaps TLS) before passing the application-layer connection to one of several servers behind it. This is more difficult with QUIC, but QUIC also provides routing features like a Connection ID visible in every packet. This draft describes ways for servers and load balancers to cooperate in the creation of Connection IDs, so they can be used for routing by the load balancer without compromising QUIC's privacy and security guarantees.
Datagrams
While QUIC runs over UDP, its application interface looks much more like TCP or SCTP: a series of reliable, in-order byte streams. For many applications, that's perfect -- reliable delivery is necessary to many protocols. But other protocols are more sensitive to latency than loss, and it's better to skip lost data rather than retransmit it.
The DATAGRAM extension will add the ability to send unreliable datagrams within a QUIC connection -- the application learns whether the datagram was received or not, and can decide which lost data needs to be retransmitted.
Further into the future
From the beginning, QUIC has been exciting to Akamai as a platform for future protocol development. Akamai has developed and deployed several UDP-based transports over the years to improve application and user performance. These protocols have faced similar challenges of network filtering and security protocol development. Proprietary protocols are difficult to widely deploy.
Akamai was the first third party to deploy Google QUIC when it became available. We have actively contributed to the specification and the open source ecosystem over the course of development, including supplying work to make OpenSSL compatible with IETF QUIC. This work has developed into a joint effort with Microsoft -- quictls, which is used in Microsoft's MsQuic stack as well as in Node.js and cURL.
IETF QUIC provides a new jumping-off point. When Akamai wants to experiment with partners or customers to deploy new transport optimizations, we have the QUIC framework, its transport invariants, and its security infrastructure as a vehicle for that research.
In conclusion
The IETF QUIC transport RFCs being published is a big deal! Those of us who've spent the past several years working toward this milestone can take a moment to celebrate this accomplishment, while those of you who are just beginning to experience the benefits have something to look forward to.
However, there's still more work to be done in the world of QUIC, and we'll get right back to it after the party concludes.
