Paul: Hi there and welcome to PodRocket. I'm your host, Paul, and today we are fortunate to have Robin Marx on the call. So, Robin is a solutions architect at Akamai and has a PhD in computer science, so he's one of the guys that we're going to love to talk to talk about what's bleeding edge and down to the nuts and bolts. Robin Marx: Hey Paul. How are you? Paul: I'm good, thanks. So Robin, today we're going to get into HTTP/3, which what is this? This is a protocol, right, we're going to be talking about. Robin Marx: Yeah, it's the third version of the protocol you all know and love and use daily, HTTP. It's the most recent version of it. Obviously it's been in development for about 10 years by this point, and it's finally been standardized three weeks ago, so it's kind of time that people start catching up to what it is and how we can use it. Paul: So it's the most modern one, but it's been around in development for 10 years. Robin Marx: Yeah, exactly. So if people might remember, HTTP/2 isn't that long ago, it's only finalized in 2015, like seven years ago, but even then they were already working on the inner layers of what is now HTTP/3. So, I always like to say HTTP/2 is like a transitional version to what we really wanted to end up with, which is H3. Yeah. Paul: It's almost like it takes the same amount of time to develop these very impactful protocols as they come out as it does for the drug companies to come out with a new treatment. It's like a decade. Robin Marx: Yeah, that's true. That's actually one of the reasons that HTTP/3 is so different from HTTP/2, because we want to speed this up for the future. What we actually did was make a lot of changes in not just HTTP, the top protocol, but everything beneath that. We changed things in a protocol stack like protocols like TCP and TLS. We updated those as well to make sure that we can be much faster implementing new protocols in the future going forward, yeah. Paul: Yeah, we should definitely get into what some of the specifics about HTTP/3 are. I mean, just quickly fast forward to anybody that's listening and is excited about this. I mean, HTTP/3 is based on UDP, right, Robin? Instead of the other more reliable TCP transport layers of the other ones. Robin Marx: Yeah, so that's a statement that's technically correct, but not really useful. Loosely correct I would say. The thing is that there is a very big thing between HTTP/3 and UDP, which is called QUIC, right? QUIC is really the new thing. HTTP/3 is not that much different from HTTP/2, but QUIC is very, very new, very different. QUIC is actually what some people call TCP 2.0. It's like a re-imagining of what TCP could be in a modern network. QUIC has reliability, it has flow control, congest control, everything we know from TCP is also in QUIC. The fact that it runs on top of UDP really has nothing to do with your typical TCP versus UDP differences that we all learn about in classes. UDP is faster, because it doesn't have a connection set up, that kind of stuff. That's not really true here, because QUIC adds all of that overhead back on top. The real reason we use UDP is because it makes QUIC easier to deploy, because most devices on the internet, they expect two things on top of IP, UDP and TCP. If you would put QUIC next to them, a lot of devices would just freak, and so we said, "Okay, let's just put it on top of UDP instead." Most devices will just let UDP pass. So it's just easier to deploy, that's really the main reason we're using UDP here. Paul: Yeah, so it sounds like there's a re-imagining and implementation of some sort of TCP, some reliability on the underlying language of UDP. I wanted to just kind of outline that this is based on that lower level, on UDB, because maybe it'd be useful, if it's all right with you, to draw a line between HTTP/1, two and three, before we get into the nuts and bolts, just to create a lineage for our listeners. So HTTP/1, classic old web, based on TCP, and then we go to HTTP/2. Maybe could you talk about some of the difference between HTTP/1 and two? But very quickly, don't even get too detailed. Robin Marx: Yeah, so those differences are massive, very big difference there. The main reason for HTTP/2 at that point was a single webpage contains a lot of different resources, HTML and JavaScript, CSS, and all of those need to be sent on a single TCP connection ideally. TCP is not really made to understand that there are different resources on a single connection, it thinks it's all just a single file that it's downloading. So, HTTP/2 added that abstraction inside. That was a big problem with HTTP/1. HTTP/1 you could only send a single file over a single TCP connection at a time, and so you had some 30 workarounds where you would open six to 30 TCP connections in parallel, for which we had to do domain charting and resource bundling and all these kinds of nasty basically workarounds to get that to perform in any type of a good way. Paul: Is some of that tax that we're experiencing from the RTT, the round-trip time, for opening these connections and you have to do, oh, I have to do 50 of them? Robin Marx: Yep. Paul: Okay. Robin Marx: That is definitely one part of it, and then another point that was starting to become a problem was packet loss. Because all of these connections are going to start ramping up their bandwidth usage at the same time and they're going to cause bandwidth contention at the same moment, and so you're going to get massive loss at a certain given point of time. That's one of the other reasons you really want to have as few connections as possible. Paul: From maybe exponential back-offs- Robin Marx: Yeah, exactly. Paul: Mounting up and then going, "Ah, no more." Gotcha. Robin Marx: Yep. Paul: So yeah, so HTTP/2, when you say we can do multiple files in one connection. Kind of manifesting the actual underlying structure about what's happening there, it's kind of like a stream. Is that the right way to think about it? Robin Marx: Yeah. Yeah, that's the terminology we use. The stream comes from a byte stream. So even HTTP/2 doesn't really know if a stream is HTML or CSS or JavaScript, it doesn't care, right? It just knows this is an independent thing that the browser can then use and interpret as it wants. So, that's why they're call byte streams. This is actually terminology from TCP, but TCP is just a single byte stream, and now we want multiple byte stream on top of that single byte stream. That's what HTTP/2 does, yeah. Paul: So HTTP/2, you can kind of go connect to your guy over there, send everything that needs to be sent just as bytes and then close your connection, no need to go back and forth. All right, so now we're in the new space, HTTP/3. So the main difference we talked about is we're building up from UDP, but realistically there's some reliability that you folks are tackling and implementing, and you said that was called QUIC. Would you say QUIC is a one-to-one mirror of the reliability that TCP provides? Or what are some of the corners, I guess, that you cut and the decisions that get made in this design process? Robin Marx: Yeah, so it's not a one-to-one mirror, but the concepts are very, very similar. It's still fully reliable, so it's not like packets can get lost. You still have acknowledgements and retransmissions and you still have all the other things that we know from TCP, but there are of course big differences. One of the main differences there has to do with these streams that we just talked about. So HTTP/2 added this concept of multiple independent streams, but it then still had to map it onto a single stream in TCP, and that led to all kinds of inefficiencies. This is what is called the head of line blocking problem, which we don't really have to go into today probably, but a lot of inefficiencies there. What QUIC basically did was take that concept of these multiple streams and brought it down into the transport layer protocol. So TCP thinks it's just a single file, but QUIC knows, oh no, we actually have multiple independent things on the same connection going on. That means that QUIC also knows, okay, I have a packet lost. That doesn't mean I have to stall my entire connection, like with TCP, it can say, oh, I just have packet loss for this stream. All the other streams are unaffected, I just need to deal with this single stream that has loss, right? In that way and in several other ways, QUIC can be more efficient about loss detection reliability. So, the high-level concepts are very, very similar. It's still a reliable drop-in replacement for TCP, but it can be much more efficient, much smarter in what it does. Paul: Is this because UDP kind of gives you this platform and sandbox to sort of say it's like a big soup of a transport layer, and we can slash this up and divvy it up and manage what is going in semantically, versus I guess in TCP, when you open up a connection in the stream, everything in TCP is stateless, right? Is that the right way to think about it? It's really like this is my one job, my one file and I'm going to take care of it, so everything else on top of that is just magic that you have to implement. Robin Marx: I think that's a fair thing. That was the case with TCP, right? So, UDP is super flexible. You can do anything you want on top, but that also means it doesn't give you much reliability. TCP was more like what will most implementations, what will most higher level protocols need is order data that is sent reliably. That's basically the service TCP provides, right? Now with UDP, because it's so flexible, you can build whatever you want on top of that. So people are surprised by this, but this has been going on for ages, right? A lot of games, a lot of video conferencing systems, they all run on top of UDP and they often re-implement reliability to some extent, or very similar things that you see in TCP on top of UDP. It's nothing new, it's just that we mostly use TCP, because you get that for free. You get all this complex logic for free, you don't have to do it yourself, but there's no reason you can't do it yourself. That's what QUIC does, but then in a very advanced way. I always like to think of QUIC, we took the last 30 years of TCP, everything we learned, we put it into one nice big package with all the best practices, and that is QUIC. Paul: Awesome. I mean, let's talk about why it's faster, maybe some of the nuts and bolts. RTT, round-trip time, with UDP it's a little bit of a different story. So you mentioned these video conferencing guys, these game applications, they use UDP. There's a reason they use UDP, that you can get faster, there's more malleability, like you said, when you're implementing things. Could you talk about how HTTP/3 tackles the speed problem within QUIC and makes your own improvements? Robin Marx: So, QUIC offers a lot of different performance features over optimizations. Like I said before, head of line blocking removal, there is something called connection migration, you have slightly better prioritization system and so on. These are things very technically interesting, but I wouldn't say those will have the biggest impact for most of the users, right? They only come out at very specific edge cases for some use cases. The one thing that will impact most of your users is what you just hinted at, which is the round-trip time, which is the connection setup at the start, right? That's one of the reasons that we use UDP, that's correct, is that it doesn't require a connection setup. This is why, for example, DNS runs over UDP, because you can send the request and get a response into a single round-trip between client and server. That's not true with TCP, right? TCP you need to do the SYN, SYN-ACK, ACK three-way handshake that costs you a full round-trip. Then if you run TLS on top of it to get the encryption that we all want so badly nowadays, that is another one to sometimes two round-trips on top. So you're already doing two or three round-trips of waiting, and only then can you send your HTTP request and start getting a response back. If the round-trip time is high, let's say your client is in Europe where I am and your server is somewhere in California, then this can take up to 200 milliseconds or more for even a single round-trip, right? That gets slow really, really fast. That's one of the core optimizations with QUIC. So with TCP, TCP and TLS are completely separate protocols. With QUIC, that's no longer true. QUIC and TLS are fused together inside of QUIC. So QUIC can only be used with TLS, it's always encrypted. This means that you can do the handshake of both QUIC, so the transport level handshake and the cryptographic handshake, the TLS setup, you can all do that in a single round-trip. So you save at least one round-trip, sometimes even two compared to your TCP typical setup. I have maybe some numbers later that I can share. We have some companies that actually tested this in practice and they find up to 400, even 600 milliseconds lower connection setup time just because of this in real networks. Then it gets better, because QUIC and HTTP/3 can then do even one RTT less with a new feature called 0-RTT where you can buy, not just the transport and the cryptographic stuff, but also the HTTP request. All of that is done in the first round-trip. So you have no more waiting time, the first packets you send already contain your HTTP request. So that will lead to substantial speed-ups, and I think that is the most important feature and the biggest reason you should be using HTTP/3 at this point. Paul: How do you guys get TLS to work with 0-RTT? That sounds amazing to me. Without the SYNs, the ACKs, and yeah, I mean, we don't want to get too into the weeds here. Robin Marx: Very good question, because that's one of the most technically challenging aspects here. The problem there is to be able to do that, you want to fully encrypt all of your first RTT, otherwise you're sending your initial ACP request in plain text and people will be able to read it, you don't want that. So, 0-RTT is really only possible from a second connection onwards. What's going to happen in your very first QUIC connection, that's going to be a normal one round-trip time for transport in TLS. After that, it's fully encrypted and then the client and server are going to negotiate new cryptographic parameters for the next connection. This is what's called a session ticket, because it's a TLS session. So you basically get what is called a resumption key, so you pretend like you're resuming the current session sometime in the future. So in practice, this means that from the second connection onward, if you have a valid session ticket, you can basically use those old cryptographic keys that you discussed earlier to encrypt already new stuff from the very, very start. That sounds too good to be true, and it is, because it has a lot of edge cases and a lot of security issues, things like replay attacks, and it's difficult to deploy this after load balancers and all those kinds of things. Very true, but it's doable, and it does provide you with a nice additional boost. Paul: Is there some sense of a cryptographic key tied to time? Robin Marx: Exactly. So the session tickets, they have a certain lifetime, a certain time to live. So it's typically, I think in modern deployments, about a day. So, you have to do it within a day or it expires and you fall back to a normal connection setup. There are other protections there as well. Some deployments won't allow you to use 0-RTT if you change IPs or you change networks in between. There are several other ways of mitigating a new risk, any security risk that you might encounter with 0-RTT. Paul: I mean, the most dangerous people out there are the ones who are smart enough to keep up with the modern news and transport layers that are being built so they can understand how to sneak in and do these types of things. So, it's always important to reframe ourselves to talk about these very important things that people listening are going to be implementing them. Emily: Hey, this is Emily, one of the producers for PodRocket. I'm so glad you're enjoying this episode. You probably hear this from lots of other podcasts, but we really do appreciate our listeners. Without you, there would be no podcasts. Because of that, it would really help if you could follow us on Apple Podcasts, so we can continue to bring you conversations with great devs like Evan You and Rich Harris. In return, we'll send you some awesome PodRocket stickers. So, check out the show notes on this episode and follow the link to claim your stickers as a small thanks for following us on Apple Podcasts. All right, back to the show. Paul: How do you think implementing HTTP/3 will fall on the plate of people? Is it difficult? Is it hard? I mean, from my personal experience, you talked about HTTP/2, and right before this call I was [inaudible 00:18:51] over what a difficulty it was to get HTTP/2 to work in a serverless function because of the streams. Robin Marx: Yeah, yeah, yeah, yeah, yeah. I have bad news for you, it's only going to get worse. Yeah, because for HTTP/2, you only need to deal with HTTP/2, but now you need to deal with HTTP/3 and QUIC, and inherently also TLS, because it's inside of QUIC. Where, for example, TCP and TLS, you typically get much easier in modern stacks because they're older. So the recommendations I have up to now always given is that if you want to use it today, use an existing deployment from ACDM. A content delivery network. Something like Akamai, CloudFlare, Fastly, they all have very mature deployments that you can use with the click of a button. It's really that simple. They have implemented this, they even implement all the complex features that you might ever want to use, because they have been working on this for five to six years since the beginning. If you really want to do this yourself, I wish you the best of luck. There are options, but they are very hands-on. For example, your typical servers that you might use, let's think of an Nginx or an Apache or an OTS or something like that, they all don't yet have implementations or the implementations they have are still in beta. They're very unstable and it will take quite a while for them to become production ready, I think. There are some servers that do have it. One of the good one is Caddy, which is often mentioned. Another one that I would recommend is H2O, which is the server that Fastly uses internally. That's a very, very good server to use. But those are not your typical servers you would have in a production setup, so you would have to mix and match this with your existing HTTP/1 and two deployments. Not always easy. That's doable, but you're going to have to read up a lot on how to properly set this up, and you will probably lack advanced features like 0-RTT, at least if you want to do them securely, because that's difficult to do yourself. Then for what you mentioned, trying to integrate this yourself into a custom app or a custom piece of software, also possible, because there are many open source libraries. But there, again, you're going to have to spend quite a bit of time getting to know what exactly this is, because the APIs are very different, the capabilities are very different and you really need to know what the protocols are doing underneath to really get the speed boosts that you might want. If you don't get the speed boosts, there's really no real good reason to switch to HTTP/3 and it's easier to stick with HTTP/2. So, it's a lot of work to really get it out of there. I expect this to improve over time, let's say two to three years, but at this point it's too new, I would rely on existing implementations, existing deployments I should say, yeah. Paul: So, it sounds like one of the big optimizations you're getting here when you're talking about, okay, well, if you're not getting an optimization, it's not even really worth it in the first place, so these optimizations we're talking about are mainly framed in our conversation around RTT and how these get carried out and performed. Is there a vector we're missing here that you would say has important weight to the speed saving? Robin Marx: Yeah, so as I said before, there are several other performance features that are definitely in there and that will have a big impact, but only for a subset of the users. In web performance, for example, we like to talk about the p99 or the p99.99 percentiles, which means one in every 100 or every one in 10,000 users will have a big impact from that. One thing, for example, is what I mentioned earlier, the better way that QUIC can do loss detection and re-transmission will be useful in some heavily congested cellular networks, that kind of stuff, right? So, that's one aspect. The second aspect that you might want to start using HTTP/3 now is for future improvements. So right now the RTT and the connection setup is the most important one, but this is going to change in the future. They are talking about adding new features. One of the ones that I'm particularly interested in is called MultiBot, where you can use Wi-Fi and cellular at the same time. So now you'd switch between them, but then you can use them at the same time to get better bandwidth or more robustness. That's coming. There is something called FEC, forward error correction, where you prevent three transmissions from being needed by sending some additional data in the first place, that kind of stuff. That is all coming. If you switch to HTTP/3 now, you can very easily tap into that later. That being said though, I think HTTP/3 is mostly going to be useful in the client side of the connection, like the last mile typical setup. If you want to start doing inter-data center or CDN edge server to your origin server kind of traffic, there is probably not going to be worth it to switch to HTTP/3, at least not yet. Probably down the line, but not yet. You won't get a lot of huge benefits there, unless you're using very specific setups that use very specific features in the protocols. But I haven't seen people talk about that yet, for now it's mostly client to edge server where it's being deployed and where it's most useful. Paul: So your point about congested networks, I mean, that's a very potentially pertinent use case, even for an internal setup. If you have something, big data applications where you're having millions and billions of requests being sent, do you think HTTP/3 could help in that internal sort of setup remove the cruft and the extra time from the RTTs, from everything? Yeah, that's a lot of requests that need to get made, it's insane. Robin Marx: In that case, definitely, but again, that is a very specific setup that not many people have, right? In that case, you also need to look at other features like ECN and the upcoming L4S and that kind of stuff. QUIC definitely has support for those things, and this kind of use case is supported in QUIC, but again, I don't think most people listening hopefully, hopefully are managing that kind of network, because woe become them. Paul: Now, I have a very specific question to go along with this. So if you're in big data and you're trying to maybe populate a Snowflake table or S3 buckets or whatever, if you're making millions and billions of requests, you're probably doing it wrong. What you want to be doing is batching things, you want to be sending it in one request and then kind of finding that sweet spot. That comes at an overhead though. You're storing things longer in memory, you're paying for more memory, you're paying for, I don't know, just more ... there's always a trade-off between your run time efficiency and the amount of memory you spend, those are the two big. So if we have HTTP/3 and all this cruft is removed, do you think we can reasonably start to think about removing batching? Would that be something in your mind in the future that could be potentially looked at from this new protocol? Robin Marx: I think with the concept of independent streams, at the transport layer, I don't think HTTP/3 will be the big driver there, but QUIC definitely could be, but it will require some use case-specific logic on top to make proper use of these underlying mechanisms to make this work just fine. I do think QUIC has, again, the capabilities, the raw capabilities to make this kind of thing possible. Is it able to do it right now with the current implementations? I doubt it, but yeah, I think that's definitely an avenue worth exploring for that kind of use case. Paul: That would be very interesting. I just feel like everything is batched everywhere, and it causes a lot of system design complexity and bug surface area that could otherwise be avoided. So yeah, what are your hopes about where HTTP/3 will make the biggest impact? So I think you kind of touched upon this briefly, you were saying, okay, the client side, those RTTs. Do you see any business use cases in your mind that we haven't touched upon that excite you? Either in video streaming, because they already use UDP their own custom solutions. Robin Marx: Yeah, video streaming is a difficult one, because we have serious established protocols, like WebRTC that just work and it will be difficult to port them over to QUIC, even if we need that down the line. What I am personally most interested in, what I'm personally most excited about is two things. One of them is both protocols that are built on top of HTTP/3 or use HTTP/3. The first one is called WebTransport, which is kind of like the new version of WebSockets in the browser, if you know about that. WebTransport is basically access to HTTP/3 and QUIC in the browser, but also unreliable data in this same connection. So up until now you really have almost no way of doing unreliable data in the browser, unless you're using WebRTC special data channels, which is kind of annoying. What we have with QUIC, because it runs on top of UDP, we can implement pure unreliable datagrams in the same QUIC connection that also does reliable data at the same time. So, it's like a single encrypted connection that does both reliable and unreliable stuff. I think this will be very interesting for many different use cases. The main one that comes to mind is gaming. You could start doing that finally without custom logic. Video streaming could be one, if you really want to do custom media streaming on top of that and many other use cases. I think WebTransport will be somewhat revolutionary in that respect. That's one part, and then the other part is called MASQ, which is kind of an effort to do proxying. So secure proxying, VPNing on top of QUIC. So, using QUIC as an encrypted tunnel to proxy different kinds of traffic through there. That's a bit technical. It is very interesting and it has a lot of benefits on top of proxying over TCP or plain UDP even. For example, if you might have heard about the new Apple feature called Private Relay, where they let you browse through anonymous proxies. It's kind of like anonymizing your IP and your user by sending your traffic through proxies. That's all QUIC-based, that's all MASQ-based basically. So it's like a whole new suite of ways of doing secure encryption through a QUIC tunnel, which offers a lot of new ways of doing this that were impossible with current TCP options. I'm very excited about that. They're talking about using this heavily in, say for example, the 5G backbone implementation, that kind of stuff. So yeah, lots more there to come, except for just HTTP/3 for the browser or for the webpage loading use case. Paul: Yeah, this sounds like it's a whole paradigm shift in how we're thinking about reliable transport as a networking concept. Robin Marx: Yeah, and that's exactly the intent, right? QUIC is intended to replace TCP in the long run. It makes things a lot more flexible and potentially performant, but that means we also need to update some of the applicational layer stuff. HTTP/3 was the first of those and we're now slowly doing this for other things as well. Paul: Some of our, yeah, mentalities. I remember the first thing I learned when I took networking class was UDP, that's where the bad people live. They'll figure out how to mess you up with UDP so it's blocked everywhere. Robin Marx: That's one of the big problems we face with QUIC today, a lot of people still have that mindset. Even though QUIC, it actually integrates a lot of mitigations for a lot of known attacks. It has protections in place to some UDP implication attacks, for example. People have thought about this, it should be quite safe to use. Not 100%, but quite safe to use, but a lot of network administrators still don't really follow that story yet and they will simply block QUIC, especially because a lot of firewall applications don't have proper support for it yet, and so they'd rather be safe than sorry. So yeah, that's one of the big challenges that we have, and I think that's going to be around for years and years and years, but the thing is the big deployments already use QUIC and HTTP/3 right now. All the CDNs, the YouTubes, the Facebooks, Amazon is coming on, that kind of stuff, that should provide enough counterweight I think in the long-term to force people to switch eventually, right? Paul: If they're doing it, they must know what they're doing, right? Well, we're running up on time, Robin. This has been a very interesting talk. I always love getting to the weird networking stuff. Even though, I mean, this is not weird, this is going to be the new standard coming out. It's out and it's going to be in our lives if we know it or not already. So, where can people find you on Twitter or whatever social platforms you're on, if they want to find out about your work, what you're doing? Robin Marx: Yeah, it's very easy. My Twitter handler is @programmingart, which I made when I was still in web design kind of circles. So, @programmingart. I mainly share QUIC and HTTP/3 and Protocol news there, and then I promise, I promise I will start my own blog soon, which I've been saying for 10 years probably. But if you look me up on Google, I would say Robin Marx, QUIC and HTTP/3, you will find a lot of blog posts I did for other blogs, you will find a lot of videos I did on YouTube. So my content is definitely out there, but it can take a while to find. Paul: Just to name drop some of the talks, so you did a married to HTTP/3 talk that people can- Robin Marx: That's my most recent one, that's not on YouTube yet. That was for Smashing Conference San Francisco. That will be out in let's say about two to three months, I think. Paul: Then Internet on Mars is not maybe directly related, but that was another one. That one caught my eye, because it raises this whole question of we all like net neutrality, but what happens when Mars is there, man? I don't want those guys to be blocking my traffic, so that's going to be an interesting conversation when that comes up. Robin Marx: Exactly. Internet on Mars was my very first conference talk I ever did at the start of my PhD. I'm still very, very proud of that one. I tried to be a little bit funny, but like you said, the questions it raises are I think very important. It's about centralization, it's about privacy, it's about censorship. All of those things taken into the concept of an interplanetary internet, but still very, very actual here on earth right now as well. Paul: It's like taking a magnifying glass to the problem though, because these are such small intervals that it's hard for human emotions to really get tied to decision-making processes. Yeah, I thought it was a really elegant way to frame the question. If I ever have a child, that's the talk I'm going to send them to to learn about the internet. All right, Robin. Well, thank you for your time, and hopefully some people can start to get tuned on to the new HTTP/3 stuff and learn about it. Robin Marx: Absolutely. I would say if people have more questions, please let me know on Twitter. I'm very happy to help in any way I can. Paul: Awesome. Thank you, Robin. Robin Marx: All right, thanks Paul. See you later. Paul: See you. Speaker 4: Thanks for listening to PodRocket. You can find us at PodRocketPod on Twitter, and don't forget to subscribe, rate and review on Apple Podcasts. Thanks.